Change the default plugin behavior in perf tests