Testing is more than running the software

This is old news to some most, but I'm going to state it anyway.

"Testing is more than running the software"

Today I finished a bit larger test project that I've been working on for the last month (it's "done-done"... it even has documentation!).

The test project is test automation for doing performance measurements when building the Android source code, and the solution is integrated into our CI system (ElectricCommander of course).
I'll write a longer blog piece on the project in itself, but for now I just want to touch up on the aforementioned.
When developing performance testing and gathering measurements, I have long since learned to not trust all my assumptions, in this scenario I am the (test) developer that implements what, how and in which context we do performance measurements.

In these situations I want as many eyes on my solution as possible so that errors and mistakes in assumptions I make can be corrected.

After all these numbers is usually used by many stakeholders, developers use them to improve their code, product managers use them for decisions about the product backlog/roadmap, and marketing/sales might even use them with customers.
So after having everything in place I spent a lot of time writing up an internal wiki page where I described the tool, which I so cleverly named AATP (AutomatedAndroidPerformanceTests... pure genius), described the purpose, the implementation, the hardware setup, the measurements we take, and so on.
Part of that is because I want other people at my company to be able to use it and extend upon it, but also to have as much transparency as possible so that people can point out where I'm wrong.
And sure enough a kick-ass tester Peter L. Guilhamet who'm I respect (now for some reason head of DevOps... go figure) sent me an email an hour later pointing out several not-so-serious issues, but also some that were big issues that I had missed along the way, all in all 14 issues!
He had done so solely by looking through my email and documentation, and found all the issues.
Not once had he executed my tool.
"Testing is more than running the software"
These issues will be fixed on Monday, that is if I can wait until then.
But after that I will feel a lot better about the test solution, and others will probably continue to scrutinize it as well.
So, who test the testers assumptions? Why your other testers of course!
Never be afraid to disclose how you test, and thus take responsibility for your testing.
We all make mistakes and that's ok, that's why the test profession exists.

You can find this and other blog posts by Kristoffer Nordstrom at http://contextdriven.blogspot.com.