While wondering over internet, I just come across the "mutation testing" and here I blog about it. The blog largely said about one such framework "PIT tests".
When we say my code coverage is 80%, that only mean, in simple term, your unit tests cover 80% of your code base. Do they really tell you what your tests are really testing? - No. To answer this, there is a thing that called as "mutation testing". The framework I am talking about is PIT tests; they said it’s like automation tests for your tests. Cool!
How does it work?
Well in plain simple language, it mutate (modify the byte code at runtime) some logical part, decision taking part of your code and then run tests against such modified code, if your test fail (called as mutation killed), your test is OK, if its passed (called mutation survived), beware, there might be some bug hidden there. So, more the mutation killed more good your tests are.
Example:
The simplest example might be,
if ( i < 5) return "hi"; else return "bye";Consider you write tests for it, and all are passing. Now, when pit tests executes over such tests, they mutated to this runtime as
if ( i < 5) // operator < mutate to <=
return "hi";
else
return "bye";
And tests are rerun over such mutated code, if you really test such methods for boundary condition; everything will be good otherwise bug will occurs.
There is few default mutators that are applied while mutation testing using PIT.
That’s all, give it a try and see if its suits in your organization culture ;).
What’s missing?
Only thing I felt that need to improve for PIT tests are the HTML report visibility, the report doesn’t clearly states how many test got the problem, means how many tests are killed and how many are survived. I think this info should be aggregated on index page.
Comments