Avoiding Logic in Tests
Hello! Today, let’s discuss the problem of adding logic in tests.
Consider the following code, which calculates the total savings (or economy) we can realize by applying a discount percentage to a list of prices1:
Now, we want to write a unit test to validate this economy function. We might want to reuse the same logic as the function itself to compute the expected result in the test like this:
The test passes ✅, and everything looks good—until we deploy to production and realize there’s a bug! Can you guess what’s wrong in both the function and why the test didn’t catch it?
If we print the results of the tests with the three prices and a 10% discount, we get -17.5. Yet, as it’s an economy value, we expected a positive value from this function. The issue lies in the negative sign of our economy function:
Instead of calculating a positive discount, the logic applies a negative value.
Why didn’t we catch this bug in our tests? To calculate the expected economy, we duplicated the same flawed logic in the test itself, thus reproducing the bug in the test.
Let’s rewrite the test with a hardcoded expected value. In this case, it would become apparent that something was wrong in our code:
=== RUN TestEconomy
wrong economy: want 17.5, got -17.5This example highlights an important principle in testing: avoid adding logic to tests, especially logic that mirrors the function under test.
A test should:
Use hardcoded values for expected results whenever possible.
Favor simplicity and readability, even if it means some repetition.
Be written in a way that prevents the reader from doing mental computation to verify the test’s correctness.
A test shouldn’t:
Mirror the code logic it is meant to validate.
Include elements like loops, concatenations, or complex calculations that could obscure the intended result.
Tomorrow, we will discuss code coverage.
The examples are in Go.
The underlying principle is double checking. If you get to the same answer through two methods, you’re much more confident.
> Be written in a way that prevents the reader from doing mental computation to verify the test’s correctness.
The validity of the above statement hinges on the definition of "mental computation".
> Include elements like loops, concatenations, or complex calculations that could obscure the intended result.
Complex inputs/preconditions and/or complex outputs/post-conditions are often part of the requirements. How do you avoid complex calculations when such states/data structures have to be created for verification?
For example, when the producer of the inputs and/or the consumer of the outputs are machines, the raw data may not be trivial to either assemble for the test's (human) author or to parse by the test's (human) reader. Programmatically assembling input and/or output data may be less error prone in such cases than literally including the test-data.
State-related test-complexity can be often observed (and seems to be inevitable) with white-box testing time-sensitive functionality. Concurrency testing (e.g. mocking known potentially problematic states involving concurrent functions) is one large obvious sub-category here.