If all QA can be covered by E2E tests, why even bother writing unit and integration tests at all?

TL;DR

When designing your test strategy, make sure that the test pyramid is actually shaped more or less like the ancient Egyptians would build it: perform as many tests as possible on the lowest part of the test pyramid because it will save you money.

Unit Tests - Fast Execution and Instant Feedback

Unit tests should be the base of your construction. They're fast and provide instant feedback. Decent coverage makes refactoring nice and easy since you can make sure you didn't break anything by instantly running unit tests after every small change you introduce. Don't obsess over the percentage of coverage though. Good coverage cuts through your business logic, not simply displays a big number.

Integration Tests - The Cost Is Growing

You won't cover everything with unit tests. At some point, you'll have to move on to integration tests that verify the correctness of interactions and communication between components you've already unit-tested individually. They should still be easy to reason about and debug, but you'll have to wait some time (seconds or - worst case - minutes) for the results. Building Docker images takes time, spinning up databases takes time, and setting up additional dependencies adds even more delay. When it all adds up, you'll have the reason why those tests can't be run as frequently as unit tests: time is money, and you don't have an infinite bag of gold, do you?

End to End Tests - Expensive but Necessary

Integration tests alone won't provide enough coverage. At some point, you'll want to test the actual experience that your users interact with, i.e., perform API or UI tests. They are expensive mainly due to the complexity and scale of infrastructure required to run them. You have to maintain all dependencies required by your application like databases, authentication and authorization services, infrastructure like messaging brokers and so on. But that's not the end. You’ll also need realistic, production-like data, ensuring it doesn’t accumulate or become stale across multiple test runs. The execution time gets even longer for the end-to-end tests, but you'll spend even more time maintaining the whole environment that supports these tests. It'll cost at least an order of magnitude more than what you had to invest in both unit and integration tests; that's why you'll want to limit the coverage only to what couldn't be covered with the previous two kinds.

Conclusion

Every test type has its place, but let the economics guide your strategy. Invest heavily in rapid-feedback unit tests, be selective with integration tests, and use expensive E2E tests only where absolutely necessary. Your wallet will thank you and so will your dev team when they're not waiting hours for test runs to complete.

Recommended Reading

  • Roy Osherove - The Art of Unit Testing
  • Vladimir Khorikov - Unit Testing Principles, Practices, and Patterns

If this article helped you understand how to optimize your testing architecture and cut development costs, consider buying me a coffee ☕️