Modern Test Pyramid
The Old Test Pyramid (Unit, Integration & E2E tests) - is dead. It's got fundamental problems. That's why we're introducing the Modern Test Pyramid, so you can practice effective Automated Testing.
🔒Hello, this is Valentina with a premium issue of the Optivem Journal. I help Engineering Leaders & Senior Software Developers apply TDD in Legacy Code. This article is part of the TDD in Legacy Code series. To get these articles in your inbox every week, subscribe:
The Old Test Pyramid is dead
Most developers see the Test Pyramid in the following way:
We previously outlined the problems with the Old Test Pyramid:
Overuse of E2E tests, often reaching the Inverted Test Pyramid
No consensus on the definition of Integration Tests, and over-reliance on large scoped integration tests makes it hard to maintain
No consensus on the definition of Unit Tests, and writing unit tests that are structurally coupled to code with over-mocking, thus fragile when refactoring
The Test Pyramid does not include Acceptance Tests needed for testing Acceptance Criteria, but instead it relies on fragile E2E Tests for that
The Test Pyramid doesn’t provide us guidance for testing microservices, specifically it doesn’t even mention Component Tests & Contract Tests which are crucial for microservice testing
Modern Test Pyramid
Now let’s look at the Modern Test Pyramid:
We can also view it as a Test Spectrum where we illustrate the test spans:
Let’s see the test types - why they are used, what they target and span:
Smoke Tests
We use Smoke Tests to verify that the System is up-and-running.
Smoke Tests target the System - they span the System and External System Stubs or External System Test Instances.
E2E Tests
We use E2E Tests to verify System behavior, however the scenarios are limited.
E2E Tests target the System - they span the System and External System Test Instances. They are executed on the E2E Environment.
Acceptance Tests
We use Acceptance Tests to verify behavior of the System. We have the possibility to execute any scenario.
Acceptance Tests target the System, and they span the System and External System Stubs.
External System Contract Tests
External System Contract Tests are used to verify that we can communicate with External System Test Instances and that their contract matches our expectations.
External System Contract Tests target (and span) the External System Test Instances.
Component Tests
We use Component Tests to verify the behavior of Components (e.g. verifying behavior of Frontend, Microservice #1, Microservice #2, Microservice #3)
Component Tests target the Components, and span Component Dependency Stubs (e.g. targeting Microservice #1 and Microservice #2 Stub and External System A Stub)
Contract Tests
We use Contract Tests to verify communication between a Component and its Dependencies.
Common example is Consumer Driven Contract Testing
Unit Tests
We use Unit Tests to verify application-side business behavior.
Narrow Integration Tests
We use Narrow Integration Tests to verify presentation behavior and infrastructure behavior within the Component.
The Modern Test Pyramid solves problems
The Modern Test Pyramid SOLVES the problems of the Old Test Pyramid in the following ways:
Problematic E2E tests
In the Old Test Pyramid, E2E Tests are problematic because they are limited regarding scenarios when testing System Behavior. The tests are often fragile and non-deterministic.
In the Modern Test Pyramid, we solve that problem through Acceptance Tests (together with External System Contract Tests) which enables us to fully test System Behavior in a deterministic, reliable way.
Problematic Integration tests
In the Old Test Pyramid, Integration Tests are problematic because developers generally interpreted them as Broad Integration Tests. They had similar characteristics as E2E Tests - limited scenarios and fragility, with the additional problem that it was hard to isolate where the fault is occurring.
That’s why in the Modern Test Pyramid, we introduce Component Tests & Contract Tests to replace Broad Integration Tests. We can fully test each Component and its interactions with other Components in a reliable way.
No clue how to handle Microservice Architecture
In the Old Test Pyramid, there was no guidance how to handle Microservice Architecture. Often teams would use Broad Integration Testing for microservices, then waste time debugging which microservice was at fault, and cause a lot of dependencies between microservice teams.
The Modern Test Pyramid contains Component Tests, which are very effective for separately testing Frontend and each Microservice. It also contains Contract Tests, which are effective for testing communication with another microservice without having both of them running.
No clue how to test Presentation & Infrastructure Logic
In the Old Test Pyramid, there’s no guidance on testing Presentation Logic and Infrastructure Logic, but only Business Logic. Sure, we’d use Unit Tests for Business Logic, but what kind of tests do we need for Presentation Logic & Infrastructure Logic? The Old Test Pyramid doesn’t provide an answer.
In the Modern Test Pyramid, we solve that problem with Narrow Integration Tests. ⬇️⬇️⬇️