Modern Test Pyramid

The Old Test Pyramid (Unit, Integration & E2E tests) - is dead. It's got fundamental problems. That's why we're introducing the Modern Test Pyramid, so you can practice effective Automated Testing.

🔒Hello, this is Valentina with a premium issue of the Optivem Journal. I help Engineering Leaders & Senior Software Developers apply TDD in Legacy Code. This article is part of the TDD in Legacy Code series. To get these articles in your inbox every week, subscribe:

---

The Old Test Pyramid is dead

Most developers see the Test Pyramid in the following way:

We previously outlined the problems with the Old Test Pyramid:

1. Overuse of E2E tests, often reaching the Inverted Test Pyramid

2. No consensus on the definition of Integration Tests, and over-reliance on large scoped integration tests makes it hard to maintain

3. No consensus on the definition of Unit Tests, and writing unit tests that are structurally coupled to code with over-mocking, thus fragile when refactoring

4. The Test Pyramid does not include Acceptance Tests needed for testing Acceptance Criteria, but instead it relies on fragile E2E Tests for that

5. The Test Pyramid doesn’t provide us guidance for testing microservices, specifically it doesn’t even mention Component Tests & Contract Tests which are crucial for microservice testing

Modern Test Pyramid

Now let’s look at the Modern Test Pyramid:

We can also view it as a Test Spectrum where we illustrate the test spans:

Let’s see the test types - why they are used, what they target and span:

Smoke Tests

• We use Smoke Tests to verify that the System is up-and-running.

• Smoke Tests target the System - they span the System and External System Stubs or External System Test Instances.

E2E Tests

• We use E2E Tests to verify System behavior, however the scenarios are limited.

• E2E Tests target the System - they span the System and External System Test Instances. They are executed on the E2E Environment.

Acceptance Tests

• We use Acceptance Tests to verify behavior of the System. We have the possibility to execute any scenario.

• Acceptance Tests target the System, and they span the System and External System Stubs.

External System Contract Tests

• External System Contract Tests are used to verify that we can communicate with External System Test Instances and that their contract matches our expectations.

• External System Contract Tests target (and span) the External System Test Instances.

Component Tests

• We use Component Tests to verify the behavior of Components (e.g. verifying behavior of Frontend, Microservice #1, Microservice #2, Microservice #3)

• Component Tests target the Components, and span Component Dependency Stubs (e.g. targeting Microservice #1 and Microservice #2 Stub and External System A Stub)

Contract Tests

• We use Contract Tests to verify communication between a Component and its Dependencies.

• Common example is Consumer Driven Contract Testing

Unit Tests

• We use Unit Tests to verify application-side business behavior.

Narrow Integration Tests

• We use Narrow Integration Tests to verify presentation behavior and infrastructure behavior within the Component.

The Modern Test Pyramid solves problems

The Modern Test Pyramid SOLVES the problems of the Old Test Pyramid in the following ways:

Problematic E2E tests

In the Old Test Pyramid, E2E Tests are problematic because they are limited regarding scenarios when testing System Behavior. The tests are often fragile and non-deterministic.

In the Modern Test Pyramid, we solve that problem through Acceptance Tests (together with External System Contract Tests) which enables us to fully test System Behavior in a deterministic, reliable way.

Problematic Integration tests

In the Old Test Pyramid, Integration Tests are problematic because developers generally interpreted them as Broad Integration Tests. They had similar characteristics as E2E Tests - limited scenarios and fragility, with the additional problem that it was hard to isolate where the fault is occurring.

That’s why in the Modern Test Pyramid, we introduce Component Tests & Contract Tests to replace Broad Integration Tests. We can fully test each Component and its interactions with other Components in a reliable way.

No clue how to handle Microservice Architecture

In the Old Test Pyramid, there was no guidance how to handle Microservice Architecture. Often teams would use Broad Integration Testing for microservices, then waste time debugging which microservice was at fault, and cause a lot of dependencies between microservice teams.

The Modern Test Pyramid contains Component Tests, which are very effective for separately testing Frontend and each Microservice. It also contains Contract Tests, which are effective for testing communication with another microservice without having both of them running.

No clue how to test Presentation & Infrastructure Logic

In the Old Test Pyramid, there’s no guidance on testing Presentation Logic and Infrastructure Logic, but only Business Logic. Sure, we’d use Unit Tests for Business Logic, but what kind of tests do we need for Presentation Logic & Infrastructure Logic? The Old Test Pyramid doesn’t provide an answer.

In the Modern Test Pyramid, we solve that problem with Narrow Integration Tests. ⬇️⬇️⬇️