Modern Test Pyramid - System Level

Many teams are stuck with slow QA, waiting days/weeks for QA Engineers to test User Stories as well as Regression Testing. The feedback loop is too slow. Can we have much faster feedback?

May 09, 2025

∙ Paid

Manual System Regression Testing = Waste!

The default approach that most companies still rely on is Manual System Testing, which is done by QA Engineers at the end of the Sprint & also in preparation for the Release.

Problem #1: Slow Delivery

Developer Time Wasted - Manual Regression Testing: During Sprints, as Developers are implementing User Stories, they also need to do some basic Manual Regression Testing to make sure their User Stories didn’t break anything. Generally, they do very basic Manual Regression Testing, it is not extensive, maybe just a few basic scenarios.
QA Engineer Time Wasted - Manual Regression Testing: At the end of the Sprint, QA Engineers need to test User Stories as well as conducting some basic Manual Regression Testing. Furthermore, prior to releasing the System, they need to execute the Manual Regression Test suite to make sure there’s no regression bugs.
Developer Time Wasted - Expensive Late Rework: Since Developers have to wait for the QA Engineers to perform the time-consuming Manual Regression Testing

The impact of the time wasted is that we have a high internal cost; we’re throwing money down the drain; we’re paying our developers & QA Engineers to waste time; the team spends more time on bugs rather than features.

Problem #2: Buggy Releases

Customer-facing Regression Bugs: Prior to the Release, the QA Engineers should do “full“ Regression Testing (and keep on repeating that until Developers have resolved all Regression Bugs). However, in practice, that is infeasible - because Manual Regression Testing is too slow; the company can’t hire an infinite number of QA Engineers; the company can’t indefinitely postpone a release due to repeated Manual Regression Test cycles. So what’s the pragmatic solution? QA Engineers are forced to cut corners, by executing only “partial“ Regression Testing, rather than “full“ Regression Testing. This causes Regression Bugs to escape to Customers, to be reported to the Customer Support Team.

The impact of customer-facing bugs is that customers become dissatisfied, leading to potential customer loss (hence lost revenue). Furthermore, it can cause poor reputation in the market, thus causing loss of potential future clients (hence losing potential future revenue).

Automated System Regression Testing = Solution

Impact #1: Faster Delivery

Developers do NOT have to do Manual Testing of their User Stories and don’t have to do Manual Regression Testing either. This reduces Developer time spent per User Story.
QA Engineers do NOT execute Manual Regression Testing. Instead, they just do Exploratory Testing (i.e. creative testing).
Developers incur LOW cost of rework. Even if Developers introduce a Bug, they will discover it quickly through Automated System Tests (within <= 1hr)

The impact of this is that we have lower internal cost of change, hence we can spend more time on features rather than bug fixes, and we can deliver faster.

Impact #2: Working Releases

No more Customer Regression Bugs! With Automated System Tests, we can be sure that the full suite is always executed.

The impact of this is that customers can trust our software, leading to increased customer retention, hence we can maintain our revenue consistently. Furthermore, it aids customer referrals and positive brand image, hence increasing the number of new customers we get, hence increased revenue.