Subscribe now

Many companies try E2E tests and Unit Test initiatives.

The might write 10 tests. 100 tests… 1000 tests.

… But did NOT achieve the goal of reducing production regression bugs.

Management is getting angry - why is time being spent on test automation, without any results to show for it?

They are right.

If we don’t achieve any outcomes, then the test automation is a WASTE of time.

Why E2E Tests don’t protect us in production

1. Limited scenarios

We can cover some scenarios:

• Testing that given valid inputs, we can place an order

But there’s many scenarios that we can’t cover because we can’t control the external systems:

• Can’t test total order total price, because we can’t control the ERP product pricing

• Can’t test that error popup when payment system is down, because we can’t force PayPal to go down

• Can’t test order discount when order is made on weekend after 5pm, because we can’t control the system clock

2. Maintenance issues

Furthermore, E2E tests are flaky and hard to maintain. If you change one thing in the UI, this can break hundreds of tests, and then you have to fix all of them. Teams often spend more time maintaining tests than actually delivering new features.

Why Unit Tests don’t protect us

Unit tests don’t protect us against regression bugs because they cover only Business Logic. They don’t cover I/O and Presentation Logic, and they don’t cover the whole user flow. That’s why you can have hundreds of thousands of tests passing, 100% code coverage… but coverage is only for business code logic, not infrastructure or presentation logic.

Why Acceptance Tests CAN protect us

Since those two approaches don’t work, what does work is Acceptance Tests.

Acceptance tests are like E2E tests but without the limitations of E2E tests.

Read more

Subscribe now

You're stuck in Legacy Code. Every day is spent fire-fighting. Manual QA is taking longer and longer. The team wants to adopt Automated Testing.

1. The teams tried E2E Tests, but they could cover only limited scenarios and those tests were constantly breaking whenever the UI changed. So they disabled the E2E Tests, and went back to relying just on Manual QA.

2. The teams tried Unit Tests, aiming for 80% code coverage. They spent a lot of time, wrote many unit tests, but QA Engineers still discovered many regression bugs. Furthermore, the unit tests frequently broke when developers did refactoring, so they had to waste time fixing the unit tests, ultimately disabling the Unit Tests. They still had to rely on Manual QA for regression bug protection.

Read more

1. Why ATDD will reduce software delivery time and reduce regression bugs, replacing manual regression testing

2. How to introduce ATDD in Legacy Code in an incremental way, while still keeping up with existing feature delivery

3. How to convince Engineering Managers, Software Engineers and QA Engineers to adopt ATDD in Legacy Code

🏆 Register now

P.S. If you’re a Paid Member, you’ll receive a 100% discount (see Event Description for instructions on how to redeem your 100% discount).

Subscribe now

During this Live Q&A, we continued answering the question:

• ​Which Test Doubles should we use in unit testing? (should we use mocking libraries or write our own test doubles) [Source]

We compared different test doubles, as well as comparing hand-written test doubles versus using mocking libraries.

Using Mocking Libraries

Many developers use mocking libraries, for example: Mockito (Java), Moq (.NET), Jest (JS/TS), etc. I was using mocking libraries for many years, too.

It’s very appealing to use mocking libraries, because you install the library and can immediately start to use it in your test. It’s also very easy how to setup with mock libraries.

Although there’s no upfront cost to setting up test doubles with mocking libraries (since no separate class is required), the issue lies in the ongoing recurring cost: for each test, we must create and set up the mock. Let’s see the impact:

• Tests have more lines (due to mock setup), which means more reading time, hence increasing maintenance costs.

• Tests are coupled to implementation, because we have to know exactly which methods will be called for each dependency; consequently, if a dependency’s interface changes (e.g., parameters change, or we need to call another method on the dependency), it will cause tests to fail with compile-time error, so we may need to fix hundreds/thousands of tests. This also means higher maintenance costs.

Using hand-coded libraries

You can see examples of hand-coded test doubles in The Little Mocker (Uncle Bob).

I remember when I first saw hand-coded test doubles, I was against the idea. Isn’t it better to use a mocking library instead?!

Read more

Subscribe now

Thank you to everyone who tuned in to my live video!

During our chat, I shared my learnings:

• Why did I shift from LinkedIn to Substack as my primary platform?

• How did I discover who the readers are that I’m writing for?

• How did I convert my work as a Coach/Consultant into the TDD series?

• Ultimately, what advice would I offer if you’re starting a Paid Newsletter?

Get more from Valentina Jemuović in the Substack app

Available for iOS and Android

Get the app

📢 On Wed 9th July 17:00 CET, I’m hosting TDD & Unit Tests - Part 4 (Live Q&A):

Register now

P.S. If you’re a Paid Member, you’ll receive a 100% discount (see Event Description for instructions on how to redeem your 100% discount).

Subscribe now

Do you want to start a Substack Newsletter?

I regret not starting Substack earlier.

If you’re a Senior Developer who wants to write a book, or share your knowledge on Substack; or if you’re a Leader/Coach/Consultant who wants to share your experience on Substack and would like some advice from me, feel free to comment below.

Leave a comment

Subscribe now

Read more

Subscribe now

During our Live Q&A Session, we addressed Alessandro’s question about TDD & Hexagonal Architecture:

Speaking about broad test at the Hexagon Ports, how this impact TDD in terms of feedback Speed?

How long take a single TDD cycle implementing a Port behaviour?

How we can we move to have short cycles?

Let’s compare the two approaches:

• Inside-out / bottom-up approach with Sociable Unit Tests

• Outside-in / top-down approach with Sociable Unit Tests

At the end: Which approach do I prefer? Why?

Read more

Subscribe now

During our Live Q&A Session, we addressed the question posted by Leandro:

What are your learnings, insights, and reflections from your experience and learning process applying TDD? I mean, like the one you mentioned in the first Q&A about sociable and solitary unit testing—particularly your reflection on initially thinking that solitary unit testing should be the way to go, but then discovering that it makes refactoring very difficult, and learning that sociable unit testing is the better approach.

Test Coupling: Behavior vs Structure

Kent Beck wrote the following:

“Tests should be coupled to the behavior of code and decoupled from the structure of code.” - Kent Beck

Tests should be coupled to Behavior

“Tests should be coupled to the behavior of code…” - Kent Beck

Requirements specify behavior, which is encoded in tests; tests are executable specifications of behavior.

Hence it is expected that when requirements change, we need to update tests to reflect the changed behavior. This naturally causes the tests to fail because the tests are expressing the changed behavior whereas the code is acting in accordance with the old behavior.

Tests should NOT be coupled to Structure

“Tests should be… decoupled from the structure of code.” - Kent Beck

To implement a certain behavior, we could structure our code in a multiple of ways. We could use one class, or maybe one class split into multiple classes. We could use if-else statements or we could use the Strategy Pattern, etc. Whichever structure we choose, it must satisfy the behavior specified.

Over time, we’re also free to make structural changes but without changing the behavior, aka refactoring (Martin Fowler):

Refactoring (noun): a change made to the internal structure of software to make it easier to understand and cheaper to modify without changing its observable behavior.

Refactoring (verb): to restructure software by applying a series of refactorings without changing its observable behavior.

Since tests should not be coupled to structure (as already said by Kent Beck) it means that tests should NOT break in response to structural changes.

This is where many developers make a mistake - they write Unit Tests which are coupled to structure!

What’s the solution? What kind of Unit Tests are decoupled from structure?

Read more

Subscribe now

During our Live Q&A Session, we answered these questions. Well, I started with the first question, and it expanded to fill the whole session.

What do most developers think an integration test is?

Suppose we have the Ordering Microservice, which communicates with PayPal, SAP and also with the Catalog Microservice, which depends on the Product Microservice.

The way that most developers will write up the integration test for the Ordering Microservice is:

• Spin up real instance of the target Microservice (Ordering Microservice)

• Spin up all of its transitive dependencies (PayPal, SAP, Catalog Microservice, Product Microservice).

This kind of test is like an E2E test… indeed, we could see an E2E test as an integration test targeting the Frontend, and spanning the whole transitive dependency graph.

Why is this kind of integration test problematic?

1. When this kind of “integration test“ for Ordering Microservice fails, we cannot isolate the fault. We don’t know if the problem is in the Ordering Microservice or PayPal or SAP or Catalog Microservice or Product Microservice, or perhaps a combined problem across all of these. All of these teams have to get together, do some tracing & debugging, to find out which team is at fault.

2. It also means that if for whatever reason an external system is unavailable (e.g. SAP test Instance is unavailable) or some dependent microservice is unavailable (e.g. Catalog Microservice is unavailable), then the Ordering Microservice team can NOT run the Ordering Microservice Integration Test, they’re blocked.

So what’s the solution?

As I explained in the Live Q&A session, the solution is:

Read more

Subscribe now

During our Live Q&A Session, we answered these questions:

1. 00:32 - Fake Agile, Estimations & Deadlines (Vivek) - How do we handle sprints, deadlines & estimations? Should estimations be based on past data or should developers do the estimations?

2. 11:12 - Smoke Tests & External Systems with Human Interaction (Mansour) How do we handle Smoke Tests whereby we have integration with auth providers (e.g. Google, Microsoft) as well as payment providers (e.g. Stripe) that require SMS code validation?

3. 15:01 - Combinatorial Explosion & Test Types (Mansour) Should we test each combination of auth provider and payment provider, or is there a more reasonable approach?

4. 24:01 - Unit Tests - Sociable vs Solitary (Alessandro) Should we write broader or narrower Unit Tests in Hexagonal Architecture?

5. 38:07 - Hexagonal Architecture & Adapter Testing (Alessandro) In Hexagonal Architecture, suppose I have an adapter that reads JSON from a file (it has some logic - checking if path exists). Should we test the adapter in isolation?

6. 42:14 - Explaining Testing & Pipelines to Business Executives (Jason) How to explain the importance of pipelines & testing to executives, who see it just as extra money and not worth the investment?

7. 50:48 - Balancing Learning & Client Work (Jason) How to balance learning by reading versus learning by doing? I spend time to help clients with these things, but don’t have time to practice myself.

8. 1:00:36 - AI Code & Critical Thinking (Mathieu) How will AI reshape the developer mindset, how to ensure that teams remain critical thinkers rather than passive consumers?

Read more

Subscribe now

I am excited to announce that I plan to start TDD Live Sessions in 2025!

Our Premium Members will be able to:

• Attend the live sessions

• Watch the replay of the sessions (as sessions will be recorded)

Before I launch these video sessions, I have some questions for you…

What’s your preferred way of learning?

Which format do you prefer?

I’m planning to start videos (might be webinars or group video calls or something, we’ll see what you prefer)

• Group video call - all participants will be visible on video/audio you're visible via video, and we can have a live face-to-face discussion

• Webinar & live text chat - I’ll be visible on video, you can write questions via chat, then I’ll read out your questions and I’ll answer it live

• Both are fine

Note: In both cases, the session would be recorded anyway.

What time do you prefer for our sessions?

I’m planning that the sessions be 1hr. In that case, which time range would you prefer?

Which topic is currently most relevant for you?

Based on your current job, perhaps you’ve tried some best practices and you have questions about it. Which of these topics is most relevant for you, and where you have questions that you’d like to ask me?

Which sequence do you prefer?

What would you find more useful for us to start with:

• Concepts (TDD, Hexagonal, etc.) - Webinars / group calls (with live chat) that cover key concepts/foundations regarding TDD, Modern Test Pyramid, Hexagonal Architecture, Clean Architecture etc… they would be much bigger picture than the TDD in Legacy Code series, and we could jump between those topics across sessions

• TDD in Legacy Code series - Webinars / group calls (with live chat) whereby we go in the sequence that’s in the TDD in Legacy Code series, we got up to Pipelines, Acceptance Tests & ATDD…. whereas topics like Hexagonal Architecture will be further down the road…

Based on this, what do you find more useful?

Do you have any suggestions for me?

I’d like to hear your feedback and comments, and any further ideas/suggestions you have, or any questions.

Leave a comment