Legacy Code is a nightmare to maintain. Escape this by introducing TDD. I created the TDD in the Legacy Code series to help you achieve this step-by-step.
I've heard many developers say that they have integration and unit tests that span the entire system to check that it's working, so why would they need more tests?
When they say they have integration tests, it generally refers to Broad Integration Tests. The problem with Broad Integration Tests is that they don't provide us with component isolation. For example, a Broad Integration test for Microservice 1 is also spanning its dependencies - which might be Microservice 2, Microservice 3 and PayPal. So a Broad Integration Test fails, we have no idea is the problem with Microservice 1 or Microservice 2 or Microservice 3 or PayPal... That's why instead of Broad Integration Tests I like to replace it with Component Tests & Contract Tests, so that Microservice 1 team gets feedback if Microservice 1 works, and each team gets feedback if their component works...
So I wouldn't say "more tests", but rather "instead of". So instead of Broad Integration Tests, we'd have Component Tests and Contract Tests.
"In my mind, TDD and Legacy Rescue are two separate workflows. Both involve test coverage and refactoring, and in both cases, refactoring is directed by the Four Rules of Simple Design (of which the first rule necessitates test coverage). The difference is in the end-goal of refactoring. In TDD we refactor relentlessly (or, in Kent Beck's words, mercilessly) until we cannot refactor anymore. In Legacy Rescue, by contrast, we follow Arlo Belshee's adage: "I don't want good. I want better ... fast."
Now, while doing Legacy Rescue, we will sometimes want to shift into the TDD workflow, particularly when we have successfully isolated our change point and are ready to implement new functionality. But by this point, we are no longer in Legacy Code.
3. Refactor the Acceptance Tests to be maintainable
4. Introduce External System Contract Tests (by migrating communication covered through E2E Tests)
5. Refactor External System Contract Tests to be maintainable
Acceptance Test Driven Development
- At this point, we're able to practice ATDD (for new User Stories, Bug Fixes) because the existing code is covered by Acceptance Tests, so then as part of ATDD we just add onto that test suite
- However, the underlying code is "still" legacy... we'll need to introduce tests within the system (Component Tests, Unit Tests), and it's only when we get to those levels that we'll be able to efficiently refactor the legacy code... It's only at that point that we've escaped legacy.
Thanks for responding, Valentina. I see that you are extending the Feathers Legacy Rescue algorithm by recognizing that you need to begin with whatever test coverage you can get. Very important.
My point remains, however, that this cannot be called "TDD in Legacy Code." Even once you get into the early steps of the Feathers algorithm, you are not (by definition) doing TDD until you are adding new functionality, and that code that you are test-driving is not Legacy Code (but rather a test-driven island in a sea of Legacy Code).
You demonstrate in your response that ATDD in Legacy Code is possible. I still maintain, however, that "TDD in Legacy Code" is a contradiction. I like to say that TDD is a healthy lifestyle, whereas Legacy Rescue is curing cancer.
(Part 2/3) "My point remains, however, that this cannot be called "TDD in Legacy Code." Even once you get into the early steps of the Feathers algorithm, you are not (by definition) doing TDD until you are adding new functionality, and that code that you are test-driving is not Legacy Code (but rather a test-driven island in a sea of Legacy Code)."
"you are not (by definition) doing TDD until you are adding new functionality" --> that's correct.
- OLD CODE: We're applying Test Last when retroactively introducing tests on Legacy Code (that's a big focus in my series). However, if we decide to do a rewrite, then we could apply TDD (I don't write much about this, but left up to reader)
- NEW CODE: We're applying TDD only when making behavioral changes, which could be due to User Story tickets, Bug tickets, which cause us to change behavior. This causes us to change code or write new code.
"and that code that you are test-driving is not Legacy Code (but rather a test-driven island in a sea of Legacy Code)"
I like how you used the terminology "island". Indeed, yes, the Legacy Code base is huge and as we're incrementally introducing tests (in whatever way, respective of TDD or TLD), it means the code is no longer Legacy Code, because by Feathers' definition, Legacy Code is code without tests, so as soon as the code becomes covered by tests (those islands of code) in either Test Last or TDD, it means that code is no longer legacy.
HOWEVER, this is where I extend it again to different levels. In Feathers book, the main mention I see is unit tests (not component tests, acceptance tests). So when he says Legacy Code means code without tests, he is implicitly referring to unit tests. Now with the extension I apply, the notion of legacy is at different levels:
0. INITIAL: Let's take some behavior, such as creating an order. There's some code in frontend, some code in backend, and it's about creating an order. Suppose starting state is that it is Legacy Code at all levels - it's Legacy Code at system level (absence of acceptance tests), it's Legacy Code at component level (absence of component tests) and it's Legacy Code at unit level (absence of unit tests)
1. LEGACY RESCUE AT SYSTEM LEVEL: So as we're peeling the onion, let's start off by introducing acceptance testability and writing some acceptance tests for creating an order. Now that the code is covered by Acceptance Tests, then the code is no longer Legacy Code at the System Level, because it's covered by tests. However, it still remains Legacy Code at the Component level and lower down. Furthermore, for any new functionality, as we practice Acceptance Test Driven Development, tests are a natural byproduct, hence any new code is not Legacy Code at the System Level.
2. LEGACY RESCUE AT COMPONENT LEVEL: Let's write some component tests for the frontend (testing form for placing order, mocking out backend) and some component tests for backend (testing POST /orders, mocking out external ERP). Now that island of code is covered by Component Tests, it is NOT Legacy Code at the Component Level .... but it is still Legacy Code at the Unit Level (because it's not covered by unit tests). Furthermore for any new functionality, as we practice Acceptance Test Driven Development, and within it practice Component Test Driven Development, both Acceptance Tests & Component Tests are a natural byproduct, hence any new code is not legacy code at the system level, and not legacy code at the component level.
3. LEGACY RESCUE AT UNIT LEVEL: Lastly, let's zoom into frontend and backend, and within each of them introduce unit testability (breaking down dependencies, could adopt HA). Now that island of code is covered by unit tests, hence it is NOT Legacy Code at the Unit Level... Furthermore for any new functionality, as we practice Acceptance Test Driven Development, and within it practice Component Test Driven Development, and within it practice (Unit) Test Driven Development, then both Acceptance Tests & Component Tests & unit tests are a natural byproduct, hence any new code is not legacy code at the system level, and not legacy code at the component level, and not legacy at the unit level.
It's only at step 3. that I could say that we fully escaped Legacy Code. And when we do reach that level, then yes, I agree in full sense, at all levels that (as you wrote) "and that code that you are test-driving is not Legacy Code (but rather a test-driven island in a sea of Legacy Code)."
(Part 3/3) "You demonstrate in your response that ATDD in Legacy Code is possible. I still maintain, however, that "TDD in Legacy Code" is a contradiction. I like to say that TDD is a healthy lifestyle, whereas Legacy Rescue is curing cancer."
That's correct. We go at different levels:
1. System Level
1.a. Legacy Rescue at System Level (cure)
1.b. TDD at System Level (health) i.e. ATDD loop
2. Component Level
2.a. Legacy Rescue at Component Level (cure)
2.b. TDD at System/Component Level (health), i.e. ATDD loop, and within it we have CTDD loops
3. Unit Level
3.a. Legacy Rescue at Unit Level (cure)
3.b. TDD at System/Component/Unit Level (health), i.e. ATDD loop, and within it we have CTDD loops, and within them we have (U)TDD loops
I'd say when we reach stage 3, that's when we have the achieved the full cure and the full health. But look, even stages 1 and stages 2 (interim stages) are much better than where we are.
(Part 1/3) "Thanks for responding, Valentina. I see that you are extending the Feathers Legacy Rescue algorithm by recognizing that you need to begin with whatever test coverage you can get. Very important."
Exactly! Yes, that's the intention.
I found Michael Feathers work to be an excellent inspiration, esp. how he does the untangling of dependencies to introduce unit tests.
I expected that idea across the levels, I view now test levels like layers of an onion, that we're spelling an onion.
So the extension I applied was to go beyond unit tests, but applying the same principle.
1. Introduce Acceptance Testability, then can write Acceptance Tests & practice Acceptance Test Driven Development
2. Introduce Component Testability, then can write Component Tests & practice Component Test Driven Development
3. Introduce Unit Testability, then can write Unit Tests & practice (Unit) Test Driven Development
I've heard many developers say that they have integration and unit tests that span the entire system to check that it's working, so why would they need more tests?
When they say they have integration tests, it generally refers to Broad Integration Tests. The problem with Broad Integration Tests is that they don't provide us with component isolation. For example, a Broad Integration test for Microservice 1 is also spanning its dependencies - which might be Microservice 2, Microservice 3 and PayPal. So a Broad Integration Test fails, we have no idea is the problem with Microservice 1 or Microservice 2 or Microservice 3 or PayPal... That's why instead of Broad Integration Tests I like to replace it with Component Tests & Contract Tests, so that Microservice 1 team gets feedback if Microservice 1 works, and each team gets feedback if their component works...
So I wouldn't say "more tests", but rather "instead of". So instead of Broad Integration Tests, we'd have Component Tests and Contract Tests.
I received an insight question on LinkedIn, from Daniel Steinberg https://www.linkedin.com/feed/update/urn%3Ali%3Aactivity%3A7265432517472645120/
"In my mind, TDD and Legacy Rescue are two separate workflows. Both involve test coverage and refactoring, and in both cases, refactoring is directed by the Four Rules of Simple Design (of which the first rule necessitates test coverage). The difference is in the end-goal of refactoring. In TDD we refactor relentlessly (or, in Kent Beck's words, mercilessly) until we cannot refactor anymore. In Legacy Rescue, by contrast, we follow Arlo Belshee's adage: "I don't want good. I want better ... fast."
Now, while doing Legacy Rescue, we will sometimes want to shift into the TDD workflow, particularly when we have successfully isolated our change point and are ready to implement new functionality. But by this point, we are no longer in Legacy Code.
Your thoughts?"
We'll we are incrementally escaping Legacy at different levels...
The stages in Legacy Code are as follows:
E2E Tests - Test Last
1. Introduce E2E Testable Architecture
2. Introduce E2E Tests
3. Refactor the E2E Tests to be maintainable
Acceptance Tests in Legacy Code
1. Introduce Acceptance Testable Architecture
2. Introduce Acceptance Tests (by migrating E2E Tests to Acceptance Tests)
3. Refactor the Acceptance Tests to be maintainable
4. Introduce External System Contract Tests (by migrating communication covered through E2E Tests)
5. Refactor External System Contract Tests to be maintainable
Acceptance Test Driven Development
- At this point, we're able to practice ATDD (for new User Stories, Bug Fixes) because the existing code is covered by Acceptance Tests, so then as part of ATDD we just add onto that test suite
- However, the underlying code is "still" legacy... we'll need to introduce tests within the system (Component Tests, Unit Tests), and it's only when we get to those levels that we'll be able to efficiently refactor the legacy code... It's only at that point that we've escaped legacy.
Thanks for responding, Valentina. I see that you are extending the Feathers Legacy Rescue algorithm by recognizing that you need to begin with whatever test coverage you can get. Very important.
My point remains, however, that this cannot be called "TDD in Legacy Code." Even once you get into the early steps of the Feathers algorithm, you are not (by definition) doing TDD until you are adding new functionality, and that code that you are test-driving is not Legacy Code (but rather a test-driven island in a sea of Legacy Code).
You demonstrate in your response that ATDD in Legacy Code is possible. I still maintain, however, that "TDD in Legacy Code" is a contradiction. I like to say that TDD is a healthy lifestyle, whereas Legacy Rescue is curing cancer.
(Part 2/3) "My point remains, however, that this cannot be called "TDD in Legacy Code." Even once you get into the early steps of the Feathers algorithm, you are not (by definition) doing TDD until you are adding new functionality, and that code that you are test-driving is not Legacy Code (but rather a test-driven island in a sea of Legacy Code)."
"you are not (by definition) doing TDD until you are adding new functionality" --> that's correct.
- OLD CODE: We're applying Test Last when retroactively introducing tests on Legacy Code (that's a big focus in my series). However, if we decide to do a rewrite, then we could apply TDD (I don't write much about this, but left up to reader)
- NEW CODE: We're applying TDD only when making behavioral changes, which could be due to User Story tickets, Bug tickets, which cause us to change behavior. This causes us to change code or write new code.
"and that code that you are test-driving is not Legacy Code (but rather a test-driven island in a sea of Legacy Code)"
I like how you used the terminology "island". Indeed, yes, the Legacy Code base is huge and as we're incrementally introducing tests (in whatever way, respective of TDD or TLD), it means the code is no longer Legacy Code, because by Feathers' definition, Legacy Code is code without tests, so as soon as the code becomes covered by tests (those islands of code) in either Test Last or TDD, it means that code is no longer legacy.
HOWEVER, this is where I extend it again to different levels. In Feathers book, the main mention I see is unit tests (not component tests, acceptance tests). So when he says Legacy Code means code without tests, he is implicitly referring to unit tests. Now with the extension I apply, the notion of legacy is at different levels:
0. INITIAL: Let's take some behavior, such as creating an order. There's some code in frontend, some code in backend, and it's about creating an order. Suppose starting state is that it is Legacy Code at all levels - it's Legacy Code at system level (absence of acceptance tests), it's Legacy Code at component level (absence of component tests) and it's Legacy Code at unit level (absence of unit tests)
1. LEGACY RESCUE AT SYSTEM LEVEL: So as we're peeling the onion, let's start off by introducing acceptance testability and writing some acceptance tests for creating an order. Now that the code is covered by Acceptance Tests, then the code is no longer Legacy Code at the System Level, because it's covered by tests. However, it still remains Legacy Code at the Component level and lower down. Furthermore, for any new functionality, as we practice Acceptance Test Driven Development, tests are a natural byproduct, hence any new code is not Legacy Code at the System Level.
2. LEGACY RESCUE AT COMPONENT LEVEL: Let's write some component tests for the frontend (testing form for placing order, mocking out backend) and some component tests for backend (testing POST /orders, mocking out external ERP). Now that island of code is covered by Component Tests, it is NOT Legacy Code at the Component Level .... but it is still Legacy Code at the Unit Level (because it's not covered by unit tests). Furthermore for any new functionality, as we practice Acceptance Test Driven Development, and within it practice Component Test Driven Development, both Acceptance Tests & Component Tests are a natural byproduct, hence any new code is not legacy code at the system level, and not legacy code at the component level.
3. LEGACY RESCUE AT UNIT LEVEL: Lastly, let's zoom into frontend and backend, and within each of them introduce unit testability (breaking down dependencies, could adopt HA). Now that island of code is covered by unit tests, hence it is NOT Legacy Code at the Unit Level... Furthermore for any new functionality, as we practice Acceptance Test Driven Development, and within it practice Component Test Driven Development, and within it practice (Unit) Test Driven Development, then both Acceptance Tests & Component Tests & unit tests are a natural byproduct, hence any new code is not legacy code at the system level, and not legacy code at the component level, and not legacy at the unit level.
It's only at step 3. that I could say that we fully escaped Legacy Code. And when we do reach that level, then yes, I agree in full sense, at all levels that (as you wrote) "and that code that you are test-driving is not Legacy Code (but rather a test-driven island in a sea of Legacy Code)."
(Part 3/3) "You demonstrate in your response that ATDD in Legacy Code is possible. I still maintain, however, that "TDD in Legacy Code" is a contradiction. I like to say that TDD is a healthy lifestyle, whereas Legacy Rescue is curing cancer."
That's correct. We go at different levels:
1. System Level
1.a. Legacy Rescue at System Level (cure)
1.b. TDD at System Level (health) i.e. ATDD loop
2. Component Level
2.a. Legacy Rescue at Component Level (cure)
2.b. TDD at System/Component Level (health), i.e. ATDD loop, and within it we have CTDD loops
3. Unit Level
3.a. Legacy Rescue at Unit Level (cure)
3.b. TDD at System/Component/Unit Level (health), i.e. ATDD loop, and within it we have CTDD loops, and within them we have (U)TDD loops
I'd say when we reach stage 3, that's when we have the achieved the full cure and the full health. But look, even stages 1 and stages 2 (interim stages) are much better than where we are.
(Part 1/3) "Thanks for responding, Valentina. I see that you are extending the Feathers Legacy Rescue algorithm by recognizing that you need to begin with whatever test coverage you can get. Very important."
Exactly! Yes, that's the intention.
I found Michael Feathers work to be an excellent inspiration, esp. how he does the untangling of dependencies to introduce unit tests.
I expected that idea across the levels, I view now test levels like layers of an onion, that we're spelling an onion.
So the extension I applied was to go beyond unit tests, but applying the same principle.
1. Introduce Acceptance Testability, then can write Acceptance Tests & practice Acceptance Test Driven Development
2. Introduce Component Testability, then can write Component Tests & practice Component Test Driven Development
3. Introduce Unit Testability, then can write Unit Tests & practice (Unit) Test Driven Development
Thanks Stephanie! Great to hear that!