Spec-Driven: When AI writes the code and the tests, green means nothing

Don't let AI invent the expected behavior. Use it to refine the requirement, then let the tests verify the code against it.

Valentina Jemuović

Oct 16, 2026

∙ Paid

⚡Spec-Driven Development in Legacy Code
Hands-on work with your teams. Limited spaces for 2027.

Book a call

AI writes code fast. Many teams then ask AI to write the tests for that code too.

That’s the problem. The tests check the code against itself. They don’t check it against what the business asked for.

It’s like a student marking their own exam. The answers always match, so they always pass. A test that can’t fail tells you nothing.

❌ Write test after code

The test comes from the code, not from the spec.

1. AI Reads Spec

AI reads the spec:

A customer gets the 10% promotion and their 20% coupon.

There are (at least) two ways to interpret the spec:

Interpretation A: 10% off first, then 20% off what is left.
Interpretation B: 10% + 20% = 30% off.

2. AI Writes Code

AI picks interpretation A: 10% off first, then 20% off the reduced price.

var promotedPrice = basePrice.multiply(promotionFactor);
var discountAmount = promotedPrice.multiply(discountRate);
var totalPrice = promotedPrice.subtract(discountAmount);

Note: AI is non-deterministic, so we can’t know which interpretation it will choose. In one run it may only think of interpretation A. In another run it may only think of B. In another it may see both and ask us to choose. And in another it may come up with an interpretation C that we hadn’t even thought of.

3. AI Writes Acceptance Test

AI reads its own code, then writes a test that mirrors the behavior of that code. It makes up inputs and answers that fit the code:

@TestTemplate
@Channel({ChannelType.UI, ChannelType.API})
void shouldApplyCouponOnPromotedPrice() {
    scenario
            .given().product().withUnitPrice(”100.00”)
                .and().promotion().isActive().withDiscount(”10%”)
                .and().coupon().withDiscountRate(”20%”)
            .when().placeOrder()
            .then().shouldSucceed()
                .and().order().hasTotalPrice(72.00);
}

4. Verify Acceptance Test Passes

The acceptance test is executed. It passes, because it was made to match the code, so the build is green.

5. Human discovers misinterpretation

The build is green, so the work goes to QA. QA reads the spec their own way. They expect $70.00 and see $72.00, so they report a bug. The bug goes to the product owner.

Only now does anyone ask the question that should have been asked first. The product owner says 30% off.

6. AI reworks code & test

The code is wrong, and the test is wrong too, because it was made to match the code. Both the code and the test must be reworked.

It was circular: the code decided the test, and then the test “proved” the code. That makes it tautological: true by construction, so it can never fail.

Optivem Journal