Register now

👋 Hello, this is Valentina with the free edition of the Optivem Journal.

Subscribe now

Markdown spec = enforced spec? WRONG.

Markdown = Spec. Acceptance Test = Executable Spec.

❌ Spec -> AI writes code -> Human manually tests

✅ Executable Spec -> AI writes code -> Automatic spec verification

99% of developers follow Spec-Driven Development (SDD) the WRONG way.

1. They write the spec in Markdown.
2. They tell AI to implement code based on spec.

But what's the problem here? Sure, AI reads the markdown file, but doesn't follow it fully! It may decide to skip instructions, or misinterpret instructions, etc.

So we can NOT trust AI!

We have to manually read the spec, manually run the system, and manually decide whether the functionality that the system implemented matches what was in the spec.

Interestingly, this is not a new problem. We've had this problem for ages, with human developers. Remember, we'd write a User Story, the developer thinks they followed it, but then QA says it doesn't work correctly - that there was something that the developer missed, or they had misinterpretations.

Well, that's exactly the same problem in Spec-Driven Development, how most people practice it.

The solution is: EXECUTABLE specs. With executable specs, we can be sure to enforce that AI actually implements the behavior that we specified. We have an automated way to check - a pass or fail. This means if AI skips instructions, or misinterprets something, we'll have a test telling us it failed. AI tries again, until the test passes. There's no way for AI to "cheat" the test.

That's the right way to do Spec-Driven Development. Convert textual specs to executable specs.

⚡Full walkthrough, with the code examples

📅 Register (free): Spec-Driven Development with ATDD & AI (Oct 14)

Register now

Book a call

“Spec-driven development” is everywhere in AI tooling right now: Kiro, GitHub’s spec-kit, Tessl. Write the spec first, then let the agent build against it. Birgitta Böckeler reviewed all three in Understanding Spec-Driven-Development: Kiro, spec-kit, and Tessl on martinfowler.com and found the catch.

In every one of them, “spec” means a Markdown document the agent is supposed to read. Nothing checks that the code matches it, and the agents don’t always follow it. As she put it: “I frequently saw the agent ultimately not follow all the instructions.”

The problem isn’t the tools. It’s that the spec is a Markdown file, and you can’t run a Markdown file against the code. To find out whether the agent followed it, a person has to read the code line by line. That’s the work you hoped the spec would save.

We’ve had a spec without this problem since 1999. Kent Beck’s first Extreme Programming book, Extreme Programming Explained (1999), already had acceptance tests, and the XP rules on extremeprogramming.org spell out how they work: acceptance tests “are created from user stories” and “should be automated so they can be run often.”

In 2010, Jez Humble and Dave Farley’s Continuous Delivery gave automated acceptance tests their own chapter: “Acceptance tests are a crucial stage in the deployment pipeline: They take delivery teams beyond basic continuous integration. Once you have automated acceptance tests in place, you are testing the business acceptance criteria of your application, that is, validating that it provides users with valuable functionality.”

An acceptance test is the spec and the check in one: it describes what the system should do, and it fails until the system does it.

An agent can ignore a Markdown file.

It can’t ignore a test that won’t pass.

An acceptance test doesn’t call the system directly. It calls a DSL, and the DSL calls two kinds of drivers. System drivers talk to the system through its UI or API. External-system drivers talk to the external systems it depends on, such as an ERP.

In acceptance test-driven development (ATDD), the Red step is where you write that test, before any system code exists. When an AI agent does the work, the Red step is split into four small steps, and a human reviews each one.

In every box, the AI writes the code, a human reviews it, and then it’s committed.

Before Red: the acceptance criteria

Example: placing an order in an online shop.

It starts with a ticket. Its acceptance criteria are agreed before any code is written:

Scenario: Calculate the base price from unit price and quantity
  Given a product with unit price 20.00
  When I place an order for 5 of that product
  Then the order is placed successfully
  And the order’s base price is 100.00

Inside the Red step

1. Write Acceptance Tests

The agent turns the acceptance criteria into a test. The test is written in a DSL, the test code’s own vocabulary (placeOrder().withQuantity(...)). Here’s the acceptance test, from PlaceOrderPositiveTest.java:

@TestTemplate
@Channel({ChannelType.UI, ChannelType.API})
void shouldCalculateBasePriceAsProductOfUnitPriceAndQuantity() {
    scenario
            .given().product()
                .withUnitPrice(20.00)
            .when().placeOrder()
                .withQuantity(5)
            .then().shouldSucceed()
            .and().order()
                .hasBasePrice(100.00);
}

Read more