Harness Engineering: Keeping AI-Generated Code Correct and Coherent
Harness engineering is the practice of surrounding AI-assisted code generation with deterministic tooling, agent-based review, and periodic entropy checks to keep AI-generated code correct and coherent over time. Originating from Birgitta Boeckeler's article on martinfowler.com, the concept adapts the test harness idea to enforce not just functional correctness but also architectural decisions, naming conventions, and security constraints. The framework comprises three components: context engineering (maintaining a HARNESS.md knowledge base for the AI), architectural constraints (verification slots using deterministic tools or agent reviews), and garbage collection (scheduled checks for accumulating entropy). The plugin implements these in three enforcement loops—advisory, strict, and investigative—and adds a self-improving dimension where the harness learns from its own operational history.
The harness does not trust the programmer. It verifies.
- ricardobeat
The term harness engineering does not come from Birgitta Boeckeler's article in martinfowler.com. It was first used by Mitchel Hashimoto months earlier in his 'My AI adoption journey' blog post [1], and immediately picked up by OpenAI [2] a week later.
Also worth noting that this text is not only entirely AI generated, but meant for other agents to read, not us humans.
- iTokio
It’s fascinating to live through a the emergence of a new technology and to see people trying to make sense of it as they go.
I personally think that a lot of the words used around AI, harness, SKILLS, agents, RAG.. are make up words or close to it, words that do not have profound semantics, even though people are trying to, often after the fact, make sense of them.
It’s just popular words that are different enough that people like to use them to claim a new knowledge, or to market a product.
But we could use AI tooling instead of harness in the abstract, and it would be better to use more precise terms for more concrete use cases, agent loop, CLI, IDE..
It was a fun game at a time where papers were competing for attention, but now that it has become a proven technology, I hope we can find more precise and meaningful words.
- gardnr
Here’s the original article this one is based on: https://martinfowler.com/articles/harness-engineering.html
And here is the original author on a podcast talking about it: https://open.spotify.com/episode/4FxEdjXldNhoh67KYVmbDu
- davedx
This part seems relevant - they make explicit the difference between their HARNESS.md document and AGENTS.md. It's actually interesting: https://habitat-thinking.github.io/ai-literacy-superpowers/p...
- lacoolj
Not saying this is AI-gen but it's very dense and doesn't read like something I could glean info from easily