Docs as Tests & AI

Have you ever discovered that your AI chatbot was confidently wrong-thousands of times-because of one outdated paragraph? Do you even know where to check?

Documentation errors used to stay local. One user reads something wrong, gets stuck, opens a ticket. But when your docs feed chatbots, RAG systems, or coding agents, that wrong paragraph is included in every answer for every related question, and for every user, with complete confidence. One stale API endpoint. One parameter renamed three sprints ago. One screenshot of a button that no longer exists. AI doesn't sanity-check your documentation. It amplifies it.

Docs as Tests & AI picks up where Docs as Tests left off. Manny Silva shows how to validate documentation when AI is generating it, consuming it, or both. The book covers territory the original didn't: the difference between deterministic and probabilistic testing, validating the content that feeds AI systems, writing documentation that gives AI agents reliable process knowledge, and building systems that detect and fix errors automatically.

Manny Silva has spent fifteen years working with companies including Apple, Google, and Skyflow on technical documentation. He created Doc Detective, an open-source tool for testing documentation against real products, and codified the Docs as Tests strategy. After that book published, he interviewed practitioners at Anthropic, AWS, Elastic, Mintlify, Avalara, and others to document how teams were navigating AI in their documentation workflows. Each team was figuring it out on their own. This book assembles what they learned.

What You'll Learn:

• Why AI makes documentation errors more expensive than before, and how deterministic and probabilistic testing address them differently.
• How to validate documentation from any source-human writer, LLM, recording tool, or AI coding assistant-so it reflects what your product actually does.
• How to write documentation that gives AI agents reliable process knowledge (project descriptions, agent definitions, orchestration patterns, skills, and plans), and how to test it the same way you test product docs.
• How to migrate from probabilistic to deterministic testing as your understanding of what to check matures.
• How to build self-healing systems that detect documentation failures, propose fixes, and verify repairs, with governance that keeps automation from going sideways.

• Technical writers using AI to generate documentation who need a way to verify that generated content actually describes the product, not what the model assumed it would be.
• Documentation engineers building testing infrastructure who want to know when deterministic testing applies, when probabilistic is appropriate, and how to build self-healing pipelines.
• Product managers and developer experience teams whose documentation powers AI chatbots or search systems, where one wrong paragraph means thousands of wrong answers.
• Developers and engineers who contribute to documentation and want to apply the same testing discipline they use on code.
• QA professionals ready to extend their testing practice to the documentation users and AI systems depend on.