Talk to enough vendors this year and you'll hear a version of the same pitch. Flip a switch, and your knowledge base updates itself. No more stale articles, no more scrambling after a product change. AI handles it.
It's a good pitch. It's also not quite what's actually happening, even at the companies building the technology, and not what the research on AI agents doing real work actually shows.
Take Fin Operator, the AI agent Intercom (now rebranded as Fin) launched in May to manage its own customer-facing AI agent. Part of its job is updating documentation when the product changes.
That's real, and it's a meaningful step. But every recommendation Fin Operator makes needs a person to approve it before it goes live, "pull request" style, the same way an engineer reviews a colleague's code before it merges.
"Every change that Operator recommends — whether it is an edit to a help article, a rewrite of an AI guidance rule, or the creation of a new QA monitor — appears as a proposal with a full diff view. Users can inspect, edit, and approve each change before it takes effect. Nothing goes live without a human clicking 'Apply.''"
That approval gate is worth sitting with. A company built around AI agents, with every incentive to claim full automation, still put a human in the loop. That's not a company hedging its bets for marketing.
That's a company that ran into the same problem everyone building this eventually runs into.
Andrej Karpathy described something similar this year: an architecture where an LLM maintains its own knowledge base as an evolving set of markdown files, ingesting new information, compiling it into existing structure, checking for contradictions. People called it a knowledge base that heals itself.
It's a genuinely interesting idea, and one worth building toward.
and here is the full architecture of the LLM Knowledge Base system covering every stage from ingest to future explorations. https://t.co/Wmn48gB0g0 pic.twitter.com/ObJet8Esfu
— himanshu (@himanshustwts) April 2, 2026
What happens when it's not standard?
Here's the part that doesn't show up in the pitch. Self-healing works well when the thing being healed follows a general rule. A lot of documentation doesn't.
Custom plans are a good example.
Plenty of teams run configurations where the standard advice doesn't apply, where the right move is to manually override a setting instead of following the default path.
An AI that generalizes "here's the answer" from what works for most accounts will get that case wrong with complete confidence. It won't flag the exception. It won't know there was one.
That's not a hypothetical edge case either. Carnegie Mellon's TheAgentCompany benchmark, which tests AI agents on the kind of multi-step office tasks documentation maintenance actually involves, put successful task completion around 30 to 35 percent.
A November 2025 Stanford and Carnegie Mellon study comparing 48 professionals against four AI agent frameworks across 16 realistic multi-step tasks.
They found the same pattern from a different angle: agents alone worked faster, humans alone worked to a higher standard, and pairing the two based on their respective strengths beat letting the agent run solo. The finding wasn't "don't use agents." It was "don't use them unsupervised."
That's the actual tradeoff nobody puts in the pitch deck.
Is it worth the trade-off?
Fully automated documentation sounds like it saves a technical writer's time. In practice, that's only true if the AI is never wrong. The moment it is, someone still has to catch it, which means reading everything the AI touched anyway.
At that point you're not writing documentation. You're auditing someone else's mistakes, on a schedule you don't control. That defeats the entire point of automating in the first place.
Become an expert in all things Knowledge Base with our monthly newsletter. No spam, just expert content, delivered.
None of this means the idea is wrong. It means the current honest answer is: review it.
Have a person check what the AI changed before it goes live, the same "pull request" instinct Fin Operator already builds in, and the same split the Stanford and Carnegie Mellon research points to.
It's also the principle behind Stale, our own feature for flagging outdated articles: notify a person, never touch the content automatically. Not forever, necessarily. As the tooling gets better at recognizing its own edge cases, that review step can get lighter.
But eventually isn't now, and pretending otherwise doesn't serve anyone writing docs for a real product with real edge cases.
We're not neutral on this. We're a documentation company that already treats AI agents as a real audience. That's the same reason we built an MCP server, so tools like Claude and ChatGPT can read a knowledge base directly.
We're also testing ways AI can make a technical writer's job easier. Not by replacing the judgment calls, but by taking the repetitive parts off their plate, so there's more time left for the calls that actually need a person. More on that soon.
For now, if someone tells you their knowledge base updates itself with no review step, ask what happens the day it's wrong. If they don't have a good answer, that's the blind spot.
