Why We Took Our Time Introducing AI Into HelpDocs
AI is everywhere, but not all AI is helpful. Here’s why we took a measured approach before adding AI into HelpDocs.
When AI first started making waves, it felt like every SaaS company rushed to bolt it onto their product. AI-powered chatbots, AI-written content, AI search—it was everywhere. And a lot of it was… not great 😅
Honestly, it was making us sweat a bit.
It seemed like AI came out of nowhere, and suddenly, there was pressure to integrate it into everything. We saw competitors rolling out AI features overnight, often with mixed results.
Some implementations were flashy but unreliable, leading to frustration rather than improved experiences. We didn’t want to fall into that trap.
We took a different approach.
Our content in monthly bitesized emails
Get our best content delivered straight to your inbox.
SubscribeWhile we were excited about the potential, we also knew AI had serious limitations. It needed a lot of context to work well, it tended to hallucinate facts, and it wasn’t always the best fit for knowledge bases 🫢
So, instead of diving in headfirst, we waited. We observed how AI evolved, tested it internally, and only moved forward when we knew it could genuinely help our users.
Early AI Results Were Rough
When GPT-3 launched in 2020, it was impressive—but also wildly inconsistent (we even wrote about it here).
AI-generated content often lacked accuracy, and in knowledge bases (where clarity is everything), that wasn’t acceptable to us.
Research showed that early large language models (LLMs) required significant fine-tuning and additional context to produce reliable responses. One study from MIT found that AI models made errors in about 30% of factual knowledge tasks, meaning they could mislead users instead of helping them (source).
We didn’t want to introduce AI just for the sake of it.
It needed to genuinely improve the user experience—so we held off. Instead, we watched the space closely, analyzing how AI performed in other knowledge-based systems. We saw companies struggle with accuracy, user frustration over incorrect responses, and the need for human oversight.
River Sloane
Rather than rushing out a half-baked feature, we committed to taking our time and doing it right 🧑🍳
We Don't Want a Pure AI Search Engine
AI-powered search sounds futuristic—but in reality—AI fabricates answers more often than we'd like.
Unlike a traditional keyword search, AI search tools tend to "hallucinate"—confidently making up information when they can’t find a good match. A 2023 Stanford study found that AI-generated responses were inaccurate up to 27% of the time, depending on the model and dataset (source).
For HelpDocs, that was a dealbreaker. Customers use our search to find trusted, company-approved information—not AI-generated guesses. Instead of making AI the backbone of search, we wanted it to work alongside a structured, non-AI search engine.
That meant waiting for AI models that were safer and more context-aware.
We’re still working on getting AI into search, but it needs to be built on top of a robust search engine (like the one we started rolling out last year), not as a replacement for it. AI should enhance search—not override it with fabricated answers.
That’s why we’ve been experimenting with AI-assisted ranking and intent detection rather than just throwing AI-generated responses into the mix. We want AI to help users get to the right article faster, not confuse them with uncertain responses.
Why We Chose to Wait for Anthropic’s Claude
When we started looking for an AI model to integrate, we knew we wanted something more reliable and ethical. That’s why we waited for Claude, the AI model from Anthropic, which prioritizes safety and factual accuracy over raw creativity.
Unlike some other AI models, Claude is trained with “constitutional AI” principles, meaning it has built-in safety rules that make it less likely to generate harmful or misleading information.
River Sloane
In a 2023 evaluation, Anthropic’s Claude was found to hallucinate less than OpenAI’s GPT-4 in critical applications, making it a better choice for knowledge-based search (source).
By waiting for Claude to mature, we made sure we weren’t just jumping on the AI hype train. We wanted a tool that actually added value—without compromising accuracy. We also liked Anthropic’s commitment to transparency and safety, aligning with our belief that AI should assist rather than replace human expertise 💖
AI in Content Creation: Where It Made Sense First
While AI search posed challenges, content creation was where we saw the most potential. Writing help articles is time-consuming, and AI could assist without replacing human judgment. But we didn’t want to introduce generic AI-generated text—we wanted AI to work seamlessly with our editor.
It took almost a year of testing to make sure AI-generated content was actually helpful.
We worked on integrating native elements like callouts and tables into AI-generated drafts, so users wouldn’t have to manually reformat everything. We also spent months refining how AI structured information—ensuring that it mirrored how real technical writers craft documentation.
The rewrite feature was even more complex.
Instead of a total rewrite that ignored user intent, we had to make sure it kept what users wanted while improving clarity and structure. Getting that balance right took time—but now, it’s a powerful tool for making articles more readable without losing key details.
Most importantly, we wanted to ensure it was clear that AI in Content Creation generates drafts, not final content.
River Sloane
The expertise of technical writers, engineers, and support teams is the foundation of every great knowledge base. Their creativity, problem-solving, and deep product knowledge aren’t replaceable—AI is just here to help streamline their workflow.
We built our AI tools to empower knowledge base creators, not to remove them from the process.
Context is key when it comes to help articles—so having an expert writing high-quality AI outlines can make or break an AI help article draft. AI can assist in structuring information, but the human touch is what ensures accuracy, relevance, and clarity.
Without that, AI-generated content can feel generic or even misleading.
AI, Done Right
We believe AI can be a helpful assistant, not a replacement for structured knowledge. That’s why we took our time, tested different approaches, and ultimately chose a model that aligns with our values.
Now, we’re excited to introduce AI-powered enhancements that actually work—without sacrificing trust