Building an Internal AI Copilot for Support Teams
The brief from one client was explicit: this tool assists agents, it doesn't replace them, and it doesn't talk to customers directly. That framing shaped every design decision that followed, because a copilot that support agents trust enough to actually use looks very different from a customer-facing chatbot optimized for autonomy.
The single highest-value feature turned out to be the least flashy: real-time suggested answers surfaced alongside the ticket, pulled from the same knowledge base agents already searched manually, but ranked and summarized so agents didn't have to read three long articles to find the relevant paragraph. That alone cut average handle time by 28% in the first full month post-launch, more than any of the fancier features we'd planned.
We built an explicit accept-or-edit tracking mechanism into every suggestion, logging whether an agent used a suggestion as-is, edited it, or ignored it entirely. That data became the most useful signal we had for improving the system — suggestions with high edit rates pointed us straight at knowledge base articles that were outdated or poorly written, problems the copilot surfaced indirectly that no one had been tracking before.
Trust took longer to build than the technology did. Early on, agents largely ignored the suggestions, and interviews revealed why — a couple of confidently wrong suggestions in the first week had been enough to sour trust for weeks afterward. We added visible confidence indicators and source links to every suggestion, and usage climbed steadily over the following month as agents developed a feel for when to rely on it.
The metric the client cared about most in the end wasn't a technical one — it was new-agent ramp time. New hires using the copilot reached average performance benchmarks in roughly two-thirds the time of the cohort before it launched, because the tool effectively encoded institutional knowledge that used to take months of osmosis to pick up from more experienced teammates.
