Most companies treat internal AI as an interface problem.
They build a chatbot over company data, connect it to a few systems, give employees a place to ask questions, and call the result an intelligence layer.
I came to the opposite conclusion after building versions of this model twice, first at SoundCloud and later at Honeycomb.
The chatbot was never the product. The product was the evidence system underneath it.
The useful part was not that someone could type a question into Claude or another conversational interface. It was that the system could connect information the company had previously kept separate, preserve the context around that evidence, distinguish observation from inference, and give different teams a useful view of the same underlying reality.
At SoundCloud, that meant connecting customer support conversations with product behavior and business intelligence. At Honeycomb, I expanded the same architecture across Sales, Marketing, Product, Research, Support, and competitive intelligence.
The domains were different. The operating model was almost identical.
That repetition changed how I think about internal AI systems. The strongest ones are not really chatbots, and they are not really agents either. They are organizational learning systems.
SoundCloud: when support became product evidence
The SoundCloud work started with a much more operational problem.
Customer support had enormous amounts of qualitative data spread across Zendesk tickets, Otto chatbot conversations, Help Center content, surveys, spreadsheets, and other support systems. The company also had the quantitative side of the story through its existing business intelligence and product analytics infrastructure.
Those worlds were largely separate.
Business intelligence could tell us that usage declined, retention moved, a cohort behaved differently, support volume increased, or a funnel broke. Support could tell us what customers were confused about, what they expected to happen, the language they used to describe a problem, whether they were angry, where they got stuck, and what happened when they tried to resolve it.
One side could often tell us what happened. The other could help explain why. The opportunity was to connect them.
That sounds obvious in retrospect, but it required changing the way we thought about support data. A ticket could no longer be treated only as a unit of operational work. It was also a piece of product evidence.
The challenge was that the evidence was messy. A human ticket might use one taxonomy. Otto might classify the same issue differently. Help Center content might use another name. Tagging could be inconsistent. "Subscription problem," "Go+ billing," "charged twice," and "money taken twice" might all describe the same underlying issue while appearing as different categories in reporting.
Before we could build a useful intelligence layer, we needed to make those interactions legible to one another.
I began thinking about each customer interaction through a broader 5W1H structure: who was experiencing the problem, what they were trying to do, when and where it occurred, why it appeared to happen, and how they were trying to resolve it.
The point was not to turn every ticket into six fields for the sake of having six fields. The point was to normalize enough context that support, Product, and analytics could talk about the same problem consistently. That became foundational.
Fix the information before asking AI to reason over it
One of the patterns I saw repeatedly was that poor information architecture gets misdiagnosed as an AI problem.
The Help Center was a good example. If an automated support system depends on retrieving the right knowledge, messy content structure creates retrieval failures that can look like model failures. Duplicated articles, inconsistent language, unclear taxonomy, weak routing, and missing content all make the AI look worse than it is.
So some of the most valuable work had very little to do with sophisticated models.
We reorganized Help Center content. Weekly ticket volume dropped roughly 10% in Q1. Automation reached around 80% in Q2. Search success on the redesigned experience reached approximately 90%, with a goal of getting people to relevant help within three clicks. Better routing pushed lost tickets below roughly 1%.
Those outcomes reinforced something that later became central to my AI Transformation work: not every AI problem needs an AI solution. Sometimes the right answer is content. Sometimes it is taxonomy. Sometimes it is routing. Sometimes it is product UX. Sometimes it is code. And sometimes it is an LLM.
Choosing among those options is part of the work.
Otto became both a service layer and an evidence source
Otto, SoundCloud's support chatbot, was initially useful because it could answer customer questions and prevent unnecessary tickets. But the more interesting thing was what every interaction told us.
A successful conversation told us which intents could be resolved automatically and which content worked. A failed conversation told us where the taxonomy broke, which knowledge was missing, which questions were being misunderstood, and where the system was creating loops or unnecessary escalations.
Instead of evaluating the bot only on whether it contained a conversation, I wanted to understand the downstream outcome. Did the customer actually resolve the issue? Did they escalate? What happened when a ticket was created? Did the support agent reach a different conclusion? Was the original intent classification correct? Was the right article surfaced? Did the customer loop through the same answer multiple times? What did CSAT or BSAT tell us afterward?
That led us toward evaluating the complete support journey rather than a single bot decision. Combining bot interactions with downstream ticket outcomes improved our decision accuracy by roughly 20%.
That became another principle I carried forward: evaluate the system, not just the model.
A containment rate by itself can be misleading. A bot can technically contain someone because the customer gives up. Lower ticket volume can mean better self-service. It can also mean people stopped trying. The metric only becomes useful when it is interpreted alongside resolution, escalation, satisfaction, routing, content gaps, and downstream behavior.
Support became a continuous discovery source
Once the support evidence was more structured, the Product opportunity became obvious. Instead of a PM periodically asking Customer Operations what customers were complaining about, Product could interrogate the evidence directly.
Which issues around this feature are increasing? What language do customers use to describe the problem? Is this a real recurring problem or a handful of noisy tickets? Which users are affected? Is it a bug, an education problem, a policy problem, or a missing capability? What happened in the journey immediately before the ticket? What questions should we investigate in research? Which users would make useful interview participants?
That changed the role support data could play in Product. It was no longer just a queue of things that had gone wrong. It became one source of continuous discovery.
That did not mean replacing User Research. It meant giving researchers and Product teams a better starting point. If support had already captured hundreds of examples of a problem, the research team did not need to begin by asking whether the problem existed. They could begin with harder questions about why it existed, which users experienced it, what customers were actually trying to accomplish, and whether the solution they requested addressed the real need.
Then we connected it to business intelligence
This was where the system became materially more useful.
SoundCloud already had a quantitative business intelligence layer. I worked toward merging the qualitative support intelligence into that environment so people did not have to choose between a dashboard that told them what happened and a separate support tool that might explain why.
The combined system could reason across product usage, business outcomes, support interactions, Product and Engineering documentation, Marketing content, and other internal context.
A PM could move from a question like "why did adoption fall for this cohort?" to a much richer investigation: adoption fell for this cohort, so what support themes appeared during the same period, what are those users saying, did anything change in the product, does the documented workflow match what customers appear to experience, and are there Help Center searches or failed support intents around the same step?
That is a different kind of business intelligence. Quantitative systems are extraordinarily good at measuring behavior. Qualitative systems are good at explaining context, expectations, frustration, intent, and language. Combining them makes both stronger.
It also creates useful contradictions. Product analytics may show low adoption while support conversations show that customers are repeatedly trying and failing to use the capability. Marketing may have set an expectation that the product experience does not consistently fulfill. Product documentation may describe intended behavior that looks different from what customers report.
The system should not smooth those contradictions into a neat summary. The contradiction may be the most important finding.
From reporting to sensing
I did not want the SoundCloud system to remain purely reactive. If a particular support theme began increasing quickly, the system should eventually identify that change without requiring someone to notice it in a weekly report. That created a path toward anomaly detection and proactive triage.
Imagine support contacts around uploads suddenly increase. Traditional reporting can show the spike. A more connected intelligence system can cluster the conversations, identify the common language, determine when the pattern started, connect it to relevant product context, compare it against telemetry, and surface the likely scope of the issue.
The handoff to Product or Engineering becomes much better. Instead of "customers are complaining about uploads," the investigation can begin with something closer to: this issue began increasing yesterday, appears concentrated in this workflow, affects this cohort, and these are representative customer reports and related product signals.
The AI is not replacing incident investigation. It is improving the starting point.
Honeycomb: the same architecture in a different domain
When I joined Honeycomb, I found myself building a very similar system against a different kind of evidence.
Instead of support conversations being the dominant qualitative source, Honeycomb had an enormous stream of GTM conversations. The company was generating more than 100 Gong calls a day.
Those conversations contained competitive information, buying criteria, objections, pain points, feature expectations, customer language, perceived strengths and weaknesses, and the reasons customers were leaning toward one option or another.
The conventional response would have been to build better competitive battle cards. That initially sounded reasonable to me too. Then the scale of the evidence made the weakness in that model obvious.
A battle card assumes competitive knowledge changes slowly enough to be researched, documented, published, and periodically refreshed. That was not true. Sales was learning every day. Product was shipping. Marketing was testing different narratives. Support was seeing different problems. Competitors were releasing products and changing positioning. Buyers were changing the way they described the category.
No static document could keep up. The more I worked on it, the more convinced I became that the battle card was not the product. It was another view over an evidence system.
The company already knew more than it could use
Honeycomb had competitive and customer information spread across Gong, Salesforce, Snowflake, product usage, Support, User Research, Product and Engineering documentation, Marketing content, Slack, and public market sources. Each function saw a different slice.
Sales knew which competitors were appearing in deals and which objections were becoming harder to overcome. Product knew what customers were actually using. Research had deeper context from a smaller set of users. Marketing understood category language and positioning. Support saw friction that might never appear in a sales conversation.
Every team could be correct about its own evidence and still reach the wrong conclusion. A salesperson might hear the same feature request three times and reasonably conclude that the market wants it. Product might look at usage and see that almost nobody uses the adjacent capability that already exists. Research might discover that customers are asking for a familiar competitor feature because they lack better language for the underlying problem.
That is not a failure of any individual team. It is a failure of the information architecture. The company did not need more data. It needed to connect what it already knew.
Gong changed the economics of analysis
A human competitive intelligence function can sample sales calls. Managers can listen to important opportunities. PMs can join customer meetings. Researchers can conduct interviews. Marketing can ask AEs what they are hearing.
What humans cannot realistically do is listen across thousands of conversations, retain the details, connect those conversations to deal outcomes, compare patterns across time, and continuously distinguish an isolated comment from an emerging market signal. AI changes that equation.
I was not interested in producing better Gong summaries. I wanted structured evidence. A conversation could contain signals about the alternatives a buyer was considering, the problem that initiated the search, the criteria driving the decision, recurring objections, technical concerns, missing capabilities, competitor claims, and the language the buyer used to explain what mattered.
The useful unit of analysis was not the individual call. It was the pattern across calls.
"A prospect thinks Datadog does this better" is an anecdote. "Datadog has appeared repeatedly across relevant opportunities, this concern is increasing, Honeycomb tends to win when one criterion dominates the evaluation and lose when another one does" begins to look like organizational knowledge. The difference is evidence density.
One evidence layer, several decisions
The next step was to stop treating competitive intelligence as a Sales-only problem. Sales, Marketing, Product, and Research often needed the same evidence for different decisions.
An AE needed to understand what a competitor meant in a specific live opportunity. Marketing cared about buyer language, competitive narratives, and whether the company's positioning actually appeared in winning deals. Product needed to understand whether a recurring competitive complaint represented a genuine product problem. Research needed to know which patterns deserved deeper qualitative investigation.
A single feature request could therefore mean four different things. Sales might need an objection-handling strategy. Marketing might have a positioning problem. Product might have a capability gap. Research might discover that the requested feature was simply the customer's proposed solution to a different underlying job.
A static battle card collapses those interpretations into one document. A shared evidence layer preserves them.
That became the operating model: structure evidence once, then let each function interrogate it according to the decision it needs to make.
The interface mattered less than the reasoning underneath it
Claude became an important part of the Honeycomb system because it gave people a natural interface for asking questions across the evidence. But I never thought the model itself was the strategic asset. The asset was the structure underneath it.
A Sales leader should not need to know which Snowflake table contains a field, where Product keeps release notes, which Gong calls matter, or which internal document explains a capability. They should be able to ask a business question and have the system assemble the relevant evidence.
The problem with simply handing everyone a powerful general-purpose model is that the organization can quickly end up with twenty different versions of the same analysis. Different prompts, different assumptions, different source material, different definitions.
So I started turning repeatable analytical workflows into reusable Claude skills. The broader GTM environment included skills around Discovery and Multithreading, Whitespace Analysis, Win/Loss Mining, Deal Hygiene and Enablement, and Board or QBR narratives. Competitive intelligence overlapped heavily with win/loss because many of the useful questions involved repeated loss reasons, objections, competitor signals, and repeatable win themes.
Those skills lived in a shared, version-controlled GitHub repository. That mattered because analytical methods that influence real business decisions should not live in one person's chat history. They should be inspectable, reusable, improvable, and owned.
The model was becoming the interface. The reasoning was becoming infrastructure.
From asking questions to detecting change
At both SoundCloud and Honeycomb, the more mature version of the system was not simply a better search experience. It was a sensing system.
I began thinking about integrations through four capabilities: read, write, trigger, and monitor. Read lets the system answer a question. Write lets it update a system of record. Trigger lets it initiate another workflow. Monitor lets it recognize when something meaningful changes.
Most internal AI tools stop at read. The more interesting systems eventually reach monitor.
At Honeycomb, imagine a competitor changes pricing, mentions of that competitor increase in Gong, several active opportunities raise the same objection, and Honeycomb has recently released something directly relevant to the concern. A static competitive program depends on somebody noticing each of those signals. A connected system can recognize that the combination matters.
At SoundCloud, the equivalent pattern might be a sudden increase in support contacts around a particular workflow, correlated with a product change and a decline in a related usage metric.
Different domains. Same architecture.
Trust is part of the system
The biggest risk in systems like these is not that they fail to retrieve information. It is that they turn weak evidence into strong conclusions.
That is why I cared so much about provenance, evaluation, and the distinction between fact, observation, pattern, inference, and recommendation.
"Fifteen prospects mentioned this competitor" is evidence. "Mentions increased month over month" is a pattern. "This competitor is becoming strategically dangerous" is an inference. "We should change the roadmap because of it" is a recommendation. Those statements require different levels of confidence.
The same is true in support. "Ticket volume increased" is measurable. "Customers are confused by this workflow" is interpretive. "This design caused the increase" is causal. "We should rebuild the workflow" is a recommendation.
AI makes it very easy to present all four statements in the same confident tone. The architecture has to preserve the difference. That means keeping conclusions attached to their underlying evidence, exposing contradictions, showing uncertainty where it exists, and evaluating whether the reasoning actually matches human judgment.
Confidence should come from evidence, not from how polished the paragraph sounds.
The shared lesson
SoundCloud and Honeycomb looked like very different projects on the surface. One involved Customer Operations, Product discovery, and business intelligence. The other involved competitive intelligence, Sales, Marketing, Product, and Research.
Underneath, I was building the same thing twice.
At SoundCloud: customer interactions, then structured qualitative evidence, then quantitative product and business data, then reasoning, then Support, Product, and Research decisions, then new evidence.
At Honeycomb: sales, customer, and market interactions, then structured evidence, then GTM and product data, then reasoning, then Sales, Marketing, Product, and Research decisions, then new evidence.
The pattern was consistent enough that I stopped thinking of these systems as support agents, competitive agents, or even business intelligence agents. They were organizational intelligence systems.
They took fragmented signals the company was already generating, made them legible to one another, preserved enough context to reason responsibly, and gave people a common way to interrogate the evidence.
The chatbot was only the surface. The real product was the connective tissue underneath it.
And once that exists, the interesting question stops being "what can we ask the AI?" It becomes "what can the organization finally understand that it could not see before?"