The other two case studies are about systems I built. This one is about judgment in the room—three copy decisions on a fast-moving AI product, each a different call: when to restructure, when precision carries legal weight, and when to change nothing at all.
Agent Analytics is Pendo's product for measuring how AI agents perform in real customer conversations—a fast-moving area full of concepts that didn't exist two years ago: LLM-as-judge evaluation, conversation sampling, tracked issues and use cases. I'm the embedded content designer. UI copy, in-product guidance, and the language customers rely on to understand what an AI is doing with their data.
Writing for an AI product is a different job than writing for a dashboard. The concepts are unfamiliar, the product changes weekly, and some of the copy carries real trust and legal weight. Three moments from this work, each a different kind of judgment.
Users define "goals" for their AI agents by describing what success looks like—and what they write directly determines the quality of the LLM-generated evaluation rules built on top of it. The design question: keep one big description box, or break it into guided questions ("What does a successful outcome look like for the visitor?").
I named the tradeoff in the design discussion: guided questions would extract better information, but at higher cognitive load—and the surrounding UI was already text-heavy. My call: I preferred the guided-question direction, but recommended shipping the current version, watching what users actually produce, and moving to guided questions if the open-ended results proved too vague. We trimmed the supporting text, swapped the generic "Description" label for the single most important guiding question, and cut a tip panel that wasn't earning its space.
Copy structure is a product decision with measurable consequences. Ship and learn beats blocking on the theoretically better version.
The shipped call: one field, with the most important guiding question doing the label's job.
Agent Analytics processes customer conversations, which means the copy explaining what Pendo stores had to be precise. The engineering underneath: visitor identifiers are hashed, raw conversation content is summarized rather than stored. My job was translating those guarantees into customer-facing language accurate enough to survive security review and clear enough that a non-technical admin could trust it—without overpromising in either direction.
This is the copy with the highest cost of error. Too vague and customers assume the worst. Too confident and you've made a legal claim engineering can't back.
The line I sweat most is the second sentence of the "never store" option: "Summarized data still follows your standard retention period." It's the sentence that keeps the first one honest—deleting raw data isn't deleting everything, and saying so is exactly what makes the deletion promise trustworthy.
The words carrying the legal weight: what Pendo keeps, for how long, and whether conversations stay tied to individual visitors.
A tooltip explained why certain experiment metrics were unavailable: "Switch to conversations mode to enable." The product manager tried to improve it: "These metrics require full conversation capture. Update your agent's capture mode in settings." and an AI design reviewer separately flagged the original: instructional copy with no path to act, since switching modes means abandoning the experiment you're mid-way through creating. Both critiques were fair. Both fixes kept the same flaw: they told the user to go do something. ("Settings" was also carrying too much—subscription settings, application settings, or agent settings? The reader can't tell, even when the writer can.)
Realistically, nobody abandons an experiment setup to change their agent's capture mode—that's a big, deliberate configuration decision, not a tooltip-sized detour. So the right move wasn't a better instruction. It was reframing instruction as explanation: "Only available for agents capturing full conversations." The copy stopped telling users to do something they wouldn't do, and started telling them why the thing they're looking at is off.
The AI reviewer's critique was useful. Its solution was wrong. Knowing the difference is the job.
Three drafts of one tooltip—the fix wasn't better instructions, it was changing the copy's job.
The call, live: assessment, proposal, pushback, alignment—in four messages.