From noise to signal: an alerting system for CSMs
Sciene's Account Flagging is the foundation of Companion, a triage platform for Customer Success teams. The story of getting it right by first getting it wrong.
(vs. 34% at v0)
to directors weekly
(across 250 CSMs)
Companion is Sciene's suite for Customer Success operations, spanning three modules: Account Flagging, an AI Email Hub, and a Meeting Hub. Account Flagging is the foundational one, the daily first look CSMs open every morning, and the anchor point everything downstream relies on.
CSMs at Sciene's enterprise clients each manage 10 to 20+ accounts. Their day starts with the same question: what's on fire? Without a system to answer it, critical issues, sales drops, ad spend spikes, listing suspensions, delayed payments, only surface when the client calls angry. Teams patched the gap with an external dashboard sitting outside the product ecosystem, useful, disconnected.
I open five tabs before I even know which client to think about first.
Six weeks of interviews: 12 CSMs across account tiers, plus 3 directors. I mapped how they actually scan for problems, which data they trust, and when they escalate. I led with the workflow, not with UI ideas.
Instead of transcribing 12 hours of interviews by hand, I used Claude to cluster transcripts into pain patterns. What would have taken 3–4 days took an afternoon, and let me spend the rest of the week pressure-testing those patterns against the raw source instead of just producing them.
The first prototype surfaced every alert the system could generate. It looked comprehensive. In testing with CSMs, it was noise. The AI insights layered on top were often technically correct and functionally useless.
I built a Claude artifact hooked to Databricks that pulled acknowledgment data from v0. Within a week the pattern was clear: only 34% of alerts were being acknowledged inside 24 hours. Two-thirds were being ignored, snoozed indefinitely, or drowned. The AI wasn't hallucinating, it was faithfully surfacing everything the data allowed. That was the problem.
The reframe: meaningful, not comprehensive.
I sat down with CSMs and their directors to define what "meaningful" meant, threshold by threshold. Then worked with the AI engineers to encode that back into the alerting logic. The design and the model met in the middle.
v1 shipped with three alert categories (Performance, Financial, Business), filters that reflect how CSMs actually segment their morning, and AI insights on each alert that explain why it fired: frequency, historical context, most-impacted metrics. Acknowledgment states (OK, snooze, resolved) closed the loop.
For layout exploration, I generated 4–5 rough React prototypes in Cursor and tested them with CSMs the same day. Threshold logic was co-authored with the AI engineers, I'd prototype an alert rule, they'd stress-test it against production data, we'd iterate until the false-positive rate was liveable.
Account Flagging was the first surface, not the only one. Companion spans three modules, so the alerting patterns had to hold up across an AI Email Hub and a Meeting Hub, both also designed by me.
I built the design system from zero alongside v1: tokens, a component library, accessibility standards, and handoff documentation, adopted across product and engineering. Alert states, severity treatment, and acknowledgment controls became shared primitives rather than one-off screens. That is why v2's Director Dashboard shipped without redesigning anything from scratch.
The system was documented for two audiences: people and AI agents. Semantic token names, component descriptions written as usage rules, and states enumerated in predictable structures mean an LLM assistant can compose valid product UI from the library the same way an engineer does. The docs double as the prompt.
v1 gave individual CSMs their morning triage. v2 gave directors their team.
The Director Dashboard surfaces MRR under alert across all CSMs, resolution rates, help-requested loop, and priority ranking by workload. Every metric traces back to the same alerting system.
CSMs can flag an account as "asked for help." Directors see the ask surface in their view; the CSM sees it acknowledged in theirs. Solo dashboard, two-role system.
Developer handoff docs and Figma annotations were drafted alongside the design work with Claude, not after. That kept the spec in sync with the prototype and cut handoff friction to near zero.
The pivot taught me the real job of AI in product design: not to surface more, but to make fewer things matter. The v1 works because we let the AI hide most alerts, and made sure the ones that survived were worth a CSM's attention.
AI is an accelerant, not a shortcut. This project is where I first proved that to myself, on both sides of the screen.