Cherry v1 is deliberately demo-scale: one user, one browser, no database. That was the right scope to prove the synthesis-and-judgment core. But a feedback platform serving hundreds of submitters and dozens of teams breaks in specific, predictable places. This page names each break honestly and gives the v2 answer — because knowing where your own system fails is the difference between a demo and a platform.
BreakCorrections — the human judgment Cherry learns from — persist in localStorage. One user, one machine. A colleague's Cherry knows nothing your Cherry learned, so the system can't compound judgment across a team, and trend detection only diffs your own prior runs.
FixA persistent store (Postgres + the warehouse sync in the integration architecture): corrections, snapshots, and lifecycle states become org-wide. Every correction carries provenance — who, when, why — because shared ground truth that can't be audited isn't ground truth. Which immediately raises the governance question ↓
Breakv1 has one reviewer, so "ground truth" is whatever they said. At scale, the Product PM corrects an issue's severity down while the Support lead corrects it up. Last-write-wins silently destroys one person's judgment; averaging destroys both.
FixThree rules, in order. (1) Scope before you adjudicate: most "conflicts" aren't — severity-for-Product and severity-for-Support are different jobs, which is what persona views already model. A correction applies first to the corrector's view. (2) Ownership decides shared fields: for fields with one true value (owner, disposition), the issue's owning team's reviewer has authority; others' corrections attach as stakeholder annotations, not overwrites. (3) Disagreement is signal, not noise: genuinely contested calls get flagged contested — both corrections visible — and routed to a steward's adjudication queue. Every adjudication becomes a labeled eval case, so governance doesn't just resolve conflicts, it manufactures golden data.
BreakAnyone with the URL is "the reviewer." There are no teams, no roles, no separation between who may view a triage, who may correct it, and who may send a ticket to Slack.
FixSSO, workspaces per team, and role-scoped authority: viewers, correctors (whose judgment feeds learning), and senders (who fire external actions). Authority maps to the autonomy ladder below — the permission system is the guardrail layer.
BreakThe per-IP rate limit, daily cap, and 24-hour result cache are in-memory — per serverless instance. They vanish on cold start and don't coordinate across instances. Fine for a portfolio tool; a hole at scale.
FixShared state (Redis / Vercel KV) for limits and cache, plus real cost governance: the bill is users × runs/day × tokens × price — at 800 users it's a budget line someone must defend. Levers: prompt caching (live in v1), result caching, model right-sizing (small models enrich and tag; the frontier model synthesizes), context trimming. Budget alerts at 50/80/100%, and the metric that ends arguments: cost per triaged issue vs. the human hour it replaces.
Breakv1 keeps no record of model calls. When a user says "Tuesday's triage was weird," there is nothing to replay — the diagnosis is a shrug.
FixLog every call — full input, output, model version, latency, cost — replayable. Text is nearly free to store (a year of transcripts costs less than one video call's recording); the real constraint is policy: retention windows, PII, and honoring zero-retention contracts, which is exactly what the data-terms flags enforce.
BreakGolden files and labeled cases are authored and graded by one builder. At scale that's a single point of judgment: the evals encode one person's definition of "correct."
FixEval governance: labeled sets get an owner and a review cadence; production corrections flow in weekly as candidate cases (adjudicated conflicts first — see 02); the LLM judge is recalibrated against human labels quarterly. Two eval kinds run in CI: the contract gate (check.mjs — is the output well-formed?) and the accuracy gate (classify.mjs — did it put issues in the right box?). A wrong-but-well-formed answer passes the first and fails the second; you need both.
BreakEditing the system prompt in v1 deploys to 100% of users at once. With 800 users, that's betting the whole fleet on every wording change — and "rollback" means digging through git at 11pm.
FixPrompts become versioned config, not code. The rollout ladder: shadow (new prompt runs silently alongside old, outputs compared offline) → 5% of users → 25% → 100%, each stage gated on metrics (correction rate, receiver-usefulness, eval scores). Rollback is a pointer flip — seconds, no deploy. Never change the prompt and the model version in the same step, for the same reason as the ladder below: an unattributable failure forces a total rollback.
Breakv1 asks the person pasting to declare the data terms. Honest for a demo, but at scale nobody should re-read an NDA at triage time — and nobody will.
FixTerms are captured once, at contract signing, as structured CRM fields (usable for analytics? for learning? retention window?). Every feedback record carries its account ID and inherits the flags; the pipeline enforces them mechanically, default-deny when flags are missing — so the failure mode is "overly careful," never "learned from something we shouldn't have." New contracts get a standard data-use clause; the top accounts get a one-time backfill.
BreakSearch-the-web and paste are two doors; the richest signal — support tickets, Gong calls, CRM loss reasons, the warehouse — never walks through either on its own. Manual intake is a documentation tax, and taxed data goes stale.
FixThe connector architecture: thin adapters normalize each source into the shape paste-mode already accepts (the core never changes), work items file into Linear/Jira, and status flows back automatically. For warehouse volume, cheap SQL shrinks millions of rows to the slice needing judgment — the model reads the slice, never the millions.
BreakOne builder knows how Cherry runs, fails, and turns off. Owners move on — new roles, new teams, well-earned vacations — and a system without an operability floor leaves with them.
FixMinimum viable operability before the user base widens: a one-page runbook (what it is, how to tell it's broken, how to turn it off, who inherits it), a named backup owner who has run it once, a documented kill switch, and a failure taxonomy — each known failure mode with its detection, severity, and containment. Scale is a privilege operability has to earn.