Implementation worksheet · 5 min read
A Retention Cohort Reporting Contract
Retention numbers disagree because four definitions float free: who's in the cohort (signup date vs activation date — activation-based curves run higher and hide onboarding losses), what counts as retained (any login vs core action — the activity bar), how windows work (calendar weeks vs rolling N-day, unbounded vs bounded), and the minimum segment size worth reading (under ~50 accounts, curves are noise). The contract is one page that pins all four, gets signed by product, marketing and CS, and becomes the citation for every retention claim. After that, a 'retention improved' claim either references the contract or it doesn't count.
Three teams, three retention numbers, one board meeting — the standard failure. None of the numbers is wrong; they're answers to different unwritten questions. The fix is boring and organizational, which is why nobody does it and everybody argues.
Put it into practice
1. Pin the cohort entry event
Signup-cohorts measure the whole funnel including onboarding losses; activation-cohorts measure the retained product experience. Both are legitimate — the contract picks one as the headline and names the other as a secondary view, so nobody swaps them mid-quarter to flatter a launch.
2. Set the activity bar at the core action
'Opened the app' retention flatters everyone and predicts nothing. The bar is the action your value proposition names — the message sent, the report run, the campaign shipped. Expect the honest curve to drop when you raise the bar; that drop is information, not a bug.
3. Fix the window convention
Rolling 7-day windows from each account's entry date, or calendar weeks — either works, mixed is chaos. Decide bounded (active IN week 4) vs unbounded (active in week 4 OR later); unbounded curves only rise over time and quietly rewrite history in dashboards.
4. Set the segment floor
No curve gets cited below ~50 accounts per cohort; small-segment 'insights' are how noise becomes strategy. Small segments aggregate up (monthly cohorts instead of weekly) until they clear the floor.
5. Publish the contract next to every dashboard
One page: entry event, activity bar, window rule, floor, owner, review date. Every retention chart links it. Disagreements now amend the contract instead of forking the metric.
The four decisions
Copy this structure into your review document and record your observed result for each row.
| Decision | Option A | Option B | Contract says |
|---|---|---|---|
| Cohort entry | signup date | activation date | |
| Retained = | any login | core action | |
| Window | calendar weeks | rolling from entry | |
| Bounded? | active in week N | week N or later |
A failure worth checking
The mid-quarter definition swap: a launch underperforms on signup-cohort retention, so the deck quietly switches to activation-cohort numbers — which run higher by construction. Nobody lied; the metric just forked. Six months later no historical comparison is valid and the retention conversation resets to zero. The contract exists to make this swap visible and expensive.
Common questions
Which definition should be the headline number?
Activation-based with a core-action bar, bounded windows — it measures the product experience you can actually improve week to week. Keep the signup-based curve as the funnel-health view. What matters more than the choice is that it's written down and stable.
How do we handle seasonality in cohort comparisons?
Compare cohorts to the same-period prior year where the business is seasonal, and annotate known events (pricing changes, big launches) directly on the curves. The contract should name which annotations are mandatory — an unannotated anomaly becomes next quarter's myth.
Basis and scope
This is a proposed implementation method using illustrative examples, not a measured benchmark or a customer case study. Prepared with AI assistance. Validate product-specific behavior against current documentation and your own test environment.