Implementation worksheet · 5 min read
An End-of-Month Lifecycle Learning Repository Template
Keep one entry per learning, not per test: a one-sentence claim at the scope actually tested, the evidence behind it (test IDs, dates, absolute effects with intervals), where it holds (segment, channel, lifecycle stage), a strength grade (replicated, single test, observational only, inconclusive, contradicted), the decision it informed, known limits, a revisit date and an owner. At month end, spend an hour: add entries for the month's readouts, inconclusive ones included, re-grade the entries new evidence touches, and retire those past their revisit date without fresh support. New test proposals cite the entries they build on.
Scope: a lifecycle team that runs experiments and journey changes across email, push, in-app and SMS, and wants what it learns to outlast the people who learned it. Starting state: results scattered across slides and threads. Boundary: learnings about lifecycle messaging from tests and before-and-after packs; product research lives elsewhere. Intended outcome: a searchable set of graded claims, curated monthly, that stops the team re-running settled tests or treating unsettled ones as settled.
Put it into practice
1. Write the claim at the scope that was tested
'For self-serve trial workspaces in their first 14 days, a day-2 push raised activation', not 'push works'. Broad claims get applied where they were never tested, which is how one good result becomes a policy for every segment.
2. Grade the strength explicitly
Replicated: two or more tests agree. Single test: one test with an interval that excludes zero. Observational only: before-and-after evidence without randomization. Inconclusive: the test couldn't separate effect from noise, recorded with the effect it could have detected. Contradicted: later evidence disagrees.
3. Record inconclusive and negative results
They are the entries that save the most time later, because they stop someone re-running a test that already couldn't be read, or already failed. A repository of wins only also tends to overstate the effects in it, since the results chosen for recording were chosen for looking good.
4. Link the evidence rather than pasting it
Test IDs, dates and readout documents, with effects as absolute differences and intervals. The entry summarizes; the evidence stays where it was produced.
5. Run the month-end hour
Add an entry for every readout this month, re-grade entries that new results touch, retire entries past their revisit date with no new support, and flag contradictions for the weekly review.
6. Set revisit dates
Channels and audiences change. A learning about open rates from before Apple's Mail Privacy Protection, which Apple says prevents senders from seeing whether a message was opened, may not hold now. Six to twelve months is a rule of thumb for revisiting, not a standard; shorten it for anything that rests on opens.
7. Worked example (illustrative, synthetic numbers)
In September the team adds L-014: 'A day-2 push raises 14-day activation for self-serve trial workspaces'. Evidence: T-031 in July, +1.8 points (+0.4 to +3.2), and T-047 in September, +1.2 points (-0.3 to +2.7). Grade: single test with a consistent but inconclusive follow-up, so not replicated. The same session retires L-022, 'first names in subject lines raise engagement', because its only evidence was a 2025 test measured on opens.
Learning repository entry
Copy this structure into your review document and record your observed result for each row.
| Field | What to record | Illustrative entry (L-014) |
|---|---|---|
| ID and claim | Stable ID and a one-sentence claim | L-014: a day-2 push raises 14-day activation |
| Scope | Segment, channel, lifecycle stage, market | Self-serve trial workspaces; push; first 14 days |
| Evidence | Tests or evidence packs, with IDs and dates | T-031 (July), T-047 (September) |
| Effect and interval | Absolute effect and 95% interval, per test | +1.8 points (+0.4 to +3.2); +1.2 points (-0.3 to +2.7) |
| Strength grade | Replicated, single test, observational only, inconclusive or contradicted | Single test, with a consistent but inconclusive follow-up |
| Decision informed | What shipped or changed because of it | Push step shipped to all self-serve trials |
| Known limits | Where it hasn't been tested | Not tested on sales-assisted trials or users without push permission |
| Revisit date | When it gets re-graded | March 2027 |
| Owner | Who curates it | Lifecycle analyst |
A failure worth checking
The repository of wins. Only tests that shipped get entries. A year later a new team member proposes three tests the team had already run and found inconclusive, and nobody can find the old results; two are re-run at the same underpowered size and come back inconclusive again. Meanwhile the recorded wins, each chosen for looking good, are quoted as settled facts. Verification: each month, count readouts against new entries. Every readout should produce an entry, or a written reason why it didn't.
Common questions
What tool should the repository live in?
A spreadsheet or a shared document database is enough. The schema and the monthly hour matter more than the tool; a sophisticated tool with no curation decays the same way a spreadsheet does.
How is this different from the experiment log?
The log records tests. The repository records claims, each backed by one or more tests or evidence packs, with scope and strength. One test can support several entries, and one entry can cite several tests.
Basis and scope
This is a proposed implementation method using illustrative examples, not a measured benchmark or a customer case study. Prepared with AI assistance. Validate product-specific behavior against current documentation and your own test environment.