Building a 28-day retention prior from messy Firebase exports

Firebase will happily export three names for the same home screen. One came from an iOS rename, one from an Android experiment, and one from a contractor who left. If you average “return” across those names, your 28-day retention prior is a story about instrumentation, not about people.

In the studio we treat the export as a suspect document. The first artefact is not a model. It is a one-page dictionary: canonical event, aliases, first seen, last seen, and whether the event still fires after the latest store build.

Missing days are information

A blank Wednesday is often an SDK outage, a sampling change, or a developer who paused debug mode incorrectly. Imputing it with a weekly average pretends the outage did not happen. For a prior, we prefer to mark the day as unobserved and to shorten the horizon rather than invent a number that will later look like demand.

What a prior is allowed to claim

A 28-day prior should say: given the users who opened in week zero, what fraction do we expect to open again inside four weeks, after weekday and holiday structure is acknowledged? It should not say “engagement is healthy.” It should not absorb a campaign that only ran on weekend two unless you explicitly keep campaign users in the base.

Write the inclusion rule in a sentence a designer can challenge. “We count a return if any of screen_home, HomeView, or main_tab fires.” If that sentence feels embarrassing, the prior is already doing too much work.

A small, stubborn method

Collapse aliases. Drop days you cannot observe. Compute return at 7 and 28 on the remaining users. Compare payday cohorts to non-payday cohorts before you pool them. If the two disagree by more than you can explain, do not pool them just to get a single slide.

This is slower than a dashboard tile. It is also the reason Retention Horizon spends two weeks on dictionaries. Teams that skip it still ask us later why the forecast “drifted” after a rename.

Related reading: weekday seasonality and the usage forecasting brief.