What is cohort analysis?

What problem does cohort analysis solve?

Aggregate metrics mix new and existing users, so a change in the mix looks identical to a change in behavior.

A site where retention is improving every month can show a falling aggregate retention rate, if it is acquiring new users faster than before. Each cohort performs better than the last, and the blend gets worse because recent cohorts are larger and younger. The aggregate reports decline while the product improves.

Cohort analysis removes the ambiguity by never mixing groups. January's cohort is followed as January's cohort forever, and comparing it against February's is a comparison of like with like. Aggregate metrics answer how the business is doing right now. Cohorts answer whether it is getting better, and the two diverge whenever the audience is changing.

How do you build a cohort table?

Three components.

Cohort definition. The shared starting point. Usually first-visit month or week. Can also be acquisition channel, first product purchased, or the version of the site experienced.

Time axis. Periods since the cohort started, not calendar dates. Week 0, week 1, week 2. This alignment is what makes cohorts comparable.

Metric. Retention, revenue, repeat purchase rate, or conversion. For CRO work it is usually conversion, measured over a window long enough to cover the sales cycle.

The result is a triangular table: rows are cohorts, columns are periods since start, and each row is shorter than the one above because recent cohorts have had less time. Read down a column to compare cohorts at the same age. Read across a row to see one cohort's trajectory.

Two reading errors to avoid. Comparing incomplete periods, where the newest cohort's final column covers a partial week and looks artificially low. And ignoring cohort size, where a tiny cohort with an extreme rate distorts the visual pattern.

Where does it apply to CRO?

Less directly than to product analytics, and it earns its place in three situations.

Verifying that a shipped change lasted. Cohorts arriving before and after a redesign, tracked over months, show whether the improvement held. A holdout group is the stronger version of the same idea, since it keeps an unexposed comparison running instead of trusting that the earlier cohort was otherwise similar.

Separating channel quality from page performance. Cohorts by acquisition channel reveal whether a falling conversion rate reflects a worse page or a worse traffic mix. This is the most common CRO use.

Long B2B cycles. When conversion happens weeks after first visit, session-scoped metrics miss it entirely. Cohorting by first-visit month and tracking conversion over the following quarter captures what a session-based funnel cannot.

Related terms

Segmentation analysis · Holdout group · Funnel analysis · Revenue per visitor · Novelty effect

Service: Conversion Rate Optimization.

FAQ

What is the difference between cohort analysis and segmentation?

Segmentation splits users by an attribute at a point in time. Cohort analysis groups them by a shared starting point and follows each group forward. Segmentation is a snapshot, cohorts add the time dimension.

What cohort size do you need?

Enough that the metric is stable within each cohort. Small cohorts produce volatile rates that look like signal. If weekly cohorts are noisy, use monthly.

Why does aggregate retention fall when every cohort is improving?

Because recent cohorts are larger and younger, so the blend weights toward users who have had less time to return. Each cohort can beat the one before it while the aggregate declines. Read the cohort table before treating an aggregate drop as a regression.