heldout

Working notes on growth and strategy, plus four decisions I keep re-arguing until the numbers stop flattering whoever asked the question.

note 000

The question nobody asks afterwards

Every growth team I have watched runs the same meeting. Somebody presents a campaign. There is a slide with reach on it, a slide with click-through, and a slide with revenue in the period. Everyone nods. Nobody asks the only question that matters, which is whether any of that revenue would have arrived anyway.

It is not that people are lazy. It is that the question is genuinely uncomfortable, because the honest answer is usually "most of it". The users who open a push notification about a product are disproportionately the users who were already thinking about that product. You targeted intent, then took credit for it. The campaign performed brilliantly and changed nothing.

This page is two things stuck together. The first half is what I actually believe about growth, written the way I would argue it at a whiteboard rather than the way it gets written in a deck. The second half is four strategy questions I have worked through properly, with the arithmetic visible, because a recommendation without arithmetic is just a preference wearing a suit.

Every number in the case studies is a worked assumption, not a company's real accounts. The point is the structure of the argument and where it would break. Change an input and see which conclusions survive. Most of them don't.

note 001

Growth is four numbers, not a department

Strip away the tooling and a consumer business is four quantities multiplied together. How many people arrive. What fraction of them do the thing once. What fraction keep doing it. How much each one is worth while they do. Everything a growth team ships moves one of those four, and most teams can't tell you which one they moved last quarter.

The reason this matters is that the four are not equally elastic and they are wildly unequal in cost. Acquisition is the one everyone reaches for because it is purchasable. You can spend money on Monday and see users on Tuesday. It is also the one with the worst economics, because you are bidding against every other company for the same attention, and the price only goes up.

Activation is usually the cheapest win in the building and the least fashionable. If half the people who install never complete the first meaningful action, you have already paid for those users. Fixing the first session costs engineering time rather than media spend, and the gain compounds into every cohort you acquire afterwards.

Retention is the one that decides whether the company exists in three years, and it is the hardest to move with marketing, because retention is mostly the product being good. I have seen a great deal of lifecycle messaging deployed against a product that people simply did not want to come back to. It buys you a few weeks of flatter curve and then the curve remembers what it was doing.

the uncomfortable version

If your retention curve does not flatten, no amount of acquisition fixes the business, it only changes how fast you find out. A leaky bucket with a bigger tap is a wetter floor.

note 002

Lift, luck, and the holdout

A holdout is a group of users you deliberately do not message, chosen at random from exactly the same pool as the ones you do. It is the cheapest instrument in growth and the one most often skipped, usually with the argument that withholding a campaign from ten percent of users costs revenue. That argument is only correct if the campaign works, which is the thing you were trying to find out.

The discipline is harder than it sounds, because the temptation is always to hold out the users you care least about. The moment your holdout is systematically different from your treated group, you have measured the difference between two kinds of user rather than the effect of your campaign. Random assignment is not bureaucracy. It is the whole instrument.

The second trap is running the test at a size where the effect you are hoping for is invisible. If baseline conversion is two percent and you are hoping for a ten percent relative improvement, you are trying to see the gap between 2.0 and 2.2 percent. That needs a lot more users than people assume, and if you do not have them, the honest move is to stop and pick a bigger swing rather than run an underpowered test and read the noise.

Can you even see it?

Rough sample size per arm for 80% power at 95% confidence. Move the dials and find out whether the test you are planning can detect the thing you are hoping for.

2.0%
10%
200,000

brick = treated · green = held out

The widget is a rule of thumb, not a statistics package, but the shape of it is the lesson. Small baselines and small hoped-for lifts are brutally expensive to measure. This is why "we tested it and it was flat" so often means "we could never have seen it either way".

note 003

Cohorts are not segments

People use the words interchangeably and then get confused by their own charts. A segment is a group defined by an attribute: city, device, plan, age. A cohort is a group defined by when something happened to them, usually when they signed up. The distinction matters because segments answer "who" and cohorts answer "is this getting better".

Blended metrics hide almost everything interesting. If your overall retention is flat across a year while you triple acquisition, the honest reading is that your newer cohorts are retaining worse and being masked by the older ones. Anyone looking at the blended line sees stability. Anyone looking at the cohort grid sees the business quietly changing shape.

The practical habit: never report a rate without saying which cohort it belongs to and how old that cohort is. "Conversion is 4.1 percent" is not a fact. "Week-four conversion for the March cohort is 4.1 percent, against 5.3 for January" is a fact, and it is also a much more alarming sentence, which is usually why the first version gets presented.

The grid you should be able to draw from memory

cohortsizewk 1wk 4wk 12read
Jan41k38%21%17%curve flattens, healthy
Feb52k36%19%15%slightly worse, watch it
Mar96k31%13%8%paid push brought in worse users

March looks like a triumph on a blended dashboard, because it is by far the biggest number in the size column. It is the month the business got worse.

note 004

Where the funnel lies to you

The classic funnel drawing implies that the biggest absolute drop is the biggest problem. It usually isn't. The biggest drop is often a stage that is doing its job, filtering people who were never going to buy. The expensive leak is normally further down, where users have already shown intent and then hit something stupid.

What I look for is the step where the drop is large and the users leaving had already demonstrated they wanted the thing. Someone who added to basket and then vanished at payment is a different animal from someone who bounced off the homepage. Both show as a drop. Only one of them is worth an engineer's month.

visit1,000,000 browse240,000 basket38,000 paid9,900 -76%  cheap filter -84%  cheap filter -74% and these people already wanted it the two biggest drops are the two least interesting ones
Illustrative numbers. The 84% drop before basket is a filter. The 74% drop after it is a bug, a fee revealed too late, or a form that fails on a mid-range Android handset.

A useful test when someone shows you a funnel: ask what it costs to move each step by one percentage point, and what that point is worth. Most funnels have exactly one step where those two numbers are not close, and that step is the roadmap.

note 005

Payback, and why LTV is a story

Lifetime value is a forecast dressed as a measurement. It requires you to assume a retention curve out to a horizon you have not reached, and the assumption is doing all the work. A company with eighteen months of history quoting a three year LTV is telling you about its optimism, not its unit economics.

Payback period is the honest cousin. How many months until the gross profit from a cohort covers what you paid to acquire it. It uses only data you already have, it cannot be inflated by a flattering tail assumption, and it maps directly onto whether you need to raise money.

CAC payback, month 7 never pays back m0 m7 m18
Same acquisition cost, two channels. The lower curve is the one that looks fine on a blended LTV slide because someone assumed the tail keeps climbing.

The rule I hold to: if you cannot show me the payback month from realised data, I will treat the LTV number as marketing. This is not pedantry. Companies die in the gap between the assumed tail and the real one.

note 006

Seasonality eats your experiment

In a lot of consumer categories, demand is not smooth. It arrives in spikes attached to festivals, paydays, weather and school terms. Run a campaign into a spike and your conversion numbers will be spectacular. Run the identical campaign three weeks later and it will look like a failure. Neither result tells you anything about the campaign.

This is why the before-and-after comparison is close to worthless in seasonal businesses, and why the holdout is not optional there but essential. A control group living inside the same week as the treated group absorbs the spike automatically. A baseline period from last month does not.

There is a subtler version that catches people out. During a spike, a large share of your converters were going to convert regardless, so the ratio of incremental to total conversion collapses even as total conversions rise. The campaign genuinely gets less effective exactly when it looks most effective. Budget decisions made on the headline number during a festival tend to be exactly backwards.

a thing worth saying out loud in an interview

Sizing a holdout during a demand spike is harder, not easier, because you need it to be big enough to detect a smaller relative effect against a noisier baseline. If someone tells you they will just compare against last week, that is the moment to push.

note 007

When to stop testing and just ship

Experimentation has a cost that nobody puts on the slide, which is calendar time and the opportunity you did not take while you were measuring. Not everything deserves a test. A test is worth running when the decision is expensive to reverse, when the effect is plausibly small enough that you could be fooled, and when you have enough traffic to see it.

If the change is cheap, obviously directionally right, and reversible in a day, ship it and watch the dashboards. If the change is a rebrand, a price move, or anything touching the first session for every new user, test it properly, because the cost of being wrong is paid by every cohort that follows.

The failure mode I see most is the middle ground, where a team runs a six week test on something they would have shipped anyway and calls the delay rigour. Ask what you would do if the result came back flat. If the answer is "ship it", you have already made the decision and you are buying comfort rather than information.

case study 008 · category entry · grocery retail

Should Tesco enter the beer market?

market sizingvertical integrationprivate labelright to win
client
A large UK grocer with roughly a quarter of national grocery spend and an established own-label operation.
question
Should it move from reselling third-party beer to owning production, either by acquiring a brewer or building its own brand at scale?
decision
Capital allocation. Roughly £150m either way, and it is slow to unwind.
figures
Worked assumptions for the purposes of the argument, not the company's accounts.

First, the question behind the question

"Should we enter the beer market" is badly posed, because Tesco is already in the beer market. It is one of the largest sellers of beer in the country. What is actually being asked is whether to move up the value chain from distribution into production, and that is a different question with a different answer.

So the real decision splits into three options that get conflated in the room. Acquire a brewer. Build an own-label beer served by contract brewers. Or do neither and keep negotiating harder with the brands you already stock.

Sizing, roughly

The arithmetic does not need to be precise to be decision-useful. UK adults, call it 54 million. Beer drinkers, maybe 55 percent. Of beer volume, a bit over half is now bought for home rather than in pubs, and that share moved permanently after 2020. Take-home beer spend lands somewhere in the region of £7bn a year. A quarter share of that flows through this grocer already, so roughly £1.75bn of beer moves across its tills.

lineassumptionvalue
Take-home beer, UKmarket~£7.0bn
Share through this grocer25%~£1.75bn
Current margin as reseller~8%~£140m
Realistic own-label share of its beer15%~£260m revenue
Margin on own-label~22%~£58m
Margin given up on displaced brands8% of £260m-£21m
Net annual gain if it works~£37m

That is the number the whole case turns on, and it is smaller than people expect. Roughly £37m of incremental margin against £150m of capital, before any of it goes wrong. Payback is four years on a flawless execution, which is not the base case for anyone's first brewery.

Does it have a right to win?

Two things are genuinely in its favour. It owns the shelf, which is the scarcest asset in the category and the reason most challenger beers die. And it owns the data: it knows exactly which of its customers buy beer, how often, at what price point, and what else lands in the same basket. A new brewer would pay enormous sums for that and still not have the shelf.

What it does not have is the thing beer is actually sold on. Beer is an identity purchase in a way that own-label pasta is not. People buy the brand in front of other people. The categories where grocer own-label has won hardest are the ones consumed privately and judged on price. The categories where it has struggled are the ones carried to a barbecue.

attractive & can winValue take-home lager, multipack, price-led shoppers. Own-label already wins here in adjacent categories.
attractive, can't winPremium craft and imported lager. The margin is real and the brand equity is not purchasable on a grocer's balance sheet.
unattractive, can winLow and no-alcohol. Grocer brand carries no stigma here, but the category is small and the majors are already fighting for it.
unattractive & can't winOwning a brewery outright. Capital heavy, cyclical, unionised, and it makes every brand you stock treat you as a competitor.

The thing that kills the acquisition option

This is the part that usually gets missed in the room. The moment a grocer owns a brewer, every other brewer it stocks is negotiating with a competitor. Those relationships are worth far more than £37m a year in promotional funding, listing support and supply priority. You are risking the profitable relationship to chase the smaller margin.

recommendation

No to acquiring a brewer. Yes to a contract-brewed own label, confined to the value tier. Same margin opportunity, a fraction of the capital, no vertical conflict with the brands that fund your promotions, and it is reversible in eighteen months if the numbers disappoint.

What would change my mind: if own-label penetration in the value lager tier tests above 20 percent rather than 15, the acquisition case starts to clear its cost of capital. That is a six month, low-cost test using existing shelf. Run it before spending £150m.

case study 009 · adjacency · consumer fintech

A savings app that wants to sell jewellery

cross-selllifecycleincrementalityseasonality
client
An Indian micro-savings app. Twenty million users save small amounts into digital gold. It has launched a direct-to-consumer gold jewellery brand on the side.
question
How do you grow the jewellery business off the savings base without wrecking the savings business that feeds it?
tension
The two products want opposite things from the same user. One wants them to accumulate. The other wants them to spend.

The structural problem nobody wants to name

A savings product measures success by balance retained. A jewellery product measures success by balance converted into an order. These are the same rupees. Left alone, the jewellery team will cannibalise the savings team's core metric and both will report a good quarter using different denominators.

So before any campaign work, the businesses need one shared number. The candidate I would argue for is gross profit per user per year across both products, because it is indifferent to which product the money lands in and it stops the internal argument about whose rupee it was.

Who actually buys

The instinct is to blast the whole base. That is wrong for a reason that is specific to gold. Savings balance is not purchase intent. A user with a large balance has demonstrated patience, which is close to the opposite of impulse. The buyers are more likely to be found where balance intersects with an occasion.

cohortsignalhypothesistest first?
Milestone holdersbalance crosses a round numberBalance feels "spendable" at a thresholdyes
Occasion-proximatebought gold in the same week last yearAnnual gifting habit, highly predictableyes
Browsersopened jewellery tab, no orderIntent already shown, blocked on price or trustyes
High balance, no browselarge balance onlySaver identity, will resent the askno, hold back
New saversunder 60 daysHabit not formed, spending ask breaks itno, protect

The last two rows are the ones that make this a strategy question rather than a campaign question. Excluding users is a growth decision. Every message sent to a saver who does not want to be sold to is a small withdrawal from the reason they trusted you with money in the first place.

The measurement trap

Festival season is where this business makes its money, and it is exactly where attribution falls apart. A campaign sent in the week before a major gifting festival will report enormous conversion. Most of those people were buying gold that week regardless. The campaign's true contribution is the difference against a randomly held-out group living through the same festival, and it will be a fraction of the headline.

I would insist on a permanent holdout, not a per-campaign one. Two percent of the base, never messaged about jewellery, refreshed annually. It costs a little revenue and it is the only way to answer the question the board will eventually ask, which is what the entire lifecycle programme is worth.

recommendation

Grow the jewellery business off intent signals, not balance, and protect the saving habit explicitly. Start with the three cohorts where intent is already visible, keep new savers entirely out of the jewellery programme for their first sixty days, and run a permanent holdout so the festival numbers can be discounted honestly.

The number I would take to the board is incremental gross profit per user per year against that holdout. Everything else on the dashboard is activity.

case study 010 · pricing & packaging · subscription

Annual-only pricing in a monthly market

pricingcash conversionchurn opticselasticity
client
A consumer subscription app, monthly plan at ₹299, roughly 400,000 paying users, monthly churn around 8 percent.
question
Should it push hard into annual plans, or go annual-only?
why it is being asked
Someone in the room noticed that annual subscribers churn less, and drew the wrong conclusion from it.

The inference error at the centre of this

Annual subscribers do churn less. They also self-selected into the annual plan because they already expected to stay. Converting a wavering monthly user into an annual one does not give them the loyalty of the existing annual cohort. It gives you their money earlier and a refund request in month three.

This is the same mistake as taking credit for a campaign that targeted intent. The correlation is real. The causal claim smuggled in beside it is not.

What annual actually buys you

Three things, and they are worth being precise about. Cash up front, which matters enormously if you are burning. A twelve month window in which the product has time to become a habit. And a churn number that looks better because the decision point has been moved, not removed.

monthly ₹299annual ₹2,399note
Effective monthly price₹299₹20033% discount to buy the commitment
Expected months paid~12.512.0at 8% monthly churn
Revenue per subscriber₹3,738₹2,399annual is worse per head
Cash in month 0₹299₹2,399eight times the cash

At 8 percent monthly churn the average monthly subscriber already lasts a bit over a year, so the annual plan as priced destroys revenue per head and buys cash. That is a financing decision dressed as a pricing decision, and it should be made by whoever knows the runway.

Why annual-only is the wrong version of a right idea

Removing the monthly option raises the entry price by eight times at the exact moment a user is least sure. In a price-sensitive market, the top-of-funnel loss will swamp the retention gain, and you will not notice for two quarters because your churn metric will be beautiful while acquisition quietly collapses.

recommendation

Keep monthly. Offer annual at the renewal moment, not at signup, and price the discount at what the cash is actually worth. The user who has already paid three times is the one whose annual conversion is closest to genuinely incremental.

Test design: randomise the annual offer at month three against a holdout, measure twelve month revenue per user rather than churn rate, and pre-commit to that metric before the results come in. Churn will improve either way and it will not tell you anything.

case study 011 · expansion · quick commerce

New city, or deeper in the old one?

density economicscontribution marginexpansion
client
A delivery business, profitable at contribution level in its first city, roughly breaking even in its second.
question
Next ₹40 crore of growth capital: a third city, or more dark stores in the first?
real question
Whether the profitable city is profitable because of the city or because of density.

Density is the whole business

In delivery, the cost that matters is minutes per order, and minutes per order falls as orders per square kilometre rises. A rider who completes three drops in one trip has a third of the delivery cost of one who completes one. This is why the first city looks like a different company from the third: it is not a better market, it is a denser one.

Which means the honest framing is not "city A versus city C". It is "buy density where we already have demand, or buy a new demand pool at the worst point on the density curve".

optionorders/day addedcontribution/ordermonthly contributiontime to breakeven
6 more stores, city 19,000₹34₹92 lakh~9 months
Launch city 312,000₹-6₹-22 lakh~26 months

The new city wins on the vanity metric, which is orders. It loses on the metric that determines whether you need to raise again, which is when the money comes back. Boards get shown the first table and asked to approve the second decision surprisingly often.

The argument for the new city anyway

There is one, and it is not financial. Competitors take cities and do not give them back. If a rival establishes density in city three first, the cost of entering later is not the same cost, it is a much higher one against an incumbent. Expansion in this category is partly a land grab and pretending otherwise is naive.

So the question becomes whether city three is genuinely contested. If two competitors are already scaling there, the land grab argument is spent and you are buying the worst version of the market. If it is open, the strategic case can outweigh the payback case, but it should be argued as such rather than smuggled in behind an orders forecast.

recommendation

Density first, with a small defensive position in city three rather than a full launch. Put the bulk of the capital where contribution is already positive, and take three stores in the best-performing corridor of city three to hold the option open.

Revisit in two quarters against one trigger: if a competitor crosses roughly 20 percent share in city three, the option has expired and the full launch becomes a defensive necessity rather than a growth choice.

note 012

How I'd run the first ninety days

Weeks one to three, no campaigns. Rebuild the funnel from raw event data rather than trusting the dashboard, because the dashboard encodes somebody's old assumptions. Find out what the real conversion rate is, cohort by cohort, and where the expensive leak sits.

Weeks four to six, establish the holdout before shipping anything, because you cannot retrofit a control group onto a campaign that has already run. Get agreement in writing on the one metric that decides whether the programme worked, while nobody yet has a result to defend.

Weeks seven to twelve, run the three biggest swings rather than twelve small ones. Underpowered tests on marginal ideas are how a quarter disappears with nothing learned. Ship the winners, kill the losers publicly, and report incremental contribution rather than campaign volume.

If any of this is the argument you are currently having internally, I am happy to have it with you.

lkrajath22@gmail.com  ·  +91 91875 15572  ·  Bangalore