The Lifecycle Brief · Issue 003
The proof you need to make a good decision isn’t the same as the proof you need to claim you caused something to happen.
Brian Heying
September 14, 2026 · 12 minute read
A few years ago I spent a lot of time on failed payments within the SaaS product I managed. A card expired, a payment bounced, the billing contact changed jobs and nothing was updated until I reconciled the open invoices each month. The service kept running. The accounts still existed. The money just didn’t come in. We routinely threatened account suspension for failed payments but followed through far less frequently.
I helped build and improve the system that went after those missed payments. The transactional emails, the retry logic, when to ask and how many times, the handoff to the people who could fix a broken payment method. Billing. Collections. It recovered more than $250,000.
Over roughly the same period, while a lot of other lifecycle and growth work was running, total monthly churn moved from around 5% to around 4%.
I was reporting dunning revenue captured and churn reduced. Both KPIs most likely showed up on the same slide in my monthly reports.
I don’t recall stating explicitly that the dunning system caused the churn reduction. In fact, part of the process was dealing with cancellation requests from customers trying to avoid collections. I couldn’t support a causal claim to churn reduction, as much as I wish I could have. But I also never went out of my way to test whether it was true.
That hesitation was some rigor I was trying to maintain. More than that, it was an inability to prove something that might’ve been true.
The recovered revenue was sufficient to report on its own. It was a widely viewed success.
Every dollar the system brought back was money owed for a service the customer had already used. I don’t know how much of that $250,000 would have come in anyway. I didn’t have a holdout for that either. But the system recovered several times what it cost to run, which gave us a lot of room to be wrong about the exact incremental amount and still make the same decision. That was all the reason I needed to keep it running and continue optimizing it.
Now for the other claim. The dunning system caused monthly churn to fall from 5% to 4%. It might have. It’s certainly a causal statement. It says this specific thing produced that specific movement, and that without it the movement wouldn’t have happened.
A failed payment didn’t automatically become churn. We rarely suspended service, so recovering a payment didn’t mechanically keep an account active. The softer theory was that better billing transparency gave customers a clearer understanding of their account standing and maybe made them more likely to stick around. There were reasons to doubt that too. Collections sometimes prompted customers to cancel.
I didn’t have anything to support that. Other work was running at the same time, and we weren’t holding any group of customers back from payment recovery, so there was nothing to compare against. Holding a group back would have meant deliberately not attempting to collect money owed, which is tough to justify to anyone.
The proof you need to make a good decision isn’t the same as the proof you need to claim you caused something to happen.
I had the first. I didn’t have the second. That previous work, plus a couple recent conversations, particularly around lifecycle attribution in B2B sales-assisted SaaS, is what prompted me to pull on this thread.
eBay wanted to know what its paid search advertising was actually producing.
The normal reporting said it was producing quite a bit of sales and revenue. Somebody searched, clicked an ad, and bought something, and the sale got counted as a result of the ad. Pretty straightforward attribution.
But using that number to say the ad produced the sale turns it into a causal claim. It was also the number being used to decide whether to keep spending on paid media.
So eBay turned some of that advertising off in certain regions and watched what happened to sales in those areas where the ads stopped running.
When they turned off non-brand search ads, revenue barely moved at all, nowhere near what the reporting predicted. The reporting had been counting sales the advertising didn’t produce. A lot of those customers were going to buy anyway.
The attribution had become the evidence for the spending decision. eBay’s decision was big enough to be worth testing, and the test is what showed the number couldn’t be trusted. They didn’t know that going in.
Here’s the other side of that same coin:
If the advertising was working the way the reporting said, new app installs should have dropped noticeably once it stopped. But they didn’t.
Nobody had produced a clean accounting of what every advertising dollar had been doing. What Uber had was a huge removal without the expected drop in installs. For the decision actually in front of them, which was whether to turn the spending back on, they had all the evidence they needed.
The two cases arrived at the same problem from opposite directions. eBay deliberately created a comparison to test what its reporting was claiming. Uber made a massive spending change for other reasons and learned from what didn’t happen afterward.
Both of those are consumer businesses with massive volume and the ability to get feedback quickly. My work is primarily in B2B, where sales cycles can be long, fewer deals close in a given period, and the buying decision most often runs through a committee.
I went looking for evidence that B2B lifecycle treatments create incremental pipeline. We’re talking the nurture programs before the sale, but also reactivation campaigns and anything else administered to active and dormant prospects. I couldn’t find anything publicly available that put a credible incremental pipeline or revenue value on a program like that.
There were, however, two pieces of B2B research I found useful.
How well the nurture programs worked depended on the situation, including how long the sales cycle ran and whether there was already a relationship with the prospect. The programs were valuable under some conditions and not others.
One of the most interesting findings was that in some cases, meetings went up while sales conversions didn’t.
A metric can improve without the business outcome improving. That feels like an obvious statement, just like more emails opened don’t automatically mean more sales. But if measurement stops prematurely at meetings booked, there’s no way to tell whether the program produced more sales or just more meetings.
There was another complication. The nurture treatments also changed how salespeople interacted with prospects. The treatments weren’t only working on the buyer. The nurture programs gave salespeople more information about the prospect, which changed the quality of those interactions.
Zendesk took roughly 400 leads its system had scored as sales-ready and another 400 or so randomly selected leads that hadn’t been qualified, and had the Sales team work both groups over a quarter.
Neither group of leads outperformed the other on initial engagement, re-engagement, or opportunities closed as won.
That doesn’t prove there was no difference between the groups. With roughly 400 leads per group and a single quarter of data, a smaller difference could have gone undetected. But if one group had performed dramatically better than the other, the test was much more likely to have caught it.
The more useful piece to this is the question they asked. They weren’t trying to work out the total economic value of lead scoring to the company. That would mean putting a dollar figure on what the scoring system returns against what it cost to build and run it. They were asking whether a salesperson should trust the score when deciding which lead to call first. Their evidence answered that second question. It couldn’t have answered the first question.
Picking a question your evidence can actually answer is a skill. It’s certainly one I’m still honing.
If the decision and the claim can require different standards of evidence, the next question is what standard the decision itself requires.
Here’s an example: sales-led B2B SaaS. It has a large pool of leads and accounts demand gen has already paid to acquire. The prospects engaged at some point, but didn’t progress, and have since gone quiet. Lifecycle looks at that pool and sees stranded demand worth trying to recover.
Previously, I would’ve started by asking how much of the resulting pipeline lifecycle treatments should get credit for. That’s a question about the internal scoreboard, and it skips a more immediate question: is this opportunity worth testing?
So that’s where I’d start now.
Then: what would we need to see to keep investing in it?
A plausible answer to the second question could be seeing more of the inactive leads we treated get picked up by Sales and move into active opportunities than the leads we left alone.
But we’d also need to watch what happened after that. More opportunities doesn’t mean much if they don’t eventually produce more sales. Habel’s meetings-without-conversion finding is exactly what we’d want to avoid.
Now we’re at the evidence. Depending on how much volume is in that pool of inactive leads and how much of it we can hold back, that could be an account-level holdout, a phased rollout, or something else entirely.
A holdout can give us a comparison group: accounts we deliberately leave alone so we can see what happens without the treatment. A phased rollout might be easier to run, but will be harder to read because everything else is changing at the same time.
There’s another complication we’d have to design around. The treatment itself may change how Sales allocates attention. If treated accounts start surfacing more signals, reps may spend more time on them and less time on the holdout. An account-level holdout isn’t automatically clean in that environment. The design has to account for how Sales will actually work both groups.
Use evidence strong enough for the decision being made. The available data and what the organization can realistically do will largely determine what that looks like. And commit to the outcome being measured before the results start to come in.
Now for the conclusion: if the marketing treatment gets meaningfully more inactive leads picked up by Sales and moved into active opportunities, and those opportunities continue producing better outcomes downstream, that could be enough to keep investing in it.
If we’re still early in the test, that still wouldn’t be enough to put a defensible dollar figure on incremental pipeline. A well-designed holdout followed far enough downstream could eventually support that stronger claim. We just haven’t earned it yet.
Those are two different statements, and so far the evidence only supports the first.
Now assume the test works. More inactive leads are getting picked up by Sales, more are becoming active opportunities, and the downstream results still look good.
The next decision is bigger. The CRO proposes putting another $500,000 into it. Now we’re talking about additional headcount to run and expand it, a data enrichment contract to fill in the accounts missing usable contacts, and data engineering time to wire the account-level orchestration into the CRM properly.
The first decision was easier to reverse. If the treatment turned out to be worth less than it appeared, the loss was mostly the effort already spent. A $500,000 investment is quite a bit harder to take back.
The cost of being wrong went up. That’s a reason to want stronger evidence before making the commitment.
At that point, I’d want to know more than whether the first test cleared its initial hurdle. I’d want to see the result hold up over a longer window and farther downstream. If the investment case depends on a specific amount of incremental pipeline or revenue, then the evidence has to be strong enough to support that claim too.
The distinction between decision and claim hasn’t disappeared. The claim has become part of the decision.
It’s the same underlying opportunity, albeit with a more sophisticated system, a different decision, and a higher standard of evidence altogether.
Lifecycle practitioners don’t need the strongest possible evidence for every decision. They need evidence strong enough for the decision they’re making. Deciding whether to keep running a program they could turn off next month isn’t the same as deciding whether to put $500,000 behind it.
The cleanest design on paper isn’t always available either. Sometimes, dare I say often in B2B, there just isn’t enough volume to hold a group back, or enough time before the decision gets made, or enough tolerance for treating one set of customers differently from another.
The evidence also determines what can responsibly be said about the result. Influenced, associated, attributed, incremental, and caused aren’t interchangeable. The evidence has to support the language being used and the decision being made.
The sequence can be:
The decision to build, keep, kill, or scale often gets made quietly, or by default, while the louder conversations more often happen over credit and causality.
There’s no doubt that the payment recovery system I was managing was worth the effort. It recovered more than $250,000. I still can’t tell you what share of that churn reduction it accounted for, if any at all.
Building, improving, and continuing the dunning system was the decision. Stating that the system caused churn to fall from roughly 5% to 4% would have been the claim. I had enough evidence for the decision. I didn’t have enough for the claim.
One email per issue. Unsubscribe anytime.
Working through something similar?
I’m interested in talking with founders and operators building customer-growth capability, and hearing what they’re seeing.