The Lifecycle Brief · Issue 002
A study of how much AI-powered personalization is actually worth building
Brian Heying
August 31, 2026 · 13 minute read
Marketing technology keeps making it easier to personalize more of the customer experience.
More data, more dimensions, and more ways to segment customers. More ways to change what each customer sees, when they see it, and what happens next. And increasingly, AI can make some of those decisions for us.
The question I think we should be asking is whether all of that personalization is even worth building.
Because the ability to distinguish between two customers doesn’t tell us that treating them differently will create more value. And proving that they should be treated differently still doesn’t tell us that AI is the best tool to accomplish it.
Those are separate decisions.
Imagine this scenario: you work at a field-service SaaS company like Housecall Pro, ServiceTitan, or Jobber. Two customers have told you what they’re trying to accomplish with your product.
Customer A wants to get paid faster. Customer B wants to book more jobs.
You have two different customers with two different stated goals.
So you personalize.
Customer A gets pushed toward payments setup. Customer B gets pushed toward online booking.
That’s a reasonable first move, but it’s not a justified one… yet.
Because there are at least three ways it could be wrong.
Maybe customers who say they want to get paid faster are already using payments. Maybe the ones who say they want to book more jobs are already using online booking. Or maybe the stated goal is real, but the treatment you chose doesn’t actually help them achieve it.
Different customers give us a reason to test different treatments, but they don’t tell us which treatment is going to perform best.
I’ve run into this before. At Equifax, customers chose a monthly subscription based on how many I-9 verifications they expected to process for new hires. But hiring doesn’t happen like clockwork. I added smaller plans for lower-volume customers and annual options that could better absorb changes in hiring throughout the year. Conversion and trial-to-paid performance improved over that period, but several things changed at the same time, so I don’t attribute the movement to the new plans alone.
But what customers did next exposed something else. Some bought the minimum subscription while barely using the I-9 capacity included with it. They weren’t trying to find the right amount of capacity. They were paying for access to other capabilities in the platform they actually valued.
The question stopped being how many volume options we should offer and became why we were making customers predict a number in the first place. Eventually, I designed the volume tiers out of the model altogether, moving to a subscription for platform access plus consumption-based pricing.
Back to Customers A and B: the first thing I’d want to know is whether the difference they told me about shows up in their behavior.
Do customers who say they want to get paid faster actually use the product differently from customers who say they want to book more jobs?
If they don’t, I’d be cautious about building much around the distinction.
If they do, now I have something worth investigating.
But even that isn’t enough.
I’d want to know whether the treatment I’m considering changes the behavior I care about. If I send the get-paid-faster group toward payments setup, do they actually adopt payments at a higher rate? If I send the book-more-jobs group toward online booking, do they actually enable it?
And then I’d want to know whether that behavior produces the outcome the customer originally cared about.
Did payments adoption actually shorten time-to-payment?
Did online booking actually produce more booked jobs?
And finally, I’d want to know whether any of that matters to the business.
Did customers who got paid faster retain better, expand more, or create more value?
Did customers who booked more jobs become healthier accounts?
That creates a fairly simple evidence chain:
The farther I can move down that chain, the more complexity I’m willing to justify.
None of that means a controlled experiment with statistical significance is required to treat two customers differently. That’s altogether separate from what evidence should exist at all, and they’re worth keeping separate. Customer research, support conversations, behavioral patterns, plain operational judgment: all of that is legitimate evidence, at any size of company. Defensible decisions get made on exactly that kind of evidence all the time, with zero experiments attached.
Less certainty doesn’t mean you can’t act on the evidence you have. It should change how much you’re willing to build around it.
If the evidence is reasonably compelling, test it. Send those two customers through different onboarding treatments and see what happens. That’s cheap, easy to watch, and easy to undo. What you probably shouldn’t do is spend six months building infrastructure around the same evidence before you know whether the underlying assumption holds.
The less certain you are, the easier the decision should be to reverse if you’re wrong.
I found a study that makes this concrete and I’m going to walk through it, because the mechanics of it are compelling, more so than the headline number.
Researchers Walter Zhang and Sanjog Misra worked with a food delivery platform running a randomized promotional experiment across over 1.2 million customers. Using 126 pieces of information about each customer, they built a statistical model of how a customer’s expected profit to the company changes depending on which promotional treatment they receive. That let them estimate, for every individual customer, which treatment would work best for someone with their specific profile. They added up what total profit would look like if everyone actually received their own personally optimal treatment. Let’s call that the full-personalization ceiling. Rounded to the precision used in the study, those individually optimal offers resolve to 2,206 distinct treatment levels. It’s worth being precise that this was a model’s estimate of a ceiling, not a live experiment where 1.2 million people each received a genuinely unique treatment.
Then they asked a more useful question: what if the company isn’t allowed a unique treatment for every customer, but is instead limited to a lesser number of common treatments? They tested increasing the number of allowed treatments, one at a time, and tracked how close expected profit got to that ceiling.
One thing worth knowing before looking at the numbers: they’re measured as profit gained over doing no targeting at all, not as raw revenue. So even the one-treatment row below isn’t a zero point. It’s the incremental profit from the single best blanket promotion the company could run.
| Number of treatments | Expected incremental profit |
|---|---|
| 1 | $808,920 |
| 2 | $829,300 |
| 3 | $834,085 |
| 4 | $836,669 |
| 5 | $838,450 |
| 10 | $840,985 |
| Full personalization | $842,366 |
Advancing from a single treatment to two produces the majority of the jump. A third and fourth add real but shrinking amounts. By five, the company was already capturing more than 99.5 percent of the profit available under full personalization. Going from five treatments all the way to ten, doubling the number of treatments available, bought nearly nothing on top of that.
That’s wild. The first few levels of personalization captured almost all of the available value. Everything after that was optimization around the edges: diminishing returns.
There’s another way to look at the same result. Once the company had found the best single treatment, the entire remaining profit opportunity between that and full personalization across 2,206 treatments was a little over $33,000.
Whether that $33,000 is worth pursuing depends on what it costs to capture it. That’s the part the percentage alone can’t tell us.
Here’s the takeaway: five treatments isn’t some magic number. This was one experiment, on one food delivery platform, using promotional offers and a specific set of customer data. There’s no reason to assume a B2B SaaS company would see the same curve, or that five treatments would get you anywhere close to the same result.
But what the study does reveal is the shape of the problem. A company can keep finding more ways to distinguish between customers long after those distinctions have stopped producing much additional value. And that’s the point where personalization stops being a question of what the company can do and becomes a question of whether building the next layer of complexity is actually worth doing.
There’s also a cost to this that’s not easy to see.
Gartner surveyed 1,464 B2B buyers, as well as consumers, about their experience with personalized marketing. 53 percent reported that a personalized experience had actually been a negative one. Among that group, people were 3.2 times more likely to regret the purchase they’d made, and 44 percent less likely to buy from that company again. I want to be exact about what this is and isn’t stating: the sample combines both B2B and consumer respondents, so these aren’t B2B-only figures, and the survey shows an association between bad personalization and worse outcomes, but that’s not proof that the personalization itself caused those outcomes.
Still, the spirit of the finding deserves more attention than the specific numbers. A generic, one-size-fits-all message can miss a customer cleanly. It says, in effect, this wasn’t built with you in mind. A personalization attempt that gets the customer wrong says something entirely different: we hear you. We just weren’t listening. My take is that the second kind of miss is a lot harder to walk back than the first. Put yourself in the customer’s seat: you know they tried to understand you, had information about you, and still got it wrong.
Gartner’s survey doesn’t prove my claim. But it does establish that personalization isn’t a neutral option sitting around waiting to be perfected while nothing else happens. There’s potential downside to getting it wrong. That means attempting it at all deserves more scrutiny before any of it even gets built.
If a personalized treatment reliably beats the default for customers who show a clear signal, that’s proof the treatment earned its complexity. It doesn’t yet say anything about whether AI earned its.
If the same company above finds that routing by a customer’s stated objective works, and the whole system amounts to a couple of rules:
…then that can be the system. It’s three rules, maybe a few more once the exceptions get written down. A person can maintain that, explain it to a coworker who asks how it works, and change it in an afternoon if it turns out to be wrong. There’s no reason to add a model on top of something that small just because a model is available.
This is not an argument against AI. It’s an argument that AI also has to earn its place the same way the personalization did, on its own evidence, not by inheriting credit from a decision it wasn’t involved in making.
The place AI actually starts to earn something is when that little rule set grows up.
Let’s picture the same company a year later. It’s not one objective driving the routing anymore, it’s an objective and a role. The account owner sees one thing. An office manager who only handles scheduling and invoicing sees another. A technician using the app in the field sees a third. Multiply objective by role and a system that looked like two paths can become a dozen, and each of those needs its own sequencing, its own timing, and its own rule about what gets suppressed so the different workflows don’t both decide to email the same person in the same hour about two different things.
The instinct at that point is to keep building: another workflow, one more branch, a few additional exceptions for the case someone just found. What that actually does is just shift the complexity around. It never really removes it. Each individual message stays simple. The system responsible for deciding who’s eligible for which treatment, in what order, and what to hold back when two rules disagree gets harder to see and maintain. Nobody writes that part down anywhere either. But the complexity is still there, whether a person or a model is making the call.
This is where asking whether AI can decide better than the rules isn’t actually the correct question. The better question is whether AI actually reduces the total complexity of running the system, or just shifts that complexity somewhere less visible.
AI doesn’t arrive complexity-free. Standing it up means:
Weigh that honestly against forty accumulating workflows and hundreds of interacting rules, and AI can absolutely come out on top as the simpler system to operate. Weigh it against an idealized, frictionless version of what AI could theoretically do, and it probably never does.
There’s really no way to precisely calculate this. Forty workflows and an AI governance layer won’t easily reduce to the same number. You’re comparing them to make sure they both get counted, instead of judging one system by its costs and the other by its promises. A big part of what belongs in that ledger is how hard each option would be to unwind if the judgment underneath it turns out to be wrong. A rule can be rewritten in an afternoon. A system built around a mistaken assumption about what customers want is a whole different kind of commitment. It’s worth weighing that difference honestly instead of treating both options as equally reversible.
I want to be clear. I’m not arguing against personalization, nor am I arguing against using AI to deliver personalization. I’m trying to treat each as a claim that has to earn itself separately, with the evidence getting stronger as the thing you’re building gets more complex and harder to undo.
The difference in what customers want is definitely worth investigating. It isn’t, by itself, permission to build around it. Proof that a personalized treatment works is absolutely a reason to deliver it. On its own, though, it’s not a reason to hand an AI model the keys. And even a model that can make decisions well isn’t automatically a model that’s simpler to run than the alternative it’s replacing.
What ties it together is my preference: easy to say and harder to actually put into practice. Don’t chase the most personalized system a company could build. Pursue the system with the least complexity that can still capture enough value to justify itself.
I think too much of what gets called an ‘AI-personalization initiative’ skips past that principle and naively starts building. And after working through the evidence above, I think that’s the wrong place to start.
One email per issue. Unsubscribe anytime.
Working through something similar?
I’m interested in talking with founders and operators building customer-growth capability, and hearing what they’re seeing.