TapCub is live — analytics, insights and live chat, free on one platform
Growth & Marketing

Web Personalization Steps: From Segment to Measured Lift

The six steps of web personalization, run on one worked example from segment to a holdout test, with sample size, significance and a lift range turned into money.

Sophia Bennett Published Nov 20, 2026 11 min read Growth & Marketing
Sample data: 10,000 visitors per group. Measured relative lift +17.1%; the 95% interval is roughly +3% to +31%, so the variant's true rate is plausibly between 4.34% and 5.50%.Sample data

The steps of web personalization are: set one goal and metric, check what data you may use, pick a segment, design one variation, test it against a holdout and measure the lift before rollout. In the worked example, a cart banner lifted conversion from 4.20% to 4.92%, a 17.1% gain that plausibly ranges from 3% to 31%. The range, not the point, decides whether to ship.

What you will learn

  • Run six steps in order: goal, data and consent, segment, one variation, holdout test, lift with a range
  • Size the segment first: at a 4.2% baseline you need about 17,000 visitors per group to detect a 15% lift
  • Report lift as a range and price the low end; roll out only if it covers the build and upkeep

What are the steps of web personalization?

Web personalization has six steps: set one goal and metric, check what data you may use, pick a segment, design one variation, test it against a holdout, and measure the lift before you roll it out. The order matters. Teams that start with the variation (step 4) and measure afterwards end up with a page that feels personal and a number nobody trusts. This article follows one example through all six steps with the arithmetic shown. For the wider concept and its advanced forms, see hyper-personalization; for how shoppers experience it, see personalized shopping experience.

StepQuestion it answersWhat you produce
1. GoalWhat should change, measured how?One metric and a target
2. Data and consentWhat may we use?A list of allowed signals
3. SegmentWho gets the change?A rule and a size
4. VariationWhat do they see instead?One change, written down
5. Holdout testDid it cause the change?Control and variant results
6. Lift and rolloutIs it worth keeping?A decision with a range
StepQuestion it answersWhat you produce
1. GoalWhat should change, measured how?One metric and a target
2. Data and consentWhat may we use?A list of allowed signals
3. SegmentWho gets the change?A rule and a size
4. VariationWhat do they see instead?One change, written down
5. Holdout testDid it cause the change?Control and variant results
6. Lift and rolloutIs it worth keeping?A decision with a range

Skipping the holdout in step 5 is the most common failure.

Step 1: What is the one goal, and how will you measure it?

Pick a single metric tied to a business outcome, with a baseline. Example used throughout (sample data): an online shop wants more orders from returning visitors who added to cart but did not buy within seven days. The metric is order conversion rate for that group, and the baseline is 4.2%.

Why bother? McKinsey's Next in Personalization 2021 survey, with 1,013 US adults in September 2021, reported that 71% of consumers expect companies to deliver personalized interactions and 76% get frustrated when this does not happen. That tells you personalization is worth attempting, but not whether your change works. Only your own metric can say.

Step 2: What data may you use?

List the signals you actually have and are allowed to use: page views, referring source, previous visits, cart contents, a logged-in identity. Anonymous visitors do not have a profile, so the signal must be something you can observe in the session or in a first-party identifier.

Check the rules for your audience. Under Article 21 of the EU's GDPR, a person has the right to object at any time to processing for direct marketing, including profiling, and once they object their data may no longer be used for that purpose. Build the opt-out into the design, and apply the stricter rule if you serve several regions. If you are not sure that a signal is allowed, leave it out of the first test.

Step 3: Who is in the segment, and is it big enough?

Write the rule as an if-statement a colleague could check: returning visitor, cart value above $40, no purchase in 7 days. Then size it before you build anything.

Sample data: this rule matches 20,000 visitors in the test window. With a 4.2% baseline, how many are needed to detect a lift? For 80% power at the 5% significance level, a standard two-proportion calculation gives about 17,000 visitors per group to detect a 15% relative lift (4.2% to 4.83%), and about 9,800 per group to detect a 20% relative lift. With 10,000 per group the test can reliably find lifts of about 20% or more. If the segment is too small, widen the rule (drop the cart-value threshold) or run the test for longer.

Step 4: What single change do they see?

Change one thing, so a result can be attributed. Sample data: visitors in the segment see a banner under the header, "You are $12 away from free shipping," based on their cart value, while the control group sees the usual page. Write down the hypothesis before launch: the banner reduces surprise at shipping cost, so conversion in this segment rises. A hypothesis tied to a known reason is easier to learn from than "personalized banner."

Keep the control page identical to what visitors saw before the test, so the comparison measures the banner and nothing else. Two guardrails keep the variation honest: show nothing that reveals sensitive data (a banner that mentions a health purchase is a privacy problem, not a feature), and keep the experience coherent if the segment rule stops matching halfway through a visit.

Step 5: How do you test it against a holdout?

Randomly split the segment, not the whole site. Half sees the personalized banner, half does not. The holdout is what turns a before-and-after story into a measurement, because both groups live through the same weekdays, promotions and weather.

Sample data from the test, 10,000 visitors per group:

GroupVisitorsOrdersConversion
Control10,0004204.20%
Variant10,0004924.92%

Keep the split for the whole planned period. Stopping the moment the result looks good inflates false positives. Decide the end date from the sample size in step 3.

Step 6: How do you calculate lift and decide to roll out?

Compute absolute lift, relative lift and a range.

  • Absolute lift: 4.92% − 4.20% = 0.72 percentage points.
  • Relative lift: 4.92 ÷ 4.20 − 1 = 17.1%.
  • Significance: pooled conversion = (420 + 492) ÷ 20,000 = 4.56%; the two-proportion z-statistic is about 2.44, above the 1.96 needed at the 5% level.
  • Range: the 95% interval on the difference runs from about 0.14 to 1.30 percentage points, which is a relative lift of roughly 3% to 31%.

The honest summary is "17% in this test, plausibly anywhere from 3% to 31%." Convert the low end into money before deciding. Extra orders in the test period: 492 − 420 = 72. At $24 gross margin per order that is 72 × $24 = $1,728; at the low end of the range (a 3.4% relative lift, about 14 orders) it is about 14 × $24 = $336, a fifth of that. If the build and upkeep cost less than the low-end figure, roll out. If they cost more than the point estimate, do not. Then keep a small holdout after the rollout, so you notice if the lift fades.

app.tapcub.com

Conversion in the test and the lift range

sample data, % converting
Control4.20%
Variant (measured)4.92%
Variant, low end of range4.34%
Variant, high end of range5.50%
Sample data
Sample data: 10,000 visitors per group. Measured relative lift +17.1%; the 95% interval is roughly +3% to +31%, so the variant's true rate is plausibly between 4.34% and 5.50%.

How long should the test run?

Work it out from traffic, not from a feeling. Sample data: the segment produces about 5,000 qualifying visitors a week. A test with 10,000 per group needs 20,000 visitors in total, which is 20,000 ÷ 5,000 = 4 weeks. Detecting the smaller 15% lift needs about 17,000 per group, so 34,000 ÷ 5,000 = 6.8 weeks, which you round up to 7.

Run whole weeks so every weekday appears the same number of times, and avoid starting on a promotion day. Watch for a novelty effect: a banner can win in week one because it is new, so compare the first and last weeks. If the lift in the last week is much smaller, trust the later number.

What goes wrong in web personalization?

  • No holdout. A before-and-after comparison cannot separate the banner from the season.
  • A segment too small to test. Fewer than a few thousand visitors per group only detects huge effects.
  • Several changes at once. If you change copy, offer and layout together, you will not know which one worked.
  • Personalizing on sparse data. A rule built on one page view guesses wrongly often enough to annoy people.
  • Stale rules. A rule that worked in winter may not in summer; recheck with a small standing holdout.

How do you do the measurement side in TapCub?

Take the example above. TapCub does not serve personalized content or split traffic; it measures the segment and the outcome. The split and the banner happen in your site or your personalization tool.

  1. Define the goal. In Web analytics › Goals set the purchase as a goal, so the baseline of 4.2% comes from the same data you test against.
  2. Build the segment. In All-platform analytics › Users › Segments & tags create a rule segment from conditions such as did or did not do an event and properties. Anonymous visitors have no profile and are not in segments; identify logged-in users first so the segment can include them.
  3. Record which group each person saw. Send the group name from your site as an event or user property (for example banner = control or variant). The segment rule can also use a random-bucket condition if you want the split to live in TapCub.
  4. Read the outcome. Open All-platform analytics › Behavior › Event analysis, chart the purchase event and use the breakdown by that property to compare the two groups over the full planned period. Event analysis can break counts and users down by event or user properties.
  5. Mark the rollout. Add a note in Configuration › Annotations on the day you roll out, and keep watching the same funnel.

Bots and AI crawlers are counted separately, so crawler traffic does not distort the two conversion rates.

FAQ

What are the steps of web personalization?

Set one goal and metric, check which data you may use, pick a segment, design one variation, test it against a holdout group, and measure the lift with a range before rolling out. Skipping the holdout is the commonest failure.

How do you personalize a website?

Define a segment with a rule such as returning visitors with items in the cart, show that group one specific change, and compare conversion with a randomly held-out group. Roll out only if the low end of the lift range pays for the work.

How big does a personalization test need to be?

At a 4.2% baseline, about 17,000 visitors per group to detect a 15% relative lift and about 9,800 per group for 20%, at 80% power and 5% significance. Smaller segments only detect larger effects.

It depends on the data and the region. Under GDPR Article 21 people can object to profiling for direct marketing at any time. Build in an opt-out and check the law where your visitors live.

How do you measure the success of personalization?

Compare conversion for the personalized group against a randomly held-out group over the same period, and report the lift as a range. In the sample, 17.1% was the point estimate and roughly 3% to 31% the range.

Sources

Was this article helpful?

We use your answers to fix and expand this page.

Sophia Bennett

Product Adoption Consultant

Explores customer onboarding, live conversations, and self-service, turning common usage questions into practical guidance and improvements.

Measure the segment and the outcome

Goals, segments and event breakdowns to compare the groups you test.

See your data clearly. Find your growth.

Every click, backed by data. Install one line of code and see your first numbers in a minute.

No credit card · Free plans for analytics and chat · Cookieless analytics