The Experimentation Roadmap Template (Free)
A ready-to-copy experimentation roadmap template: the columns that matter, PIE and ICE scoring, and the sample size check almost every backlog forgets.
📚 This article is part of the guide How to Build an Experimentation Culture (2026).
An experimentation roadmap is the prioritized, dated queue of tests the team will run, with hypothesis, primary metric, required sample and estimated duration on every line. It is not a backlog of ideas: a backlog accumulates, a roadmap commits. This article gives you the template ready to copy, explains how to score each item with PIE or ICE, and covers the column almost no prioritization framework includes and that decides whether the item is truly executable: statistical feasibility. It has a live prioritizer and calculator, plus a worked example where the highest-potential item in the backlog drops from fourth to last place once duration enters the math. This is one of the articles in the guide on how to build an experimentation culture.
What an experimentation roadmap is (and is not)
The most expensive confusion in this area is treating backlog and roadmap as synonyms. They are two artifacts with different jobs:
- Backlog of ideas: everything that showed up (a stakeholder request, a finding from a session recording, a hunch from the sales team). No order, no commitment, and growing is normal.
- Experimentation roadmap: the slice of the backlog that has already been scored, sized and ordered. Every item has a written hypothesis, a defined primary metric, a computed sample and an execution window. Here, growing without limit is a symptom.
The practical difference shows up in the planning meeting. With a backlog, the conversation is “what do we think will work”. With a roadmap, it is “the next item in the queue starts Monday and runs 3 weeks”. The first conversation repeats every month, the second one ends in five minutes.
The experimentation roadmap template: the columns that matter
Copy the table below into your spreadsheet or knowledge base. Every column exists to answer a question that usually shows up mid-execution, when it is already too late:
| Column | What to fill in | Why it exists |
|---|---|---|
| ID | Short unique code (EXP-014) | Reference the test in a dashboard, commit and report without ambiguity |
| Page or flow | Where the change happens | Group items competing for the same traffic (two tests on one page interfere) |
| Hypothesis | Because I observed X, I believe Y will cause Z, measured by W | Prevents the test with no prediction, which teaches nothing on a tie |
| Primary metric | Exactly one, chosen before running | Blocks metric swapping after seeing the result |
| Guardrails | Metrics that must not get worse | Protects revenue, churn and complaints while the primary improves |
| Current baseline rate | Today conversion on that metric | Feeds the sample size calculation directly |
| Target MDE | Minimum effect worth detecting | Defines the size of the test; without it there is no timeline |
| Sample per variation | Output of the calculator | Turns feasibility into a number instead of an opinion |
| Estimated duration | Total sample divided by daily traffic | Turns the queue into a calendar |
| Score (PIE or ICE) | Average of the three scores | Orders the queue with an explicit criterion |
| Status | Idea, prioritized, running, done, archived | Shows work in progress and caps WIP |
| Result and decision | Winner, tie or loser, and what was done | Becomes the team learning repository |
Two columns usually spark debate and both are worth defending. Guardrails exists because almost any experiment can improve one isolated metric at the expense of another: a more aggressive popup increases email capture and increases complaints. Result and decision exists because a roadmap without memory becomes a conveyor belt that repeats already-finished tests as soon as team turnover erases what was learned from them.
A starter roadmap, filled in
An empty template is easy to nod at and hard to start. Here are five lines of a plausible roadmap for an ecommerce team, filled with the level of detail the columns expect. Note that two of the five carry a decision that is not “run it as proposed”:
| ID | Page or flow | Primary metric | Baseline | Target MDE | Sample per variation | Duration | Score | Status |
|---|---|---|---|---|---|---|---|---|
| EXP-021 | Product page | Add to cart rate | 3.2% | +20% rel | 13,015 | 11 days | 7.7 | Prioritized |
| EXP-022 | Cart | Checkout start rate | 3.2% | +12% rel | 34,885 | 28 days | 5.7 | Prioritized |
| EXP-023 | Signup flow | Purchase rate | 1.1% | +15% rel | 67,375 | 53 days | 7.0 | Reformulate, metric too low in the funnel |
| EXP-024 | Checkout button | Checkout completion | 3.2% | +20% rel | 13,015 | 11 days | 7.7 | Running |
| EXP-025 | Home | Product page visit rate | 3.2% | +12% rel | 34,885 | 28 days | 5.0 | Raw backlog, score too low for the slot |
The two rows that are not simply “prioritized” are the point of the whole exercise. EXP-023 has a strong score and an impossible timeline; EXP-025 fits the calendar comfortably and does not deserve the slot. A roadmap that only ever produces “prioritized” is not filtering anything.
How to score: PIE and ICE
The two most used frameworks score each item from 1 to 10 on three dimensions and, in the original formulation of each, take the simple average of the three:
| Framework | Dimensions | Origin | Where it fits best |
|---|---|---|---|
| ICE | Impact, Confidence, Ease | Credited to Sean Ellis, in a growth context | Teams prioritizing varied initiatives, not only page CRO |
| PIE | Potential, Importance, Ease | Created by Chris Goward, at WiderFunnel, for CRO projects | CRO teams that need to weigh how much traffic passes through each page |
A warning before choosing: two different arithmetics circulate for the same frameworks, one that averages the three scores and one that multiplies them. The average is the formulation described in the original sources of each, it is the one this article uses, and it is the one the prioritizer below computes. If you adopt the multiplying version, the scale changes (1 to 1,000 instead of 1 to 10) and the final order can change with it, so pick one arithmetic and keep it across the whole roadmap.
The real difference is in the second letter. In ICE, Confidence asks “what is the chance this hypothesis is right?”, meaning how much evidence supports it (Growth Method, ICE Framework). In PIE, Importance asks how much the traffic of that page is worth. Chris Goward himself defines the most important pages as those with the highest volume and the most expensive traffic to buy (Goward, Practical Ecommerce). A brilliant redesign of a page receiving 200 visits per month has high Potential and low Importance, and that is exactly the kind of item PIE pushes down the queue and ICE does not.
Neither is precise, and it is important to say that out loud: they are subjective scores whose job is to make disagreement explicit, not to produce a true number. Kohavi and Thomke, in the Harvard Business Review, argue precisely that not even experts in the field reliably predict which changes will win, and that this is why it is worth testing ideas that opinion-based prioritization would discard. The real gain of scoring, then, is not the final order: it is the conversation it forces when two people give 3 and 9 to the same dimension of the same item.
Score your backlog now, choosing the framework in the selector, and export the ranking:
| # | Experiment | Potential | Importance | Ease | Score | Remove experiment |
|---|---|---|---|---|---|---|
| - | - | |||||
| - | - | |||||
| - | - |
Nothing here leaves your browser: the list is never sent anywhere and the CSV is built locally. The score is the average of the three ratings, and the ruler is for comparing ideas against each other, not for guessing which one will win the test.
The column missing from almost every framework: statistical feasibility
PIE and ICE answer “is this worth it?”. Neither answers “is this executable?”. And that second question fails a lot of well-scored items.
A test only ends when it accumulates enough sample to detect the effect you are chasing. That sample depends on the baseline rate, the minimum detectable effect (MDE) and the statistical rigor you choose. With the sample in hand and the page traffic, duration becomes a simple division. The guide on how to run an A/B test step by step covers the calculation in detail; what matters here is its effect on the queue.
Look at three plausible items from the same backlog, all with 18,000 visitors per week in the tested flow:
| Item | Baseline rate | Target MDE | Sample per variation | Duration |
|---|---|---|---|---|
| New product page layout | 3.2% | +12% relative | 34,885 | 28 days |
| Same layout, bigger ambition | 3.2% | +20% relative | 13,015 | 11 days |
| New signup flow (conversion to purchase) | 1.1% | +15% relative | 67,375 | 53 days |
Sample per variation by the normal approximation for two proportions (95% confidence, 80% power, two-sided); duration for 2 variations and 18,000 visitors per week.
The third item is the illustration of the problem: nearly two months of execution occupying the entire flow. It may well be the highest-potential item in the backlog, and approving it still means spending the whole quarter on a single test. The way out is not to ignore the statistics, it is to reformulate the item: chase a bigger effect (which changes the nature of the proposed change), move the primary metric one level up the funnel (measure signup start instead of purchase), or apply the change on a flow with more traffic.
Compare the second and third rows carefully, because they contain the entire lesson. The second item is not “better” than the third in any business sense: it is the same page with a more ambitious target, and a more ambitious target is cheaper to prove, not more valuable. Raising the MDE lowers the sample, but it also means the test will report nothing when the true effect is real and small. That is the trade the duration column forces you to make on purpose rather than by accident.
Compute the sample and duration for every line of your roadmap:
Two-proportion normal approximation, 2 variations (50/50). Tweak the inputs and watch it update live.
Worked example: the queue changes when duration enters
An ecommerce team scored six items with PIE. Ordered by score alone, the queue would look like this (average of Potential, Importance and Ease):
| Item | P | I | E | PIE score | Estimated duration |
|---|---|---|---|---|---|
| New signup flow | 9 | 8 | 4 | 7.0 | 53 days |
| Social proof on the product page | 7 | 8 | 8 | 7.7 | 11 days |
| Free shipping highlighted in the cart | 8 | 7 | 7 | 7.3 | 14 days |
| Category page redesign | 8 | 6 | 3 | 5.7 | 28 days |
| New checkout button copy | 4 | 9 | 10 | 7.7 | 9 days |
| Video on the home page | 6 | 4 | 5 | 5.0 | 35 days |
By raw score, the order would be: social proof and button copy tied at 7.7, then free shipping (7.3), then the new signup flow (7.0). Now bring in the duration column and the real execution queue changes shape:
- Quarter by score, ignoring duration: social proof (11 days), button copy (9 days), free shipping (14 days), signup flow (53 days). Total: 87 days, and the quarter ends with 4 completed tests, the last of which ate more than half the calendar.
- Quarter by score divided by duration: button copy (9 days), social proof (11), free shipping (14), category redesign (28). Total: 62 days, four completed tests, and roughly 4 weeks of calendar still free for a fifth test or a confirmation rerun.
The highest-potential item (signup flow) was not discarded, it was postponed and reformulated: with the primary metric changed from purchase to signup start, the baseline rate rises, the sample drops and it returns to the queue next quarter with a viable duration. That is the decision the duration column makes possible and that the score alone hides.
What to do when an item does not fit
“Reformulate it” is easy to write and vague to execute. There are exactly four levers, and each one costs something specific. Choosing consciously among them is most of the craft of running a roadmap under a traffic constraint:
| Lever | What it does to the sample | What it costs |
|---|---|---|
| Raise the target MDE | Cuts the sample fast (going from +12% to +20% on a 3.2% base drops it from 34,885 to 13,015) | The test becomes blind to real but smaller effects |
| Move the metric up the funnel | Cuts the sample by orders of magnitude, because the baseline rate is higher | You are measuring a proxy, and the proxy has to be documented as such |
| Merge similar pages into one test | Adds their traffic, cutting duration proportionally | The result is an average across pages that may behave differently |
| Accept a longer window | Nothing, the sample is unchanged | The flow is occupied, and every other item in the queue waits |
The fourth lever is legitimate and underused. Two well-sized tests per quarter beat six tests cut in half, which mostly produce false positives. What is not legitimate is the fifth option nobody lists in a table and everybody uses: keeping the original timeline and stopping the test as soon as the dashboard shows something. That is the peeking problem, and it converts a feasibility problem into a credibility problem.
Cadence: how many items and how often to review
A useful roadmap is the size of the real capacity of the team, and that capacity is limited by traffic before it is limited by people. If your flow supports two tests per month, a queue of 40 prioritized items is not planning, it is fiction: by the time item 20 arrives, the context that justified its score will have changed. The article on how many A/B tests you should run per month shows how to compute that capacity from traffic, and it is that capacity that should define the size of the queue.
A cadence that works well in practice:
- Monthly review: reorder the queue with what completed tests have taught, retire items that lost context, and pull new items from the raw backlog.
- Review after every completed test: a result changes the estimated confidence of neighboring items. If the social proof hypothesis lost on the product page, the equivalent item on the category page deserves a lower score before entering the queue.
- Work-in-progress limit: one test at a time per screen or flow. Large programs run dozens of simultaneous experiments without a problem, because they touch different parts of the product; the caveat applies to two tests touching the same elements. In that case you choose between running both mutually exclusive, where each gets half the traffic and takes twice as long, or letting the same visitor fall into both, where the two changes can combine in a way neither isolated reading predicts (the classic example is each test pushing the buy button slightly further down, so the visitor who got both never sees the button). Kohavi, Tang and Xu cover the design of programs with many concurrent experiments in Trustworthy Online Controlled Experiments.
Common mistakes in an experimentation roadmap
| Mistake | Warning sign | Fix |
|---|---|---|
| Queue much larger than capacity | 40 prioritized items, 2 tests per month | Size the queue by traffic capacity, leave the rest in the raw backlog |
| Item with no written hypothesis | “Test the red button” | Write evidence, change, expected effect and metric before scoring |
| Ignoring duration when prioritizing | An 8 week item entered as a “quick win” | Add the duration column and order by score divided by duration |
| Score inflated by whoever proposed it | Every product team item has Potential 9 | Score in pairs or in committee, and record the disagreement |
| No guardrail defined | Conversion went up, nobody checked refunds or churn | Define 1 or 2 guardrails per item before running |
| Roadmap with no result column | Nobody remembers why the item was archived | Record result and decision on every completed line |
| Two tests on the same flow | Two simultaneous changes in checkout | One test per flow, always |
Make this automatic with Donnu
An experimentation roadmap only holds up when every line has real numbers behind it: a computed sample, an estimated duration and an honest verdict at the end. That is exactly the part Donnu automates: you define the hypothesis and the primary metric, Donnu sizes the test against your real traffic, tells you how long it will take before you approve the line, and returns the result with its confidence interval instead of a green badge. The roadmap stops being a spreadsheet of intentions and becomes a queue with a reliable timeline.
Start a 14 day free trial and size the next item in your queue before approving it. For the rest of the process, see how to build an experimentation culture and how to write an A/B test hypothesis.
References
- Growth Method. PIE Framework: prioritise marketing tests by Potential, Importance and Ease. Describes the framework created by Chris Goward at WiderFunnel. growthmethod.com/pie-framework.
- Growth Method. ICE Framework: the original prioritisation framework for marketers. Describes the framework created by Sean Ellis. growthmethod.com/ice-framework.
- Goward, C. Use the PIE Method to Prioritize Ecommerce Tests. Practical Ecommerce. practicalecommerce.com.
- Kohavi, R. & Thomke, S. The Surprising Power of Online Experiments. Harvard Business Review, 2017. hbr.org/2017/09/the-surprising-power-of-online-experiments.
- Kohavi, R. ExP Platform: accelerating innovation through trustworthy experimentation. Reference material on experimentation programs at scale. exp-platform.com.
Read also:
Frequently asked questions
- What is an experimentation roadmap?
- It is the prioritized queue of experiments the team will run, with the hypothesis, primary metric, estimated sample size, duration and status of each item. Unlike a backlog of ideas, a roadmap has already answered the question "does this fit our traffic, and in how long", so every line in it is a test that can start executing without a new discussion.
- What is the difference between PIE and ICE for prioritizing experiments?
- Both score each idea from 1 to 10 on three dimensions and, in their original formulation, take the average of the three scores (some implementations circulating online multiply instead of averaging, which changes the scale and can change the final order). ICE (Impact, Confidence, Ease) is credited to Sean Ellis, in a growth context, and works for any kind of growth initiative. PIE (Potential, Importance, Ease), created by Chris Goward at WiderFunnel for conversion optimization projects, swaps Confidence for Importance, which measures how much traffic and revenue flow through the page in question. In practice, PIE prevents the roadmap from investing in promising improvements on a page almost nobody visits.
- What does almost every experimentation roadmap forget?
- Statistical feasibility. An item can score high in any framework and still be impossible to run, because the page traffic will not close the required sample within an acceptable timeframe. Before approving an item, compute the sample per variation and the estimated duration; if it comes out beyond 6 to 8 weeks, the item needs to be reformulated (bigger effect, a metric higher in the funnel, or a page with more traffic), not merely scheduled.
- How many items should an experimentation roadmap have?
- Enough to cover two or three months of real execution capacity, no more. A roadmap with 60 ideas and a capacity of 2 tests per month is a wish list that ages: priorities change before the queue moves. Size the queue by the capacity your traffic supports and keep the rest as a raw backlog, with no dates.
- How often should the roadmap be reviewed?
- A monthly review is usually enough to reorder the queue with what completed tests have taught, plus a quick review whenever a test finishes. Results change the estimated confidence and potential of neighboring items, so reordering after each conclusion prevents the team from executing a queue that is already obsolete.
- Should the roadmap include tests that are expected to lose?
- Yes, when the loss itself is informative. An item worth running is one where both outcomes change a decision: if winning ships a change and losing closes a recurring internal argument, the test earns its slot. The items to cut are the ones where a loss changes nothing, because someone will propose the same idea again next quarter regardless of the result.