Shopify · CRO · Testing

A/B Test Scoring SheetFree Resource

Score every test idea across 8 variables: location, exposure, research, ease, device, action type, intent, and workstream. Highest score runs first.

ToolCRO Framework
AudienceStore owners · AgenciesType: Free Guide
01Test Scoring Framework

Test Location

Where on the site does this test run?

LocationScore
Sitewide30
Multiple locations30
Product Detail Page20
Cart20
Any Checkout step20
Bundle20
Homepage10
Product Listing Page10
Edge Case0

Tests that touch more of the funnel score higher. Sitewide and multi-location tests reach every visitor, maximum exposure, maximum potential impact.

02Test Scoring Framework

Traffic Exposure

What percentage of your visitors will actually see this test?

ExposureScore
100%30
75–99%20
50–75%10
Under 50%0

A test only 20% of visitors see can't move the needle on overall RPV. Prioritize tests with broad reach.

03Test Scoring Framework

Research Indication

What's backing this test idea?

Research BasisScore
All 4, AI + Follow-up + Qual + Quant50
AI + Follow-up Test40
Follow-up Test + Qualitative30
Follow-up Test + Quantitative30
Follow-up Test20
AI20
Qualitative + Quantitative20
Qualitative only10
Quantitative only10

The more evidence behind a test, the higher it scores. A follow-up to a test you already ran, where you know something works and you're dialing it in, is the highest-confidence bet outside of having all four signals aligned.

04Test Scoring Framework

Ease of Implementation

How hard is this to build?

DifficultyScore
Very Easy40
Easy30
Average20
Hard0

Ease gets the highest individual weight in the formula. A high-impact, low-effort change always beats a high-impact, high-effort change when everything else is equal. Build the easy wins first.

05Test Scoring Framework

Test Device

Which devices does this test run on?

DeviceScore
All devices30
Mobile only20
Desktop only10

Mobile-first, all-device tests reach more visitors. Desktop-only tests have a smaller addressable audience in most niches.

06Test Scoring Framework

Action Type

How is this change being deployed?

TypeScore
A/B Test20
Direct Change10

A/B tests generate learning, not just change. A direct change tells you what you did. An A/B test tells you what worked and by how much, that data compounds over time.

07Test Scoring Framework

Change Intent

What is this test trying to do?

IntentScore
Fix40
Optimization30
Redesign25
Addition20
Merchandising15
Offer15
Content10
UX System10

Fixes score highest because they address something broken. A broken thing losing you money every day is a higher priority than adding something new that might make you more. Fix first, optimize second, build third.

08Test Scoring Framework

Workstream

Where does this test sit in the overall project?

WorkstreamScore
Baseline Fix40
Experiment30
Structural Addition20
System Definition10

Baseline fixes, the foundational things that should have been right from the start, always run first. No point running experiments on a broken foundation.

How to Use It

  • Score every test idea across all 8 variables.
  • Add the totals.
  • Sort highest to lowest. Run in order.

The formula doesn't tell you what to test. Research does that. The formula tells you which of the things worth testing to do first.

The Scoring Sheet Is Ready to Use

Open it, select options from the dropdowns, and the Total Score column updates automatically. Sort that column highest to lowest, that's your test queue.

  • Dropdown selectors for all 8 variables on 50 rows
  • Auto-calculating total score per row
  • Score Reference sheet with all variables and point values
  • Alternating row shading, score column highlighted in orange

Open in Google Sheets ↗