Test Plans: The Testing Overview

The Testing Overview is the home of Test Plans: instead of inventing one experiment at a time, you pick a goal and Split Test Pro maps your funnel, ranks test ideas by projected impact, and lays them out on a timeline — then keeps the plan updated as results come in.

You’ll find it as Testing Overview in the app’s sidebar, between Dashboard and Experiments.

Setting Up Your Program

Before you have a plan, the queue area shows a Plan your testing program card with a Set up your testing program button and a note of the cost: “Uses 5 AI generations”, with what you have left this month. The button opens a three-step wizard — Confirm, North star, Generate — right on the page:

If a generation fails, nothing is used from your allowance — the generations are refunded automatically, and a Retry button appears.

The Page at a Glance

The Testing Overview has these parts, top to bottom:

• Stat banners — Running, Upcoming, Completed and Revenue. The first three count your experiments and show the change over the last 7 days; click one to filter the roadmap to that status, and click it again to show everything. Revenue shows the estimated monthly value of the winners you’ve shipped (the program impact from Global Goals) with its change on last month — “No revenue source connected” until a revenue source is set up, and “Ship a winner to start tracking impact.” until there’s something to count.
• Experiment Roadmap — every dated experiment on one day-by-day timeline, grouped into Running (including paused tests), Upcoming (scheduled tests and dated drafts), Completed and Archived. A Today rule marks the current moment and the Today button scrolls back to it; the Status menu filters the groups. Click a bar to open a card with the test’s key facts. Scheduled and draft bars can be dragged along the timeline to new dates, which you then confirm in the date picker.
• Testing Queue — the tests waiting to be built or started, in four columns: Test, Area, Status and Target Date. Click a column heading to sort by it; click again to reverse. Each row’s ⋯ menu offers Build Test (or Open Test once the test exists) and Remove from Queue. The queue also has a Create Test button for adding your own tests.
• Program cards below the queue — a proposed plan update when one is waiting (see plan updates), the plan’s revision history, and suggestions for improving your plan’s data.

The queue’s own ⋯ button, Queue options, holds the view actions — Clear Sorting and Export CSV — and the program actions: Regenerate plan, Autopilot, Surfaces and Global goals. Regenerating shows the same data-readiness checklist and costs 5 AI generations.

Projected, Not Scheduled

A queue item’s Target Date is the date you set for it or, if you haven’t set one, its projected start — “Not set” when there’s no estimate yet. Projections come from a queue simulation: how long each test is expected to need on its surface’s real traffic, and what has to finish before it can start. A projected date never launches anything — planned tests start when you (or Autopilot) start them, and the projections re-flow around what actually happens. A test you schedule yourself starts at its scheduled time; see Scheduling.

Two chips flag feasibility on proposed plan changes:

• Low power — this surface’s traffic can only detect fairly large lifts in a reasonable time; smaller effects won’t reach significance.
• Traffic unknown — no traffic data for this surface yet, so duration is unknown until data arrives. Opting in to site-wide tracking fills these in for surfaces you haven’t tested.

The Queue

Each queue row carries a status tag:

• Ready — built and ready to start.
• Scheduled — it has a date set.
• Draft — planned, not built yet. A test that’s being built shows a Building… note under its name.
• Backlog — an idea the plan hasn’t lined up yet.

You stay in control of the backlog: remove anything you disagree with using Remove from Queue.

Build turns a plan item into a real experiment — variants, targeting, and goals generated from your site snapshot. Click the test’s name or choose Build Test from its ⋯ menu; the Build this test? dialog shows the cost, and Build test starts it. A build uses 1 AI generation. If it fails, the row says “Build failed — retry” and “No generations were used.”

A built test is a Draft experiment: nothing starts on its own. Open it from the queue, review it, and start it with Launch Test like any other experiment.

Plan Updates: Suggest and Approve

The plan is not static. When an experiment concludes, an anomaly is flagged, metrics refresh, or you change your north star, Split Test Pro proposes a plan update: a Proposed plan update card listing each change (Add, Drop, Move, Update, Follow-up) with the reason for it — for example, a follow-up test added because a variant won. Added tests show their surface and projected impact — High, Medium, or Quick win, never a made-up number.

Nothing changes until you approve. Every change has a checkbox: untick the ones you disagree with and the button becomes Approve N of M changes. Or press Reject to keep the plan as it is — the AI proposes again only on new evidence. Approving uses 2 AI generations; proposals are free to receive. Every decision lands in the revision history, so the plan’s whole evolution is auditable.

You also get the update by email, with a Review in Split Test Pro button. When an update only reorders or updates existing tests, the email also has a one-click Approve update button. It opens a confirmation page in Split Test Pro where you approve the update — the email never approves anything by itself. Approve links last 7 days; updates that add new tests need in-app approval.

When you’ve built a track record, you can hand parts of this loop to Autopilot — it’s off by default and earned per workspace.

Surfaces and the Site Snapshot

Open Queue options → Surfaces to edit your surface map and see each surface’s crawl status. As the dialog notes, one test runs per surface at a time; different surfaces test in parallel. It may also suggest surfaces for high-traffic pages you haven’t mapped, each with Add or Ignore. Refresh site snapshot re-crawls your pages so generated ideas are grounded in what’s actually on your site — up to 20 pages, at most one request per second, honoring robots.txt, and once per hour. See the crawler for how crawling works.

You can also manage surfaces in Settings → Surfaces (workspace owners): name each area, list its URL patterns, and use + Add surface for more — up to 20 surfaces. It’s the same surface map; experiments use it for their Area in the experiments list.

If Your Plan Changes

If your workspace moves to a plan without Test Plans, nothing is deleted. The Testing Overview shows: “Test Plans is paused on your current plan. Your roadmap and history are saved — upgrade to pick up where you left off.” The queue becomes read-only, and Re-upgrade takes you to Billing.

AI Generations

Every AI feature draws from your workspace’s one monthly AI generation allowance: Splitty drafts (1), full plan generation (5), plan updates you approve (2), building a planned test (1). The cost and your remaining balance are shown before anything is spent, and failed generations are refunded. When you don’t have enough left, Get more generations takes you to Billing.

Next Steps

• Set your north star and guardrails: Global Goals
• Let the program run itself, one earned step at a time: Autopilot
• Ship what wins: Winner Rollout
• Measure surfaces you haven’t tested: Site-Wide Tracking
• The practice behind the product: Building a Testing Program