Autopilot & the Trust Ladder

Autopilot lets Split Test Pro run the program you approved. It’s off by default, opt-in per action, and every action is logged and emailed to you. You’ll find it at Testing Overview → Queue options → Autopilot, which opens the Autopilot drawer — the only place autopilot is configured. A status dot beside the menu item shows when autopilot is on, and turns to a warning when it has suspended itself or paused an action. Workspace owners can change autopilot; members see the drawer read-only, with the note “Only workspace owners can change autopilot.”

The Four Actions

Each action is a separate toggle under the Autopilot master switch:

• Auto-stop — “Conclude tests at ≥95% confidence on two consecutive daily checks, or on futility — never earlier than 7 days and a minimum sample.”
• Prebuild — “Build the next queued tests ahead of time so they’re ready to start. 1 generation each.”
• Auto-start — “Start the next ready test when its surface frees up — only if built, safety-scanned, QA-passed, and within your caps.”
• Auto-rollout — “Serve a clean winner at 100% automatically after it concludes.” (See Winner Rollout.)

The Auto-Stop Rule

Auto-stop checks each running test from your plan at most once a day. It concludes a test with a winner only when all of these hold:

• The test has run for at least 7 days.
• Every variant, the control included, has at least 300 sessions and 15 conversions.
• No guardrail is breached.
• The same variant is at 95% or higher probability to be best on two consecutive daily checks.

It stops a test on futility when no variant reaches 95% in time: after twice the test’s estimated duration or 28 days, whichever is longer (later still if you set a later end date), and always by 60 days.

The Trust Ladder

You don’t get all four on day one. Each action is earned per workspace, matched to its blast radius — the more a mistake would cost, the more track record it takes to unlock. Progress counts over a trailing 90 days.

Action	Unlocks when
Auto-stop	1 experiment has completed in this workspace.
Prebuild	2 plan items built with the manual Build button pass Launch QA without manual edits.
Auto-start	3 approved plan updates, an edit rate of at most 20% over the last 5 decided proposals, and 2 plan-created experiments manually started and completed without SRM or anomaly flags.
Auto-rollout	1 planned winner is manually rolled out.

A locked action shows a lock and what’s left to earn:

• Auto-stop: “Complete 1 experiment to unlock.”
• Prebuild: “Build 2 plan items with the Build button first — 1 of 2.” (with your count)
• Auto-start: “Unlocks with 3 approved plan updates with light edits (1 of 3) and 2 plan-built tests run start-to-finish (0 of 2).” (with your counts)
• Auto-rollout: “Roll out 1 winner manually to unlock.”

Trust can also be lost. If an autopilot-driven test triggers an anomaly or a rollout goes wrong, the action is demoted — paused with a note explaining the cause and the criteria to re-earn it, plus a See what happened link to the log.

Limits and Kill Switches

Autopilot is bounded in several independent ways:

• Master switch — one toggle turns all automatic actions off. Off means off: “All automatic actions are off.”
• Daily action cap — “At most this many automatic actions per day.” The default is 3 (adjustable from 1 to 20).
• Monthly autopilot generations budget — “Autopilot stops (and emails you) instead of spending past this.” The default is 10 generations a month. Autopilot never buys add-ons or overages.
• Self-suspension — “Autopilot suspended itself after 3 consecutive failures. Review the log, then re-enable.” Nothing automatic runs until an owner presses Re-enable.
• Anomaly hold — “Auto-start is paused — an anomaly was flagged on an autopilot-started test. Acknowledge it to resume.” Press Review anomaly to acknowledge it.
• Held for review — a test is never auto-started when the safety scan flagged its generated code, or when it was created by an automatically applied plan update. It waits in the drawer’s Held for review list — “is held for review — the safety scan flagged its generated code.” or “is held for review — it was created by an automatically applied plan update.” — until you press Review & approve.
• Fleet pause — if something is wrong on our side, the drawer shows “Autopilot is temporarily paused by Split Test Pro operations.”

Auto-start also starts a test only when it passes Launch QA.

Plan changes that add or drop tests always require your approval — autopilot can auto-apply only low-risk reorder and follow-up updates, and even that respects everything above.

The Action Log and Emails

Every autopilot action lands in the drawer’s Action log with a timestamp, and an email digest goes to the workspace owner after each action tick (“Autopilot acted on your testing program”), listing every action plus anything held for your review.

Autopilot also emails you on its own when it:

• pauses an action — “Autopilot paused Auto-start” (or whichever action was demoted);
• suspends itself — “Autopilot suspended itself”;
• hits its budget — “Autopilot stopped before spending more”.

Every autopilot email includes a one-click Pause autopilot link, so you can stop autopilot from your inbox without logging in — nothing else changes.

Manual Actions Always Win

Autopilot reconciles against reality instead of fighting it. If you manually start, stop, or edit an experiment, autopilot treats your action as the truth and re-plans around it — it never reverts or blocks something you did by hand.

Next Steps

• The plan autopilot executes: Testing Overview
• What auto-rollout actually does: Winner Rollout
• The guardrails that constrain every decision: Global Goals