Splitty, the AI Test Builder

Splitty is the AI test builder in Split Test Pro. It opens on your live site, asks what you want to improve, suggests test ideas, and builds the whole test — variants, targeting and traffic split — without you writing code. You reach it with the Create with AI button on the Dashboard or the Create Using AI item in the Experiments menu.

This page is the tour. If you only want to launch something today, Your First Experiment has the short version.

Where Splitty Runs

Splitty does not live in the dashboard. Starting a test opens a new tab on your own site in preview mode — the real page, rendered by your real theme — with the Splitty chat docked beside it. What you build is applied to that page as you go, and nothing changes for your visitors until you launch.

The preview bar at the bottom of the page has a menu. It holds the Preview variant switcher, so you can flip between the control and each variant on the page, and Exit Preview, which leaves preview mode and takes the chat with it.

The chat panel starts docked on the right. Drag its header to move it anywhere, drop it on the left or right edge to dock it there, or let it float; a floating panel also resizes from its edges and corner. Minimize chat folds the panel into the preview bar, and the bar brings it back.

Your preview session renews itself while you work, so a long conversation does not time out. If it cannot renew — for example, your computer slept past the session’s 30-minute life — the panel locks and shows Your preview session expired — reopen it from the dashboard. with a Reopen link. Nothing is lost: the conversation and the draft are saved, and the Reopen link lands you back where you were.

Starting a Test

On the Dashboard, the banner Build your next test with AI carries the Create with AI button.
On Experiments, the Create Test button starts the same flow. Its menu also lists Create Using AI and Create Manually.

Either entry opens your homepage in preview with Splitty ready. Navigate to any page you want to test — preview mode follows you across your site — and build from there. You are only asked for a Page URL when Split Test Pro cannot work out your site’s domain yet.

The draft appears in your Experiments list at once, named AI Draft Test — followed by today’s date. Close the tab and it is still there; delete it if you abandon the run.

If you don’t see Create with AI, ask your workspace owner — the AI builder is switched on per workspace, and on HTML sites only the workspace owner sees the Dashboard banner.

Answer the Questions or Just Ask

Splitty greets you with Use the chat or the questions to get started. Both routes end in the same place.

The questions. The question card walks through ten short questions. Each has a quick “no, decide for me” answer, so you can skip what you don’t care about:

What is the test name?
What goals are you trying to improve?
What is the test hypothesis?
Where should this test run?
When should the test activate?
Which devices should see this test?
Who should see this test?
What should be tested?
When should this test run?
Generate test ideas, or save as a draft?

Just ask. Type one message instead — “make the add-to-cart button stand out on mobile” is enough. Splitty reads it in one shot: every detail you mention is captured, and every blank takes a default — this page, page load, all visitors, start now. There is no follow-up interview. The chips under the chat list what was captured and what was assumed; tap a chip to reopen that question on the card and change the answer.

Finish with Generate test ideas to see what Splitty proposes, or Save as draft to have Splitty write the test straight from your answers.

Ideas and Build Test

Splitty suggests up to five test ideas for the page, revealed one at a time. Each card shows the hypothesis, the proposed change, the primary metric, the audience and the effort. Thumbs up or down record feedback; a thumbs-down removes the idea (with Undo).

Click Build Test on the idea you like — the caption reads Uses 1 generation. A checklist tracks the build in the chat:

Create the experiment draft
Configure both variants
Set targeting and traffic
Validate configuration

Nothing is changed on your live website while it builds. When the build lands, the page reloads on the new variant and the review card appears.

After a build, the remaining idea cards switch to Add as a variant, so a second idea becomes another variant of the same test instead of a new one. Asking Splitty to create another variant for this test brings up to five fresh variant ideas with the same button.

Review Your Test

The Review your test card is the built test’s summary and its editor. It lists:

Experiment name and Hypothesis
Preview URL — the page the preview shows
Default goal, the Tracked goals chips and What is tested
Funnel
Total traffic — the share of visitors entering the test
Activation (On page load or Custom event, with its Event name)
URL targeting, Devices and Audience targeting
The variants, their traffic split and their code
Schedule — When it runs is Manually or Schedule, with Start date and End date when scheduled

Each field saves on its own: edit it, click its Save, and Saved flashes beside the label. A failed save shows the server’s message inline and keeps your value for another try.

When you are happy, click Launch Experiment. A sheet asks Launch this experiment? — confirm with Launch Experiment or go back with Keep reviewing. The success card follows (“A small change. A new possibility.”), with Pause Test / Resume Test and Back to dashboard, which opens the test in the admin. A scheduled launch waits for its start date instead.

Editing in Chat

After a build, the chat becomes an editor. Each message you send is one targeted edit — “make the headline shorter”, “only show this to returning visitors”, “move the badge above the price” — applied at once to the draft. Splitty replies with what was saved (“Saved — …”), the review card highlights those fields, and the preview reloads where the change needs it. For a change to a variant’s code, Splitty then checks the page before it says more: once the variant’s script has run and the elements it changed have settled, the reply gains a second line — “…and it’s showing on the page”, or “…but I can’t see it on the page yet” with the reason and an Undo that restores the variant’s previous code for free. The intent check’s verdict appears as a chip on the same reply. If the edit lands on a variant other than the one you are previewing, the preview switches to that variant first; if the code editor holds unsaved changes, Splitty asks you to save or discard them before it edits.

Edits are scoped to the variant you are previewing, so switch variants in the preview bar before asking for a change on a different one. Every chat edit costs one generation.

A big change — a multi-step wizard, a navigation rewrite — can take a few minutes to write. While Splitty works, the send button reads Stop generation. Stop only stops the chat waiting: nothing is undone and no generation is refunded, and if the reply finishes anyway it is in the thread the next time you reload. Closing the tab works the same way — reopen the preview and the reply is there.

When Splitty Can’t Finish a Change

Sometimes a build or a chat edit does not come back. Splitty says what happened in the chat, in one line, and a failed attempt never uses a generation — the line ends with This didn’t use a generation. and the footer count stays where it was.

That change was too big to write in one go — try one piece at a time. The reply ran past the size Splitty can write in a single turn. Ask for a smaller part of the change, then the next.
Splitty’s AI service is busy — wait a moment and try again. The AI service was overloaded. Splitty already retried once for you.
That took too long, so I stopped — try a smaller change. The reply did not finish within ten minutes.
I couldn’t reach the AI service — try again in a minute. The AI service answered with an error.
I couldn’t produce a usable change — try rephrasing. The reply came back but could not be applied.
I can’t help with that request. Splitty declined the request.
Lost the connection — check it and try again. Your browser could not reach Split Test Pro.
Something went wrong on our side — try again. Anything else.

Attachments

The paperclip in the composer attaches files to your next message — a screenshot of the page you have in mind, a brief, your notes. Up to three files per message, 5 MB each, as PNG, JPG, WebP, TXT or Markdown. You can also drop files anywhere on the chat panel or paste an image into the composer. A file that has since expired on the server shows as Attachment expired on the old message.

History

Each build starts a new chapter. The chat shows a divider — This test has its own chat, so the conversation you just had is saved in your chat history. — with View history, and folds the earlier messages away. Every change is logged in the same thread: Saved · and a time when you edit a field on the review card, Splitty updated · and a time when a build or a chat edit changes the test.

View history opens the Chat history sheet, which lists this test’s chapters and versions. Open a chapter to jump back to it. The history is per test; closing the tab and reopening the test later brings it back.

Quick Start Tour

The first time Splitty opens on your site it plays a short tour: Hi, I’m Splitty!, then how to move the panel, where the questions live, how to attach files and where the history is. Click Show me around to take it or Skip to go straight to the chat; Got it closes it at the end. To watch it again, click Splitty’s mascot and answer Show me to Want a quick tour of the chat again?

Generations

When the allowance runs out, Splitty tells you so and points you to the add-on; nothing you have built is lost.

Improve with AI in the Wizard

Splitty also helps inside the manual wizard. Each step carries a Splitty banner with an Improve with AI button:

General — Make this hypothesis stronger. With a name typed and nothing else, one click drafts the whole step; with a hypothesis written, it turns it into a specific, measurable prediction. The hypothesis field receives only the prediction itself, with goals named as they appear in your catalog (for example “Purchases”); anything Splitty wants you to know, such as what to make more specific or that traffic is low and the test may need to run longer, appears as a note under the banner and is never saved into the hypothesis. The banner appears once the test has a name.
Targeting — Reach the right visitors: targeting recommendations based on your experiment.
Variants — Build a high-impact variant: turns your hypothesis into a ready-made change for the selected variant (add a variant first).

Improve with AI and Regenerate cost one generation each; Undo restores your own values for free. Out of generations, the banner reads You’re out of AI generations and nothing is changed.

Next Steps

Build and launch a test end to end: Your First Experiment.
What to check before you launch: Launch Checks.
Read the results once data arrives: Reading Your Results Dashboard.