Troubleshooting

When Split Test Pro doesn’t behave the way you expect, the cause is almost always one of a small number of usual suspects. This guide walks through the diagnostic flow for the most common problems.

Variant Doesn’t Apply

The single most reported issue. Work through this checklist in order.

1. Confirm the script is loading

Open your site in DevTools → Network tab → reload → filter for experiment.js.

• You see a 200 response → script is loading. Skip to step 2.
• You see a 404 or 403 → the API key is wrong. For HTML, recopy the snippet from Settings → Script & API. For Shopify, the script lives in the Theme App Extension — see step 3.
• You don’t see a request at all → the script tag isn’t in the page’s HTML. For HTML, verify the snippet is pasted in <head> of every page (or via your tag manager firing on All Pages). For Shopify, see step 3.

2. Confirm targeting matches the current page

The variant only activates if the current URL matches the experiment’s targeting rules.

• Open the experiment and go to the Setup tab’s Targeting step. Compare its page rules with the exact URL you’re testing.
• On the Variants step, set that URL as the Preview URL and click Edit on site. The page opens in preview with the variant applied, so you can confirm the variant itself works on that page.
• If the variant looks right in preview but not on the live page, the targeting rules don’t match the URL. Adjust targeting or test on a URL the rules cover.

See URL Targeting for the exact behavior of each operator.

3. (Shopify) Confirm App Embeds is enabled

The most common Shopify-specific cause:

1. Online Store → Themes → Customize on your live theme.
2. App embeds icon (puzzle-piece) in the left sidebar.
3. Under App embeds, turn on Experiment (listed under Split Test Pro).

If you recently switched themes, the new theme may have started with App embeds OFF. See Shopify Theme App Extension.

4. Check device targeting

If the experiment has device targeting enabled, you need to be on a matching device class. A mobile-targeted experiment won’t activate on a desktop browser. Check the devices on the Setup tab’s Targeting step.

5. Check the assignment cookie

Open DevTools → Application (Chrome) or Storage (Firefox) → Cookies. Look for splittestpro-experiment.

• Cookie present, contains the right experiment ID → the visitor is bucketed but the variant CSS/JS isn’t applying. Skip to step 6.
• Cookie present, but contains a different variant ID → you’re in the control. Try a fresh incognito window to get re-bucketed.
• Cookie missing → assignment never happened. Check console for JS errors that may have interrupted the script.

6. Check for theme specificity issues

If the cookie is correct but the variant still doesn’t render visibly, theme styles are overriding your variant CSS. See Specificity Escalation in Selector Cookbook.

7. Check console for errors

DevTools → Console. Red errors that occurred before the script ran can interrupt initialization. Look especially for errors from theme scripts that load early in <head>.

Conversions Not Tracked

The variant applies fine, but the Results dashboard shows no conversions.

HTML: snippet missing on the conversion page

The snippet must be on the page where the conversion event fires (typically the thank-you / confirmation page). If the conversion happens on a different domain or a page where the snippet isn’t installed, no conversion is recorded.

Fix: install the snippet site-wide.

HTML: event key mismatch

The event you fire must match the goal’s event key exactly. trackConversion("Signup") won’t match a goal defined as signup. Case-sensitive, exact-string match.

Fix: open Settings → Conversion goals, copy the exact event key, paste it into your trackConversion call.

HTML: trackConversion not actually firing

Open DevTools → Network tab → filter for analytics. Trigger the conversion. You should see a POST to /api/html/experiment/analytics.

• No POST → your code isn’t reaching the trackConversion call. Add a console.log to verify the line executes.
• POST returns 4xx → the payload is malformed or the API key is wrong. Check the response body for the error.
• POST returns 200 but Results still shows zero → see “Results not updating” below.

Shopify: Web Pixel not connected

In your Shopify admin, go to Settings → Customer events. Split Test Pro should show Connected. If it doesn’t, uninstall and reinstall the app.

Shopify: customer privacy / consent

In regions where Shopify’s customer-privacy banner is active, visitors who decline analytics consent don’t get pixeled — and therefore don’t appear in any experiment results. This is expected behavior.

Results Not Updating

You’re sure data is flowing in, but the Results dashboard isn’t reflecting it.

Cache lag

Results are cached server-side and may take a few minutes to reflect new events. Click Refresh on the Results page to force a recompute. For very large datasets, refresh can take 1–2 minutes. If a refresh lands with Revenue is still computing — try Refresh again shortly. in the comparison card, Performance by variation or the Revenue breakdown, the revenue query did not finish in time: the other sections are fresh, the last saved revenue numbers are kept, and the next refresh fills them in.

Stale browser state

Hard-refresh the page (Ctrl+Shift+R / Cmd+Shift+R) to bypass the browser cache. Sometimes the page itself is showing a cached older state.

Wrong workspace

If you have multiple workspaces, the dashboard is scoped to the currently-selected one. Confirm you’re in the workspace where the experiment lives.

Variant CSS Shows Briefly, Then Disappears

The variant flashes in then is replaced by the original styling. This means a theme script is reapplying styles after the variant CSS injects.

Two fixes:

1. Use !important more aggressively in the variant CSS. The theme’s late-applied styles are usually inline (style="...") — only !important beats those.
2. Switch to a JS variant that uses MutationObserver to re-apply the change whenever the theme overwrites it.

See JavaScript Variants for the MutationObserver pattern.

Variant Causes Layout Shift

Variant CSS that changes element sizes can cause cumulative layout shift (CLS), where the page jumps as the variant applies. Mitigations:

• Pre-allocate space in the variant CSS (min-height, padding).
• Avoid changing display modes (block ↔ flex ↔ grid) at runtime — those reflows are jarring.
• For full-page redesigns, use a redirect variant instead of CSS — visitors arrive at the redesigned page directly with no shift.

Variant Works in Incognito but Not Normal Window

Cookies persist your variant assignment across visits. If you’re testing in your normal browser, you keep seeing the same variant. To verify, either:

• Open a fresh incognito window.
• Delete the splittestpro-experiment cookie in DevTools and reload.
• Use the preview cookie to force a specific variant — see Screenshots and Preview.

Single-Page App: Variant Works on Initial Load Only

Split Test Pro evaluates targeting and applies variants on the initial page load. SPA route changes via history.pushState re-fire page-view goals but don’t re-run targeting. See Single-Page Apps for the workaround.

“Site not connected” Banner on HTML

While Split Test Pro can’t find the snippet on your site, the app shows a Site not connected banner: “Add the Split Test Pro snippet to your site to start testing.” The check fetches your site’s home page HTML from our servers and looks for the snippet’s literal data-api-key="…" attribute.

If the banner stays:

1. Confirm the snippet is in the HTML of your public home page on the domain in Settings → Workspace (not just a staging URL).
2. Confirm the API key in the snippet matches the one in Settings → Script & API.
3. Open your site with DevTools → Network tab and confirm experiment.js loads with a 200.

Shopify: Experiment Worked Yesterday, Not Today

Most likely cause: you published a new theme, and the new theme has App Embeds OFF.

Re-enable App Embeds on the published theme — see step 3 above. This is the single most frequent regression report.

AI Analysis Returns Errors or Unhelpful Output

Common scenarios:

• “Analysis temporarily unavailable. Try again in a few minutes.” → the AI service is rate-limited or temporarily down. Wait a few minutes and retry.
• “Results have been updated since this analysis was generated. Click Regenerate for fresh insights.” → the analysis you’re reading was written before the latest results. Click Regenerate under it.
• The analysis is generic or wrong → the experiment’s data is too thin for the AI to draw meaningful conclusions. Wait until more data accumulates, then click Regenerate.

Opening a saved analysis is free. A fresh one (including Regenerate) uses one AI generation from your workspace’s monthly allowance, and once the allowance is used up the request is refused until it resets.

Errors in Browser Console About SplitTestPro

If you see window.SplitTestPro is undefined in the console, the script isn’t loaded yet. Common causes:

• Script tag is at the bottom of the page (it should be in <head>).
• Tag manager fires the script on a later trigger than All Pages.
• A third-party blocker (ad blocker, privacy extension) is blocking the request.

Check the Network tab to confirm the script is actually loading.

When to Contact Support

If you’ve worked through this guide and the issue persists, contact support with:

• Your site domain (HTML: Settings → Workspace; Shopify: your store’s .myshopify.com domain) and the experiment page URL.
• The exact symptom (with a URL we can reproduce on if possible).
• Browser version and DevTools console output if you have them.

Most issues are resolvable with the site domain and experiment page URL alone. To get in touch, open Help & Docs in the app and choose Email support, or write to support@splittestpro.com.

Next Steps

• Run through the FAQ for common questions: Frequently Asked Questions.
• Re-verify your install end-to-end: Installing on an HTML Site or Installing on Shopify.
• Confirm the variant editor has what it needs: Variant Types Overview.