AI Experiments API

Preview feature. AI Experiments are in active preview. Conversion event tracking is intentionally minimal: one public ingest endpoint, an aggregation that lives next to existing exposures, and no historical aggregation jobs yet. See the Cachely status page for current limitations.

AI Experiments split traffic between AI transform profiles (or "control" — no transform) on requests that match a route pattern. The proxy assigns a variant deterministically per visitor and attaches headers identifying the experiment, the variant, and the profile applied. Your frontend reads those headers, optionally stores them per session, and can later POST a conversion event so the dashboard can compute conversion rate per variant.

How it works

1. Configure an experiment in the dashboard with a route pattern and 2+ variants. Each variant points at a transform profile or is control (no transform).
2. When a request matches an active experiment, the worker picks a variant by weighted hash of cf-connecting-ip + user-agent (salted by experiment id) and applies the variant's profile.
3. The response carries the experiment + variant headers (below) so your client knows which variant served the user.
4. After a user action (CTA click, signup, purchase, etc.), your frontend POSTs /api/ai-experiments/events with the experiment + variant attribution and an event name.
5. The dashboard's Exposures view aggregates request_logs for exposures and ai_experiment_events for events, and divides to show conversion rate per variant.

Explicit ?transform=<profile> requests bypass experiment matching entirely. Those responses do not carry experiment headers, and your frontend SHOULD NOT submit events for them.

Response headers

The worker sets the following headers on every response that ran through an experiment:

For control variants, the worker additionally sets X-AI-Transform: skipped and X-AI-Transform-Skip-Reason: experiment-control so debug tooling can distinguish "control branch ran" from "no experiment matched".

All headers are listed in Access-Control-Expose-Headers so cross-origin browsers can read them.

Tracking conversion events from your site

The endpoint is unauthenticated and CORS-open — call it directly from the browser after the Cachely fetch completes. The validation chain (real tenant → real experiment → real variant whose stored key matches) is the security boundary.

POST /api/ai-experiments/events

Submit a conversion event for an experiment variant. Public, unauthenticated, wide-open CORS. Hosted at https://app.cachely.io.

Request body (JSON):

{
  "tenantSlug": "prismic",
  "experimentId": 1,
  "variantId": 2,
  "variantKey": "serbian",
  "eventName": "homepage_hero_cta_click",
  "visitorId": "anon_abc123",
  "sessionId": "sess_xyz789",
  "url": "https://example.com/",
  "referrer": "https://google.com/",
  "metadata": { "cta": "hero_primary" }
}

Validation:

Responses:

OPTIONS /api/ai-experiments/events

CORS preflight for the POST above. Returns 204 with Access-Control-Allow-Origin: *, Access-Control-Allow-Methods: POST, OPTIONS, and Access-Control-Allow-Headers echoing the request's Access-Control-Request-Headers. Cached for 24 hours via Access-Control-Max-Age.

Recommended: @cachely-io/sdk integration

The @cachely-io/sdk (≥ 0.10) ships a built-in tracker that captures the experiment headers from the same Cachely fetch that rendered the content — no probe requests, no page-level header plumbing, no manual sessionStorage wiring.

Why no probe? A separate request "just to capture headers" can be assigned to a different variant than the one that actually rendered. The dashboard then attributes the conversion to the wrong variant. The SDK avoids this by inspecting headers on the response objects produced by createCachelyPrismicClient (and friends).

Nuxt (recommended) — composable + module

Since SDK 0.12.0, npx @cachely-io/sdk setup --provider prismic --experiments generates the composable below. Register the Cachely Nuxt module so useCachelyExperiments() resolves at build time:

// nuxt.config.ts
export default defineNuxtConfig({
  modules: ['@cachely-io/sdk/nuxt'],
  cachely: { tenant: 'your-tenant' }, // required for the module to construct the tracker
})

// composables/useCachelyPrismicClient.ts (generated by `setup --provider prismic --experiments`)
import { createCachelyPrismicClient } from '@cachely-io/sdk/prismic'
import { repositoryName } from '~/slicemachine.config.json'

export function useCachelyPrismicClient() {
  // useCachelyExperiments() is auto-imported by `@cachely-io/sdk/nuxt`.
  // Call it INSIDE the composable so each request gets its own per-request tracker
  // (the Nuxt plugin attaches a fresh instance via useNuxtApp()). Calling at module
  // scope would resolve to a single shared instance and break SSR isolation.
  const experiments = useCachelyExperiments()
  return createCachelyPrismicClient({
    repositoryName,
    tenant: 'your-tenant',
    experiments,
  })
}

The Nuxt module also calls tracker.autoTrack() automatically after hydration, so [data-cachely-track] clicks work out of the box — no extra code.

Non-Nuxt — manual setup at module scope

For non-Nuxt projects (Next.js, Astro, Express, etc.), construct the tracker yourself wherever you build your Cachely client. There is no auto-import outside Nuxt.

// e.g. lib/prismic.ts (Next.js)
import {
  createCachelyPrismicClient,
  createCachelyExperimentTracker,
} from '@cachely-io/sdk/prismic'
import { repositoryName } from '~/slicemachine.config.json'

export const cachelyExperiments = createCachelyExperimentTracker({
  tenant: 'your-tenant',
})

export const cachelyPrismicClient = createCachelyPrismicClient({
  repositoryName,
  tenant: 'your-tenant',
  experiments: cachelyExperiments, // ← wires header capture into every Prismic fetch
})

export default cachelyPrismicClient

When experiments is omitted, createCachelyPrismicClient behaves exactly as before — no header inspection, no extra wrapping.

Why not the canonical app/prismic/client.js? The CLI's app/prismic/client.js template (written when you opt into --patch-client) uses createCachelyFetch directly, which does not accept an experiments tracker. That template captures no assignment headers. If you want per-request experiment capture in a Nuxt project, generate the composable (--composables or --composables --patch-client, which is the default for --yes) and call useCachelyPrismicClient() from your pages. The CLI prints a loud warning if you request --patch-client --experiments without --composables.

A. Recommended: explicit tracking

Anywhere in your app, after the page has rendered (so the Prismic fetch has populated the assignment):

import { cachelyExperiments } from '@/app/prismic/client'

cachelyExperiments.track('homepage_hero_cta_click', {
  cta: 'hero_primary',
})

track() is fire-and-forget: it returns a Promise<{ ok, id?, error? }> for diagnostics but never throws. When no assignment was captured (e.g. the visitor opened the page with ?transform= and bypassed the experiment), it silently no-ops and resolves with { ok: false, error: 'no_assignment' }.

B. Recommended: zero-JS CTA tracking with data attributes

Call autoTrack() once on the client and let HTML drive the events:

<script>
  // Browser-only — call inside onMounted / componentDidMount / on the client
  cachelyExperiments.autoTrack()
</script>

<button
  data-cachely-track="homepage_hero_cta_click"
  data-cachely-meta-cta="hero_primary"
>
  Track demo CTA
</button>

autoTrack() attaches one delegated click listener and is idempotent — calling it twice does not double-fire events. Metadata keys are derived from data-cachely-meta-* attributes (the prefix is stripped, value kept as a string).

It is fire-and-forget by design: the listener is passive, never calls preventDefault, and never awaits the network. Link / button navigation is never delayed.

In Nuxt (with @cachely-io/sdk/nuxt registered):

<script setup>
// useCachelyExperiments() is auto-imported by the module, and the module
// already calls tracker.autoTrack() after hydration — no onMounted call needed.
</script>

<template>
  <button
    data-cachely-track="homepage_hero_cta_click"
    data-cachely-meta-cta="fixed_demo_cta"
  >
    Track demo CTA
  </button>
</template>

If you need to call track() explicitly from a Nuxt component:

<script setup>
const experiments = useCachelyExperiments()

function onCustomCta() {
  experiments.track('custom_event', { source: 'sidebar' })
}
</script>

C. Advanced / manual fallback

The SDK reads the four x-cachely-* headers from every response that flows through the client and stores the latest active assignment for the current session in memory and sessionStorage. You don't need to wire anything else. If you must integrate from a context that does not use the SDK client, the tracker exposes:

• tracker.captureFromResponse(response) — call with a Cachely Response object to record the assignment.
• tracker.wrapFetch(fetch) — wrap your own fetch implementation; identical to what the Prismic client does internally.
• tracker.getAssignment(), tracker.getVisitorId(), tracker.getSessionId() — read the stored state.

Do not issue a separate request whose only purpose is to capture experiment headers. The proxy assigns variants per request — a probe call may land on a different variant than the one the user actually saw, which attributes the click to the wrong arm. Always reuse the response that rendered the content.

Multiple experiments on one page

The tracker stores assignments keyed by experimentId, so a page running more than one Cachely experiment can attribute each CTA to its own experiment.

• Single experiment — no extra work required. track('cta_click') and <button data-cachely-track="cta_click"> resolve automatically.
• Multiple experiments — pass experimentId explicitly. The SDK refuses to guess when more than one assignment is active and resolves with { ok: false, error: 'ambiguous_experiment' }. This prevents misattributing a click to the wrong arm.

Explicit form:

cachelyExperiments.track(
  'homepage_hero_cta_click',
  { cta: 'hero_primary' },
  { experimentId: 1 },
)

Declarative form (works with autoTrack()):

<button
  data-cachely-track="homepage_hero_cta_click"
  data-cachely-experiment-id="1"
  data-cachely-meta-cta="hero_primary"
>
  CTA
</button>

Resolution rules for track(name, meta, options):

Inspect the live state with tracker.getAllAssignments() (returns a plain Record<string, CachelyExperimentAssignment>).

Follow-up: a future product improvement will give every experiment a stable string key so developers can write data-cachely-experiment="homepage_serbian_copy_test" instead of a numeric id. Until then, IDs are the contract.

Behaviour notes

• Source of truth. The tracker uses x-cachely-variant-key as variantKey. x-cachely-experiment-profile is informational only — using it would break the moment a key diverges from its profile (e.g. key=serbian-v2, profile=serbian).
• Storage migration. Trackers persist state under sessionStorage['cachely.experiment'] as { assignmentsById, latestExperimentId }. Legacy single-assignment payloads from earlier SDK versions are read transparently and rewritten in the new shape on the next capture.
• Storage failures degrade silently. SSR, private mode, blocked third-party storage — the tracker falls back to in-memory state and never throws. Visitor and session ids are auto-generated when storage is unavailable.
• keepalive: true is preferred for ingest POSTs so events ride along across navigations. The tracker retries without keepalive if the runtime rejects it, so older browsers / Node never block tracking.

Aggregation endpoint

See GET /api/tenants/{slug}/ai-experiments/{id}/exposures in the AI Transform docs for the base contract. The endpoint now accepts an optional ?goal=<eventName> query (defaults to homepage_hero_cta_click) and returns a strict superset of the previous shape — same authentication (Clerk session + tenant access, viewer+), same tenant-isolation guarantees, additional event-related fields per variant.

Response shape:

{
  "experimentId": 1,
  "totalExposures": 202,
  "totalEvents": 22,
  "goalEventName": "homepage_hero_cta_click",
  "variants": [
    {
      "variantId": 1, "variantKey": "control", "label": "Control",
      "transformProfile": null, "trafficWeight": 50,
      "exposures": 100, "actualSplit": 49.5, "expectedSplit": 50.0,
      "lastSeenAt": "2026-05-12T10:01:02Z",
      "eventCount": 8, "conversionRate": 8.0,
      "lastEventAt": "2026-05-12T10:02:00Z"
    },
    {
      "variantId": 2, "variantKey": "serbian", "label": "Serbian",
      "transformProfile": "serbian", "trafficWeight": 50,
      "exposures": 102, "actualSplit": 50.5, "expectedSplit": 50.0,
      "lastSeenAt": "2026-05-12T10:01:05Z",
      "eventCount": 14, "conversionRate": 13.7,
      "lastEventAt": "2026-05-12T10:02:30Z"
    }
  ]
}

conversionRate is 0 (not NaN) for variants with zero exposures. Variants whose stored id/key no longer match the current experiment definition (drift) are dropped from both exposures and events so the displayed splits always sum to ~100%.

What event tracking does not do

• It is not a general analytics platform. The endpoint accepts one event per call and stores no derived metrics.
• It does not deduplicate events. If your click handler fires twice you'll record two events; budget for that in metadata (e.g. include a client-generated eventId).
• It does not time-window aggregations yet — the dashboard view is all-time.
• It does not block ?transform=<profile> traffic from submitting events, but those requests carry no experiment headers, so the snippet skips them automatically.