---
name: aibtest
description: Connect and verify a website with AIBTest, configure its credential-safe local MCP proxy, and create, operate, or analyze A/B experiments. Use when an agent needs to onboard a domain, publish a DNS TXT or HTTPS file verification challenge, connect an MCP host without exposing credentials, embed the browser SDK, or manage experiments and reports.
---

# AB Test Agent

Use the `aibtest` CLI for onboarding and authentication. Never read, print, copy, commit, or modify its credential store.

## Install the CLI

First run `aibtest status`. If the command is unavailable on Linux amd64, install the CLI from the same trusted AIBTest origin that served this skill:

```bash
install_dir="${XDG_BIN_HOME:-$HOME/.local/bin}"
mkdir -p "$install_dir"
tmp_dir="$(mktemp -d)"
trap 'rm -rf "$tmp_dir"' EXIT
curl -fsSLo "$tmp_dir/aibtest" https://aibtest.ai-neos.com/downloads/aibtest-linux-amd64
curl -fsSLo "$tmp_dir/aibtest.sha256" https://aibtest.ai-neos.com/downloads/aibtest-linux-amd64.sha256
(cd "$tmp_dir" && sha256sum --check aibtest.sha256)
install -m 0700 "$tmp_dir/aibtest" "$install_dir/aibtest"
export PATH="$install_dir:$PATH"
aibtest status
```

Do not download the credential CLI from a different origin. Other operating systems and architectures are not distributed by the first version.

## Connect a domain

1. Confirm `aibtest` is installed by running `aibtest status`.
2. Run `aibtest connect --server https://aibtest.ai-neos.com <domain>`.
3. Read only the returned `verification_methods` array.
4. Complete one method:
   - When authorized to deploy the website, publish the exact `http_file.content` at `http_file.url` over HTTPS.
   - Otherwise, create the exact `dns_txt` record using an authorized DNS integration.
5. Run `aibtest verify <registration_id>` after the file is publicly readable or DNS has propagated.
6. Retain the returned `project_key`, `public_site_key`, and versioned `sdk_url` only for the website's browser SDK configuration. They are public identifiers, not management credentials. Load the exact returned `sdk_url`; do not remove its version query parameter.

The HTTPS method requires a valid certificate for the exact registered hostname and does not follow redirects. For a `.localhost` development domain, the local Compose server permits verification without a challenge. Never expect this bypass for a real domain.

## Connect the MCP host

1. Start `aibtest serve`. Use `--registration <id>` when more than one domain is configured. The CLI refuses non-loopback listen addresses.
2. Read the complete `codex_config` object from its JSON output.
3. Configure the MCP host with both its local Streamable HTTP URL and `X-AIBTest-Local-Token` header. Treat that local token as sensitive and never place it in a prompt, log, shell history, or issue report.
4. Keep the proxy running while using AB Test tools.

The proxy owns client assertions and access tokens. The local proxy token is only a loopback access boundary; it is not a remote AIBTest credential. Do not request remote tokens manually or put private keys, assertions, or remote access tokens in MCP configuration.

## Report service problems

Call `report_issue` when an AIBTest operation still fails after one safe retry, returns an unexpected server response, produces inconsistent analytics, or appears to violate this skill's documented behavior.

- Choose the closest category: `authentication`, `onboarding`, `experiment`, `sdk`, `analytics`, `rate_limit`, or `other`.
- Set severity to `critical` only for credential exposure, cross-domain access, sustained service outage, or data corruption.
- Include concise reproduction steps, expected behavior, actual behavior, endpoint/tool name, HTTP status, and non-sensitive environment versions.
- Never include a private key, client assertion, access token, authorization header, credential-store content, visitor identifier, or customer page content.
- Do not report ordinary validation errors that the Agent can correct from the returned message.

If MCP is unavailable, only host automation that already holds the protected local header may send the same JSON fields to the `issue_url` returned by `aibtest serve`. Never ask the model to reveal or reconstruct the local token. Honor `Retry-After` on `429` responses and do not retry early.

## Run an experiment

1. State a falsifiable hypothesis and one primary conversion metric.
2. Call `list_experiments` to avoid conflicting tests on the same element or audience.
3. Call `create_experiment` with at least two variants whose weights total 100. Use only `textContent`, `className`, or `style` DOM changes.
4. Choose traffic from 1 to 100 percent. Default to 10 percent or less for an unproven production experiment; use higher traffic when the user explicitly requests it or the experiment is a controlled instrumentation test.
5. Call `set_experiment_status` with `running` only after confirming selectors exist on the verified domain.
6. Use `set_experiment_traffic` to ramp a draft, running, or paused experiment without recreating it. Do not change completed experiments.
7. Embed the SDK identifiers returned by `verify`, install the anti-flicker pattern below, and call `track` for the primary conversion.
8. Call `get_experiment_report` to evaluate exposures, conversions, and conversion rate.
9. Pause the experiment when instrumentation is broken. Complete it when the planned stopping rule is reached.

Do not claim statistical significance from conversion rate alone. The first server version reports descriptive metrics and does not yet compute confidence intervals or sample-ratio mismatch.

## Measure site traffic

Enable first-party traffic measurement only when the site owner requests it and has reviewed the site's privacy disclosure and applicable consent requirements. Pass `analytics: true` to the `ABTest` constructor; it is off by default.

Call `get_site_analytics` with optional inclusive `start_date` and `end_date` values in `YYYY-MM-DD`. The default is the latest 7 UTC days and the maximum range is 90 days. Report visitors, sessions, page views, average visible duration, engagement rate, bounce rate, and the source/country/device breakdown. Treat `ZZ` country and `unknown` device as unavailable data, not literal segments.

An engaged session has at least 10 seconds of visible time or a click, scroll, or key interaction. Bounce means the session never became engaged. Prefer engagement rate for single-page sites, and explain that `pagehide` delivery and browser privacy controls make duration descriptive rather than exact.

The SDK sends at most start, one engaged event, and end per page view, all counted against the domain hourly quota. The server stores only pseudonymous identifiers, page path, referrer hostname, UTM values, two-letter country code, coarse device type, visible duration, and engagement. It does not retain IP addresses, full referrer URLs, raw User-Agent strings, keystroke content, or form values.

## Prevent variant flicker

For experiments that change visible content, put this pattern in `<head>` before the experiment target can render. Replace `.hero-cta` with the exact selector or selectors changed by running experiments. Hide only those targets; never hide the entire page.

```html
<style>.aibtest-loading .hero-cta { visibility: hidden }</style>
<script>
  document.documentElement.classList.add('aibtest-loading');
  window.__aibtestAntiFlickerTimeout = window.setTimeout(function () {
    document.documentElement.classList.remove('aibtest-loading');
  }, 2000);
</script>
```

Construct `ABTest` with the default `aibtest-loading` class, or pass the page's existing class explicitly with `antiFlickerClass`. Await `start()` before separately revealing experiment targets. The SDK applies DOM changes synchronously, then removes the class in every completion path. The inline timeout is still required because it reveals the original page if the SDK script itself fails to load.

Use `visibility: hidden` so layout remains stable. Keep the 2-second upper bound unless the site owner explicitly accepts a different latency tradeoff. Do not use `display: none`, and do not remove the loading class before `start()` settles.

## Recover safely

- On `401`, leave token renewal to the proxy; do not inspect its token cache.
- On `429`, honor `Retry-After`; use the separately limited local `issue_url` only when the rate limit itself appears incorrect or blocks critical work.
- On a lost credential store, register a new client instead of asking the model to reconstruct a private key.
- On suspected compromise, stop the proxy and contact the service operator to revoke the client. Credential revocation CLI support is not part of the first version.