Split

Promptfree, for everyone, and the only version there is

Copy the prompt for Claude CodeCopy the prompt for CodexCopy the prompt for Cursor

Copy

Build me feature delivery with measurement, replacing Split — and know what makes this ALMOST.

Flags are a weekend. The thing that earns the price is **rollouts that are measured rather than watched**: every release tied to metrics, with guardrails that halt it automatically when something degrades. That needs correct statistics, and correct statistics under continuous monitoring is where homemade versions produce confident wrong answers.

STACK
- Node 20+ with Fastify for the control plane
- SQLite through better-sqlite3 for flags and definitions
- Your existing warehouse or database for the metrics. Do not build an analytics store
- A small SDK per language, evaluating locally
- Caddy in front

THE DATA MODEL
- flags: id, key, kind, variations_json, default_variation, is_active
- environments, and rules per flag per environment: position, condition_json, variation, rollout_json
- rollouts: id, flag_id, environment_id, target_percent, started_at, current_percent, step_percent, step_interval_minutes, status, halted_at, halt_reason
- assignments: id, flag_id, unit_id, variation, at — append-only, and the record every measurement depends on
- metrics: id, name, kind, query_sql, unit_column, value_column, is_guardrail, direction, minimum_effect
- results: id, rollout_id, metric_id, computed_at, variant_stats_json, method, verdict
- Assignments must be recorded from the first minute. Everything else can be recomputed; that cannot

EVALUATION
- Deterministic bucketing: hash the unit key with the flag key and a salt. The same user always gets the same variation everywhere, with no coordination
- The SDK holds the rule set in memory and evaluates locally. No network call in the request path
- It loads from a local cache first, then updates. If the service is unreachable it keeps serving the last known rules indefinitely rather than falling back to defaults — that fallback is how a flag service takes your application down with it
- Updates pushed over server-sent events with a slow poll as a fallback, so a kill switch takes effect in seconds
- Percentage rollouts stable when the percentage changes: a unit in the first ten per cent stays in when it becomes twenty

THE MEASURED ROLLOUT, WHICH IS THE PRODUCT
- A rollout is a schedule: five per cent, then twenty, then fifty, then everybody, with an interval between steps
- At each step, guardrail metrics are computed for the exposed group against the rest. If one degrades beyond its threshold, the rollout halts automatically and alerts. It does not roll back on its own unless you configure it to — but halting is automatic, always
- Guardrails are the ones you always care about: errors, latency, crashes, conversion. Defined once and attached to every rollout by default
- The primary metric is whatever the change is meant to improve, with a minimum effect stated in advance
- A sample-ratio mismatch check at every step: if the exposure split is significantly off, the rollout is broken and the numbers mean nothing. This catches more real bugs than any other check

THE STATISTICS
- Use a published method valid under continuous monitoring — a sequential test, or a Bayesian approach with a stated prior. **Not** a fixed-horizon test looked at after every step, which will call a neutral change significant roughly a third of the time
- Report an interval, never a point estimate, with the method named beside every number
- Compute the sample size needed for the minimum effect before starting, and show progress towards it
- Multiple metrics mean multiple comparisons; correct for it or say plainly that you have not

DISCIPLINE
- Every flag has an owner and an intended lifetime. Temporary flags are the point; four hundred permanent ones is a codebase nobody can reason about
- A stale-flag report — fully rolled out, or unchanged for months — and a code search showing where each is referenced so removal is a small task
- Every flag change audited with before and after values. A flag change is a production change

OPERATIONS
- .env: DATABASE_PATH, WAREHOUSE_URL, BASE_URL, SESSION_SECRET, SDK_KEY_SALT
- Migrations on boot, each once; nightly backup off the machine
- The control plane independent of the applications it serves, or they share an outage
- Health endpoint reporting connected SDKs, rule version, and any rollout with a ratio mismatch

WHAT MATTERS MOST
Local evaluation that survives the service being down, and automatic halting on a guardrail. Turn the control plane off and confirm every application keeps serving; then degrade a metric on purpose and confirm the rollout stops.