SwitchNow

How SwitchNow decides

SwitchNow compares tools for one job at a time, with deterministic rules over cited, reviewed evidence. No model is asked to invent a score, and when the evidence is thin the answer is an explicit abstention.

Coding-agent engine v4.0, decision rules switch-v1

What SwitchNow decides

One question: for this job, should you keep your current tool, switch, or wait for better evidence? A brief names the task, your current tool, up to two priorities, and any hard constraints.

Coding agents, assistants, and API models are separate lanes. Evidence never crosses between them, and a model’s results reach a product only through an exact, dated relationship.

CHOOSE
You have no incumbent and one option clears the publication gate for your brief.
KEEP
No qualified challenger clears the switching threshold against your current tool.
SWITCH
A qualified challenger clears the threshold overall and on a priority you selected, without lower confidence.
TIED
The leaders are within the uncertainty band; SwitchNow will not invent a winner.
INSUFFICIENT EVIDENCE
Coverage, independence, or confidence is too thin. This includes a current tool with no published scorecard.
NO MATCH
Every candidate fails a hard constraint, or lacks a verified fact needed to prove it qualifies.
VERIFYING
The candidate field is verified, but the independent comparison has not cleared review yet.

Evidence it accepts

Every displayed claim keeps its exact URL and the exact passage or JSON pointer behind it. Duplicate and syndicated sources count once per independence group.

Independent benchmarks
Reproducible results under a verified reuse licence, approved by source policy. They are the only evidence that can rank performance, and only within the same benchmark, version, and metric.
Exact configuration verification
Official evidence that names the exact product release, model, and mode being compared. Model-only results are never inherited by an agent.
Official facts
Documentation establishes features, limits, prices, platforms, and lifecycle state, never that one tool performs better than another.
Qualitative claims
Structured strengths and weaknesses can lower confidence when credible sources contradict each other. They never create a score.

The publication gate for coding agents

A coding-agent decision is published only when every condition holds. Anything less is shown as verification in progress or as insufficient evidence.

  1. Field verified

    • The tool is part of the field SwitchNow tracks for the job.

  2. Setup reviewed

    • An exact, reviewed product, model, and mode configuration.
    • Official verification that names that exact configuration, from evidence that is still fresh.

  3. Benchmarked

    • At least two independent quantitative benchmark families, and direct coding-agent evidence.
    • All three required dimensions (visual fidelity, functional correctness, and autonomy) with at least 80% weighted coverage.
    • Fresh, source-policy-approved observations with uncertainty bounds.
    • No unresolved credible contradiction, and medium or high confidence.

  4. Call published

    • At least two qualified configurations from different products.

Approving evidence and approving a recommendation are separate, audited human actions. Unknown required facts fail closed: a candidate that cannot prove it meets a constraint is excluded, with the reason shown.

When it says switch

Switching has a cost, so a challenger must earn it: it needs to lead your current tool by at least five comparison points overall and ten on a priority you selected, with no lower confidence and no new constraint failures. Within two points the result is a tie.

Priorities boost at most two existing rubric dimensions; they never add new scores. If your current tool has no published scorecard, the answer is “insufficient evidence”, not “keep”. The comparison points stay internal. SwitchNow shows the verdict and its reasons, not a rating.

Freshness and history

Each source policy sets how long its evidence stays fresh. Freshness is a gate, not a hidden decay: stale evidence stays visible but cannot help a candidate win.

Published decisions are append-only. A rollback creates a new activation instead of rewriting history, and setting an evidence date in the brief shows the decision as it stood then.

What SwitchNow will never do

  • Publish a universal leaderboard or anonymous star ratings.
  • Let a language model or embedding create a measurement, a score, or published evidence.
  • Show demo or seed rankings in production. A failed database is an explicit error, not a fallback.
  • Treat a product’s own marketing as proof that it performs better.
  • Charge you, or create paid model, evaluation, or email usage on your behalf.