Test and verify

There are four layers, from fastest to most real. Run the first three before every change; run the fourth when you touch anything a browser would notice.

1. The four gates

CI runs these, and AGENTS.md asks for them after every change:

cargo fmt --all --check
cargo clippy --workspace --all-targets          # zero warnings; CI adds -- -D warnings
cargo test --workspace
cargo run -p xtask -- purity                    # add --build for the CC=/bin/false proof

purity reads cargo metadata and fails on a denylisted crate (ring, aws-lc-rs, openssl, libsqlite3-sys, zstd-sys, libz-sys), a links = key, or a cc/cmake/bindgen/pkg-config build dependency. The only packages allowed to break those rules are listed in C_BINDING_EXCEPTIONS in xtask/src/purity.rs, each with an ADR. Adding npm tooling for the docs or the Bench does not touch the Rust dependency tree: docs-site/ and webapp/ are outside the Cargo workspace.

2. Progress bars

If you change parity/status.json, regenerate and check the README bars:

cargo run -p xtask -- progress
cargo run -p xtask -- progress --check

The progress section of the README sits between <!-- progress:start --> and <!-- progress:end --> and is generated. Never edit it by hand, and never edit PARITY.md (cargo run -p xtask -- parity-matrix makes it).

3. The differential harness

Replays recorded C-server exchanges against the Rust binary and diffs them. No Docker needed for replay.

cargo run -p xtask -- parity-run                       # all scenarios, builds janus first
cargo run -p xtask -- parity-run --scenario echotest   # one scenario
cargo run -p xtask -- parity-run --bin target/debug/janus --scenario admin
cargo run -p xtask -- parity-media --golden            # the media plane against the recorded document (no Docker)
cargo run -p xtask -- parity-sip                       # needs Docker and python3
cargo run -p xtask -- parity-lua                       # needs Docker and python3
cargo run -p xtask -- capture --scenario echotest      # record a new C trace (needs Docker and the oracle image)

Read the harness page for what is compared and Add a scenario to extend it.

4. Janus Bench, with real browsers

The replay compares JSON. It cannot say that Chrome connects and hears audio. The Bench does, with a real RTCPeerConnection, either by hand in a browser or headless:

cd webapp
npm install
npm run e2e                  # builds janus, serves the app, drives Chrome, prints pass or fail
npm run e2e -- --only echotest,videoroom
JANUS_BIN=/path/to/janus npm run e2e -- --no-build

The runner starts the binary with a generated config, serves the production build of the app, opens it in Chrome through Playwright, runs every check, and writes webapp/e2e-results/report.json and report.md. It exits non-zero when a check fails. See Janus Bench and the recorded results.

Write a check of your own

A Bench check is a function in webapp/src/checks/. It gets a context with sessions, test media and a step log:

{
  id: "echotest.media",
  group: "EchoTest",
  plugin: "janus.plugin.echotest",      // skipped when the server does not list it
  title: "WebRTC echo with audio and video",
  proves: "Offer/answer, ICE, DTLS-SRTP and RTP work end to end.",
  async run(ctx) {
    const conn = await ctx.session();
    const h = await ctx.step("attach", () => conn.attach("janus.plugin.echotest"));
    // ...
  },
}

Each ctx.step records pass or fail with a time; a step that throws fails the check and the page or the report shows exactly which one.