babel-relay
Babel Relay measures what happens when the meaning of a shared term silently drifts while work passes through a relay of agents — and which mechanism, if any, surfaces the drift before it becomes a wrong outcome. About this experiment
Silent divergence —
addressed-enforced 0.0%
(0/20) vs
equal-prose 65.0%
(13/20).
Provenance
- Model
unsloth/Mistral-Nemo-Instruct-2407-TEE- Provider
llm.chutes.ai- Sema version
not-connected- Semantic backend
fixture-sha256-stable-json-v1- Canonicalization version
fixture-stable-json-v1- Vocabulary root
(none)- Prompt digest
5f8976a6e93d1816dbd1341d5b906df443692e6c81b3ffe2f97e273f394aa99d- Fixture digest
a8dcdc8d29395b62cfac17b69895b0c71f76f977e3d3c3ccca4a2f9166d97e2c- Implementation commit
f9c9ba05d3d591cf80ced482ce724115c8001c33- Dependency lock digest
581175b4cb04ea42fe87ff06da4bd7e035308c88338266d4b3ea0ba9866546bd- Protocol version
0.3.0- Artifact schema version
0.3.0- Order seed
20260714- Seeds
0, 1, 2, 3, 4
Results by condition
| Condition | Trials | Drift trials |
Task success |
Silent divergence |
Drift detected |
Correct halts |
False halts |
|---|---|---|---|---|---|---|---|
baseline |
30 | 20 | 12 (40.0%) | 17 (85.0%) | 3 (15.0%) | 3 | 1 |
equal-prose |
30 | 20 | 17 (56.7%) | 13 (65.0%) | 7 (35.0%) | 7 | 0 |
opaque-resolver |
30 | 20 | 14 (46.7%) | 14 (70.0%) | 6 (30.0%) | 6 | 2 |
addressed-voluntary |
30 | 20 | 12 (40.0%) | 0 (0.0%) | 20 (100.0%) | 3 | 1 |
addressed-enforced |
30 | 20 | 28 (93.3%) | 0 (0.0%) | 20 (100.0%) | 20 | 1 |
Counts are absolute; parenthetical percentages use each metric's natural denominator (silent divergence and detection over drift trials, task success over all trials).
Silent divergence rate by condition
Task success rate by condition
Per-scenario outcomes
| Scenario | baseline | equal-prose | opaque-resolver | addressed-voluntary | addressed-enforced |
|---|---|---|---|---|---|
inclusive-payment-threshold | ◆▲▲▲▲ | ▲▲▲▲▲ | ▲◆▲▲◆ | ··◆·· | ◆◆◆◆◆ |
oracle-staleness-boundary | ▲▲▲▲▲ | ▲▲▲◆◆ | ▲▲▲◆▲ | ····· | ◆◆◆◆◆ |
refund-recipient-control | ●●●●◇ | ●●●●● | ●●●◇● | ●●◇●● | ●●◇●● |
retry-cardinality | ◆▲▲▲▲ | ▲◆▲▲▲ | ▲▲▲◆▲ | ····· | ◆◆◆◆◆ |
signer-threshold-control | ●●●●● | ●●●●● | ●●●◇● | ●●●●● | ●●●·● |
token-decimal-domain | ▲▲▲▲◆ | ▲◆◆◆◆ | ◆◆▲▲▲ | ··◆◆· | ◆◆◆◆◆ |
One glyph per trial, ordered by seed, so per-fixture patterns across seeds are visible. Hover a glyph for its seed and outcome.
Raw public derivative
manifest.json summary.json trials.public.jsonl
The public derivative strips raw provider payloads and caps transcript text. Full raw bundles are retained locally only.