Experiments
Five experiments are integrated into main. They answer different questions and should be read independently.
| Experiment | GPU | What the runner does | Result |
|---|---|---|---|
| Laya / ModernBERT | T4 | Runs choice, score, and noul decision outputs with synthetic cases. | T4 JSON |
| Kev-0.5B | T4 | Uses the official decision model and adapter, then compares packed and separate reads. | T4 JSON |
| Jevlike | T4 | Trains a tiny scorer briefly, evaluates held-out synthetic data, and runs a shuffled-context control. | T4 JSON |
| SemIf / Qwen3.5-4B | L4 | Reads next-token logits for declared single-token options and applies option-local softmax. | latest L4 JSON |
| OpenJev NLI 4B | L4 | Scores fixture premise/hypothesis pairs with contradiction, entailment, and neutral outputs. | L4 JSON |
Evidence status
The Laya and OpenJev JSON use status=ok; Kev, Jevlike, and SemIf use status=success. In every case, inspect the adjacent runtime or hardware object for the actual GPU. A requested T4/L4 value by itself is not evidence of allocation.
The catalog also keeps failure records:
- Kev keeps the initial dependency mismatch beside its successful rerun.
- SemIf keeps the pre-fix optional-package import failure.
- Jevlike keeps the earlier authentication blocker.
Failures explain the path to a successful run. They are not extra successes.
JevDash capture evidence
The five model paths have a completed JevDash Level 1 capture set outside Git. The experiment pages below are the public conduits for the runners and checked-in evidence; MP4 files are intentionally external and no public video URL is claimed here.
| Model path | Public conduit | What to verify |
|---|---|---|
| Laya | experiment README | Fixed game revision, real T4 adapter path, and simulation-time labeling. |
| Kev | experiment README | T4 trajectory, state-preserving presentation replay, and sanitized manifests. |
| Jevlike | capture guide | Input truncation makes the game result invalid for ability evaluation; replay is HUD-only. |
| SemIf | experiment README | Real L4 option-logit control and fixed game marker. |
| OpenJev NLI | experiment README and evidence | L4 NLI control, manifests, judgment log, representative frames, and decode evidence. |
These are independent single episodes, not a ranking. Video time is simulation time with synchronous inference waits omitted. Kev and Jevlike presentation replays preserve the recorded trajectory and state and change only the HUD rendering.
Jevlike has a confirmed input-path defect: its 192-byte ByteCollator prefix is identical across all 27 decisions, as are the probability vector and right_jump action, before a 3.6-second death. The scratch scorer was trained only on synthetic badge-selection data. The other four paths are not confirmed to have the same defect, but their game connections are not a completed comparative ability evaluation.
Comparison boundaries
Do not rank these rows by raw latency or probability. The models, prompt formats, option counts, batch shapes, precision, and fixture semantics differ. SemIf scores are conditional option scores, OpenJev outputs are NLI probabilities, and Jevlike accuracy is a short synthetic experiment with a shuffled-context control.
Jevlike timing fields retain their measurement boundary in the result schema. Do not compare any first-inference field across experiments unless the runner definitions and setup boundaries are equivalent.