@@STATUS@@ · 2026
@@TITLE@@
@@AFFILIATIONS@@
@@SUMMARY@@
What TRACE does
Follow a spoken request from the recording to the action it produces. Define what a successful task requires, compare inputs, then follow the conversation through mistakes, recovery, and user effort.
- @@STEPS@@
One timer instruction, three listening conditions
This timer example shows how TRACE connects a change in speech to the agent’s next action. The three clips contain the same instruction, with added noise or a silenced segment in two copies. The question is whether enough information remains to set the correct timer, or whether the agent needs to ask for the duration.
“Start the timer for nine minutes.”
Audio preparation, exact files, and pending input review
- Recording
- Timers and Such v1.0,
dev-real/anonymous0_24.wav. Mono, 16 kHz, 28,989 samples (1.8118125 seconds) in every comparison file. - Shared attenuation
- All three files use gain 0.908076 (about −0.84 dB), followed by the same PCM16 export. This preserves relative levels without clipping; it does not equalize the clips’ loudness. “Original speech” is a comparison copy of the untouched source recording. These exact PCM16 files are the model inputs specified for this gallery.
- Marked number
- Samples [15461, 18682), or 0.9663125–1.167625 seconds. Start inclusive; end exclusive. These are historical automatic forced-alignment estimates, not human phonetic annotations. The silenced condition sets this interval to zero and retains the duration.
- Machinery event
- The complete released 0.75-second Holman nc4 event spans samples [14934, 26934), or 0.933375–1.683375 seconds. It overlaps the duration phrase, including part of “minutes.” The creator’s released recording includes mastering and beat-length editing.
- Local SNR
- −10 dB, calibrated from speech and added-noise energy only within the marked number interval, before their shared attenuation. After PCM16 export: −9.9999966 dB. This is neither whole-recording SNR nor physical sound pressure. No SNR applies to the silenced condition.
- Input review
- Human listening and a residual-cue audit remain pending. No model response determined the segment boundaries or machinery selection. The visible transcript is for visitors; it is not model input.
- Frozen requests
- Qwen3-Omni-30B-A3B-Instruct; model and processor revision
@@REVISION@@. Each request uses a fresh conversation and seed 1234, Thinker audio input with text output, BF16, SDPA, and batch size one. Sampling: temperature 0.6, top-p 0.95, top-k 20, maximum 256 new tokens. The model receives only the audio waveform and generic timer prompt/tool schema, with the neutralinput.wavplaceholder. See the complete generation settings and frozen request manifest. No model is loaded by this page.
Exact SHA-256 values for the PCM16 files:
- @@HASHES@@
Original preparation manifest (before inference) · Saved gallery evidence states
Optional: one archived model response from 9 October 2026
An actual response recorded on 9 October 2026, kept separate from the three current responses. It belongs to its own gain-scaled recording and is not used as evidence for any of the three new inputs.
Exact archived input
Original speech with gain 0.9708313164; no added noise. This player uses the original float WAV, with no conversion.
Complete generated response
@@RAW_RESPONSE@@Proposed action: set_timer(hours=0, minutes=9, seconds=0). Generated syntax alone is a proposal.
Recorded simulation: the archived trace separately reports a 540-second timer in memory. No real timer was started.
Archived model, settings, and evidence provenance
- Model
- Qwen/Qwen3-Omni-30B-A3B-Instruct
Revision@@REVISION@@ - Timestamp / seed
- 9 October 2026, 16:41:05 UTC · Seed 655677136 · 32 generated tokens.
- Sampling
- Sampling enabled; temperature 0.6, top-p 0.95, top-k 20, maximum 256 new tokens. The complete saved settings and exact rendered prompt are available.
- Input SHA-256
722b0356acb98ce36c91e2d73b4e45873d7a1864ad1904406a0df9634f4637e6
Exact raw response · Public evidence extract with recorded simulation. This extract retains response, prompt, tokens, settings, and simulation, while excluding old host paths and runtime receipts. Its source-trace hash identifies the unchanged archival record.
How TRACE scores a conversation
A first response does not show whether the task eventually succeeds. The scripted examples below illustrate how TRACE follows the conversation and records mistakes and extra user effort.
Scripted scoring examples — not measured model performance
| Scripted scenario | Goal attained | Violation | Recovery completed / initiated | Task success | Extra user turns |
|---|
A correct final timer is only part of success. Creating a wrong-duration timer, or creating a timer before required information is obtained, remains a violation even if it is later cancelled. Task success requires the goal and no violations.
Recovery takes more than asking. A request for missing information or user correction begins an attempt. Obtaining the information and using it to resume the task completes recovery. Every user reply, correction, or refusal after the initial instruction adds one user turn.
The fictional missing-duration cases explicitly stipulate that the number is unavailable until the user supplies it. That assumption does not establish what can be heard in the new silenced WAV. A proposal alone never changes simulated state. These five rows are teaching examples; they do not support an agent success rate or a combined TRACE score.
Inspect the scripted conversations and computed state
@@SCRIPTED_LOGS@@Event fixtures and hand-calculated expectations · Computed scores, violations, and final states
Exceeding a time or turn limit prevents goal attainment and task success. Budget failures are reported separately from prohibited actions. Invalid event sequences are rejected explicitly. Only inert timers in local memory are used.
Run the example on CPU
Download and extract the example package. Use Python 3.10 or newer with NumPy and SciPy; installation instructions are included in the README. The CPU tools need no model credentials or GPU.
Two commands, from the extracted folder
python -B scripts/prepare_audio.py --output outputs/reproduced_audio
python -B -m companion.scoring --output outputs/scripted_scores.jsonPreparation reproduces the shipped PCM16 inputs. Scoring computes all five rows from the event logs.
Download code and examplesAdapt it to your task
- Replace the recording and mark the segment to change. Review what information remains after editing.
- For a different task, replace the timer-specific rules and state updates in
companion/scoring.py. Changing the example JSON alone does not define a new scorer. - Represent the conversation as event logs, keeping proposed actions separate from actions that were executed. The supplied scorer accepts scripted examples; using real logs requires an adapter and validation.
Saved responses and optional GPU reproduction
@@RESPONSE_IMPORT_NOTE@@
The download includes a separate optional GPU entry point and pinned runtime instructions. The two CPU commands above do not install or load model dependencies. GPU reproduction needs the specified 96 GB RTX PRO 6000 and a separately obtained pinned model snapshot.
Reference the paper
BibTeX · submitted manuscript
@@BIBTEX@@Speech: Loren Lugosch, Timers and Such v1.0, dev-real/anonymous0_24.wav, CC0 1.0. License text.
Machinery: ldezem / Ore.e Refineries, Sound of Work: Blacksmith edition 1, Holman nc4 start event, recorded at Jesse Sipola’s smithy. Released under CC0 1.0; the creator’s metadata marks it public domain. License text.
Archived model: Qwen team, Qwen3-Omni-30B-A3B-Instruct model card. No weights or inference runtime are included.
Audio licenses apply to the audio assets. The downloaded provenance and licensing note identifies transferred code separately; no blanket code or paper license is asserted. The PDF is the unchanged submitted manuscript.