On the same engine, ten decisions read as single tokens take about 92 ms. Generating them as JSON takes about 1.57 s, and the cascade also spends about 411 ms on transcription.
Research · Submitted to ICASSP 2027
DuplexJev
DuplexJev is a System One model for speech, in the spirit of Jev's decision models but deciding straight from raw audio. It turns a frozen speech LLM into a decision engine for full-duplex voice agents. Turn-taking, which filler to play, and who is speaking are each read as a single-token distribution over option letters. Nothing is decoded. The product name is Speech-to-Decision. State of the art in paralinguistic understanding on public benchmarks (LISTEN, StressTest and MSPB combined: 50.2 vs 47.8 for Gemini 3.8 Flash and 45.0 for Qwen3-Omni), ahead of Gemini, GPT (realtime flagships included) and Qwen3-Omni. The latest release (Oct 22) hears Japanese: error falls from 33% to 12%, Korean and Vietnamese to a quarter to a third of before, and Mandarin laughter, breath and stressed words are heard much more accurately. Not affiliated with TypeSafe (Jev).
The paper is submitted to IEEE ICASSP 2027. Acceptance has not been announced. Preprint: arXiv 2610.02638.
State of the art in paralinguistic understanding
It hears what the flagships miss
On LISTEN (EACL 2026), the public benchmark built to test whether audio models listen or just read the words, together with StressTest and MSPB, on 1,035 questions with none from our training corpora, DuplexJev-32B-Para v1.3 ranks first with 50.2 (Gemini 3.8 Flash 47.8, Qwen3-Omni 45.0), ahead of every model we tested, including Google's and OpenAI's realtime voice flagships, which both trail their own offline models.
| Model | LISTEN, 4 audio-only conditions | Neutral words, voice only | Emotion† | Meaning from stress† |
|---|---|---|---|---|
| DuplexJev-32B-Para (open) | 37.3 | 45.0 | 91.0 | 90.5 |
| Qwen3-Omni-30B | 33.9 | 33.5 | 46.0 | 36.5 |
| Gemini 3.8 Flash | 33.1 | 31.0 | 40.5 | 79.5 |
| GPT-audio-1.5 | 31.6 | 24.5 | 35.0 | 27.5 |
| Gemini 3.8 Live (realtime) | 30.9 | 25.0 | 27.5 | 55.0 |
| Gemini 3.1 Pro | 30.1 | 25.0 | 32.5 | 51.5 |
| GPT-realtime-2.1 (realtime) | 25.6 | 15.5 | 31.5 | 26.0 |
Same questions; each closed model asked once per question. Gemini 3.8 Flash is ahead on StressTest (89.0 vs 71.5), and Qwen3-Omni and Gemini are ahead when the words themselves carry the emotion. Both GPT models and Gemini 3.8 Live mostly answer "neutral" to emotion questions. † Test sets from corpora in our training data (held-out speakers or sentences). Turn-taking: on LiveKit eot-bench, fewer false cut-offs than Smart Turn v3.2 in all 14 languages at 300 ms. Scripts and every answer
Results
92 ms
~90%
90% / 91%
Decisions without decoding
With the cross-attention connector, gender and emotion both reach about 90%. Transcript distillation alone leaves them near 55% and 28%.
Spoken QA with the last-layer connector is 90%, against 91% when the model reads the transcript. The cross-attention connector drops spoken QA by one point (83% to 82%).
Method
One forward pass for every question
- 01 Audio One utterance, or a batch of calls.
- 02 Frozen ASR encoder An off-the-shelf encoder such as Qwen3-ASR. Weights stay frozen.
- 03 Connector The only trained piece. Last layer, or cross-attention over three layers.
- 04 Frozen LLM One forward pass. Shared prefix, zero decode steps.
- 05 Single-token options Each question is a distribution over its letters. The max probability is the confidence.
Any ASR encoder and any frozen LLM join through a small connector. Supervision is on the answer token that is read out, while content keeps transcript distillation, so the model can hear what the transcript never wrote down.
Release
Code, weights, and the product demo
The code, training recipe, batched inference pipeline, and the bilingual spoken-QA set qa100 are open. The deployable complete model is DuplexJev-4B.
The full-duplex demo is the product experience. It is not an online reproduction of this paper.