Reflex-1

Overview

Many agent decisions end with a choice: which queue receives a request, which tool fits a task, or which action comes next. Reflex-1 scores the choices your application supplies in a single forward pass. It returns a probability distribution that the application can use to make that decision.

The research follows the decision-model direction explored by Jev, with a distinct architecture and supervised training procedure. A 28-layer context encoder and a six-layer candidate encoder feed a shared scoring head. The current checkpoint updates all 420,778,370 parameters.

The latest fine-tuned checkpoint is the default public download. Its model card contains current evaluation results and CPU measurements. The dated studies and recordings below retain their original checkpoint identities.

Current results

The October 5 release completed 21/30 native MiniWoB++ browser episodes with a fixed controller. It scored 92.93% on 495 Banking77 examples. These are reserved subsets from task families represented in training, not full benchmark scores.

On an Intel Xeon Platinum 8558 CPU, Banking77 inference took 375.2 ms median and 452.1 ms p95 with four compute threads. Peak process memory was 2.17 GiB, including loading, warmup and inference.

The full evaluation is still incomplete. BoolQ, RACE and the public JevBench diagnostic regressed from the previous checkpoint; native Dino reached the 60-second target in 0/16 attempts. Current GPU latency has not been measured. The published evaluation records the checkpoint, conditions and remaining gaps.

How it works

Candidate answers are supplied with each request. The model supports 1–255 choices per question, within its token limits, and several questions can share a packed state. A support application can supply its own queues; an agent can supply descriptions of the tools it has available. New decision types need their own accuracy tests.

The model scores the supplied options; the application constructs tool arguments and decides which actions are permitted. State text is capped at 512 context-tokenizer tokens and can be shortened further by the request budget. Each choice accepts up to 512 option-tokenizer tokens. Inspect state_truncated when context may have been shortened.

Examples

The examples and decision-quality comparisons below use the October 3 snapshot. Speed and resource measurements are labelled separately. Each recording keeps its observed ending, including failure.

Chromium Dino

This recording uses Chromium’s original chrome://dino game. Reflex-1 receives the currently visible obstacle geometry and chooses run, jump or duck. Native keyboard events execute those choices while the game clock continues during inference. The video shows actual browser frames at normal speed.

Reflex-1 · Original Chromium43.6 SEC · SILENT VIDEO

The two attempts with this snapshot ended in collisions after 42.31 seconds and 30.41 seconds; neither reached the 60-second limit. The longer attempt is shown first. These are independent random courses, so they do not support a paired model comparison. The complete second attempt remains available below.

Watch the second Dino attempt

Reflex-1 · Original Chromium31.8 SEC · SILENT VIDEO

ViZDoom

The game is native ViZDoom’s Defend the Center scenario at skill 5. Reflex-1 chooses actions from visible object bounds, health and ammunition. This is a structured-state controller; it does not infer actions from pixels. The recording below is the originally predeclared example, shown as a single Reflex-1 view.

Reflex-1 · Native ViZDoom36.5 SEC · SILENT VIDEO

Across all 32 reserved episodes, this snapshot averaged 18.88 kills and 22.47 seconds of survival. Every episode ended before the 45-second limit. The chart retains every evaluated external model and the full episode counts.

Browser workflows

These are locally hosted synthetic WebGym sites distributed with Laya Browser. The browser states were captured in real Chromium, and the original task checker decides success. Reflex-1 selects actions and targets; a shared Qwen3-1.7B helper supplies typed text and dropdown values. The outcome measures that complete system.

Reflex-1 · Synthetic WebGym30.1 SEC · 4× · SILENT VIDEO

Reflex-1 · Synthetic WebGym8.1 SEC · 4× · SILENT VIDEO

The restaurant task succeeds; the shopping task fails. Both use the same predeclared task seed and retain the recorded ending. Playback is 4× the original wall time, replaying captured browser states with the terminal hold preserved. These examples are synthetic-site development results, not MiniWoB++, WebArena or live-site scores.

Reflex-1 completed 13/35 tasks in this study; Laya Browser completed 17/35. All attempts, failures and model-request errors remain in the denominators. A later pilot through the official MiniWoB++ environment completed 0/30 tasks with this snapshot, so synthetic-site results should not be read as success on that benchmark.

Decision quality

Typed Decisions covers customer support, invoice processing, security alerts and agent traces. The evaluation preserves original state groups and asks each model the same five judgments through a common choice interface. Reflex-1 answered 1,508/2,000 judgments correctly (75.4%).

On the reserved CLINC cohort, Reflex-1 answered 3,680/4,490 intent and out-of-scope requests correctly (82.0%). For unfamiliar-tool selection, it answered 72/87, below the strongest external result of 82/87. Both strengths and remaining gaps appear below.

These are common-choice adaptations with declared cohorts, not official leaderboard submissions. External models were evaluated as shipped; they did not receive Reflex-1’s task-specific training. The figures retain the original uncertainty intervals, token budgets and truncation counts.

Speed and resources

The measurements in this section describe the October 2 checkpoint. Current checkpoint measurements are available in the model card linked above.

H200 request latency

We replayed 120 distinct requests twice per model on a shared NVIDIA H200. They came from five game and browser environments. Every model received the same inputs, with one request in flight at a time.

reflex-1’s median was 27.5 ms and its p95 was 30.5 ms. It was faster than OpenJev in FP32 and Laya in the separate FP32 replay. Native Laya used BF16 autocast and had a slightly lower median. The shared machine and sample size limit what we can infer from close results.

The timer includes tokenization, model scoring, output normalization and loopback HTTP. It excludes loading and three warmup requests. Answer and option caches were disabled. We did not measure concurrent serving throughput in this run. OpenJev here is the public DeBERTa checkpoint; it is not the hosted Jev service.

M4 CPU latency

The CPU test used a MacBook Air M4, FP32, and batch size one. Each model used one PyTorch intra-op thread, one inter-op thread, and a BLAS thread limit of one. Each task had 24 distinct cases, each repeated twice. Model order was balanced across eight execution blocks.

Banking77 asks the model to choose among 77 intents. reflex-1 took 345.2 ms median; OpenJev took 1,332.6 ms in the same run. Across the five tasks, reflex-1’s medians ranged from 256.2 to 463.4 ms.

Laya was faster on several smaller-choice tasks. Its native Banking77 budget truncated the options, so its timing covers a reduced candidate set. Kev used CPU reference kernels with unmerged adapters. Background laptop activity contributed to the wider tail latencies.

This test includes request construction through normalized output, with the model already loaded. The CPU and GPU tests use different requests; their timings do not give a direct CPU-to-GPU speedup.

Familiar-task accuracy

reflex-1 scored 95.0% on Banking77 and 92.2% on Emotion. It led the compared checkpoints on both, tied the best score on AG News and SST-5, and scored slightly below OpenJev on BoolQ.

These were 500-example diagnostics per task. The task families were included in training, and we had inspected the subsets during development. Training budgets and prior exposure differ between models. Accuracy was measured separately from latency, on more examples.

The general checkpoint completed 0 of 80 trials with our custom browser controller. The result exposes a substantial gap between familiar-task classification and autonomous control. Transfer to unfamiliar questions and probability calibration remain open evaluation problems.

Training and storage

The serving model has 420,778,370 parameters after merging adapters. ModernBERT encodes the state and questions; a frozen MiniLM encoder represents the choices. The shared head combines their representations.

The main adaptation run updated 9,036,290 parameters over 6,000 steps, using 290,657 records from eight task families. It took 75 minutes 7 seconds on one H200, including development screens and checkpoint saves. Base-model pretraining, data preparation, projection fitting and earlier experiments are additional costs.

The extracted FP32 model takes about 1.69 GB on disk. The separate CPU resource replay measured 1.50 GiB peak process RSS, including framework allocations, tokenizers, loading, warmup and inference. GPU memory usage remains unmeasured.

Model access

The public Hugging Face release contains the 1.68 GB FP32 weights, both tokenizers, and a self-contained Transformers implementation under Apache 2.0. No account, API key, or GitHub access is required. Use Python 3.12:

python -m pip install "torch==2.8.0" "transformers==5.17.0" "safetensors==0.8.0"

import torch
from transformers import AutoModel

torch.set_num_threads(1)
model = AutoModel.from_pretrained("gai-labs/reflex-1", trust_remote_code=True)

decision = model.predict(
    state="The customer was charged twice for one card payment.",
    question="Choose the matching issue.",
    options=["duplicate charge", "lost card", "unknown fee", "cash withdrawal"],
)[0]
print(decision.choice)

Load the model once and reuse it. The model card documents batch inference, multiple questions, offline loading, and release hashes. The development repository remains private; the public package includes the code needed for inference.

The original training used RACE and SciQ, whose source terms include noncommercial restrictions. The model card documents their use and pinned revisions. The provenance audit does not establish how those terms apply to trained weights.

Recordings and reproducibility

Use the immutable October 3 Hub revision to reproduce these recordings. The charts above retain the measured outcomes, including retention regressions. Each recording identifies its environment, input format and playback speed.

The current release has a separate evaluation linked from its model card. These recordings and studies retain their original results.