DFIP: Decision-Focused Inference Path for System One Models

Decision models return a choice, probability, or score. Even a short answer can involve repeated CPU dispatch, GPU launches, and calculations for output tokens the application never uses.

In our DFIP paper, we study where reducing this work makes inference faster. We compare four models: PPLX-Decider, Liquid d1-3B, Clef-Flash, and GLiNER2.5-Decide.

What we tested

We applied established inference techniques at the boundaries each model supports: CUDA graph replay, candidate-token projection, and batching requests with compatible schemas and similar input lengths. Each accelerated path was compared with native inference, including native GPU extensions or the model's released compiler settings.

We also checked the answers. Preserving a model's winning choice does not guarantee that its probabilities, scores, or confidence thresholds stay the same.

Latency benchmarks

We measured short yes/no requests with the models already loaded and warmed up. Lower latency is better; the speedup divides each model's native latency by its DFIP latency.

Source: Tables 2 and 3 in our paper. Latencies are geometric means of within-container medians; the confidence intervals bootstrap paired container medians. These timings exclude network transport and cold starts.

For PPLX-Decider and Clef, we installed the optional native CUDA kernels in both paths. We completed five of five Decider runs and four of five Clef runs; Clef's fifth failed during native preflight. Liquid completed three of three runs against its released compiler setting. Its confidence interval spans 1×, so we found no clear improvement on that workload.

PPLX-Decider ran on a B200; Clef and Liquid ran on an L40S. Each row compares a model with its own baseline, so the table does not rank the models against one another.

Batching and answer consistency

In a separate GLiNER experiment, we measured a 1.40× improvement over compiled native FIFO batching on a diverse trace of 96 records. That result measures completion time for a batch of work, rather than the single-request latency above. It comes from one worker and five paired warm timings, with no independent-worker confidence interval. The exposed labels and confidence passed our audit. Table 5 reports both the eager and compiled comparisons.

We also audited 3,024 decisions per model for Decider and Clef. Both preserved all 1,112 returned categorical choices, but Clef changed probabilities and scores. Matching the winning answer alone does not establish that a confidence-based routing policy will behave the same way.

Serving limits

Changing external traffic reversed the original Decider and Clef replay gains through capture costs, queues, dropped requests, and a retained CUDA error. Clef also changed some confidence-threshold outcomes. GLiNER's resident gain did not improve its tested arrival tail latency.

DFIP is a research preview. The packaged client uses native inference by default; the acceleration experiments have separate research entry points.

Paper, code, and related model

• Read the paper on arXiv for the methods, answer audits, benchmarks, and serving limits.
• Explore DFIP on GitHub for the client, code, raw results, and reproduction instructions.
• Explore Security-One-27B on Hugging Face, our related decision model for security triage. DFIP supports it through the client; the paper's four-model study does not evaluate it.