Bulk Pass: batch extraction over CSVs on the new fast, cheap models

No one ships the specific product: a CLI that takes a CSV and a JSON schema, batches rows through a diffusion LLM on one local GPU, validates the outputs and sends failed rows to an autoregressive fallback. The pieces are all there, though. SGLang, dInfer, Fast-dLLM and llama.cpp give you diffusion inference. Bespoke Curator, Instructor and Sculpt give you bulk, schema-validated runs with retries, mostly on autoregressive models. Inception's Mercury API gives you fast, cheap diffusion inference with structured outputs and no GPU. So the moat is thin: Curator could add a dLLM backend easily, and many indie teams will prefer Mercury's hosted API to running a GPU. A new entrant could still win with an opinionated, local-first tool that turns a CSV into validated output with no setup. It would pick the dLLM model and decoding settings for the user, handle schema validation and the fallback, and report the cost and speed gain over running a model like Qwen through vLLM. That only pays off if diffusion models really are faster and accurate enough on short, schema-shaped outputs. That is unproven. The fastest local figure I found was d3LLM's self-reported 289 tokens per second on an H100, and "thousands of tokens per second" is Mercury's hosted figure, not a local one. I also found no public trace of "dlmrt", the runtime the pitch names. The benchmark idea is the clearer opening. Only scattered academic papers exist, such as the multi-label classification study and ParallelBench, plus vendor blog posts. There is no maintained public leaderboard comparing diffusion and small autoregressive models on extraction and classification. Building one would also bring in users for the CLI and test whether it is worth building. The risk is that the main runtime projects or Inception publish their own.

• SGLang diffusion LLM support (LMSYS)

  Production-style serving for diffusion LLMs like LLaDA 2.0 inside SGLang. It is a general engine with no CSV, schema or fallback workflow, but it is the most likely backend a user would reach for instead.

• dInfer (inclusionAI / Ant Group)

  Open-source dLLM inference framework with batched inference for throughput. Infrastructure only.

• Fast-dLLM (NVlabs)

  Training-free acceleration for Dream and LLaDA using block-wise KV cache. Research-oriented, not a data tool.

• llama.cpp diffusion CLI

  llama-diffusion-cli runs Dream, LLaDA, RND1 and (via Unsloth guides) DiffusionGemma locally. It generates text but has no batch-over-CSV, schema validation or retry logic.

• DiffusionGemma local guide (Unsloth)

  Shows a big-lab diffusion model that people can already run locally, which lowers the barrier for DIY bulk pipelines.

• Bespoke Curator

  Python library for bulk inference and dataset annotation with a vLLM local backend, caching and retries. Closest workflow competitor; it is autoregressive-focused but could add a dLLM backend.

• Inception Labs Mercury API

  Hosted commercial diffusion LLM with structured outputs, an OpenAI-compatible API, around 1,100 tokens per second and roughly $0.25/$0.75 per million tokens. A cheap hosted substitute for running dLLMs locally.

• Sculpt

  Schema-based structured extraction library with parallel batch mode. Not local-GPU or diffusion specific.

• Instructor batch processing

  Popular structured-extraction library with batch support across providers. Adjacent: covers the validate-and-retry layer.

• Discrete Diffusion LMs Are Training-Free Multi-Label Classifiers (arXiv)

  Academic comparison of LLaDA-8B against Qwen2.5-7B on classification with mixed results. The closest thing to the proposed quality benchmark, but a one-off paper, not a maintained leaderboard.

• ParallelBench (ICLR 2026)

  Studies quality trade-offs of parallel decoding in dLLMs versus autoregressive models. Relevant to the benchmark idea but not focused on extraction or classification.

• dLLM-Serve (arXiv)

  Research serving system for diffusion LLMs built on Nano-vLLM. Shows active systems work on dLLM throughput.

Checked 2026-10-17