LLM Engineering Digest

LLM Security & Agent Environments

• ToM-SB: ToM-SB is an environment where defender LLMs compete against attacker LLMs trying to access sensitive...

New paper SPPO adapts PPO at the sequence level for long-horizon reasoning tasks in agents. Join the discussion on this breakthrough.

Breakthrough in LLM security via belief manipulation:

• Double-Agent Defenders: RL builds theory-of-mind (ToM) to steer attackers to wrong info,...

Trend alert: Production LLM tools like Glassbrain fill Promptfoo's gaps in debugging real failures.

• Visual traces & replay: Inspect agent steps,...

Two papers advance on-policy distillation for efficient LLM post-training:

• Lightning OPD proposes offline on-policy distillation for large...

Nemotron 3 Super debuts as an open, efficient Mixture-of-Experts hybrid Mamba-Transformer model optimized for agentic reasoning. Check the paper discussion for details.

New research proposes Block Diffusion Draft Trees to speed up speculative decoding in LLMs. Join the discussion on this paper for inference efficiency gains.

You Only Judge Once introduces multi-response reward modeling in a single forward pass for RLHF pipelines. Streamlines efficiency in LLM alignment—join the discussion.

New Benchmarks & Evals

• 🔥 SPEED-Bench: SPEED-Bench provides a unified and diverse benchmark for speculative decoding.
• Confident AI Eval Tools:...

Benchmark & Eval Updates

• 🔥 SPEED-Bench: SPEED-Bench introduced as a unified and diverse benchmark for speculative decoding.
• Confident AI:...

SPEED-Bench is a unified and diverse benchmark for speculative decoding – key eval tool for LLM inference efficiency in vLLM-like deployments. Join the discussion.

Confident AI is the best LLM evaluation tool in 2026, covering every use case like RAG and agents. Top 7 list essential for benchmarking LLM pipelines.

PPO and GRPO run on-policy in LLM training, generating rollouts for one gradient update then discarding them—"this is crazy"! Meta's praised replay buffer paper pushes for better off-policy efficiency.

New survey dives into attention sinks pathology in Transformers:

• Utilization: Practical uses of sinks
• Interpretation: Understanding their role
• Mitigation: Strategies to address them

Join the discussion on this paper for architecture insights.

Rising practical guides for agentic apps:

• Gemma 4: Python tool calling loop—catch response, execute local code, feed back.
• OpenClaw: Fully local...

Bridging agent theory and infra:

• Theoretical gap: Agent loops enable parallel tool calls but lack structural parallelism guarantees, as LLMs may...

Andon Labs kicked off with AI controlling a vending machine at Anthropic's office, then their own office— even hiring to build a gym. Now, they've...

Audio Flamingo Next debuts as next-generation open audio-language models targeting speech, sound, and music. Join the discussion on the paper page to explore this multimodal leap.

Low-Precision Inference Recipes

• 🔥 Hugging Face NVFP4 & MXFP8 on B200: Hugging Face shares findings on achieving good speedups with NVFP4 and...

Paradigm shift: Neural Computers (NCs) make a neural network the running computer itself, folding computation, memory, and I/O into latent state .

-...