AI Red Teaming Hub

Anthropic's researchers had Claude write training data meant to make a small research model misaligned while passing its own classifier. The...

Scaling attack generation to 10,000 agents shows volume is achievable, but the headline result establishes less than it seems.

• Monitor labels, not...

The paper reports that its SafeTulu filter reduced GCG jailbreak success rates by up to 22% across three model families. On held-out HarmBench...

Agents can expose data while pursuing legitimate tasks, with no attacker involved. Builders can turn these failures into testable controls:

• Publish...

Penetration testing asks whether the known doors are locked. But most attacks on AI systems persuade rather than break the technology, so a clean...

Once weights leave the developer's hands, later safeguards may not follow them. Mistral says updates reach only its managed API and future...

A TMLR 2026 survey maps agentic security across 274 papers, organized around applications, threats, and defenses.

• Autonomy widens the attack...

CrowdStrike says the China state-backed group Vapor Panda used the ARTEX agentic pentesting framework alongside DeepSeek, with about 140 sensitive...

The article argues that defensive action is not free. An adversary who understands detection logic can generate hostile-looking activity and have the...

Two starting points for planning hands-on AI red-team exercises:

• A six-part blog series covers exercise design, prompt engineering, and enterprise...

Tracebit reports that context bombs cut malicious agent success by 91% and raised the cost of each successful attack 23 times. These are findings from...

A backdoored model can pass standard accuracy benchmarks and run in production for months before its trigger fires. Red teams need to look beyond...

Both sources point to the same gap: a red-team result is only meaningful within its stated boundaries.

• Scope first: One report recommends publicly...

Spectral Signature methods identify poisoned data by analyzing feature representations through eigenvectors, which makes them a candidate for...

ProArch's AI red teaming scope offers a practical checklist for builders, since AI systems add a new attack surface beyond traditional pentests.

-...

The 2026 reporting separates AI used by human operators from AI built into malware:

• Operator use: One campaign ran a workflow with 44 AI agents,...

The paper argues that static plans stay blind to victim replies, which wastes turns. Its SEMA-Auto conditions each turn on session history and decides...

Evasion attacks leave the model untouched and change what it sees. For red teaming, that puts the focus on real-world inputs and decision points:

-...

Capability evaluations rely on the assumption that hidden behaviors can be elicited. This research challenges that assumption by embedding encrypted...

The author argues AI security agents earn autonomy the way a new colleague earns responsibility. The path looks like this:

• Start with human...