You signed up for my ML-security work somewhere along the way, and then heard nothing for about a year. This is the first issue of what that becomes. It has a name now, and it's different from what you signed up for, so here's the honest version.

What Quantitative AI Security is

It lives on the line between academic research and applied AI/ML/LLM cybersecurity. Almost nobody stands there. Researchers publish claims they never have to run. Practitioners run tools whose claims they never get to check. So I do two things on that line.

First, I check the field's claims. Four questions, every time: Can I validate this claim? Can it even be validated? Where's the code proof? Is it actually feasible in a real security environment, at scale? Papers, vendor claims, benchmarks. Every answer comes with a number, code you can run, or a prediction written down before the result is known.

Second, I build my own ideas and ship them in the open. They're all free, runnable, local. And they get held to the same four questions. My tools don't skip the rubric because they're mine. That's what "quantitative" means here: same questions, both directions. When I'm wrong, it's on the record.

For now, Quantitative AI Security is this newsletter, weekly on Sundays. There's more coming under the name, and I'll tell you what when it's ready to be useful, not before.

What happened

My job ended two weeks ago. I have about six months of runway, and I'm spending all of it on one thing: building open-source AI cybersecurity tools and agents, in public, week by week, in this newsletter.

What you'll get every week

1. Applied. Research most of the field never sees, explained in plain language, then rebuilt to see if it holds. The four questions, answered. Papers with dead code links are a specialty.
2. The build log. Detect-Forge and whatever comes after it, in the open: the product, the agents that run the company, the first customer, and every decision behind them. Held to the same four questions.
3. The numbers. Installs, stars, readers, customers, income, runway. Weekly, including when it hurts.

What exists today

Being honest about the starting line is the whole point, so:

• detect-forge went live on PyPI this week. It tests whether your detection rules still catch what they claim, not just whether they can fire. Four commands (stale, coverage, backtest, audit), one config, one CI step, and nothing leaves your environment. pip install detect-forge.
• A public, MIT-licensed clean-room reimplementation of a peer-reviewed ML intrusion-detection model. It's the one whose official code link returns a 404. That's next week's issue.

What I won't do

No hype vocabulary. No claims without code or data behind them. No pretending something shipped when it didn't. When I'm wrong, the correction runs in the next issue with the same prominence.

Next week

What happens when an ML model treats attack chains as regime changes instead of anomalies? A 2026 paper tried it and reported 98% F1. I rebuilt the model from scratch. The 98% turned out to be the least interesting number in it.

If you know one person who'd want to watch this, a detection engineer, someone building with agents, someone who's tried to start a company with no money and no team, forward this to them. That's the only growth strategy I have, and it's the only one I want.

James

P.S. If you maintain detection rules: pip install detect-forge. It's free, it runs locally, and it will tell you which of your rules are already dead.

Quantitative AI Security · Issue #1 · Unsubscribe · Forwarded this? Subscribe here