Sunny Chokshi: LLM Security Notes

2 posts

Detection rate alone ranks the worst LLM code reviewers first

When I tested 16 open LLM code reviewers, the first table in my paper showed which of them separate secure code from vulnerable code at all. It is worth reading on its own. The setup I used 180 Python

Oct 28, 20264 min read1 view

Oct 28, 2026
Oct 28, 20263 min read3 views
A hex literal made LLM code reviewers raise false alarms
I wanted to know whether an LLM code reviewer follows what the code does or what the code looks like. Cryptographic API misuse is a clean place to ask, because the same insecure choice can be written
3 min read3 views

Latest articles

Detection rate alone ranks the worst LLM code reviewers first

A hex literal made LLM code reviewers raise false alarms