{“lastseen”:”2025-11-01T20:25:48″,”description”:”Over the past few days I wrote a new Go implementation of ML-DSA, a post-quantum signature algorithm specified by NIST last summer. I livecoded it all over four days, finishing it on Thursday evening. Except\u2026 Verify was always rejecting valid signatures.\\n \\n \\n $ bin\/go test crypto\/internal\/fips140\/mldsa\\n — FAIL: TestVector (0.00s)\\n mldsa_test.go:47: Verify: mldsa: invalid signature\\n mldsa_test.go:84: Verify: mldsa: invalid signature\\n mldsa_test.go:121: Verify: mldsa: invalid signature\\n FAIL\\n FAIL crypto\/internal\/fips140\/mldsa 2.142s\\n FAIL\\n \\n\\nI was exhausted, so I tried debugging for half an hour and then gave up, with the intention of coming back to it the next day with a fresh mind.\\n\\nOn a whim, I figured I would let Claude Code take a shot while I read emails and resurfaced from hyperfocus. I mostly expected it to flail in some maybe-interesting way, or rule out some issues.\\n\\nInstead, it rapidly figured out a fairly complex low-level bug in my implementation of a relatively novel cryptography algorithm. I am sharing this because it made me realize I still don\u2019t have a good intuition for when to invoke AI tools, and because I think it\u2019s a fantastic case study for anyone who\u2019s still skeptical about their usefulness.\\n\\n\\u003e Full disclosure: Anthropic gave me a few months of Claude Max for free. They reached out one day and told me they were giving it away to some open source maintainers. Maybe it\u2019s a ploy to get me hooked so I\u2019ll pay for it when the free coupon expires. Maybe they hoped I\u2019d write something like this. Maybe they are just nice. Anyway, they made no request or suggestion to write anything public about Claude Code. Now you know.\\n\\n## Finding the bug\\n\\nI started Claude Code v2.0.28 with Opus 4.1 and no system prompts, and gave it the following prompt (typos included):\\n\\n\\u003e I implemented ML-DSA in the Go standard library, and it all works except that verification always rejects the signatures. I know the signatures are right because they match the test vector.\\n\\u003e \\n\\u003e YOu can run the tests with \\”bin\/go test crypto\/internal\/fips140\/mldsa\\”\\n\\u003e \\n\\u003e You can find the code in src\/crypto\/internal\/fips140\/mldsa\\n\\u003e \\n\\u003e Look for potential reasons the signatures don’t verify. ultrathink\\n\\u003e \\n\\u003e I spot-checked and w1 is different from the signing one.\\n\\nTo my surprise, it pinged me a few minutes later with a complete fix.\\n\\nMaybe I shouldn\u2019t be surprised! Maybe it would have been clear to anyone more familiar with AI tools that this was a good AI task: a well-scoped issue with failing tests. On the other hand, this is a low-level issue in a fresh implementation of a complex, _relatively novel_ algorithm.\\n\\nIt figured out that I had merged `HighBits` and `w1Encode` into a single function for using it from Sign, and then reused it from Verify where `UseHint` already produces the high bits, effectively taking the high bits of w1 twice in Verify.\\n\\nLooking at the log, it loaded the implementation into the context and then _immediately_ figured it out, without any exploratory tool use! After that it wrote itself a cute little test that reimplemented half of verification to confirm the hypothesis, wrote a mediocre fix, and checked the tests pass.\\n\\nI threw the fix away and refactored `w1Encode` to take high bits as input, and changed the type of the high bits, which is both clearer and saves a round-trip through Montgomery representation. Still, this 100% saved me a bunch of debugging time.\\n\\n## A second synthetic experiment\\n\\nOn Monday, I had also finished implementing signing with failing tests. There were two bugs, which I fixed in the following couple evenings.\\n\\nThe first one was due to somehow computing a couple hardcoded constants (1 and -1 in the Montgomery domain) wrong. It was very hard to find, requiring a lot of deep printfs and guesswork. Took me maybe an hour or two.\\n\\nThe second one was easier: a value that ends up encoded in the signature was too short (32 bits instead of 32 bytes). It was relatively easy to tell because only the first four bytes of the signature were the same, and then the signature lengths were different.\\n\\nI figured these would be an interesting way to validate Claude\u2019s ability to help find bugs in low-level cryptography code, so I checked out the old version of the change with the bugs (yay Jujutsu!) and kicked off a fresh Claude Code session with this prompt:\\n\\n\\u003e I am implementing ML-DSA in the Go standard library, and I just finished implementing signing, but running the tests against a known good test vector it looks like it goes into an infinite loop, probably because it always rejects in the Fiat-Shamir with Aborts loop.\\n\\u003e \\n\\u003e You can run the tests with \\”bin\/go test crypto\/internal\/fips140\/mldsa\\”\\n\\u003e \\n\\u003e You can find the code in src\/crypto\/internal\/fips140\/mldsa\\n\\u003e \\n\\u003e Figure out why it loops forever, and get the tests to pass. ultrathink\\n\\nIt spent some time doing printf debugging and chasing down incorrect values very similarly to how I did it, and then figured out and fixed the wrong constants. Took Claude definitely less than it took me. Impressive.\\n\\nIt gave up after fixing that bug even if the tests still failed, so I started a fresh session (on the assumption that the context on the wrong constants would do more harm than good investigating an independent bug), and gave it this prompt:\\n\\n\\u003e I am implementing ML-DSA in the Go standard library, and I just finished implementing signing, but running the tests against a known good test vector they don\u2019t match.\\n\\u003e \\n\\u003e You can run the tests with \\”bin\/go test crypto\/internal\/fips140\/mldsa\\”\\n\\u003e \\n\\u003e You can find the code in src\/crypto\/internal\/fips140\/mldsa\\n\\u003e \\n\\u003e Figure out what is going on. ultrathink\\n\\nIt took a couple wrong paths, thought for quite a bit longer, and then found this one too. I honestly expected it to fail initially.\\n\\nIt\u2019s interesting how Claude found the \u201ceasier\u201d bug more difficult. My guess is that maybe the large random-looking outputs of the failing tests did not play well with its attention.\\n\\nThe fix it proposed was updating only the allocation\u2019s length and not its capacity, but whatever, the point is finding the bug, and I\u2019ll usually want to throw away the fix and rewrite it myself anyway.\\n\\nThree out of three one-shot debugging hits with no help is _extremely impressive_. Importantly, there is no need to trust the LLM or review its output when its job is just saving me an hour or two by telling me where the bug is, for me to reason about it and fix it.\\n\\nAs ever, I wish we had better tooling for using LLMs which didn\u2019t look like chat or autocomplete or \u201cmake me a PR.\u201d For example, how nice would it be if every time tests fail, an LLM agent was kicked off with the task of figuring out why, and only notified us if it did before we fixed it?\\n\\n\\n\\nFor more low-level cryptography ~~bugs~~ implementations, follow me on Bluesky at @filippo.abyssdomain.expert or on Mastodon at @filippo@abyssdomain.expert. I promise I almost never post about AI.\\n\\n## The picture\\n\\nEnjoy the silliest floof. Surely this will help redeem me in the eyes of folks who consider AI less of a tool and more of something to be hated or loved.\\n\\n\\n\\nMy work is made possible by Geomys, an organization of professional Go maintainers, which is funded by Smallstep, Ava Labs, Teleport, Tailscale, and Sentry. Through our retainer contracts they ensure the sustainability and reliability of our open source maintenance work and get a direct line to my expertise and that of the other Geomys maintainers. (Learn more in the Geomys announcement.) Here are a few words from some of them!\\n\\nTeleport \u2014 For the past five years, attacks and compromises have been shifting from traditional malware and security breaches to identifying and compromising valid user accounts and credentials with social engineering, credential theft, or phishing. Teleport Identity is designed to eliminate weak access patterns through access monitoring, minimize attack surface with access requests, and purge unused permissions via mandatory access reviews.\\n\\nAva Labs \u2014 We at Ava Labs, maintainer of AvalancheGo (the most widely used client for interacting with the Avalanche Network), believe the sustainable maintenance and development of open source cryptographic protocols is critical to the broad adoption of blockchain technology. We are proud to support this necessary and impactful work through our ongoing sponsorship of Filippo and his team.”,”published”:”2025-11-01T18:10:13″,”modified”:”2025-11-01T18:10:13″,”type”:”filippoio”,”title”:”Claude Code Can Debug Low-level Cryptography”,”source”:””,”references”:””,”id”:”FILIPPOIO:4B0E04D990771751784DE1A55F1CB2E0″,”bulletinFamily”:”blog”,”cwe”:null,”cvelist”:[],”sourceData”:””,”sourceHref”:””,”cvss”:{“score”:0,”severity”:”NONE”,”vector”:”NONE”,”version”:”NONE”},”cvss2″:{},”cvss3″:{“version”:””,”vectorString”:””,”baseScore”:0,”baseSeverity”:””,”attackVector”:””,”attackComplexity”:””,”privilegesRequired”:””,”userInteraction”:””,”scope”:””,”confidentialityImpact”:””,”integrityImpact”:””,”availabilityImpact”:””,”cvssV3″:{“version”:””,”vectorString”:””,”baseScore”:0,”baseSeverity”:””,”attackVector”:””,”attackComplexity”:””,”privilegesRequired”:””,”userInteraction”:””,”scope”:””,”confidentialityImpact”:””,”integrityImpact”:””,”availabilityImpact”:””}},”href”:”https:\/\/words.filippo.io\/claude-debugging\/”,”category_name”:”News”,”post_link”:””,”product”:””,”version”:””,”vendor”:””,”ai_description”:””,”ai_severity”:””,”ai_vendor”:””,”ai_product”:””,”ai_version”:””,”ai_score”:0}<\/p>\n","protected":false},"excerpt":{"rendered":"
{“lastseen”:”2025-11-01T20:25:48″,”description”:”Over the past few days I wrote a new Go implementation of ML-DSA, a post-quantum signature algorithm specified by NIST last summer. I livecoded it…<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4],"tags":[6,8,12,162,13,33,7,11,5],"class_list":["post-24390","post","type-post","status-publish","format-standard","hentry","category-category_news","tag-cve","tag-cvss","tag-exploit","tag-filippoio","tag-news","tag-none","tag-security","tag-tapic","tag-vulnerability"],"yoast_head":"\n