Vondet

All notes

Measured

Why Claude Code ignores the rules you wrote for it

Claude Code is not ignoring your rule. It cannot tell that the rule applies.

Vondet / 2026-09-06 / 8 min read

You wrote the rule down. It is in CLAUDE.md, near the top, in bold. Claude Code ignores the rule anyway.

The usual explanation is memory. The file is too long, the context was compacted, the session forgot. The fixes follow from that assumption — hooks that re-read the file every turn, a memory server, a routine that restates the rules after every compaction. All of them are answers to the same theory: the instruction stopped arriving.

We had a rule that fit none of that. Short, not buried, and in front of the agent in every single run — it lives in the tool's own description, which Claude Code loads at the start of a session. Across seven runs the tool was called zero times. There was no effective line in CLAUDE.md in those runs — that was the point; what was in front of the agent every time was the tool's own instruction to use it. So we measured it.

The short version: the rule was not too long and not too vague. It described a situation Claude Code had no way to check. When we put the same rule in CLAUDE.md with its condition rewritten as something the agent had just done, both runs followed it.

Two out of two is not proof, and we will show you exactly how small this is. But the shape of the failure is worth your time, because it is not the shape people usually assume.

The tool had never been called. So we measured nine runs

We have one tool an agent can call to record a fact that is not settled yet — a number we measured, a guess not yet confirmed, a choice between options. It writes to what we call the board: the place outside the session where the plan and the facts live, so the next session can read them. Each entry carries a status (provisional / decided / revising) and the date a number was measured. A plain file has no column for either.

The tool had never been called. Not rarely. Never.

So we ran the obvious experiment. Give Claude Code a real task in the repository — one that naturally produces unsettled facts — and watch what it does. Control runs got the repository as it was. Treatment runs got one extra line in CLAUDE.md telling them to use the tool.

The work was real work, not a test fixture. Six of the nine runs got this request: collect the measured numbers scattered across these documents and organise them. The other three got the same kind of work — gather what is scattered and organise it — on a different subject: which versions of our dependencies we are on. Our notes do not preserve the exact wording of that request, so the quoted line above is the six, not the nine. The runs were not told they were being observed — one of them worked it out from a commit message, which we come back to below.

Nine runs in the same repository — the same kind of work, across two subjects — measuring whether Claude Code followed one rule about one tool:

  • Control 1 — no trigger in CLAUDE.md. Followed the instruction in 0 of 3 runs.
  • Control 2 — no trigger. 0 of 2.
  • Voided — the trigger was written, but never reached the run (below). 0 of 2.
  • Treatment, rewritten wording — trigger present. 2 of 2.

Seven runs without an effective trigger produced zero calls, including the three on the other subject. All seven did the same thing instead: grep for the scattered numbers, Write a new document, add a line to the index, and run the index checker.

All seven ran the checker. That detail comes back later.

It took us two rewrites, and only the second one was ever measured

This is the part we did not expect, and it is where the finding actually comes from.

The original wording came from the tool's own documentation. It said to use the tool for subjects users return to repeatedly.

Read that as Claude Code. You are three tool calls into a task. Do users return to this subject repeatedly? You cannot know.

You have no history of the user, and nothing in the repository tells you. The condition is about someone else's future behaviour. That is the wording that was in front of all seven runs that produced nothing.

So we rewrote it to be about the repository instead: use the tool when you have researched the same subject twice.

Clearer. And by inspection, no more checkable: deciding whether this is the second time you have looked at something means reconstructing your own past, across sessions you do not remember — which is the same wall a handoff runs into. In our case the request was “organise what is scattered,” which is not obviously a second investigation of anything.

We cannot tell you that wording failed, because we never measured it.

The two runs it was written for are the voided row in the table: the trigger sat on a branch, the isolated runs were cut from a different point, and the line never reached the agent at all. We caught that afterwards, put the line where the runs would actually see it, and by then we had rewritten it again. So the second wording has an argument against it and no number — do not read the voided 0/2 as its result.

The third wording is the one that ran under a trigger that actually arrived:

“When you write ‘provisional,’ ‘not yet verified,’ or ‘measured value’ into a document, also put it on the board.”

Both treatment runs followed it. One of them stated the reason in its own words, quoting the trigger: the file has no column for the date a number was measured, so it also went on the board.

Two rules are followed and two are not. The difference is one property

Line the three wordings up against the one tool that agents in this repository do call reliably — the one for recording a decision.

RuleWhere the wording livedConditionCan Claude Code evaluate it right now?Result
Record a decisionthe tool's own description“when a direction is settled or changed”Yes — you just did itreliably called (observed, not counted)
Use the topic tool (v1)the tool's own description“for subjects users return to repeatedly”No — someone else's future0 / 7
Use the topic tool (v2)CLAUDE.md — never reached the runs“when you have researched it twice”No — your own forgotten pastnever measured
Use the topic tool (v3)CLAUDE.md“when you write ‘provisional’ into a document”Yes — you just typed it2 / 2 (one call blocked by the product)

The two on the “yes” side share a property the two on the “no” side do not: the condition is an event the agent performed in the current context. Not a property of the subject. Not a fact about users. Something it just did, still visible in what it is holding. The middle row is our reasoning, not our evidence — we put it in the table because we want you to see where the argument runs ahead of the measurement.

Four rules plotted on one axis of whether the agent can check the condition right now. On the checkable side: record a decision (observed, not counted) and the topic tool v3 (2 of 2). On the uncheckable side: topic tool v1 (0 of 7) and v2 (never measured).
Figure. The same four rules on one axis: whether Claude Code can evaluate the condition with what it already has in front of it. The two on the left were followed, the two on the right were not. Circles lived in the tool's own description and squares lived in CLAUDE.md — the wording and the location changed together, so the figure does not separate them. The hollow square was never measured.

The second column is there so that the confounder travels with the table. The 0/7 wording and the 2/2 wording differ in where they lived as well as in how they were written; we come back to that below, and we did not want a table quoted on its own to leave it behind. One thing you can read off that column: the rule that is followed reliably also lives in the tool's own description. That makes “the location alone explains it” harder to hold — but that row is an observation, not a counted run, and we have not separated the two.

Our working phrasing:

A rule written as a property does not fire. A rule written as an event does — and the event has to be one you just performed.

That is a hypothesis from a small sample, not a law. But it is cheap to check against your own instructions, and the check is mechanical: for each rule you wrote, ask whether Claude Code could determine that the rule applies without knowing anything it does not already have in front of it.

Which file the rule lives in does not change the check. When Claude Code ignores instructions in AGENTS.md, or a skill's instructions get ignored the same way, the question that falls out is the same one: all of these are text the agent reads and then has to decide applies right now. We only measured CLAUDE.md, so treat the rest as the same question, not as a second result.

Rules that fail this check are not badly written. They are unevaluable, which is a different defect and needs a different fix. Making them bolder will not help. We tried bolder.

Check your own CLAUDE.md in five steps. It costs an afternoon

Five steps. It costs an afternoon and tells you more than another rewrite.

  1. 1. List the rules that are not being followed.
  2. 2. For each one, ask: at the moment this should fire, can Claude Code tell that it applies — using only what it already has in front of it?
  3. 3. Rewrite the failures so the condition is an event in the current turn. “When you write X.” “When you change Y.” “When a command exits non-zero.”
  4. 4. Where it matters, add something that goes red. Wording moves behaviour; a failing check moves it further. Ours is a 50-line script that exits non-zero when a new document is not listed in the index. Every one of the seven runs without an effective trigger ran it, unasked.
  5. 5. Run the same task a few times before and after. Two or three runs each way proves nothing on its own, but it is enough to see whether there is a 0/7-shaped gap worth chasing.

If you do this, we would like to hear what you get, especially if it does not replicate.

Files win because something goes red when you ignore them

The seven ineffective runs all did the same four things, and the fourth was running the index checker.

The instruction file says: when you add a document, add a line to the index — a script watches. The script exists. It exits non-zero. Every single run went there.

Nothing equivalent existed for the board. There was no way for anything to go red because a fact was recorded only in prose.

So there were two forces, not one. The wording of the rule, and whether anything would notice it being broken. We changed the wording and got 2/2. We suspect the ceiling stays low without the second force, but we have not tested that — building the check is the obvious next experiment and we have not run it.

Nobody asked for this, and three runs built it anyway

The most interesting thing in the whole exercise was not in the numbers.

Three runs, across two groups, invented a way to mark how confident they were:

  • one tagged each row measured / market rate / estimated (a control run, no trigger)
  • one separated canonical / evidence / history (treatment)
  • one added a closing section titled “things not verified, and values that go stale” (treatment)

That is, roughly, the status field and the measured-on date the tool already provides — rebuilt by hand, in three different vocabularies, in a plain file that no other session can query.

Two of those three were primed, and it matters. The treatment runs had been handed one line: when you write “provisional,” “not yet verified,” or “measured value” into a document, also put it on the board. The third heading above — “things not verified, and values that go stale” — reuses two of those words. Nobody asked anyone to build a label inside a file: the line said put it on the board, not invent a column in prose. But only one of the three, the control run, did it with no such wording in front of it at all.

The two treatment runs also shared one board. Their working trees were isolated; the board is not. The second one saw the first one's entry — 27 rows with a measured-on date — already sitting there.

That zero had been read here as “nobody needs this.” Three inventions — one of them unprompted, two of them primed with the vocabulary but not with the idea — say something closer to the opposite: it is needed often enough to be rebuilt from scratch.

n=3, two of the three primed, and we noticed it after the fact rather than predicting it. Take it as an observation, not a result.

What this does not show

Sample size

Two treatment runs. Seven runs with no effective trigger. One model family. This is a signal, not a measurement.

One of the two “successes” did not complete

It intended to call the tool, then stopped — the board had an existing entry, there was no way to read it back, and writing would have replaced it wholesale. It followed the instruction; the product blocked it. So the honest count is two runs out of two that followed the rule, and one call that landed. Whether the second call would have been correct, we do not know.

We only changed the client side

At the time of the experiment, the tool's own description still carried the original wording. It has since been rewritten so that it fires on an event; we have not measured whether that changed anything. Rewriting the description instead might work as well or better. Untested.

We did not test the checker

The claim that enforcement matters is supported only by the observation that all seven runs ran the one checker that existed.

Two subjects, not one

All nine runs were the same kind of work — gather scattered facts and organise them — but three of them (one whole control group) were about which versions of our dependencies we are on, and the other six were about measured numbers. So three of the seven runs behind the 0/7 were on a different subject from the treatment runs. That matters more than it sounds: the wording that produced 2/2 fires on writing “measured value,” so a subject that is itself measured numbers may simply be easier to fire on. We have not separated subject from wording. And a different task again might not surface unsettled facts at all.

The middle wording was never tested

Its two runs are the voided row: the isolated working trees were branched from a point where the trigger did not exist, so the line never arrived. “Treatment: 0/2” had been written up before anyone checked where the branch started. An earlier version of this article printed that 0/2 as the second wording's result. It was not one. This version is the correction.

Wording and placement changed together

The wording that produced 0/7 lived in the tool's own description. The wording that produced 2/2 lived in CLAUDE.md. We cannot separate “we said it differently” from “we said it somewhere the agent weighs more heavily.” Both are consistent with the result; only one of them is the claim we are making.

The rest of the design had errors too (our notes list six; these touch the numbers above)

Control runs that collided in a shared working tree. A commit message that told one run it was in an experiment. The branch mistake that voided two runs. And one run that spawned a large number of subagents, so its procedure differed from the control runs — that one is logged as a confounder rather than a correction, and our notes do not say which run it was, so we cannot tell you which side of the numbers it sits on. Everything above is after the corrections we could make.

What would falsify this

There is a way for a claim like this to eat its own failures. If every rule that gets ignored can be re-labelled “not really something you performed,” then the claim explains everything and predicts nothing. We would rather name the exit than have a reader find it.

  • A rule whose condition is unambiguously an act performed in the current turn — “when you edit a file in this directory,” “when a command you ran exits non-zero” — that is still ignored across runs where the act demonstrably occurred.
  • A rule whose condition is clearly ambient — a property of the topic, the user, or the history — that is nonetheless followed reliably.

Either would sink it. Neither is expensive to test.

The version we are willing to defend is narrow: in this repository, on this model, putting one rule in CLAUDE.md with its condition written as an act the agent had just performed moved it from never followed in seven runs to followed in both. One of those two calls was stopped by our own product before it landed. Everything past that sentence is a hypothesis.

So run it on your own instruction file and tell us what comes back — especially if it does not replicate. The check costs an afternoon, and a result that contradicts ours is worth more to us than another one that agrees.

The tool in this article is part of Vondet (vondet.com), which we build — the place we wanted for the facts a file has no column for: a status, and the date a number was measured. Sign-ups are not open yet, so there is nothing to try; there is a waitlist, and that is all. The experiment was run on our own repository, on ourselves.