I told an AI writer, in capital letters, never to use em dashes. The next three articles it wrote had three each.
The setup is ordinary. It's a writing tool that drafts articles for a client, and it runs on a stack of plain-English instructions: a voice guide, an editorial spec, a posting schedule. All of it goes to the model on every run. One line in the spec bans the em dash outright. It doesn't say "use fewer" or "avoid where possible." It's a hard rule in caps, with one exception for text a person submitted themselves.
Three articles, nine em dashes.
I counted my own
My first instinct was the usual one: the wording must be too soft, so make it stronger.
I'd already done that once. An earlier version of the spec said no em dashes as a tic, which is perfectly clear to a person and accomplished nothing.
So instead of rewriting the rule a third time, I counted the character in the documents that carry the rule. Twenty-three.
Fourteen of them were in one guide, and I'd typed every one of them myself, in the same sitting where I was annoyed the model wouldn't stop using them.
The prompt was demonstrating the exact thing it banned. One line said don't, and twenty-three said this is how we write here. The model went with the majority, which is about what I'd do if someone handed me a style guide full of the thing it told me to avoid.
Rules in a prompt are cheap and examples are expensive. When they disagree, the examples win, because they're concrete and they're everywhere, and the rule is one sentence.
The files weren't the whole prompt
I cleaned the documents, got the count down to a handful, and the output went to zero. I measured it and moved on. Then the em dashes came back.
The second pass found where they were coming from, and it wasn't any file I could open. The prompt is assembled at runtime, and part of it is a list of candidate links the writer is allowed to use, rebuilt fresh on every run. Each candidate was formatted as title, em dash, web address, and a typical run offers fifteen of them.
Fifteen, plus two in the surrounding instruction, is seventeen em dashes in the same message as the ban, generated by my own code milliseconds before the model read the rule against them. No amount of editing the documents was ever going to fix that.
What I check now
If a model keeps breaking a rule you gave it, check your own instructions before you rewrite the rule.
- Search your instructions for the thing you banned. A word, a character, a phrasing, a heading style. Count it in everything you feed the model. If the count isn't zero, that's your bug.
- Count in the assembled prompt, not the source files. Print the exact text the model receives, with a realistic payload loaded. Mine was clean on disk and full of them at runtime.
- Write the rule in the style the rule demands. If the instruction says short sentences, write it in short sentences. The example does more work than the instruction.
- Pin it with a test. Mine asserts exactly one em dash in the assembled system prompt and zero in the built article prompt. The one survivor is in the line that bans the character, because that line can't name it without using it once.
This applies to every style instruction you give a model, punctuation or otherwise. You're teaching through two channels at once, the rules and the examples, and usually you only wrote one of them on purpose.
This post has no em dashes, by the way. It took three passes, and it's a lot easier when the document you're reading follows its own rule.