Blog / AI agents

Extract the Fact, Judge in Code

Ask a model whether the budget is big enough and it decides, quietly, what big enough means.

Kershey Cariño · · 4 min read

In short

Every AI decision is two operations: pulling a fact out of messy input, which needs a model, and applying a rule to that fact, which doesn't. Ask for the conclusion and the model does both, and the number inside the rule stops existing anywhere you can read, test, or change.

Ask a model "is this lead's budget big enough?" and it will answer. It will also, quietly, decide what big enough means.

Somewhere in that answer is a number. Ten thousand, maybe. It isn't in your code, it isn't in a config file, and it isn't in a doc. It sits somewhere between your prompt's wording and the model's weights, and nobody can tell you what it is.

every AI decision is two operations

One operation turns messy input into a fact. That part is genuinely fuzzy, and a model is the right tool for it. The other applies a rule to that fact, and there is nothing fuzzy left in it.

BOTH IN ONE   "Is this lead's budget big enough?"
              the model extracts the budget AND applies your rule.

SPLIT         "Extract the stated budget as a number, or null."   <- model
              if budget >= 10_000:                                <- code

An unsplit decision is a business rule with no home.

the test for where the seam is

One question finds the seam: if I already knew this fact, could I write the rule? If yes, the fact is the model's job and the rule is code's. Cut there. If no, the judgment really is fuzzy, and that happens less often than it feels like it does.

  • "what budget did they state?" → budget >= 10_000
  • "what date did they mention?" → date < today + 30
  • "which product did they name?" → product in the catalogue
  • "how many seats did they say?" → seats >= 5 is the enterprise tier

Look at that last one. The model gives you seats, a fact that will be true forever. Code gives you tier, a policy that changes the next time sales reorganises.

Kershey and a small robot at a balance scale outside a courthouse, Kershey's hand on the beam while the robot holds one of the pans, a check mark and a gavel floating beside them.
The model can carry the weight. What counts as heavy enough is yours to set.

This isn't the opposite of letting a model choose from a fenced set. When nobody can write the rule down, fence the options and let it pick. The moment you can write the rule down, writing it down is the job.

the number that lives nowhere

The reason to cut here isn't that code is more careful. It's that a rule you wrote down has an address, and a rule the model inferred doesn't.

the questionmodel decidessplit
who owns the thresholdnobodycode, one line
can a non-engineer change itnoyes, it's a number
can you test it without a modelnoyes, the rule is a pure function
the business moves it to 25,000re-prompt, re-eval, re-shipedit a constant
why did it decide that?ask the model, hoperead the line

The first row is the one that matters. If the model decides "big enough," the number ten thousand exists in no file anyone can open. Nobody can tell you what it is, nobody can change it, and nobody notices when it moves.

what a real one looks like

Most decisions aren't one fact and one threshold. They're several facts and a small formula, which is exactly where the split pays.

EXTRACT   { budget: number|null,        timeline_days: number|null,
            company_size: number|null,  is_decision_maker: bool }

JUDGE     score = 3*(budget >= 10_000)
                + 2*(timeline_days <= 30)
                + 1*(company_size > 50)
                + 2*is_decision_maker

          if score >= 5: priority

Now the weights are visible. Sales can argue about them in a meeting, you can A/B them, and you can explain any single decision by printing four facts and the arithmetic. None of that exists when the model just answers "yes, prioritise."

Extract facts. Decide policy. Facts are stable, policy churns, so put the churning half where changing it costs one line.

What if the judgment needs the whole message?
Some don't decompose. "Is this abusive?" has no fact-plus-threshold underneath, because the extraction is the judgment. You can sometimes half-split it by extracting what appeared and ruling on the combination, but that's a taxonomy decision, not a threshold.
What if there's no clean fact underneath?
"Is the budget big enough" splits cleanly. "Is this lead qualified" often doesn't, because qualified isn't a fact about the lead. It's a decision the company hasn't made yet, and no prompt will make it for them.
What if the rule needs forty extracted fields?
Then extraction gets brittle and you've hand-built a worse model. Take it as a signal to reconsider the shape, not to push harder on it.

The split isn't about trusting the model less. It's about the number having somewhere to live: a line someone can open, argue with, change, and diff. Ask for the conclusion and you get an answer. Ask for the fact and you keep the rule.

Seen in a real build

Want it in a real build? A Slack request where the fuzzy half is telling a real ask from the noise, and the team's rules build everything after it. See how MARA · Auto-Ticket Creation is built.

Got something to build?

Got a rule that lives nowhere but inside a prompt? Write It Down! — contact Kershey, or See the Case Files.