why tasks live in the past
When you run /evolve (or /prune), it does not test your latest code. It rewinds to the exact moments things were broken and asks: can my skills solve these now? Here's why that's the only thing worth measuring.
The code travels back to each bug ยท the skills stay fixed at today.
Every time you fixed something, /harvest pinned a task to the commit just before the fix: the last moment the bug was alive.
base_sha: the commit right before the fix). At HEAD (today) they're all already fixed, so
there's nothing left to solve there.
It carries today's skills backwards to each bug, and checks whether they can fix it.
CLAUDE.md that were committed at the old
commit are replaced by today's before every run, so the old code never runs old instructions. All of it
happens in throwaway clones: your working copy is never touched.
Because you are not checking your code. You are checking whether Claude can solve problems.
At today's commit, every past bug is gone. Claude is handed a working repo and has nothing to do, so every task "passes" and you learn nothing about your skills.
Result: a perfect score that means nothing.
At each bug's own commit, the problem is live again, and every run proves it first: the task's check must fail before Claude starts, or the run is thrown out. Now the score is real: it shows whether your skills actually help Claude fix things it couldn't before.
Result: a score you can actually trust.
Optionally, Cortex can ask a calibrated model (Jev) what to try. It never asks it what the result was, and the two live on very different time scales.
bin/score.sh,
preflight.sh and sweep.sh have no Jev import, no Jev field and no Jev flag, and the
test suite asserts it. Turning the judge on cannot change a KEEP into a KILL.
Cortex isn't asking "does my code work today?"
It's asking "do my skills help Claude solve real problems?"
So it re-opens old, already-solved problems as fresh challenges, each at the moment it was hard, and measures how today's skills cope.