Day 75 of 80 · Tuesday · Learning

Seen is not done - the orphan sweep that recovered nine documentsConcept

A document marked 'seen' is not a document whose ink was applied. On 9-17 nine of yours were found stranded that way; the fix now sweeps for unfinished returns every poll.

Catch-up progress

75/80

Why this matters to you

Thirty-three lead decisions and twenty-one screen cards sat on disk for up to fifteen days because rm_pull recorded the pull as success. Without the sweep the next processor failure strands the next batch the same way.

rm_pull.py does two things per poll: it pulls a done-tagged document's ink from the cloud and composites it into a PDF under FOR TOM/PROJECTS/REMARKABLE/returns/<date>/, then launches rm_process.py detached (output to /dev/null) to read the ink and apply it. The moment the PDF existed, the document was written to the 'seen' list so it would not be pulled twice.

That ordering is the bug. 'Seen' was recorded at the trigger (a pull happened), not at the side effect (the ink was applied). When rm_process died on arrival - a usage limit on 9-11, another run holding the lock on 9-17 - nothing failed loudly and the document was never offered again. Automation-hygiene rule 1 names exactly this: verify the side effect.

The fix (PDB commit 38a3cbb) does not touch the seen-list. It notices that rm_process moves a PDF into _processed/ only after success, so any return PDF still sitting in its dated folder is unfinished work by definition. Each poll sweeps for those, redispatches them, counts attempts per file, stops at three, and raises one deduped Things task naming the stuck files.

Worked example

Find unfinished returns the same way the sweep does.

cd "/Users/tom/Claude/FOR TOM/PROJECTS/REMARKABLE/returns"
find . -name '*.pdf' -not -path '*/_processed/*' | sort

# what the sweep will do about them
python3 /Users/tom/Claude/PDB/scripts/rm_pull.py 2>&1 | grep -i orphan

Every PDF outside _processed/ is a return that was pulled but never finished - the definition the sweep uses.
A normal poll prints '0 orphans'. Anything else is queued for redispatch, up to three tries.

▶ Do it now

Run the find above. Expect an empty list today; the 9-17 backlog was cleared by hand.
Open PDB/tests/test_orphan_sweep.py and read the three historical shapes it covers (dead processor, held lock, grace window) and the four things it must not sweep.
Plant a fake: copy any PDF from a _processed/ folder up one level, run rm_pull.py, watch it dispatch, then delete the copy.

Gotchas

Manual-hold documents (the Codex registry) are skipped on purpose; the sweep is the legacy Claude route only.
Three failed attempts stop the loop and file a task - it will not retry forever, so read the task text.

Go deeper: The fix commit

One-card takeaway

Record success where the work lands, not where it starts.

← Prev Index Next →