False green — the sweep that reads OK while it's deadHands-on

Three systems describe jobhunt-sweep right now. The alert ledger says open incident, count 3. launchctl says exit status 1. Your health check says OK. The green one is wrong, and it's the one you read.

A false negative in monitoring — a real failure reported as healthy — is worse than having no monitor at all. With no monitor you stay appropriately suspicious. With a green dashboard you stop looking, and the failure gets the benefit of your trust. This is why your own Automation Hygiene rule 1 says verify the side-effect, not the trigger: you already knew checking the wrong thing was the danger. This is a subtler version — checking the right thing, with the wrong threshold.

The mechanism is banal. Weekly jobs need wide freshness windows, because a weekly artifact is legitimately days old most of the time. But a window wide enough to tolerate normal age is wide enough to swallow a total failure. Between the last success and the window's edge, a dead job and a healthy one are indistinguishable to a file-timestamp check. Your sweep has failed three times and is still inside its window.

The structural fix isn't a tighter threshold — that just trades false greens for false alarms. It's cross-referencing. You already maintain alert-state.json, a ledger of what's currently broken, with dedup and auto-resolve. The health checker doesn't read it. Two monitors that don't talk will eventually disagree, and when they do, the optimistic one wins by default because it's the one that stays quiet.

Worked example

The three sources, then the reconciliation:

# Source 1 — the health check (optimistic)
python3 PDB/scripts/job_health_check.py | grep jobhunt
#   OK   jobhunt-sweep: artifact fresh (2026-08-10T08:02)

# Source 2 — the alert ledger (knows the truth)
python3 -m json.tool PDB/data/alert-state.json
#   "jobhunt-sweep": { "opened_at": "2026-07-27", "count": 3, ... }

# Source 3 — launchd's own exit status
launchctl list | grep jobhunt-sweep
#   -   1   app.zorc.jobhunt-sweep      <- middle column is last exit code

# The reconciliation, in job_health_check.py:
def open_incidents():
    import json, pathlib
    p = pathlib.Path('/Users/tom/Claude/PDB/data/alert-state.json')
    return set(json.loads(p.read_text())) if p.exists() else set()

# then, wrapping every registered check:
#   ok, msg = check()
#   if ok and name in open_incidents():
#       ok, msg = False, f"{name}: open incident in ledger — {msg}"

• Six days old passes because the weekly window is wider than six days. The check isn't buggy; its threshold cannot distinguish 'not due yet' from 'dead since July'.
• The ledger has the answer and has had it since Jul 27. count: 3 means it has re-fired twice since — this is a job failing repeatedly, recorded, and reported green.
• launchctl's middle column is the last exit status. 1 means the last run failed. A third independent source, also disagreeing with the green.
• The wrapper inverts the default: an open incident overrides a passing check. Optimism now has to be earned by the ledger being clean, not assumed by a timestamp.

Gotchas

• Name the checks and the ledger keys identically or the cross-reference silently matches nothing — which fails open, exactly like Monday's settings.json.
• Auto-resolve has to clear the ledger entry on success, or a fixed job stays red forever and you learn to ignore red. Confirm the resolve path still works after this change.
• Don't fix this by shortening the window to a day. A weekly job legitimately has a six-day-old artifact most of the week; you'd get a false alarm every Tuesday.