SESSION-NOTES-2026-09-19.md

Session notes

Session notes — 2026-09-19

What was fixed

Fact path was structurally unreachable (now fixed)

/query in main.py checked corrections before facts. query_corrections.get() uses a two-tier match — exact normalized string, then keyword-overlap fallback with _KEYWORD_OVERLAP_THRESHOLD = 0.70 plus light stemming. That made every phrasing of a fact-shaped question ("what is the X of Y", "the X of Y", "X of Y") hit a correction row before the fact check could run.

Commit 42956c3 ("query: lookup confirmed facts before extraction") was correct in spirit but landed the fact check after the correction check. Result: the entire facts pipeline (facts.py, extract.py, backfill_facts.py, 24 confirmed rows, planned SOZ review UI) was built and never fired once on the query path.

Fix applied: a fact-path preflight inserted at the top of the /query handler, above the corrections block. It calls _parse_fact_query and, on a clean (entity, attribute) parse with exactly one facts.lookup() hit, returns immediately with answer_qtype="fact" and _fact_hit=True.

Non-fact queries fall through to corrections unchanged. The original fact check further down the handler is now dead-but-harmless; can be removed in a later cleanup pass.

Verified end-to-end:

• "what is the boiling point of mercury" → qtype=fact, _fact_hit=True
• "what is the capital of france" → qtype=fact, _fact_hit=True
• "What is AAPL stock" → stock pipeline, _fact_hit=False

Handoff corrections

Three claims in the previous handoff need updating:

1. "24 confirmed facts include samarium" — samarium is not in the facts table. The 18 distinct entities are: bohrium, boron, carbon, france, gold, hassium, helium, hydrogen, krypton, lithium, mendelevium, mercury, oxygen, platinum, seaborgium, silver, titanium, tungsten. The 5 distinct attributes are: atomic_number, boiling_point, capital, chemical_symbol, melting_point. Correct the entity list.

1. "Query engine looks up confirmed facts before extraction" — true, but the fact check is now also before corrections (see fix above).

1. facts.lookup latency — confirmed at ~25ms steady-state, ~340ms first call. Consistent with the known per-call connection overhead.

New findings this session

• query_corrections.py is paraphrase-tolerant by design: exact normalized match, then keyword-overlap fallback at 0.70 threshold with light Porter-lite stemming and stopword stripping. This is working as documented and is not a bug — but it means the corrections table shadows any table that doesn't run first.

• Corrections table is polluted: /debug/corrections reports 42 rows for 24 facts. Many are paraphrase variants generated during backfill (now redundant with the fact path). At least one is junk (Probably a lot like jeffery epstiens...). Optional cleanup: POST /correction/forget for the paraphrase duplicates, and definitely forget the junk entry.

• Gold has boiling_point in facts but not chemical_symbol. Some entities have incomplete attribute coverage from the extractor. Worth a coverage report before the next backfill.

Files created this session

• ultimate_test.sh — 12-section read-only diagnostic. Runs in ~30s. Now correctly reports the fact-path marker.
• ultimate_probe.py — Python-side probes called by the shell script.
• SESSION-NOTES-2026-09-19.md — this file.

Next task: SOZ review UI for facts_pending

facts_pending has 0 rows. Two build directions: A. Add a "generate pending" endpoint that runs extract.py over recent documents → real candidates → build UI against live data. B. Seed 5 fake pending rows → build UI scaffold → verify confirm/reject/ edit → clear seeds.

A exercises the full loop; B is faster to scaffold.