arxiv-source-first-paper-ingest
arXiv source-first paper ingest
Self-Evolving skill. If a step here fails or the tooling drifts, fix this file in the same change. The Post-Execution Reflection at the bottom says what to check.
The whole idea in one line: for an arXiv paper, the PDF is a rendering of something you can get losslessly, so do not read the rendering.
arxiv.org/e-print/<id> serves the authors' own LaTeX. It carries the exact formulas, their
\labels, and — critically — which formulas are stated propositions rather than intermediate
steps inside a proof. None of that survives into a PDF, so no vision model can recover it.
Measured on one real paper, OCR of the PDF reached 0.958 mean token similarity to the source and
only 49 % of its formulas compiled, against 98 % for the source. Full numbers, including a
systematic model defect that similarity cannot see, in
../../references/OCR-VERSUS-AUTHOR-LATEX-GROUND-TRUTH.md.