cat-test-incremental-loads
Test incremental loads
A full load is right or obviously wrong; an incremental load is wrong quietly, and its own
log says succeeded. The tests look at the data on both sides. Every fact is on the
documentation page it links; the same page as markdown is at the URL with index.md
appended, which is the form to read when your fetch tool summarizes HTML.
Three levels
Detect: counts per day over a window longer than the longest replication delay, after every load. Detailed: the rows on a short window, with a key, so the message names them. Full: the whole table, rarely, the only test that finds a row missed a year ago. The failure modes, the three tests, and the balance between them: https://docs.justcat.it/how-to-guides/test-patterns/test-incremental-loads/
The comparison mechanics the levels are built on, rungs 2 to 4 of the ladder: https://docs.justcat.it/how-to-guides/test-patterns/compare-data-across-systems/