Interface SyncOptions

The run's documents, shared the same way: a URL one source fetched is not fetched again.

Re-extract even when nothing changed.

Fetch from a server whose robots.txt disallows it. Two Länder publish their Drucksachen openly and disallow every client; this is the operator's decision to make. The pipeline checks every document URL against its host's robots.txt before fetching it, whichever source produced the URL, and the override is never silent: the report warns once per host.

Download no documents. A new record gets metadata only and abstains on qa; a stored one is re-extracted from the documents it already archived, so a metadata correction lands and nothing it held is lost.

Injected clock — the only place the pipeline reads time (retrieved_at).

Called once discovery is done, with the number of Anfragen the run will handle — the total a progress display counts towards. Not called when discovery fails.

Called after each record, for progress output.

The run's robots.txt policy, shared by every source of one syncSources run so a host's file is read once for all of them. Built for this source when absent; one that is given must have been built with the same ignoreRobots.

Stop early: checked before each ref, so the ref in hand is finished, the catalog is saved and the report says interrupted. ka sync aborts it on the first Ctrl-C or SIGTERM.

The disk-space guard (spaceGuard from lib-store). Given, a run whose documents to fetch would not fit — estimated from what the source already archived — is refused with StoreError after discovery and before the first download, and a run stops between two refs, like an aborted one, once a volume drops below the floor (SyncReport.lowSpace).