Since the run began, discovery included.
Time spent extracting records, and writing the records.
Time spent writing the index postings and catalog rows, once per batch
(CHECKPOINT_MS). It was inside storeMs, and was most of it (issue #30).
Requests sent again at once on a new connection, the kept-alive one having been closed by the server; not retries.
Requests sent (attempts and redirect hops), and how many were retries.
The retries by reason: a 429/503, a timeout, a failed connection, anything else (issue #31).
429/503 answers: the upstream asking to slow down.
OptionalupstreamTime inside one request, on average and at the 95th percentile; absent before the first.
OptionalupstreamTime spent waiting before requests: pacing (--min-host-interval, a source's floor) and retry backoff.
Where a run's time went (issue #14): waiting on the upstream, waiting to be polite, or extracting and storing. A slow sync could not tell these apart without
ps,iostatandnettop. Upstream numbers are the engine's (EngineMetrics) since the run began; the rest are the pipeline's, on its clock (SyncOptions.now).