Constant BLOCK_VERIFY_TIMEOUT

pub(super) const BLOCK_VERIFY_TIMEOUT: Duration;

Controls how long we wait for a block verify request to complete.

This timeout makes sure that the syncer doesn’t hang when:

• the lookahead queue is full, and
• all pending verifications:
  • are waiting on a missing download request,
  • are waiting on a download or verify request that has failed, but we have deliberately ignored the error,
  • are for blocks a long way ahead of the current tip, or
  • are for invalid blocks which will never verify, because they depend on missing blocks or transactions.

These conditions can happen during normal operation - they are not bugs.

This timeout also mitigates or hides the following kinds of bugs:

• all pending verifications:
  • are waiting on a download or verify request that has failed, but we have accidentally dropped the error,
  • are waiting on a download request that has hung inside Zebra,
  • are on tokio threads that are waiting for blocked operations.

Correctness

If this timeout is removed (or set too high), the syncer will sometimes hang.

If this timeout is set too low, the syncer will sometimes get stuck in a failure loop.

We’ve observed spurious 15 minute timeouts when a lot of blocks are being committed to the state. But there are also some blocks that seem to hang entirely, and never return.

So we allow about half the spurious timeout, which might cause some re-downloads.