Child pages
  • Resumption token
Skip to end of metadata
Go to start of metadata

See: Repositories that implement resumptionTokens must do so in a manner that allows harvesters to resume a sequence of requests for incomplete lists by re-issuing a list request with the most recent resumptionToken. The purpose of this is to allow harvesters to recover from network or other errors that would otherwise mean that the list request sequence would have to be started again.

The protocol does not mention the life span of a token. A token life span is the time a repository keeps the token stored in memory, along with the resume information. When the life span is too short, the repository does not give the harvester a reasonable time to return to complete the harvest. When this happens the repository does not comply with the protocol - see above: "must do so in a manner that allows harvesters to resume...".

Best practice: a reasonable time for a token to be kept alive is at least twenty four (24) hours. This depends on the size of the repository and the speed of the loading process and thus the resumption token life span should hold for long enough to transport the batch within that period of time.

Along with this life span there is an optimal batch size - see section "Harvest batch size".

Another aspect of the resumption token usage is the optional completeListSize attribute. This should deliver the total size of documents of the response and thus this information can be used during the harvesting process and could be compared with the total result size for control reasons (for example, is the harvest complete or broken?). Besides that, the information could be useful for maintaining the harvesting process in order to estimate the time needed.

A resumption token in an OAI response could look like this (the attributes expirationDate, completeListSize and cursor are optional):

<resumptionToken expirationDate="2008-07-14T23:00:24Z"
completeListSize="983" cursor="0">514284267</resumptionToken>

  • No labels