If you need a resource to be present from another system, but you don’t know when it will be available, it would be nice if the system whose resource you needed would send a notification to let you know when the resource is there, but if that is not the case, then you might want to try polling for the resource in a background job. This has a few benefits:
- By using a background job, you are not consuming valuable primary application processing resources to handle this task.
- If the resource you need is not present at the time of polling, you can fail the job which will put it back in the queue to be tried again later.
- When a failed job is put back in the queue it will be scheduled using an incremental backoff, so each time it fails it waits a longer before retrying the job. Background job processors like Sidekiq provide incremental backoff functionality with their default configuration.