How do you test long-running async operations (e.g. queued jobs) at the API level?

Async operations break the request/response contract: the API responds 202 Accepted before the work finishes, and you have to wait for the actual result. Three patterns cover most cases.

Pattern 1 — Poll a status endpoint. The most common. The async operation returns an id; you poll /jobs/:id/status until done:

async function waitForJob(jobId: string, timeoutMs = 30_000) {
  const deadline = Date.now() + timeoutMs;
  while (Date.now() < deadline) {
    const res = await request.get(`/jobs/${jobId}`);
    const body = await res.json();
    if (body.status === 'completed') return body;
    if (body.status === 'failed') throw new Error(`Job failed: ${body.error}`);
    await new Promise((r) => setTimeout(r, 1000));
  }
  throw new Error(`Job ${jobId} did not complete within ${timeoutMs}ms`);
}

test('async export completes', async () => {
  const start = await request.post('/exports', { data: { type: 'csv' } });
  const { id } = await start.json();
  const result = await waitForJob(id);
  expect(result.outputUrl).toMatch(/^https:\/\//);
});

Use exponential backoff if jobs are slow; fixed 1s polling for short ones.

Pattern 2 — Webhook callback. The API calls back to a URL you provide when done:

const received: any[] = [];
const server = startReceiver((req) => received.push(req.body));
const port = server.address().port;

await request.post('/exports', {
  data: { type: 'csv', callback_url: `http://localhost:${port}/cb` },
});

await waitFor(() => received.length > 0, 30_000);
expect(received[0].status).toBe('completed');

Useful when polling isn't supported. Adds the complexity of signature verification (see the webhook question).

Pattern 3 — Inspect the queue / event store directly. If your test infrastructure has access to the underlying queue (SQS, RabbitMQ, Kafka), check the message landed and was consumed. More invasive; couples tests to infrastructure.

Pattern 4 — DB inspection. Last resort — poll the database for the expected state. Bypasses API contracts, harder to evolve. Use only when no API hook exists.

Cross-cutting principles:

1. Always set a timeout. An async test that hangs forever blocks the suite and tells you nothing. 30 seconds is a reasonable default; bump for known-slow operations.

2. Test failure paths. The job can fail; assert that the failure is reported and the right errors appear in the status. Force failures via fixture data ("export with id 'fail-test' always errors").

3. Test concurrency. Two jobs of the same type running simultaneously — do they finish independently, or interfere?

4. Test idempotency. Submitting the same async job twice — do you get one job, two jobs, or an error? Document the expected behaviour and test it.

5. Test cancellation (if supported). Submit a job, immediately cancel, assert it doesn't complete and final status is "cancelled."

Anti-pattern: Thread.sleep(60000) to "wait for async to complete." Too short → flake when the job is slow. Too long → suite is unbearably slow. Polling with timeout is always better.