Message Queue Redelivery Causes Duplicate Processing

The order-processing message queue uses at-least-once delivery semantics. When a consumer fails to acknowledge a message before the visibility timeout expires — due to slow processing, a crash, or a brief network interruption — the broker redelivers the message. Because the consumer has no idempotency check, the redelivered message is processed a second time, creating a duplicate order record with a different ID but identical contents.

HighIntermediateAPI testingManual testingExploratory testing

// UNDERSTAND

// Symptoms

A single order submission creates two order records with different IDs but identical contents
Application logs show the same message_id processed twice by the order consumer, with timestamps several seconds apart
The duplicate record appears seconds or minutes after the original — consistent with a broker redelivery delay — rather than immediately as in a double-click duplicate
Duplicate records are rare and intermittent, occurring only when consumer processing time approaches or exceeds the broker's visibility timeout
The two records have created_at timestamps seconds or minutes apart, unlike a client double-submit where both arrive near-simultaneously

// Root Cause

The message queue uses at-least-once delivery: the broker redelivers any message whose ACK is not received within the visibility timeout. The consumer has no idempotency check — it processes every received message regardless of whether the same message_id has already been handled. A redelivered message triggers the full order-creation logic again.
The absence of a processed_messages table (or equivalent deduplication store keyed on message_id) means the consumer cannot detect that a given message was already processed. Processing a redelivered message creates a new database row rather than returning the result of the original operation.

// Where It Appears

E-commerce order processing pipelines that consume from SQS, RabbitMQ, or Kafka
Payment processing jobs that consume webhook events from a message broker
Notification services that dispatch emails or push notifications from a queue
Any background job consumer that performs a non-idempotent write without a deduplication check on message_id

// REPRODUCE & TEST

// How to Reproduce

01In the test environment, configure the queue consumer with a visibility timeout shorter than the consumer's processing time (e.g. set visibility timeout to 2 s; slow the consumer to take 5 s per message using an artificial delay)
02Submit a single order via POST /api/orders with a unique identifier in the body (e.g. referenceNote: 'TEST-REQUEUE-001'); note the returned order ID (e.g. 401)
03Wait 15 seconds for the broker to redeliver the unacknowledged message and for the consumer to process it a second time
04Send GET /api/orders and search for records with referenceNote: 'TEST-REQUEUE-001'
05If two order records exist with different IDs (e.g. 401 and 402) but identical contents and created_at timestamps 8 seconds apart, the redelivery duplicate bug is confirmed

// Test Data Needed

An order body with a unique identifiable field (referenceNote: 'TEST-REQUEUE-001') to make duplicates easy to find in the list
A test environment where the queue visibility timeout can be lowered below the consumer's processing time
Access to the order list API or admin view to count resulting records after the redelivery window

// Manual Testing Ideas

Lower the queue visibility timeout to 2 s in the test environment; submit an order and count the resulting records after 30 s
Check application logs for the same message_id appearing more than once in the consumer's processing log — the gap between log entries shows the redelivery delay
Compare the created_at timestamps on the duplicate records — a gap of several seconds indicates broker redelivery, not client double-submit
Force a consumer crash mid-processing by killing the worker process; restart and confirm whether the same message is reprocessed and creates a second record
Test whether a unique constraint on the order's idempotency_key column (if present) prevents the second write when redelivery occurs

// API Testing Ideas

Submit POST /api/orders with body { "referenceNote": "TEST-REQUEUE-001" }; capture the returned order ID (401)
Force a redelivery via the queue admin API: POST /api/admin/queue/requeue with { "messageId": "<original_message_id>" }
Wait 10 seconds for the consumer to process the requeued message
Send GET /api/orders?referenceNote=TEST-REQUEUE-001 and count matching records
Assert exactly one record exists (ID: 401) — a second record with a different ID confirms the consumer is not deduplicating redelivered messages

// Automation Idea

In a test environment with a controllable queue, submit a single order then requeue the same message_id via the admin API. After 10 seconds, query GET /api/orders filtered by the unique referenceNote and assert exactly one record exists. If two records with different IDs appear, the consumer is not deduplicating redelivered messages. Extend the test by lowering the queue visibility timeout below the consumer's processing delay to trigger natural redelivery.

// Expected Result

When a message is redelivered by the broker, the consumer detects the duplicate message_id using a deduplication check and skips reprocessing. No second order record is created — the original record is returned.

// Actual Result (Example)

After the broker redelivers the order message due to an ACK timeout, the consumer processes it a second time and creates a second order record. GET /api/orders returns two records — ID 401 and ID 402 — with identical contents and created_at timestamps 8 seconds apart.

// REPORT IT

Example Bug Report

Title: Broker redelivery creates duplicate order — consumer processes the same message_id twice
Severity: High
Environment: Staging environment SQS-compatible test queue Visibility timeout: 2 s Consumer processing time: 5 s (artificial delay) Test order referenceNote: TEST-REQUEUE-001
Steps to Reproduce: 01Configure the test queue with a 2-second visibility timeout
02Submit POST /api/orders with { "referenceNote": "TEST-REQUEUE-001" }; note the returned order ID (401)
03Wait 15 seconds for the broker to redeliver the unacknowledged message after the visibility timeout expires
04Send GET /api/orders and filter by referenceNote=TEST-REQUEUE-001
05Count the matching records
Expected Result: Exactly one order record with ID 401 exists.
Actual Result: Two records — ID 401 and ID 402 — both with referenceNote: 'TEST-REQUEUE-001' and created_at timestamps 8 seconds apart. The consumer processed the same message twice.
Impact: Duplicate orders cause double fulfilment, inventory miscounts, and potential duplicate charges. In high-volume systems, any consumer restart or slow ACK period produces a wave of duplicate records that are difficult to reconcile after the fact.