Developers familiar with Relational Database design often find themselves initially drawn to the familiar territory of normalization when designing data models for DynamoDB. This instinct typically leads them towards what’s known as multi-table design, where each entity or relationship resides in a separate table.

On the other hand, DynamoDB’s schemaless nature encourages a different approach: single-table design, where all entities and relationships coexist within a single table. However, it’s worth noting that these two designs represent extremes on a spectrum, rather than strict boundaries.

According to the official documentation, single-table design is often recommended. This article explores the advantages of single-table design based on practical experience.

Our project predominantly utilizes single-table design, largely influenced by 《The DynamoDB Book》, authored by advocate Alex DeBrie. Despite our commitment to single-table design, we still manage more than a dozen tables, albeit with a focus on storing related entities together.

Challenge

In a recent project, we were assigned the task to route calls to the last agent who interacted with the caller.
This requirement may seem fundamental, but it proved to be more complex than anticipated,
especially considering its integration into Salesforce via Salesforce Service Cloud Voice.

Recently, we introduced AWS Cloud Map for service discovery, primarily to retrieve queue URLs. However, after deployment, we encountered intermittent errors weeks later, logged as getaddrinfo EMFILE events.ap-southeast-2.amazonaws.com. Not all requests triggered this error, indicating a selective issue.

Upon inspection, it became apparent that we were facing a socket timeout problem, a known issue in our setup. The remedy was simple: reusing our existing agent.

Problem

In our system, queue processors must implement idempotency to prevent the double-processing of messages. Duplicate messages may arise in the following scenarios:

1. Scheduler and Message Producer: The scheduler or message producer may be triggered multiple times, occasionally rerunning due to timeouts.

2. Queue Management: If a lambda instance times out while processing a message, another instance may retrieve the same message if the visibility timeout is not properly set.

This can have terrible consequences. We aim to avoid sending duplicate emails or messages to our customers, not to mention inadvertently delivering duplicate gift cards.

So a generic idempotency mechanism is required.