AWS back-of-envelope latency measure
Network
- Roundtrip latency between two instances in the same AZ: sub-millisecond
- Roundtrip latency between two instances in a different AZ in the same Region: single-digit millisecond
- Data Transformation costs money. DynamoDB is cross-region replication by default so I assume normally it won’t cost money
- Data in/out ALB may not cost money if they are in same VPC (not sure)
Service
- DynamoDB: Single-digit millisecond
- RDS: Single-digit millisecond
- Redis: sub-millisecond
E2E
The following data come from real traffic; and since our ECS is multi-region so the data includes traffic between different AZs.
- EC2 to Salesforce: p95 1s p50 0.6s (upsert)
- EC2 to DynamoDB: p95 around 20ms p50 20ms (read&write)
- EC2 to RDS: p95 around 10ms p50 6ms (read&write)
- EC2 to Redis: p95 0.7-0.8ms p50 0.5ms (read&write)
If the dataset is relatively small (The queries are not I/O bound), especially if it’s cross-region, it may not matter at all to use cache or not since most of the time is spent in network latency anyways.