SQS

Introduction

Amazon Simple Queue Service (SQS):

• A managed message queuing service to decouple and scale microservices, distributed systems, and serverless applications

• Use Case:

  • A write buffer for DynamoDB

Feature

• Message is 256 KB in size (use a pointer to S3 for larger messages)

• Message is kept in queued from 1minute ~ 14 days, in 4 days by default.

• Can be retrieved programmatically with SQS API

• SQS Types (can not modify dynamically):

  • Standard Queue (default)

    • Nearly-unlimited number of transactions per second

    • Delivery order loosely guarenteed

  • FIFO Queue

    • Limit to 300 messages/s without batching, 3,000 messages/s with batching.

      • If more messages, use Kinesis Data Streams.

    • Delivery order guarenteed

• Visibility time out mechanism

  • If a job finishes in time, the message is consumed

  • If a job doesn't finish in time, the message will become visible again.

  • Maximum of visibility time out is 12 hours

  • Changing the visibility timeout for a message

• Polling

  • Short polling (default)

    • If no message left in the queue, return the response.

  • Long polling

    • For event source mapping (ex. attaching to Lambda Funciton)

    • Can specify batch size from 1~10

    • If no message lef in the queue, it waits for the "Receive Message Wait Time" or a message comes to return the response.

    • Change "Receive Message Wait Time", can be 0~20 seconds.

• DLQ usage

  • Set up on SQS level (not Lambda level because DLQ for Lambda is only for async invocations)

  • Or use a Lambda destination instead

• Tracking with application level must be implemented.