Packet Batching

Proxylity handles network traffic and transforms it to AWS API calls to deliver your packet data to your serverless resources. When the rate of incoming packets is high enough, making individual per-packet calls (as AWS API Gateway does for HTTP requests) becomes unnecessarily expensive. To avoid this, UDP Gateway supports batching of packets based on count, time and aggregate size so multiple packets are included with each AWS API call. The net result for you is reducing the cost of hosting high traffic services considerably (the exact amount depends on the specific AWS service capabilities).

The available batching options are:

Option	Description
Time Window	The maximum window of time to accumulate packets before sending a batch to the destination. Adjust this setting to longer periods to increase batch size, or reduce it to improve the latency of responses (if needed).
Count	The maximum number of packets to accumulate before sending a batch to the destination. For destination services with constraints on batch size (like DDB), adjusting this setting to match may be a cost and performance optimization.
Size	The maximum size in bytes of packets to accumulate before sending a batch to the destination. For destination services with constraints on payload size (like SNS and StepFunctions), adjust this setting to be less than or equal to the limit.

Batching is configured separately for each destination, allowing you to balance latency and cost on a per-resource basis. So, for example, you may configure a Lambda function or StepFunctions Destination handling requests with a 100ms batching window to ensure timely responses, and configure a Firehose or S3 Destination receiving log messages with 1000 messages per batch to minimize cost.

Proxylity uses batching with AWS APIs where available. CloudWatch Logs, SNS, SQS and DynamoDB all support batch APIs (SendLogEvents, SendMessageBatch, PublishBatch and BatchWriteItem, respectively). Keep in mind that the maximum number of operations per batch varies by service. So, for example, if your destination is configured for batching up to 100 items in Proxylity and DynamoDB is the destination service we're delivering to, a batch of 100 items will be broken into 4 calls to the DynamoDB API with 25 packet records each. For SNS it would be 10 packets, etc.

Where possible, batches are delivered to AWS services without batch APIs by combining them into a single payload. For example, when delivering to AWS Lambda the batch is provided at the function as a JSON array containing all of the packets in a single execution. Similarly, for Firehose and S3 Destinations the items are concatenated with newline delimiters into a single large string for delivery. In such cases, configure the batch size so that the total size never exceeds the service payload size limit (e.g. 1000 kiB for Firehose).

Optimizing Batching

The first step to optimize batching for your workload is to determine the acceptable latency for responses and/or processing:

Where responses are required and have timing constraints, the Time Window setting should be adjusted appropriately and count set appropriately based on workload or Destination service limits. For example, if responses must be returned withint 100ms the time windows should be set to 100ms or less to ensure timely processing.
If no responses are required, but processing is time-sensitive, the Time Window setting should be adjusted to a value that meets your latency needs.
For use cases without response or time constraints, the Count and Size settings should be adjusted to achieve larger batches without exceeding service limits.

Monitor the performance and costs over time, and fine-tune the settings as necessary to find the best balance between cost savings and processing speed.