Now Available: Lambda MicroVM Destinations

UDP Gateway now supports AWS Lambda MicroVM Images as Destinations. This is an exciting new addition that unlocks significant new capabilities and performance improvements for stateful protocols.

Our existing Lambda integration is a workhorse on our platform. As with all destinations that support replies (Request/Response) we make a deal with your code: we hand it a batch, it returns some replies. The Lambda Event Function integration is limited by how Invoke works, and your code can't count on anything being remembered between one call and the next. It's a good model, and most of what teams build on UDP Gateway fits it. But not everything.

A game server has a world running as a stateful simulation shared by many players. A device fleet has partially-completed firmware transfers and a rate limiter per client. All of it is state, and until now the honest answer using Lambda Event Functions was to put it somewhere else (Valkey, DynamoDB, S3, Momento, etc.).

Today UDP Gateway can deliver packets to AWS Lambda MicroVMs and manage stateful MicroVM instances for you.

Lambda Event Function's promise

Lambda Event Functions reuse execution environments and create and destroy them as needed to serve the request rate. A global (static) variable set during one invocation is often still there during the next. Reuse is an optimization, not part of the contract. An environment can be reclaimed at any time, so a burst of traffic may create new ones with empty globals, and each environment handles one invocation at a time, so concurrency means more environments, each with its own private copy of whatever it caches. Code cannot treat a global as the source of truth.

With functions (including Step Functions) the state necessarily lives in an external store, and processing a batch means read the session, apply the packets, write it back. For many protocols and use cases that works fine. For others, less so. The state is hot, and touched by every packet, and pays the price of a round trip to the store. When two batches for the same session land on different environments at the same moment it becomes a concurrency problem: conditional writes, version counters, retries. It gets complicated and expensive.

Tenant isolation mode, which we added support for a year ago, tightened one half of this. An environment only ever serves one tenant, but there can still be multiple environments per tenant. The environment is still something Lambda can take away between invocations, and your code still has to behave as though it will.

What a MicroVM gives you instead

A Lambda MicroVM is a full Linux environment, built from a container image you provide, that keeps running and keeps its memory between requests (until it's suspended or terminated). Mechanically it's unremarkable, which is the appeal: your application is an ordinary long-running process that listens on a port and receives HTTP POSTs. State is just variables: A session table is a map; An index is an index. No state serialization boundary is required between one packet and the next.

But you don't get automatic durability. A MicroVM lives for at most eight hours, running and suspended time combined, and it can be terminated earlier than that. Treat its memory as a very fast cache of state you could rebuild, not as the only copy of anything you can't afford to lose. If you can't rebuild it, checkpoint it to a store on your own schedule. The difference from the function model is that now you choose when to pay for that write, instead of paying it on every batch or packet.

The destination is the MicroVM image ARN, not a running MicroVM. Which brings us to the configuration options.

Affinity is the gateway's job

If you've read our piece on stateful and stateless protocols, you'll remember sticky sessions: the load balancer sends a given client to the same server every time, and everyone hopes that server stays available. Stateful designs on server fleets have always needed some version of that, and it has always been where the operational pain accrued.

The same problem exists here, and we solve it in the gateway, with one important difference. Affinity is to an environment identified by a key, not to a server. You supply the key as a binary range expression evaluated against each packet:

Arguments:
  TenantIdExpression: "u8[0] >> 6 ?? 0"
  TenantIdFormatter: hex

This one takes the top two bits of the first byte, so there are at most four tenants, and ?? 0 sends an empty packet to tenant 0 instead of failing the batch. Deployments may also key on a device ID, a session ID, or a customer prefix. Same key, same MicroVM. Batches are split by tenant before delivery and tenants are delivered concurrently.

Choosing the key is the design decision in this feature, because it sets three things at once: the unit of state, the unit of concurrency, and the blast radius. Make it too coarse and you've built a singleton with extra steps. Make it too fine and you've made it unnecessarily expensive. If you're not sure, ask what you'd shard on if this were a database.

The gateway also owns the lifecycle that goes with the key. First packet for a key starts a MicroVM. A suspended one is resumed. A terminated one is replaced. From the client's side the address and port never change, and nobody outside the gateway needs to know a MicroVM exists. Your application sees a small, boring contract:

{
  "Messages": [{
    "Tag": "opaque-correlation-tag",
    "Remote": { "IpAddress": "203.0.113.10", "Port": 45678 },
    "Local": { "Domain": "ingress-1", "Port": 15678 },
    "Formatter": "base64",
    "Data": "aGVsbG8="
  }]
}

and replies come back matched by Tag:

{ "Replies": [{ "Tag": "opaque-correlation-tag", "Data": "SGVsbG8gZnJvbSBNaWNyb1ZNLi4u" }] }

That's the same packet format Lambda destinations use, so a handler you already understand ports over without ceremony. The example is a few dozen lines of Node.js that counts packets in a variable and tells you which MicroVM answered. The counter surviving in RAM from one batch to the next is the entire demo.

Isolation by routing

In a multi-tenant process, isolation is a key security concern. Every request's access, in every code path, must align with the customer. One missed rule, one rarely-run branch, and tenant A reads tenant B's data — a flaw that too often passes code review because each line looks correct on its own.

With a MicroVM per tenant, the access decision starts when the MicroVM is run. The optional run hook payload includes the TenantId. This happens before your packet handling code runs. Access is then limited to one tenant's packets, memory and data, so tenants can't be mixed up. That's the same property Lambda Event Function tenant isolation mode gives, and it's a worthy comparison: there the isolation is per environment and the environment is disposable, here in our MicroVM integration it's per environment and the environment is durable enough to hold a session. Isolation you get by construction is cheaper than what you get by "being careful".

Some boundaries to know about to avoid surprises:

• Reaching the MicroVM. UDP Gateway holds a token scoped to port 8080 and uses it to deliver batches. The MicroVM lifecycle hooks should listen on a different port (the example uses 9000), so that token can never reach them. The run hook payload we send contains no credentials, and RunHookExtraData isn't a place for secrets either.
• What the MicroVM can reach. It runs with the execution role you name in ExecutionRoleArn, and with none, it has no AWS credentials at all (and its logs aren't forwarded, which is something to consider). Outbound network access defaults to the internet. To change that, create a Lambda network connector through your VPC and list it in EgressNetworkConnectorArns, then leave INTERNET_EGRESS out of the destination role's lambda:PassNetworkConnector permission so the role can't start one that has it.
• What isolation doesn't cover. One destination is one image and one execution role, shared by all of its tenants. MicroVMs are isolated from each other, but they're all yours, and they all hold the same permissions. If tenants need different permissions, and that can be achieved with policy Conditions, they need different destinations.

The MicroVM lifecycle

Suspend and resume. A MicroVM that sees no packets for IdleDurationSeconds is suspended and resumes automatically on the next one. The default is 900 seconds; the range is 60 to 28,800, and the top of the range means never suspend. Memory survives the trip, which is the point, but the first packet after a resume pays a minor price in latency. Pick the idle time by asking how long a tenant can be quiet before a slower first reply matters to clients.

The first packet pays for the start. The MicroVM has to reach RUNNING before it can answer, so the packet that triggers a start may be dropped or answered late (a couple seconds or more). Test with that in mind rather than being surprised that your first ncat goes without an expected reply.

Starts converge, they aren't exclusive. If the first packets for a tenant reach several UDP Gateway nodes at once, each may start a MicroVM. One is used, and the others are terminated within seconds. That's a deliberate trade: coordinating a single start across nodes on the packet path would cost more than occasionally starting a spare. The consequence for you is that /run has to be safe to execute more than once per tenant. Don't register the tenant and instance in an external system from the hook without safeguards.

Failure handling. In the case of a failed delivery, UDP Gateway looks at the MicroVM's state. A terminated MicroVM is replaced, a suspended one is resumed, and a running one is assumed good (so errors your application data don't pathologically start new MicroVMs). If a MicroVM terminates within five minutes of starting (a failing /run hook is the usual cause) it isn't replaced for five minutes, and packets for that tenant are dropped in the meantime, with the reason written to the destination's log group. That back-off exists as a second defense to prevent a broken image from turning into a start loop.

Wiring it up

The image is declared with the AWS::Serverless::MicrovmImage SAM resource, and the destination points at its ARN. A trimmed version of the example:

UdpListener:
  Type: Custom::ProxylityUdpGatewayListener
  Properties:
    # ...
    Destinations:
      - Description: Deliver UDP packets to a Lambda MicroVM
        Role:
          Arn: !GetAtt DestinationRole.Arn
        DestinationArn: !GetAtt EchoMicrovmImage.ImageArn
        Arguments:
          IdleDurationSeconds: "900"
          TenantIdExpression: "u8[0] >> 6 ?? 0"
          TenantIdFormatter: hex
          ExecutionRoleArn: !GetAtt MicrovmExecutionRole.Arn
          RunHookExtraData: '{"Example":"lambda-microvm"}'

With an unversioned ARN, new MicroVMs run the latest active version of the image. Running ones aren't replaced when you publish, so a rollout is gradual by nature. Pin a version (...:microvm-image:my-image:3) if you want the destination to hold still. The destination role is assumed by Proxylity, and because MicroVM actions are authorized against the image, scoping them to the image's ARN limits the role to MicroVMs started from that image. The full policy is in the destination documentation and the example template.

When to choose MicroVMs

Start with a function. If the state can be rebuilt cheaply from the packet, or lives comfortably in a store you were going to have anyway, a Lambda destination is simpler and has no start-up story to explain. Reach for a MicroVM when the state is hot, per-key, and touched by nearly every packet, so that loading it per batch is most of your cost. Or when the software wants to be a long-running process, because it's an existing server, a native library with real startup cost, or something with background threads. Or when per-tenant isolation is a requirement you'd rather enforce by routing than by review.

Availability

MicroVM destinations are available today in UDP Gateway, in AWS Regions where Lambda MicroVMs are available.

A couple wrinkles on the tooling side:

• At the time of writing, released SAM CLI versions can't yet zip and upload the image's source directory during sam deploy (see aws-sam-cli#9283). The example includes a small script that handles it between sam build and sam deploy, and you can drop it once SAM does this natively.
• So far, the CDK has been updated with L1 constructs for MicroVMs but L2 constructs are still in flight. We've also released L1 constructs, but are holding-off releasing our L2 destination construct for MicroVMs until the CDK's L2's release. This seems like the right choice to maintain the idiomatic library quality.

For a quick start, deploy the Lambda MicroVM example and send a few packets with ncat. Each tenant's counter climbs in the replies, and picks up where it left off after a suspend. The Lambda MicroVM destination documentation covers every argument, the lifecycle, and the run hook payload.