gRPC, Protobuf, and the Wire Format Your Gateway Allows

Anyone arriving from the service-mesh world asks this within a minute of seeing the layer from part 101: why not gRPC? It is the default east-west transport there, and the question is fair.

The answer is not preference, it is structural. A private REST API enforces requests down to HTTP/1.1, and gRPC is carried over HTTP/2 framing. Worse, every AWS surface that does speak gRPC excludes Lambda functions as targets, so switching gateways does not rescue you. But the thing teams actually want from gRPC survives all of this, because it was never the transport. It was the schema.

Why gRPC is not on the table

Anyone arriving from the service-mesh world asks this within a minute. gRPC is the default east-west transport there: a typed contract, generated clients, streaming, one connection carrying many calls. It is a fair question, and the answer is not preference. It is structural.

gRPC needs HTTP/2, and a private REST API refuses to speak it. The gRPC wire spec is titled gRPC over HTTP2 and states that it “serves as a detailed description for an implementation of gRPC carried over HTTP2 framing” (gRPC over HTTP/2). The dependency is not cosmetic. A call terminates in HTTP/2 trailers, and the spec is blunt about it: “Status must be sent in Trailers even if the status code is OK.” HTTP/1.1 has no equivalent. The spec even ships a tripwire for intermediaries that get this wrong, defining TE: trailers as “Used to detect incompatible proxies”.

Now recall a consideration from part 101: on a private API, “If you make a request using HTTP/2 protocol, the request is enforced to use HTTP/1.1 protocol.” That single line settles it. The transport this series recommends downgrades away the exact protocol gRPC is carried over. AWS never publishes a sentence saying API Gateway does not support gRPC, so do not go looking for one. The downgrade is the proof.

Lattice does not rescue you, and this is the part people get wrong. The intuitive next move is to reach for VPC Lattice, which genuinely does support gRPC. Read the constraint before you plan around it. For the gRPC protocol version, Lattice says “The only supported target types are INSTANCE and IP”, and then, plainly: “You can’t use Lambda functions as targets” (Lattice target groups). ALB carries the identical exclusion for its own gRPC target groups. The HTTP/2 protocol version has the same target-type restriction on both.

That is the pattern worth internalising: every AWS surface that speaks gRPC excludes Lambda function targets. Lattice is not a gateway swap that unlocks gRPC. If you need gRPC east-west, you are changing your compute, not your gateway.

The reason underneath is Lambda’s execution model. A Lambda function is not a server. There is no process listening on a port, and nothing to hold a connection open: “Lambda freezes the execution environment when the runtime and each extension have completed and there are no pending events” (execution environment). Everything gRPC is prized for, a persistent connection, multiplexed streams, bidirectional streaming, presumes a long-lived server holding that connection. A frozen, request-and-response environment cannot offer it. Lambda’s response streaming is a different mechanism and gives you none of it: no gRPC framing, no trailers, no client streaming.

Be precise about the claim, because a sharp reader will test it. Lambda functions cannot terminate gRPC. Lambda MicroVMs can, and their networking documentation lists gRPC among the supported inbound protocols. That is not a counterexample. A MicroVM gives you a long-lived server process on a port, which is exactly the shape gRPC needs, and its existence confirms the rule rather than breaking it.

Keep the contract, drop the transport. Strip gRPC down and most teams want the typed contract and the generated clients, not bidirectional streaming between two Lambdas. You can keep all of that. Write .proto schemas, generate the clients, and ship protobuf or JSON over an ordinary HTTP/1.1 POST through the private API. The Connect protocol is built for exactly this shape and is explicit about it: “Bidirectional streaming requires HTTP/2, but the other RPC types also support HTTP/1.1. The protocol doesn’t use HTTP trailers at all, so it works with any networking infrastructure” (Connect protocol). What you give up is streaming and multiplexing. Lambda’s execution model was never going to give you those anyway, so the loss is smaller than it sounds.

The decision rule is short. gRPC east-west is a reason to move that service onto a long-lived container behind an ALB or Lattice with IP targets. It is not a reason to switch gateways, and it is not something a private API Gateway can be talked into.

One caveat so the picture stays honest. Lambda MicroVMs, launched in June 2026, do hand each MicroVM an HTTPS URL that speaks HTTP/2, gRPC, and WebSockets. AWS positions them as isolated sandboxes for untrusted or AI-generated code, not as an east-west transport, so do not read them as AWS steering internal traffic toward gRPC. Read them as confirmation of the rule: the protocol arrives exactly when a long-lived server process does.

The wire format is not the contract

Ask a team why they want gRPC and the honest answer is rarely HTTP/2 multiplexing. It is schema discipline: one source of truth for the payload, generated clients in every language, and a CI gate that fails the build when someone changes a field’s type. That is a contract concern. gRPC bundles it with a transport, and the transport is the part Lambda cannot take. The good news is that the bundle comes apart.

Protobuf is an encoding, not a transport. You can send binary protobuf over an ordinary HTTP/1.1 POST. This is not a hack, it is specified. The Connect protocol “does not depend on framing details specific to a particular HTTP version” and its own reference shows a plain POST /service/Method HTTP/1.1 carrying Content-Type: application/proto with an uncompressed binary protobuf body (Connect protocol). AWS agrees in its own house: the Smithy RPC v2 CBOR protocol declares http: ["h2", "http/1.1"] and moves payloads as application/cbor (Smithy RPC v2 CBOR). Binary payloads do not require HTTP/2. Only gRPC’s framing does.

But API Gateway taxes the bytes, and this is the trap. A REST API is text-first by default: “By default, the RestApi supports only UTF-8-encoded text payloads.” Post a protobuf body to a route that has not enrolled the content type in binaryMediaTypes, and the gateway treats your bytes as UTF-8 text and mangles them. Enroll the type properly and you meet the second rule: “When converting a binary payload to a text string, API Gateway always applies a base64 encoding on the binary data” (binary media types). The Lambda proxy event carries a JSON body string plus an isBase64Encoded flag, so a binary body arrives base64-encoded and your handler decodes it before protobuf ever sees it.

Base64 is four characters for every three bytes. That is a fixed one-third inflation, by definition, plus a decode you pay in the callee. Protobuf usually still lands smaller than the equivalent JSON, because it starts far below it, but the platform is quietly refunding a large part of what the binary format won. On this transport, “we switched to protobuf to shrink the payload” is a weaker argument than it sounds.

So take the contract and leave the bytes. Write the schema, generate the clients, and gate breaking changes in CI. buf breaking compares a schema against its previous version and fails on a change that would break existing clients, reporting things like a field changing type from int32 to string (Buf). None of that needs gRPC, HTTP/2, or even a binary body. It is the part you actually wanted. Ship JSON over the private API, keep the schema as the source of truth, and reach for binary only where a measured payload justifies paying the base64 tax.

JSON stays the reasonable default here, and not out of laziness. It is the platform’s native currency: the proxy event is already JSON, and CloudWatch Logs, X-Ray, and a console test invoke all read it without a decoder ring. For the contract layer, OpenAPI is the standard for HTTP and JSON APIs, now at version 3.2.0 (OpenAPI Specification).

The industry’s direction is worth naming, because AWS is the proof of it. The direction is schema-first contracts with a swappable wire format. Smithy is “a language for defining services and SDKs”, an IDL AWS says has been “widely used within Amazon and AWS for over a decade” (Smithy). The contract is the IDL; the protocol is a trait you can change. And AWS is changing it: service models such as CloudWatch and GameLift now declare smithy.protocols#rpcv2Cbor alongside their legacy protocol, and CBOR itself is a full Internet Standard, STD 94 (RFC 8949). This is a migration in flight, per service and per language, not a finished switch. The lesson generalises cleanly: AWS can move its own wire format under its own SDKs precisely because the contract was never the wire format. Build your east-west layer the same way and the format stays a decision you can revisit.

Common pitfalls

1. Planning a migration to VPC Lattice in order to get gRPC while the compute stays on Lambda functions. Lattice’s gRPC target types are INSTANCE and IP only, and it says so: “You can’t use Lambda functions as targets.”
2. Posting a protobuf body to a route that never enrolled the content type in binaryMediaTypes. The gateway treats the bytes as UTF-8 text and mangles them, and nothing tells you.
3. Switching to protobuf to shrink the payload on this transport. Base64 adds a fixed one-third back, plus a decode in the callee. Switch for the contract, not the bytes.

The transport and the contract are settled. What remains is the bill, and the question of whether this whole approach is where the industry is going. Part 104 answers both.

References

• gRPC over HTTP/2 - The wire spec. gRPC is carried over HTTP/2 framing, status is returned in trailers, and TE: trailers exists to detect incompatible proxies.
• VPC Lattice target groups - For the gRPC protocol version, “The only supported target types are INSTANCE and IP” and “You can’t use Lambda functions as targets”. The reason Lattice is not a gRPC escape hatch for Lambda.
• ALB target groups - The identical exclusion on the load balancer side: gRPC target groups support instance and ip only, and “You can’t use Lambda functions as targets.”
• Lambda execution environment - “Lambda freezes the execution environment when the runtime and each extension have completed and there are no pending events.” Why there is no connection for gRPC to multiplex over.
• The Connect protocol - Protobuf over ordinary HTTP/1.1 POST, specified: the protocol “does not depend on framing details specific to a particular HTTP version” and uses no trailers.
• Protocol Buffers - “A language-neutral, platform-neutral extensible mechanism for serializing structured data”, and the field-number rules that make it compatible across versions.
• Binary media types in API Gateway - A REST API is UTF-8 text by default, and “API Gateway always applies a base64 encoding on the binary data”. The one-third tax on any protobuf or CBOR payload through this transport.
• Smithy - AWS’s IDL, “widely used within Amazon and AWS for over a decade”. The contract is the model; the wire protocol is a swappable trait.
• Smithy RPC v2 CBOR - AWS’s own binary protocol, which declares http: ["h2", "http/1.1"]. Proof that binary payloads do not require HTTP/2.
• RFC 8949: CBOR - A full Internet Standard, STD 94. The standards-track binary format AWS is migrating its own SDKs toward.
• Buf: breaking change detection - The CI gate that fails a build when a field changes type. The part of gRPC teams actually want, and it needs neither gRPC nor HTTP/2.
• OpenAPI Specification - The standard contract language for HTTP and JSON APIs, now at 3.2.0.