What's on your gRPC wire, Protocol Buffers or JSON?

The hot kid on the block for microservice APIs is gRPC, a Google developed, OpenSource binary wire protocol.

Its native serialization format is Protocol Buffers, advertised as "Protocol buffers are a language-neutral, platform-neutral extensible mechanism for serializing structured data". How does that fit into Domino picture?

Same same, but different

When old bags, like me, hear the word RPC a flood of memories and technologies come to mind:

• DCOM Microsoft's take on: like Java, but Windows only, superceded by WCF for dotNet
• Corba a standard defined by a commitee, mainly Java (and YES Domino still ships with a Corba Server)
• SOAP with our beloved (or was the word: cursed?) WSDL

There are a few more modern contenders like Apache Thrift, Apache Avro or the KF - TEE. Good to have so many open standards.

the good

Especially with SOAP the common reaction to the rise of REST was: Good riddance RPC. I'm using the term REST fast and loose here, since a lot of the APIs are more like "http endpoints accepting JSON payloads" rather than REST in the formal sense of the definition.

So what's different with gRPC, so it got adopted by the Cloud Native Computing Foundation? IMHO there are several reasons:

• It is designed by really smart engineers to run up to Google scale
• It is ground up optimized, not bothering with legacy, but betting on HTTP/2 and its wire efficiencies
• It is a compact binary protocol, making it efficient in low bandwidth and/or high volume use cases (Google scale anyone)
• It transmits data only, no repeated meta data as in JSON or XML based approaches (at least when you use Protocol Buffers)
• It focused on code generation, functioning more like an SDK than an API
• It has versioning support built in
• It uses rich structured data types (15 on last count) including enumerations. Notably absent: date/time and currency

And of course: it's the current fashion. RedHat provides a compehensive comparison to OpenAPI, as do others. Poking around YouTube I gained the impression, that most comparisons are made to REST and its limitations, almost similar to sessions about GraphQL. Mr. Sandoval tries to describe differentiators and use cases, go read it, it is quite good.

the bad and ugly

Clearly seeing the strenght, I see weakness too. Some of them are more or less relevant, lets list them out:

• SDK vs. API. Google's protoc generates code for C++, Java, Python, Objective-C, C#, JavaScript, Ruby, Go, PHP and Dart. If you fancy other languages you are out of luck. Pretty much each language can access http (even LotusScript), but for gRPC, you are limited to the provided languages (probably not an issue for mainstream programmers, except for our LotusScript fans)
• The binary protocol makes it way harder to introspect. Without the proto file, it is a lot of guesswork what goes over the wire. If you think that's not an issue, check the Domino AppDev Pack) for its proto file
• Structured data (only). Domino's documents and views are notoriously flexible. While you could define a structured data model for one specific application, generic Domino will pose a challenge

JSON over gRPC

Is there potentially a way to still use JSON, but leveraging HTTP/2 and the gRPC protocol? Turns out, it is possible. Protocol Buffers is only one (default) way to use gRPC. In an August 2018 blog post, Carl Mastrangelo outlines how a JSON binding using Google's GSON JSON library (there is also transcoding, but requiring data to be structured).

Compared to his native example that did 2400 RPCs per second, the JSON binding example comes in at 2200 RPCs per second. Unless you have a Google scale problem, that should suffice.

Ditching Protocol Buffers comes with a price: protoc will no longer generate any code stubs for you, so you have to handcraft those. In the bigger scheme of things that might just be a labour of love. Unless someone takes your OpenAPI specification and send it through an OpenAPI Generator. If memory serves that might not be as hard as it looks.

So stay tuned!