I’m going to try something different with this blog post and make it a sort of work log for a small project I’m trying to build.
You might remember Raynor - my small TypeScript marshalling library from some time ago. It allows one to automate a lot of the boilerplate around serialization/deserialization and data validation with the help of annotations. I’ve used it in a bunch of small projects and I’ve been really happy with how it works. There’s a bunch of stuff to polish and improve for a future major version, but nothing brutally bad.
This got me thinking that I should use it for other things as well. And a natural thing is as part of an RPC library. It’s fairly natural for these types of projects to go hand in hand. You can see it with gRPC or Thrift for example.
So this blog post is a rough design document for something I want to call “Raynor RPC”. It’s going to be a bit rougher than my usual fare.
The main things I want to have are:
- Keep the annotation approach of Raynor.
- Use Raynor marshalling for making sure parameters are OK at the client and server side.
- Acknowledge the semantics of RPCs - so use
Promises, but provides some of the niceties of regular calls. Allow exceptions thrown on the server to be handled by the client.
- Integrate and make use of TypeScript powerful type’s system. There will probably be a lot of lower-level magic happening, but clients shouldn’t be exposed to it.
- Have a single place where a service is defined. Client and server infrastructure should be derived by the library automatically.
- Should be transport agnostic. But I’ll be starting with HTTP as the assumed transport, JSON for data exchange and building on top of Express for server-side support. But those shouldn’t be assumptions, and adding TCP, Koa or binary data to the mix shouldn’t dramatically change things.
- Should allow for request metadata to be sent from clients and accessed on the server-side.
- Should allow for server-side middleware. Pure HTTP servers aren’t the only ones where placing authentication into middleware makes sense. Things will get fleshed out more as I develop this thing.
Now, I’d like to get the code to look like the following small example. Suppose we have a library service which allows one to query for the books available and change the titles of a particular book. We can have the
library-sdk package, which defines the entities and services. It is meant to be used both by clients of the library service, to generate the client objects and work with entities, and by the server itself, to generate server objects.
library-server application we actually define the implementation of the
LibraryService and make it available over the network as a HTTP server. It might looks something like this:
Finally, in the
library-frontend package, which implements the frontend as a SPA, we want to use the library service. Thinghs would looks like this:
Ok, so that should be a rough outline of a v1 API for client and server components. It should be pretty familiar to people who’ve encountered gRPC/Thrift before. And hopefully it should be intuitive enough for everyone else. One important thing to notice is that on the client side there isn’t anything to code in order to use this. Everything is done by the library, because all that can be done is usually very repetitive and mechanic. And I’m referring here to things like - argument marshalling, HTTP connection setup, waiting for the response and parsing it and checking that it’s valid, handling various errors etc. On the server side there’s the same concerns, but you also have to code a bit. But just the actual important parts. Once a method like
updateBooks executes, you can be sure that
title exists and is a string, or that
bookId is a non-negative integer.
Codewise, there’s some sketches at github.com/raynor-rpc - but it’s marked with version
'0.0.0' for a reason. It’s basically just enough to check that the annotations based API will work, but nothing more. As I’ll build on it I’ll post new articles and hopefully I can build a nice series out of this.
For the sake of me remembering things, here’s some other nice things to have:
- An annotation for idempotent methods. This could impact the server-side APIs that are generated - ie, have a
GET, rather than a
PUTwhen using the HTTP infrastructure.
- An annotation to mark void outputs more clearly.
- Support for “fire-and-forget” or one-way methods like in Thrift, for methods with a void output.
- Support for optional parameters.
- Some notion of API versioning and compatibility.
- Support for service proxies for use in backends for frontends.
And since we’re at the dreaming stage, we could also provide, further down the line:
- Call mechanics - timeouts, retries, pipelining, backpressure etc.
- API documentation and debug interaction UI, like Swagger.
- Built in support for monitoring, tracing and logging of API calls.
- Support for streaming request data, like the bi-directional streaming in gRPC.