Introducing capn web a new javascript rpc protocol

A new Remote Procedure Call (RPC) protocol and implementation named Cap'n Web has been introduced, written entirely in TypeScript. Designed as a spiritual sibling to Cap'n Proto, this new system is engineered specifically for the modern web stack. Cap'n Web distinguishes itself by having no schemas and minimal boilerplate, which simplifies setup and integrates smoothly with TypeScript. It operates on a human-readable serialization format based on JSON, with some pre- and post-processing to handle special data types. The system is designed to be lightweight, compressing to under 10 kB without any dependencies, and is available as open source under the MIT license. It supports multiple transports out-of-the-box, including HTTP, WebSocket, and postMessage(), and is compatible with all major browsers, Cloudflare Workers, Node.js, and other modern JavaScript runtimes. Cap'n Web is particularly well-suited for interactive web applications with real-time collaborative features and for modeling interactions across complex security boundaries. While the system is new and considered experimental, it is already being used in features such as the "remote bindings" in Wrangler, which allows a local test instance to communicate via RPC with production services.

Understanding the Cap'n Web JavaScript RPC Protocol

Remote Procedure Calls (RPC) provide a way for programs to communicate over a network by making remote communications resemble standard function calls. Instead of manually formatting HTTP requests and parsing responses, an RPC system allows a client to call a method on a "stub" object, which then handles the serialization and transmission of parameters to a server, invokes the method, and returns the result. Historically, RPC systems were often criticized for being synchronous, causing the calling thread to block while waiting for a network response, which could lead to slow or unresponsive applications. However, the modern programming landscape, with features like Promises and async/await, has addressed these foundational issues. Cap'n Web leverages these modern constructs to provide an efficient and developer-friendly RPC experience. The system is built on an object-capability RPC model, which makes it more expressive than many other RPC systems. This model supports several powerful features, including bidirectional calling, where a server can make calls back to the client. This is enabled by the ability to pass functions and objects by reference; when a function is passed over RPC, the recipient gets a stub that, when called, makes an RPC back to invoke the original function. This pattern is useful for implementing callbacks, where a client passes a function to the server for later execution. The model also allows for promise pipelining, where a developer can immediately use the promise returned from one RPC as an input to a subsequent RPC, enabling a chain of calls to be executed in a single network round trip. This approach allows developers to design network interfaces in a way that feels natural to JavaScript programmers, reducing the need to constantly translate between different mental models.

Advanced Features of the Cap'n Web JavaScript RPC Protocol

Beyond its foundational model, Cap'n Web incorporates several advanced features designed to enhance efficiency and security. One of its most significant capabilities is promise pipelining. When an RPC is initiated, it returns a special promise that is also a JavaScript Proxy object. This allows developers to chain dependent calls without waiting for the first call to complete. For instance, a client could make a call to authenticate and receive a session object, and then immediately call a method on that session object promise, all within a single network round trip. The client sends messages to the server instructing it to use the result of the first call as the target for the second one. This pattern also enables a powerful capability-based security model. In a typical scenario, a client might call an authenticate() method with an API key. Upon successful verification, the server returns an authenticated session object. The client can then perform authorized operations by calling methods on this specific session object. Crucially, it is impossible for the client to forge this session object; the only way to obtain one is by successfully authenticating. This design neatly integrates authorization into the RPC abstraction, which is a common challenge for protocols like WebSockets where headers and cookies are not easily used for authorization. For developers using TypeScript, Cap'n Web provides strong integration. An RPC API can be declared as a TypeScript interface, which can then be implemented on the server and used on the client to get end-to-end type checking and autocompletion. It is important to note that these checks occur at compile time, not runtime, so malicious clients could still send parameters of the wrong type. For situations where a persistent WebSocket connection is not necessary, Cap'n Web offers an HTTP batch mode, allowing a client to make a quick batch of calls in a single HTTP request.

How the Cap'n Web JavaScript RPC Protocol Compares to GraphQL

Modern web development has seen the rise of technologies like GraphQL, which was created to solve the "waterfall" problem of traditional REST APIs, where a client might need to make multiple sequential requests to fetch all the necessary data. GraphQL allows a client to request multiple pieces of data in a single query, improving efficiency. However, GraphQL introduces its own set of tradeoffs, including a new schema language, specialized tooling, and a declarative query model that can be awkward for chaining operations or mutations. Cap'n Web addresses the same waterfall problem but does so without introducing a new language or ecosystem; it remains firmly within the JavaScript paradigm. Through promise pipelining, developers can naturally express chained operations in standard JavaScript, such as creating a user and then immediately using that new user object to perform another action, all within a single network round trip. A key area where GraphQL excels is in handling lists of data, for example, fetching a user's friends and then fetching the profile photo for each friend. Cap'n Web offers an innovative solution for this exact scenario through a specialized .map() function that can be called on a promise that will resolve to an array. When this .map() is used, it does not send the callback function's code to the server. Instead, it uses a "record-replay" mechanism on the client to capture the pipelined calls made within the callback. These recorded instructions are then sent to the server, which replays them for each element of the array, effectively performing the transformation on the server-side without requiring additional network round trips. This allows developers to achieve complex data fetching patterns similar to GraphQL but using familiar, imperative JavaScript code.