Introducing Wasmflow

Introducing Wasmflow

July 12, 2022

Wasmflow: A Runtime for Application Builders

We’re happy to announce the release of the Wasmflow runtime. All the source code for Wasmflow and related tools are on Github. The team has done an amazing job and it’s an incredible feeling to turn this corner.

We’re excited and we’re happy to have you here 😊

What is Wasmflow?

Wasmflow is an application runtime that connects code to other code. Wasmflow can connect WebAssembly (wasm) modules to other wasm modules, wasm modules to microservices, microservices to workers, workers to wasm, and a whole lot more.

We built Wasmflow to maximize reusability. Wasmflow normalizes interfaces between platforms and environments so code plugs into other code like Lego bricks. Every component you create benefits from an ecosystem of tools because everything just fits. You can run components as CLI apps, GRPC microservices, event workers, dataflow nodes, or use them like libraries without any changes.

The more you build on Wasmflow, the more capable everything and everyone becomes.

What does Wasmflow look like?

While wasm is in the name, Wasmflow is more than WebAssembly. It’s a way to manage the flow of data through code in myriad different places and formats.

Wasmflow applications define a list of dependencies and how they connect:

---
version: 1
external:
  getting_started: reg.candle.run/candle/getting-started
components:
  hello:
    collections:
      - getting_started
    instances:
      greet: getting_started::greet
      concatenate: getting_started::concatenate
    flow:
      - <>.first_name -> concatenate.left
      - <>.last_name -> concatenate.right
      - concatenate.output -> greet.input
      - greet.output -> <>.greeting

If you save that manifest as greeting.wafl you can execute it directly with wasmflow:

$ wasmflow invoke greeting.wafl hello -- --first_name=Samuel --last_name=Clemens
{"greeting":{"value":"Hello Samuel Clemens"}}

If you don’t like the JSON, you can pass --values to get more traditional output:

$ wasmflow invoke greeting.wafl hello --values -- --first_name=Samuel --last_name=Clemens
Hello Samuel Clemens

Wasmflow manages how data flows and branches in and out of components. The inputs and outputs to a component are like network ports to a server or STDIN/STDOUT to a process, except they’re at the function and library level. They’re named, typed, and transmit on streaming data asynchronously.

Why Did We Build Wasmflow?

Wasmflow was born out of frustration. We’d worked at enough companies to recognize we are all solving the exact same problems in slightly different ways. Many times we solve the same problems over and over again inside the same company.

The frustration was easy to accept when we were dealing with different languages. If we had a library written in Go or Java, we’d naturally accept the need to rewrite it in JavaScript or Rust. When companies started organizing around single languages the problem didn’t go away. We were still modifying, adapting, and rewriting code for the different frameworks and practices of the day. Even with perfectly reusable libraries, we’d still write endless integration code to connect dependencies, backends, and APIs.

And then there was deployment and testing and securing and maintenance. It’s all nearly identical boilerplate or applied best practice.

Every minute choice we make changes the shape of our interfaces and the steps in our processes. We take the same tasks and build unique snowflakes around each of them. We spend exorbitant amounts of time and money re-solving solved problems.

The industry has been iterating towards a solution to maximize reusability for decades. We’ve gotten far. The last key may have finally landed with WebAssembly. WebAssembly unlocks a cascade of opportunities.

At first glance, WebAssembly lets us normalize dependency formats across languages. When the original language no longer matters, our duplication across languages drops by an order of magnitude. The next major issue becomes normalizing interfaces to reduce or eliminate bespoke integration code between dependencies. After that, we can reduce the duplication across dependencies by pulling commodity logic into the runtime. Complex implementations that exist for the sake of security, scalability, and testability can be generalized once custom glue is gone. We end up with a composable software platform where you build only what you need while you reuse the rest from anywhere.

We’re on the road to a dramatically different software future with WebAssembly. Wasmflow is our entry in this game.

Containers, but for Code

Over a decade ago, JavaScript sparked the hope for a unified compilation target. Node.js gave JavaScript a place on the server and asm.js made high-performance JavaScript a reality. We were so close we could smell it. Companies like Epic compiled their Unreal engine into JavaScript and the top minds proclaimed there wasn’t anything JavaScript couldn’t do. They were right, but it didn’t matter.

Turns out, JavaScript has an upper limit. No matter how fast we can make JavaScript, it has to remain JavaScript. The web is the ultimate experiment in backwards compatibility. For the most part, we can’t change what is already in browsers. But we can add to it. This industry-wide experiment to push JavaScript to its limits proved what was possible. It inspired competing companies to collaborate on a standard to add the first additional runtime to the web in 25 years, WebAssembly.

Since then, WebAssembly has seen a cycle of inflated hype and crushing reality. WebAssembly is not capable of much on its own. It can only take numbers as input and output. It can’t do anything but compute. You can’t perform actions like network requests. You can compile languages like C, Go, and Rust into WebAssembly but each individual module has to have its own copy of the language’s standard library. That duplication is a deal-breaker for the web.

Though what many see as drawbacks to WebAssembly, Wasmflow treats as features. Sandboxed executables that can’t affect the environment and encapsulate all their dependencies sound a lot like Docker containers. Wasmflow combines WebAssembly and other technologies to make composable code containers a reality. It’s like Docker & Kubernetes inside your application. And just how you can build on existing Docker images to create new containers, you can add and combine any Wasmflow component and end up with a new component. It’s components all the way down.

Zero Trust, but for Code

Zero Trust – or “deperimeterization” – is a security philosophy that treats every user, device, and service as hostile. It isn’t claiming that companies are filled with criminals, it’s limiting the damage that any single compromised resource can inflict.

Before Zero Trust, a single compromised device would wreak havoc on everything it could reach. Devices were locked down and heavily controlled. Now most people use their personal iPhones or Android devices for work. Zero Trust enabled connecting risky assets to sensitive realms.

The philosophy is great but it has a gaping hole. Developers build applications and manage systems that require high levels of trust. Yet they download and run untrusted code from random sources hundreds of times a day. Modern platforms don’t give us granular control over the dependencies we import in our applications. They absorb the privileges and access of the host application and its environment. Unfortunately those dependencies can and do become hostile without warning.

There’s no perfect solution, but there are ways to limit the damage that can occur. Wasmflow isolates code from other code, isolates memory across transactions, signs dependencies with author keys, and restricts access to the environment. All of it at per-dependency granularity. Wasmflow is bringing Zero Trust to code.

And much more

There’s a lot more to Wasmflow and all of it hinges on what is possible when everything just works together.

Checkout out the code on Github. Peruse our Getting started guide. Join our discord channel to talk with other developers. Follow @wasmflow and @candle_corp on twitter and LinkedIn.

Stop by and give us a shout. We’ve got a lot more coming and we’d love to keep in touch with you.

Written By
Jarrod Overson
Jarrod Overson