Node to Rust, Day 19: Tests and Project Structure

Node to Rust, Day 19: Tests and Project Structure

December 19, 2021

Introduction

Over the last 18 days we got our environment set up with rustup, VS Code, and rust-analyzer. We pushed through the tough parts of being a newbie Rust developer and just started learning which crates we should start depending on.

Now it’s time to set up a real project.

The code in this series can be found at wasmflow/node-to-rust

This guide is not a comprehensive Rust tutorial. This guide tries to balance technical accuracy with readability and errs on the side of “gets the point across” vs being 100% correct. If you have a correction or think something needs deeper clarification, send a note on Twitter at @jsoverson, @candle_corp, or join our Discord channel.

Creating your workspace

You don’t need to use Cargo’s workspaces, but I recommend it. Rust – like node.js – is much easier to manage when you cut your application logic into small modules. Workspaces makes that tolerable. We’re going to start an executable project which is best set up as a library first with the CLI as a wrapper over the library. This structure is easier to test and easier for you and your users to extend.

First, create a new workspace by starting in an empty directory and making a Cargo.toml with the following contents

[workspace]
members = ["crates/*"]

The members entry lists all the crates in your workspace. We configured our workspace to include everything in the crates subdirectory (which doesn’t exist yet).

Starting a library

Create a new library crate with cargo new:

$ cargo new --lib crates/my-lib

The difference between a binary crate and a library is minimal. By default, binary crates have a main.rs. Libraries use lib.rs. The cargo new template for libraries also adds Cargo.lock to the .gitignore.

The Cargo book advises that you check in your Cargo.lock for end-products (binaries, servers, microservices, etc) and omit it for libraries.

The default lib.rs template is pretty basic but gives us a new topic to talk about:

#[cfg(test)]
mod tests {
    #[test]
    fn it_works() {
        let result = 2 + 2;
        assert_eq!(result, 4);
    }
}

Unit tests! That’s right, Rust has unit testing built in. No more configuring the test framework du jour when you start a new project. No more figuring out how to run tests in new projects. It’s all the same.

Yes, this means that many tests live in your source files. No, there’s not really any other way. Rust does have integration tests which can live in a separate tests folder alongside src, but those only have access to your public APIs. If you want to test small chunks of private code, you have to do it like this.

Really? Yes, really. There are crates that extend Rust’s testing functionality, but most of them hinge around this same harness and structure.

Unit tests in Rust

The library template introduces two new attributes, #[cfg()] and #[test].

[#cfg()] is for conditional compilation. By specifying #[cfg(test)] before an entire module like below, we tell Rust to skip compiling the module unless the test flag is on.

#[cfg(test)]
mod tests {
}

The [#test] attribute marks the annotated function as a unit test. Rust’s test harness runs each of these separately and reports the results when you run cargo test, e.g.

$ cargo test
running 1 test
test tests::it_works ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests my-lib

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

Rust’s included assertions are pretty basic. You can assert something is true with assert!(), equality with assert_eq!(), or inequality with assert_ne!().

Writing unit tests

Writing your tests first is a good way to figure out what your API should look like. “Test first” is the core philosophy behind Test Driven Development. Strict TDD is a bit extreme, but writing tests that flex major API points before writing the API methods will force your brain to think about usage before implementation.

What should our API look like? Well I hear WebAssembly is pretty hot so let’s build a wasm runner. Let’s change our lib.rs to look like this:

#[cfg(test)]
mod tests {
    use super::*;
    #[test]
    fn loads_wasm_file() {
        let result = Module::from_path("./tests/test.wasm");
        assert!(result.is_ok());
    }
}

Adding use super::* to the tests module on line 3 makes it easier to use everything in the parent module without prefixes.

The Module struct doesn’t exist yet but it seems like a reasonable name for the construct that will wrap a loaded WebAssembly module. I don’t know all the methods it will need, but I bet we’ll want a function that loads a module from a local file path. Finally, loading from a file path might fail so the return value should be a Result. I don’t know exactly what’ll be in the Result but I know I’ll want it to be Ok if I’m pointing to a valid wasm file.

Test-driven development may sound strange if you’re not used to it. Strict TDD means going back and forth between tests and code repeatedly. Write a small test, then write the code that makes it pass. I find strict TDD cumbersome and excessive, but the time I spent committed to it taught me a lot about writing testable code.

You can probably recognize what running cargo test will do. It will give us a compilation error because we reference structures and functions that don’t yet exist.

error[E0433]: failed to resolve: use of undeclared type `Module`
 --> crates/wasm-runner/src/lib.rs:6:22
  |
6 |         let result = Module::from_file("./tests/test.wasm");
  |                      ^^^^^^ use of undeclared type `Module`
For more information about this error, try `rustc --explain E0433`.

We need to add our Module struct, then our from_file function. We passed the function a &str in our test, but we probably want to be anything that can be represented as a Path. This sounds familiar to when we wanted to flexibly represent Strings in Day 12: Strings, Part 2 and – guess what? – we can do the same thing with Paths:

use std::path::Path;
struct Module {}

impl Module {
    fn from_file<T: AsRef<Path>>(path: T) -> Result<Self, ???> {
      Ok(Self{})
    }
}

But now we need to figure out what kind of error we’re going to return. Since we’re loading from a file system and those methods return an io::Error we can do that for now. If you don’t need to wrap an error, don’t. Let your user deal with it.

Now we have code that runs! It doesn’t do anything useful but we’re getting there. This is our lib.rs now:

use std::path::Path;
struct Module {}

impl Module {
    fn from_file<T: AsRef<Path>>(path: T) -> Result<Self, std::io::Error> {
        Ok(Self {})
    }
}

#[cfg(test)]
mod tests {
    use super::*;
    #[test]
    fn executes_wasm_file() {
        let result = Module::from_file("./tests/test.wasm");
        assert!(result.is_ok());
    }
}

Creating a CLI that uses your library

Run cargo new crates/[your cli name] in your workspace. Naming is hard. It’s best to leave important names ’til the very end. This is a good place to put a codename if you’re creative, or use cli if you’re not.

$ cargo new crates/cli

Add the library we just created as a dependency in our Cargo.toml.

[dependencies]
my-lib = { path = "../my-lib" }

Now we can use our library by importing from the my_lib namespace.

Rust has the unfortunate policy of allowing hyphens in crate names but disallowing them as Rust identifiers. If you have a crate with a hyphen, Rust requires that you reference it with the hyphens replaced with underscores.

use my_lib::Module;

When you add this you’ll already see VS Code complaining.

VS Code complaining that an imported module is private

Our Module was not explicitly made public so we can’t import it. This is one of the many reasons why it’s a good idea to set up your projects this way. You get a first-hand view of what it’s like to actually use your library. Add pub to struct Module and fn from_file in the impl as well. We know we’ll need it right away.

pub struct Module {}

impl Module {
    pub fn from_file<T: AsRef<Path>>(path: T) -> Result<Self, std::io::Error> {
        Ok(Self {})
    }
}

Now we can import Module and use Module::from_file in our CLI.

use my_lib::Module;

fn main() {
  match Module::from_file("./module.wasm") {
    Ok(_) => {
      println!("Module loaded");
    }
    Err(e) => {
      println!("Module failed to load: {}", e);
    }
  }
}

We’ll get to the implementations soon, but we’re putting together a solid structure for any Rust project right now.

Running your CLI from your workspace

You can run your CLI from the ./crates/cli directory with cargo run, but cargo can also run commands in any sub-crate with the -p flag. In your project’s root, run cargo run -p cli to run the default binary in the cli crate.

$ cargo run -p cli
Module loaded

Perfect! We have much more to do, but we have a foundation to build off of now.

Additional reading

Wrap-up

Setting up a solid foundation is important. You’ll frequently look at Rust and think “Really? This is the way I’m supposed to do this?” It can shake your confidence and that’s what we’re here for. When you come across those moments, I’d love to hear them! We’ve all gone through it, but it’s hard to remember how alien everything felt at first now that Rust is a part of our daily lives.

If you have questions or comments, you can reach me personally on twitter at @jsoverson, the Candle team at @candle_corp, or join our Discord channel.

Written By
Jarrod Overson
Jarrod Overson