Closures

Prerequisites:

Chapter about ownership
Trait syntax
A cursory knowledge of rust async/await, as it is mentioned

Closures are a way to delegate an action for later or for elsewhere, or to allow passing actions to code that provides context. They are very similar to anonymous functions, lambda expressions, or function literals you may be used to from other languages, such as JavaScript, C#, or pretty much all functional programming languages.

The main defining feature of closures in comparison to inline functions is that they allow capturing variables, or references to variables, from the function they were defined in. We call this mechanism closing over its environment, hence the name closure.

Their syntax slightly differs from functions:

Parameters are enclosed in vertical bars (|) instead of parenthesis
The types of parameters and return types are inferred, if possible
If the body of a closure is a single expression, it does not need to be enclosed in brackets ({})

BTW: Assignments and any other expressions trailed with a semicolon are not considered expressions for the last rule, and you always have to enclose them in brackets

Here is how a closure looks and works, in simplest terms:

fn main() {
    let x = 2;

    // let's take two parameters and add them up
    // with an outside variable and a literal
    // just like in functions, the final expression
    // in a closure is considered the return value
    let my_closure = |a, b| x + a + b + 5;
    // we could also specify the types, ie. |a: i32, b: i32|

    // you call a closure like you would call a function
    println!("result: {}", my_closure(5, 10));
}

A closure that takes mutable arguments does not need to be declared as mutable.

However, let's first try a naive attempt:

#![allow(unused)]
fn main() {
let mut x = 2;

let my_closure = |mut a| { a += 5 };

my_closure(x);

println!("result: {}", x);
}

If you run this you will see that the result is 2, not 7. What gives?

Well, to get the answer, we need to go back and consider what we know about ownership. The way we have declared our closure implies it takes arguments by ownership. If the integer was not a Copy type, it would have been inaccessible after calling my_closure, since it would be moved into it and then dropped after the assignment.

However, since all primitive numeric types are Copy, a copy of x is created in the third statement my_closure(x), and it is this copy that gets assigned to and dropped immediately.

We need to amend our code to take a mutable reference to x instead:

#![allow(unused)]
fn main() {
let mut x = 2;

// we need to deref to asign to it
// Rust also requires us to specify the type here
let my_closure = |a: &mut i32| { *a += 5 };

my_closure(&mut x);

println!("result: {}", x);
}

This work as expected, and as you can see, my_closure still needs to be declared as immutable.

However, a closure needs to be declared as mutable if it mutates the environment it closes over:

fn main() {
    let mut x = 2;

    // this time we don't take x as parameter,
    // but close over it;
    let mut my_closure = || { x += 5 };

    my_closure();

    println!("result: {}", x);
}

If you try removing the mut keyword, it will stop compiling.

This is due to the way that closures work internally.

In reality, a closure is a unique, anonymous type that cannot be written out. It is roughly equivalent to a struct which contains the captured variables.

Consider the following closure:

#![allow(unused)]
fn main() {
let x = 3;
let y = 5;

let my_closure = || { x + y };

println!("result: {}", my_closure());
}

Rust will transform this into a structure that looks (roughly) like this:

#![allow(unused)]
fn main() {
struct Closure<'a> {
    x: &'a i32,
    y: &'a i32,
}
}

And then it will implement all three of the following traits:

#![allow(unused)]
fn main() {
pub trait FnOnce<Args> {
    type Output;
    extern "rust-call" fn call_once(self, args: Args) -> Self::Output;
}

pub trait FnMut<Args>: FnOnce<Args> {
    extern "rust-call" fn call_mut(
        &mut self,
        args: Args
    ) -> Self::Output;
}

pub trait Fn<Args>: FnMut<Args> {
    extern "rust-call" fn call(&self, args: Args) -> Self::Output;
}
}

Where Args: ().

As you can see, there are some dependencies between these traits. Every closure is at least FnOnce, meaning in can be called at least once, wherein the structure is consumed and the closure body can use captured variables by value/ownership.

A closure is FnMut, if it is also valid in a context, where only accessing captured environment by at most mutable references.

And finally, the Fn trait is the most restrictive, closures that implement it must also be valid in a context where only immutable references are allowed on captured variables.

Since our closure from the previous example only needs immutable references, it implements all of the above.

We can kill two birds in one stone and show how we can take closures as a function parameter and also verify that the closure above satisfies all three categories:

fn verify_fn<T>(_: T)
where
    T: Fn() -> i32
{

}

fn verify_fn_mut<T>(_: T)
where
    T: FnMut() -> i32
{

}

fn verify_fn_once<T>(_: T)
where
    T: FnOnce() -> i32
{

}

fn main() {
    let x = 3;
    let y = 5;

    let my_closure = || { x + y };

    // note that we are only passing this closure,
    // not calling it by appending () after its identifier
    verify_fn(my_closure);
    verify_fn_mut(my_closure);
    verify_fn_once(my_closure);

    println!("result: {}", my_closure());
}

As you can see, our code compiled and so the closure satisfies all three categories.

What happens, if we were to mutate one of their variables?

fn verify_fn<T>(_: T)
where
    T: Fn() -> i32
{

}

fn verify_fn_mut<T>(_: T)
where
    T: FnMut() -> i32
{

}

fn verify_fn_once<T>(_: T)
where
    T: FnOnce() -> i32
{

}

fn main() {
    let mut x = 3;
    let y = 5;

    let mut my_closure = || { x += 1; x + y };

    // note that we are only passing this closure,
    // not calling it by appending () after its identifier
    verify_fn(my_closure);
    verify_fn_mut(my_closure);
    verify_fn_once(my_closure);

    println!("result: {}", my_closure());
}

Well, as the error indicates, calling verify_fn fails with the message:

expected a closure that implements the 'Fn' trait,
but this closure only implements 'FnMut'

And further in the description:

closure is `FnMut` because it mutates the variable `x`

This tells you all you need to know. This closure is FnMut and by extension FnOnce, so it is only valid in their respective contexts.

TIP: When requiring users of your code to pass in a closure, select the trait that's the most restrictive for you and most flexible for the user. Ie. if you will be calling a closure only once, require FnOnce, if you don't need it to be immutable, use FnMut

What happens if we got rid of the verify_fn() call?

fn verify_fn<T>(_: T)
where
    T: Fn() -> i32
{

}

fn verify_fn_mut<T>(_: T)
where
    T: FnMut() -> i32
{

}

fn verify_fn_once<T>(_: T)
where
    T: FnOnce() -> i32
{

}

fn main() {
    let mut x = 3;
    let y = 5;

    let mut my_closure = || { x += 1; x + y };

    verify_fn_mut(my_closure);
    verify_fn_once(my_closure);

    println!("result: {}", my_closure());
}

You will see that now rustc has a different issue, and that is one of ownership. We could pass the previous closure to three functions and then call it because it was Copy.

This is because all of its members are Copy (because immutable borrows are Copy as mentioned in Chapter 1). When we went and mutated x, the structure is now capturing a mutable reference for x. You can play around with the above example to figure out how to make it compile and run.

Move closure

When defined, a closure can be optionally pre-pended by the keyword move. If this keyword is used, a closure captures all of the values by ownership instead of by reference, regardless of what is at most needed in the closure. This has no bearing on whether a closure is at most Fn, FnMut or FnOnce, but it does have a bearing on ownership.

Because all variables are captured by value, the closure can (usually) outlive the function it was defined in, which can help it become more flexible.

Let's quickly remember what makes a closure Copy. A move closure can also be Copy if all of the captured variables are Copy.

Consider the following example:

fn foo(mut f: impl FnMut()) {
    f();
}

fn bar(mut f: impl FnMut() + Copy) {
    f();
    foo(f);
    f();
    foo(f);
}

fn main() {
    let mut i = 0;
    bar(move || {
        i += 1;
        println!("{}", i);
    });
}

What do you think the output is? Try to apply the rules we've just presented and deducing the output to stdout without running the code.

Once you have an educated guess, you can either run the code or peep the answer here:

The correct answer is `1223`

We need to differentiate between when a closure is copied and when it is modified.

The one really important function we need to focus on in this example is bar().

Here it is annotated with what's happening, which you can use to check if your reasoning was correct, regardless of answer.

#![allow(unused)]
fn main() {
// The Copy trait bound here is key,
// since we need to Copy f into foo()
fn bar(mut f: impl FnMut() + Copy) {
    f();     // i += 1, when i = 0, -> print 1
    foo(f);  // f is copied into bar(), copy of i += 1, when i = 1 -> print 2
    f();     // the original i is unmodified, therefore i += 1, when i = 1 -> print 2
    foo(f);  // f is copied into bar(), copy of i += 1, when i = 2 -> print 3
}
}

Keep the rules of ownership, copying and moving in mind.

Async and closures

What if you need an async closure? Well, despite async/await having some years under its belt already, asynchronous closures haven't been stabilized yet.

You can enable them with #![feature(async_closure)] in crate root (ie. lib.rs or main.rs), at which point they can be used with a rather familiar syntax:

#![feature(async_closure)]

fn main() {
    let closure = async || {
         println!("Hello from async closure.");
    };
    println!("Hello from main");
    let future = closure();
    println!("Hello from main again");
    futures::executor::block_on(future);
}

One could argue that an async block would behave the exact same way in this very limited use case:

fn main() {
    let future = async {
         println!("Hello from async future.");
    };
    println!("Hello from main");
    futures::executor::block_on(future);
}

However, there is a couple important differences:

An async closure can take parameters
You can only await a future once, but you can (unless FnOnce) call an async closure multiple times, producing multiple futures.

To take an async closure as a parameter, you need to be generic both over the closure and the future, meaning at least two generic params are required.

TIP: It might be tempting to try the impl Trait syntax, but remember that it is not possible here. **Existential types are not generics, but instead refer to a single concrete type derived from function/block body. We don't have this information when specifying a parameter

Let's be generic over the Future's output to display something you might encounter:

#![allow(unused)]
fn main() {
use std::future::Future;

async fn await_it<T, F, O>(closure: T) -> O
where
    T: Fn() -> F,
    F: Future<Output = O>
{
    closure().await
}
}

As you can see, we don't need to mention the fact that it's an async closure anywhere in T's trait bound. Once again, async closures are syntactic sugar, and without having to enable the aforementioned nightly feature, you can create them with async blocks:


use std::future::Future;

async fn await_it<T, F, O>(closure: T) -> O
where
    T: Fn() -> F,
    F: Future<Output = O>
{
    closure().await
}

fn main() {
    let closure = || async {
         println!("Hello from async closure.");
    };
    println!("Hello from main");
    let future = await_it(closure);
    println!("Hello from main again");
    futures::executor::block_on(future);
}

Non-capturing closures

A non-capturing closure is a closure that does not capture any of its environment, and only works with its parameters.

Such closure can be coerced into an fn() pointer with matching signature:

fn main() {
    fn internal(x: i32, y: i32) -> i32 {
        x + y
    }

    // a fn pointer for comparison
    let add_fn = internal;

    // a non-capturing closure
    let add_closure = |x, y| x + y;

    let mut x = add_closure(5,7);

    type Binop = fn(i32, i32) -> i32;

    let bo: Binop = add_fn;
    let bo: Binop = add_closure;

    x = bo(5,7);
}

Other "automatic" traits

We have already seen that closures can be Copy.

There are three other traits that can be satisfied by closures:

Clone
Sync
Send

Similarly to Copy, a closure is Clone if all of its captured variables are either captured by immutable reference, or by value if the types of all variables are Clone. Because all Copy types also implement Clone, all Copy closures satisfy Clone also.

A closure is Sync if all of its captured variables are Sync, meaning references to them are safe to be shared across threads. A type T is Sync if and only if &T is Send.

From this we can also deduce, that a closure is Send, if all of the types that are captured by reference are at least Sync, and all of the types captured by value are Send.

The task: Pipeline Programming

Sometimes, science is too pre-occupied with whether or not we could, that we do not stop to think if we should.

For this project, it will be your task to implement three traits, that will cover all types:

#![allow(unused)]
fn main() {
trait Pipe {
    /// take Self by ownership, returning T
    fn pipe<F, T>(self, apply: F) -> T
    where
        F: /* select correct Fn trait */ -> T;
}

trait PipeMut {
    /// take Self by &mut, returning T
    fn pipe_mut<F, T>(&mut self, apply: F) -> T
    where
        F: /* select correct Fn trait */ -> T;

    /// take Self by &mut, ignoring the result of the closure, returning &self
    fn pipe_ignore_mut<F, T>(&mut self, apply: F) -> &mut self
    where
        F: /* select correct Fn trait */ -> T;
}

trait PipeRef {
    /// take Self by &, returning T
    fn pipe_ref<F, T>(&self, apply: F) -> T
    where
        F: /* select correct Fn trait */ -> T;

    /// take Self by &, returning &self
    fn pipe_ignore<F, T>(&self, apply: F) -> T
    where
        F: /* select correct Fn trait */ -> T;
}
}

Complete the definitions of these traits with correct closure trait bounds;

Implement these three traits on types, such that this program will run:

use std::env::args;
fn main() {
    args()
        .skip(1)
        .next()
        .pipe(|o| o.unwrap())
        .pipe_ignore(|s| println!("received string: {}", s))
        .pipe(|s| s.chars().map(|c| c as u8 as usize).sum::<usize>())
        .pipe_ignore_mut(|s| println!("magic number: {}", s))
        .pipe(|s| *s + 1)
        .pipe_ignore(|s| println!("incremented: {}", s));
}

For braiins, the output should be:

received string: braiins
magic number: 744
incremented: 745