Naming is hard

Tired of E0597: borrowed value does not live long enough? 😆

Sometimes you just wish you could keep the variables you borrowed in your scope just a little longer... Well, nolife has got your back! Find it on lib.rs 🛒

Of course, usually it is better to re-architecture your application, or, God forbid, to sprinkle some clones here and there until the compiler is happy, but there are times where such an option is not available, or simply too costly.

Consider trying to implement the Read trait on a ZipFile:

use std::fs::File;
use zip::ZipArchive;

pub fn zip_streamer(file_name: &str, member_name: &str) -> impl Read {
    let file = File::open(file_name).unwrap();
    let mut archive = ZipArchive::new(file).unwrap();
    let file : ZipFile<'_> = archive.by_name(member_name).unwrap();
    file
}

The issue here is that the ZipFile<'_> references the archive we created in that same function, and so we cannot return it, because by exiting the function the archive will be destroyed, and the ZipFile will be dangling.

The example is lifted from this article on self-referential structs, feel free to go read it for a different perspective on the matter, I can wait 😊

Back to our issue, what we would like to do is something like:

use std::fs::File;
use zip::ZipArchive;

pub fn zip_streamer(file_name: &str, member_name: &str) -> impl Read {
    // freeze the scope in place, not closing it, and keeping file borrowed
    let zip_scope = scope!{
        let file = File::open(file_name).unwrap();
        let mut archive = ZipArchive::new(file).unwrap();
        let file : ZipFile<'_> = archive.by_name(member_name).unwrap();
        // 👇 some magic here
        freeze_forever!(&mut file)
    };

    // store it, without any lifetime!
    ZipStreamer { zip_scope }
}

struct ZipStreamer
{
    zip_scope: Scope,
}

impl Read for ZipStreamer
{
    fn read(&mut self, buf: &mut [u8]) -> std::io::Result<usize> {
        // enter the frozen scope to read from the zip file
        self.zip_scope.enter(|zip_file| zip_file.read(buf))
    }
}

with nolife, you can!

Introducing `nolife` 0.3

The nolife logo is a Rust lifetime crossed with a red cross

nolife was born in the end of 2022, from my frustration with the existing solutions to this common problem of having some borrowed data in a scope and wanting to keep the borrowed "view" of the data, for one reason or another:

Caching views that are costly to build
Storing the state of an iterator for later use.
Hiding "technical" intermediate objects from the API (such as the ZipArchive and ZipFile in the above)

Existing solutions mostly entail writing a self-referential struct, while my initial problem is about borrows in a scope. Self-referential structs in Rust suffer from many problems:

They're horribly unsafe and easy to get wrong.
Existing crates to achieve them historically had a lot of unsoundness issues
yoke exists but has very complicated signatures involving 2 unsafe traits
As soon as you have multiple level of borrowing, the problem becomes even harder
I just want to write a function constructing my view

Now, there's something in Rust that allows turning a function into a self-referential struct: async fn. In an async fn, any await point is susceptible of freezing the future and returning to the executor. All I needed to do for nolife was then to write an executor giving you access to a reference into some part of the future after the scope got frozen in that way. That's nolife.

Using `nolife`

Using nolife requires a bit of boilerplate, but with very low complexity.

Because Rust doesn't directly allow expressing that a type has a lifetime in a generic context, we need to define an empty, dummy struct and implement the nolife::Family trait on it.

struct ZipFamily; // helper struct whose sole purpose is to implement the Family trait
impl<'a> nolife::Family<'a> for ZipFamily {
    type Family = ZipFile<'a>;
}

The nolife::Family trait serves as a proxy so nolife can transform a ZipFile<'a> into a ZipFile<'b> as necessary.

Then, you merely copy-paste your borrowing code into an async function representing your scope:

async fn zip_file(
    file_name: String,
    member_name: String,
    mut time_capsule: nolife::TimeCapsule<ZipFamily>,
) -> nolife::Never {
    let file = File::open(file_name).unwrap();
    let mut archive = ZipArchive::new(file).unwrap();
    let file : ZipFile<'_> = archive.by_name(member_name).unwrap();
    time_capsule.freeze_forever(&mut file).await // freezing the ZipFile<'a> here!
}

The function differs from our original in four ways:

It is now async.
It has an additional nolife::TimeCapsule parameter, parameterized by our dummy struct.
It now returns a nolife::Never.
The time_capsule is used to freeze a reference to the ZipFile in time.

Just with this, we can implement our ZipStreamer object and the Read trait on it!

struct ZipStreamer {
    zip_scope: nolife::DynBoxScope<ZipFamily>,
}

A nolife::DynBoxScope is the main nolife struct, and the executor of the async function we wrote above. Parameterized by some type implementing the Family trait, it contains the lifetime-erased scope.

After creation, we can access the borrowed data in the frozen scope with the enter method:

impl Read for ZipStreamer {
    fn read(&mut self, buf: &mut [u8]) -> std::io::Result<usize> {
        self.zip_scope.enter(|zip_file| zip_file.read(buf))
    }
}

Lastly, we need to return a ZipStreamer as an anonymous Reader, after initializing the scope:

pub fn zip_streamer(file_name: String, member_name: String) -> impl std::io::Read {
    let zip_scope =
        nolife::DynBoxScope::pin(|time_capsule| zip_file(file_name, member_name, time_capsule));
    ZipStreamer { zip_scope }
}

And the compiler no longer complains! That's all there is to it.

Testing

Let's make a small test executable:

fn main() {
    let mut output = String::new();
    zip_streamer(
        std::env::args().nth(1).unwrap(),
        std::env::args().nth(2).unwrap(),
    )
    .read_to_string(&mut output)
    .unwrap();
    println!("{}", output);
}

and test it:

mkdir toto
echo "this is titi" > toto/titi.txt
echo "this is tutu" > toto/tutu.txt
7z a toto.zip toto
cargo run --release -- toto.zip toto/titi.txt  # prints "this is titi"
cargo run --release -- toto.zip toto/tutu.txt  # prints "this is tutu"

It works!

Conclusion

nolife is tested using miri to look for any unsoundness I could think of. I spent the first few versions figuring out a minimal API surface, and I feel the library is now usable 🚀

The cost of using nolife should usually be two allocations, one for the scope itself, and one for the inner future. At the price of an additional parameter, you can also use a generic future to save one allocation.

I think nolife is a much more intuitive alternative to manually rolling your self-referential struct or using self-referential-struct-oriented crates like owning_ref or yoke.

Currently nolife doesn't provide any sugar over what it does. I considered adding macros to hide the asyncness of the scope and the Family trait, but adding syn to my dependencies kills compile times for gains in ergonomy that are questionable at best.

Appendix: Full `main.rs`

use std::{fs::File, io::Read};

use zip::{read::ZipFile, ZipArchive};

struct ZipFamily;
impl<'a> nolife::Family<'a> for ZipFamily {
    type Family = ZipFile<'a>;
}

async fn zip_file(
    file_name: String,
    member_name: String,
    mut time_capsule: nolife::TimeCapsule<ZipFamily>,
) -> nolife::Never {
    let file = File::open(file_name).unwrap();
    let mut archive = ZipArchive::new(file).unwrap();
    let mut by_name = archive.by_name(&member_name).unwrap();
    time_capsule.freeze_forever(&mut by_name).await
}

struct ZipStreamer {
    zip_scope: nolife::DynBoxScope<ZipFamily>,
}

impl Read for ZipStreamer {
    fn read(&mut self, buf: &mut [u8]) -> std::io::Result<usize> {
        self.zip_scope.enter(|zip_file| zip_file.read(buf))
    }
}

pub fn zip_streamer(file_name: String, member_name: String) -> impl std::io::Read {
    let zip_scope =
        nolife::DynBoxScope::pin(|time_capsule| zip_file(file_name, member_name, time_capsule));
    ZipStreamer { zip_scope }
}

fn main() {
    let mut output = String::new();
    zip_streamer(
        std::env::args().nth(1).unwrap(),
        std::env::args().nth(2).unwrap(),
    )
    .read_to_string(&mut output)
    .unwrap();
    println!("{}", output);
}

Introducing nolife 0.3

Using nolife

Testing

Conclusion

Appendix: Full main.rs

Introducing `nolife` 0.3

Using `nolife`

Appendix: Full `main.rs`