Reasons For Writing This

  1. There is a lack of documentation on ManuallyDrop<T>, which can be pretty dangerous to use if you’re not careful.
  2. I hope to fill the gap with this blog post on what I know about ManuallyDrop<T> based on my experience with it.

What is ManuallyDrop?

ManuallyDrop<T> is a marker type that signals to the Rust compiler to NOT call the destructor of T.

Context

I wrote sosecrets-rs crate and within the crate, the most important public struct is Secret<T, MEC: typenum::Unsigned, EC: typenum::Unsigned>.

It is a Secret wrapper type that reveals the secret at most MEC: typenum::Unsigned (i.e. unsigned integer) times with compile time guarantees and exposure of secrets is strictly controlled using its lifetime.

Its definition is as follows:

pub struct Secret<
    T,
    MEC: Unsigned,
    EC: Add<U1> + IsLessOrEqual<MEC, Output = True> + Unsigned = U0,
>(ManuallyDrop<T>, PhantomData<(MEC, EC)>);

It is a tuple type that simply wraps ManuallyDrop<T>. Let me explain why ManuallyDrop<T> is used.

A user of this Secret type can call its method expose_secret<ReturnType>(mut self, closure: impl for<'brand> fn(ExposedSecret<'brand, &'brand T>) -> ReturnType) -> (Secret<T, MEC, Add<EC, U1>>, ReturnType) and get a value of a completely ’new’ type, i.e. Secret with its type parameter EC incremented by ‘1’ (i.e. incremented by U1).

I want to support the usage of the Zeroize trait from the zeroize crate on the inner type T (zeroize will zero all bytes of T when T goes out of scope). Hence, I would need to implement the Drop trait for Secret<T, ...>. With a specified implementation of Drop trait, Rust does not allow you to move the inner T out of the Secret<T, ...>. Like so:

error[E0509]: cannot move out of type `Secret<T, MEC, EC>`, which implements the Drop trait
   --> src/secret.rs:140:17
    |
140 |         (Secret(self.0, PhantomData), returned_value)
    |                 ^^^^^^
    |                 |
    |                 cannot move out of here
    |                 move occurs because `self.0` has type `T`, which does not implement the `Copy` trait

Hence, to circumvent this is to wrap T with ManuallyDrop (and use core::mem::forget). This tells the compiler to NOT call the destructor of T.

Substructural Type System

Rust uses the affine substructural type system. Simply put, values of affine types can only be used at most once (i.e. 0 or 1 time) and they can be ’exchanged’, or ‘weakened’ but not ‘contracted’ 1. This is also known as ‘destructive’ move semantics 2. This is in contrast with linear types, which must be used exactly once 3. Linear substructural type system is not yet supported in Rust but there is strong interest in supporting it, especially in applications where modeling of digital assets is important (e.g. in the blockchain industry). The ‘hot potato’ pattern in Sui Move is an example of the application of linear types in the blockchain space.

Rust Safety Guarantees

I would not belabor on this because there are already a ton of good resources that explain Rust’s safety guarantees extremely well and I’m not going to do a better job than they. You can read about Rust Safety Guarantees from references like this paper.

Relevant to ManuallyDrop<T>, is that Rust never guarantees that destructors will be called 4. This is the reason why core::mem::forget is not marked as unsafe 5 and why ManuallyDrop<T> can exist.

Example

Let’s implement a type Counter<T: DoSomething, N: typenum::Unsigned> where DoSomething is a (dummy) trait and N is a type-level unsigned integer provided by the typenum crate. It has a count_up(...) method that returns itself with its N type parameter incremented by 1.

/*
[dependencies]
typenum = "1.17.0"
*/
use core::{
    marker::PhantomData,
    ops::{Add, Drop},
};

pub use typenum;
use typenum::{Sum, Unsigned, U0, U1};

struct Counter<T: DoSomething, N: Unsigned + Add<U1> = U0>(T, PhantomData<N>);

trait DoSomething {
    fn do_something_with_it(&mut self) {}
}

impl<T: DoSomething, N> Counter<T, N>
where
    N: Unsigned + Add<U1>,
    U1: Add<N>,
    Sum<N, U1>: Unsigned + Add<U1>,
{
  fn new(t: T) -> Self {
      Self(t, PhantomData)
  }

  fn count_up(self) -> Counter<T, Sum<N, U1>>
  where
      Sum<N, U1>: Add<U1> + Unsigned,
      N: Unsigned + Add<U1>,
  {
      println!(
          "Inside `Counter::count_up`, `N` before increment: {:?}",
          N::to_usize()
      );
      println!(
          "Inside `Counter::count_up`, `N` after increment: {:?}\n",
          Sum::<N, U1>::to_usize()
      );
      Counter(self.0, PhantomData::<Sum<N, U1>>)
  }
}

impl<T: DoSomething, N> Drop for Counter<T, N>
where
    N: Add<U1> + Unsigned,
{
  fn drop(&mut self) {
      println!(
          "Inside `Drop` implementation of Counter<T, {:?}>",
          N::to_u64()
      );
      // If you need to take out `T` from the `ManuallyDrop<T>` wrapper
      let inner = &mut self.0;
      inner.do_something_with_it();
      // Else, can use `ManuallyDrop<T>::drop` to drop the wrapper.
      // unsafe {
      //     ManuallyDrop::drop(&mut self.0);
      // }
  }
}

fn main() {
  use core::sync::atomic::{AtomicUsize, Ordering};

  static NUM_DROPS: AtomicUsize = AtomicUsize::new(0);
  struct DetectDrop {
      value: isize,
  }

  impl DoSomething for DetectDrop {}

  impl Drop for DetectDrop {
      fn drop(&mut self) {
          NUM_DROPS.fetch_add(1, Ordering::Relaxed);
      }
  }

  let counter: Counter<DetectDrop> = Counter::new(DetectDrop { value: 69 });

  {
      let counter = counter.count_up(); // `N` is now 1;
                                        // Inside `Counter::count_up`, `N` before increment: 0
                                        // Inside `Counter::count_up`, `N` after increment: 1
      assert_eq!(NUM_DROPS.load(Ordering::Relaxed), 1usize);
      assert_eq!(counter.0.value, 69isize);
      let counter = counter.count_up(); // `N` is now 2;
                                        // Inside `Counter::count_up`, `N` before increment: 1
                                        // Inside `Counter::count_up`, `N` after increment: 2
      assert_eq!(counter.0.value, 69isize);
      let counter = counter.count_up(); // `N` is now 3;
      assert_eq!(counter.0.value, 69isize);
      let counter = counter.count_up(); // `N` is now 4;
      assert_eq!(counter.0.value, 69isize);
      let _counter = counter.count_up(); // `N` is now 5;
                                          // Why is this `0`?
      assert_eq!(NUM_DROPS.load(Ordering::Relaxed), 5usize);
      assert_eq!(_counter.0.value, 69isize);
  }

  assert_eq!(NUM_DROPS.load(Ordering::Relaxed), 6usize);
}

What’s noteworthy is the following snippet:

fn count_up(self) -> Counter<T, Sum<N, U1>>
  where
      Sum<N, U1>: Add<U1> + Unsigned,
      N: Unsigned + Add<U1>,
  {
      println!(
          "Inside `Counter::count_up`, `N` before increment: {:?}",
          N::to_usize()
      );
      println!(
          "Inside `Counter::count_up`, `N` after increment: {:?}\n",
          Sum::<N, U1>::to_usize()
      );
      Counter(self.0, PhantomData::<Sum<N, U1>>)
  }

This function simply moves the self.0 out from the tuple struct and constructs a new Counter (Counter(self.0, PhantomData::<Sum<N, U1>>)) with its N type parameter equals to Sum<N, U1>, in layman’s terms, it is adding 1 to N at the type level.

Using the Playground, you will see the following error:

$ cargo build
   Compiling playground v0.1.0 (/home/explorer/playground)
error[E0509]: cannot move out of type `Counter<T, N>`, which implements the [`Drop`](https://doc.rust-lang.org/std/ops/trait.Drop.html) trait
  --> src/main.rs:42:17
   |
42 |         Counter(self.0, PhantomData::<Sum<N, U1>>)
   |                 ^^^^^^
   |                 |
   |                 cannot move out of here
   |                 move occurs because `self.0` has type `T`, which does not implement the `Copy` trait

For more information about this error, try `rustc --explain E0509`.
error: could not compile `playground` (bin "playground") due to 1 previous error

Two Ways to Drop

Now if we change the definition of Counter and its implementations to:

struct Counter<T: DoSomething, N: Unsigned + Add<U1> = U0>(ManuallyDrop<T>, PhantomData<N>);

impl<T: DoSomething, N> Counter<T, N>
where
    N: Unsigned + Add<U1>,
    U1: Add<N>,
    Sum<N, U1>: Unsigned + Add<U1>,
{
  fn new(t: T) -> Self {
      Self(ManuallyDrop::new(t), PhantomData)
  }

  fn count_up(mut self) -> Counter<T, Sum<N, U1>>
  where
      Sum<N, U1>: Add<U1> + Unsigned,
      N: Unsigned + Add<U1>,
  {
      println!(
          "Inside `Counter::count_up`, `N` before increment: {:?}",
          N::to_usize()
      );
      let inner = ManuallyDrop::new(unsafe { ManuallyDrop::take(&mut self.0) });
      println!(
          "Inside `Counter::count_up`, `N` after increment: {:?}\n",
          Sum::<N, U1>::to_usize()
      );
      core::mem::forget(self);
      Counter(inner, PhantomData::<Sum<N, U1>>)
  }
}

impl<T: DoSomething, N> Drop for Counter<T, N>
where
    N: Add<U1> + Unsigned,
{
  fn drop(&mut self) {
      println!(
          "Inside `Drop` implementation of Counter<T, {:?}>",
          N::to_u64()
      );
      // If you need to take out `T` from the `ManuallyDrop<T>` wrapper
      let mut inner = unsafe { ManuallyDrop::take(&mut self.0) };
      inner.do_something_with_it();
      // Else, can use `ManuallyDrop<T>::drop` to drop the wrapper.
      // unsafe {
      //     ManuallyDrop::drop(&mut self.0);
      // }
  }
}

It will now work and output this :

$ cargo build
   Compiling playground v0.1.0 (/home/explorer/playground)
    Finished dev [unoptimized + debuginfo] target(s) in 0.11s
$ ./playground
Inside `Counter::count_up`, `N` before increment: 0
Inside `Counter::count_up`, `N` after increment: 1

Inside `Counter::count_up`, `N` before increment: 1
Inside `Counter::count_up`, `N` after increment: 2

Inside `Counter::count_up`, `N` before increment: 2
Inside `Counter::count_up`, `N` after increment: 3

Inside `Counter::count_up`, `N` before increment: 3
Inside `Counter::count_up`, `N` after increment: 4

Inside `Counter::count_up`, `N` before increment: 4
Inside `Counter::count_up`, `N` after increment: 5

Inside `Drop` implementation of Counter<T, 5>

Playground

First, let’s look at the count_up(...) method, for this particular line let inner = ManuallyDrop::new(unsafe { ManuallyDrop::take(&mut self.0) });, we first take out the T from the ManuallyDrop<T> using ManuallyDrop::take, this function simply returns the T. However, there is a safety requirement, that is we have to ’ensure that this ManuallyDrop is not used again’. This is easy to ensure because core::mem::forget(self); results in the Counter<T, N> destructor being called only once (more details below).

It also calls core::mem::forget, to instruct the Rust’s compiler to NOT call the destructor of self. This is to avoid undefined behaviour.

Let’s imagine if we use Counter like so WITHOUT the line calling core::mem::forget(self);:

fn main() {
  use core::sync::atomic::{AtomicUsize, Ordering};

  static NUM_DROPS: AtomicUsize = AtomicUsize::new(0);
  struct DetectDrop {
      value: String,
  }

  impl DoSomething for DetectDrop {

      fn do_something_with_it(&mut self) {
          self.value = self.value.to_ascii_lowercase();
      }
  }

  impl Drop for DetectDrop {
      fn drop(&mut self) {
          NUM_DROPS.fetch_add(1, Ordering::Relaxed);
      }
  }

  let counter: Counter<DetectDrop> = Counter::new(DetectDrop { value: "HELLO".to_owned() });

  {
      let counter = counter.count_up(); // `N` is now 1;
                                        // Inside `Counter::count_up`, `N` before increment: 0
                                        // Inside `Counter::count_up`, `N` after increment: 1
      assert_eq!(NUM_DROPS.load(Ordering::Relaxed), 1usize);
      assert_eq!(counter.0.value, "HELLO".to_owned());
      let counter = counter.count_up(); // `N` is now 2;
                                        // Inside `Counter::count_up`, `N` before increment: 1
                                        // Inside `Counter::count_up`, `N` after increment: 2
      assert_eq!(counter.0.value, "HELLO".to_owned());
      let counter = counter.count_up(); // `N` is now 3;
      assert_eq!(counter.0.value, "HELLO".to_owned());
      let counter = counter.count_up(); // `N` is now 4;
      assert_eq!(counter.0.value, "HELLO".to_owned());
      let _counter = counter.count_up(); // `N` is now 5;
                                          // Why is this `0`?
      assert_eq!(NUM_DROPS.load(Ordering::Relaxed), 5usize);
      assert_eq!(_counter.0.value, "HELLO".to_owned());
  }

  assert_eq!(NUM_DROPS.load(Ordering::Relaxed), 6usize);
}

Using the Playground, you will see the following output:

    Finished dev [unoptimized + debuginfo] target(s) in 0.11s
$ ./playground
Inside `Counter::count_up`, `N` before increment: 0
Inside `Counter::count_up`, `N` after increment: 1

Inside `Drop` implementation of Counter<T, 0>
thread 'main' panicked at src/main.rs:96:9:
assertion `left == right` failed
  left: "�lVU\u{5}"
 right: "HELLO"
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Inside `Drop` implementation of Counter<T, 1>

It is obvious that counter.0.value is now undefined due to the drop implementation, which is explained below.

ManuallyDrop::take

impl<T: DoSomething, N> Drop for Counter<T, N>
where
    N: Add<U1> + Unsigned,
{
  fn drop(&mut self) {
      println!(
          "Inside `Drop` implementation of Counter<T, {:?}>",
          N::to_u64()
      );
      // If you need to take out `T` from the `ManuallyDrop<T>` wrapper
      let mut inner = unsafe { ManuallyDrop::take(&mut self.0) };
      inner.do_something_with_it();
      // Else, can use `ManuallyDrop<T>::drop` to drop the wrapper.
      // unsafe {
      //     ManuallyDrop::drop(&mut self.0);
      // }
  }
}

The first line let mut inner = unsafe { ManuallyDrop::take(&mut self.0) }; takes a T out from ManuallyDrop<T> and sets it as mutable. With it being mutable, its method that requires a &mut self can then be called to do whichever cleanup for T is required (inner.do_something_with_it();). The safety requirement for ManuallyDrop::take is satisfied with the same reasons mentioned above.

As inner is a named variable that owns a value of type T, when drop’s scope ends, and the value of inner will be dropped as well, hence, the value of type String in the previous example which should be the string "HELLO" is now undefined (i.e. ‘garbage’ values).

This is the reason why core::mem::forget(self); is required, it is to defer the actual dropping of T in the Drop implementation of Counter<T, N> to the ’last’ surviving Counter<T, N> whose count_up(...) method is never called.

Recall that each invocation of the count_up(...) method creates a new value of type Counter<T, N> and core::mem::forget(self) is only called within count_up(...), hence, a value of Counter<T, N> that never calls count_up(...) will have its destructor run when it goes out of scope.

ManuallyDrop::drop

If you do not require to do anything with ManuallyDrop<T>, you also can drop it directly with the following.

unsafe {
    ManuallyDrop::drop(&mut self.0);
}

This uses ManuallyDrop::drop to drop the entire ManuallyDrop<T>. Its safety requirements are quite a mouthful. It is basically saying that the memory allocated in ManuallyDrop<T> will be in a state that is NOT usable and therefore, must not be subjected to any further use. In our case, this is safe because we have used core::mem::forget to make sure the destructor of T is called only once. Otherwise, without core::mem::forget, you will get a process abort due to double free (See Playground).

Summary

  1. There are many reasons why you might want to use ManuallyDrop<T>, many times it is because you need to implement your own fn drop for your type’s implementation of the Drop trait.
  2. To safely ‘drop’ the inner T of your wrapper struct around ManuallyDrop<T> (i.e. a struct with a field of type ManuallyDrop<T>), you should use core::mem::forget to instruct the Rust compiler to NOT call the destructor of your wrapper struct around ManuallyDrop<T>, until the very last surviving value of your wrapper struct.
  3. You can drop the T in ManuallyDrop<T> either using ManuallyDrop::take or ManuallyDrop::drop. The former is useful if you need to do something with T before dropping it, and the latter is useful if you have no such need.