co_awaiting coroutines

Have you been co_awaiting the next post about the coroutines? Well after some break from writing finally it’s here.

So first of all, if you are reading this post, I assume you are already familiar with the introduction to the coroutines and description of the promise types, since this is just a continuation of a series of the posts about the coroutines. At least you should know by now how to create your own coroutine from scratch and how to communicate from the coroutine to the caller (co_return and co_yield implementations). Specifically, you should know about the Promise concept.

This time we will be mainly talking about the Awaitable and Awaiter concept and co_await operator, but we will also explain one more thing from the Promise concept, namely, the await_transform function member. Once you know all those things, you will be able to implement any coroutine you imagine. That’s good news, isn’t it? 🙂

So let’s get to work.

Some of you, that have already faced the coroutines from different languages might already know what we are going to talk about in this post. From all the things we have already said about the coroutines, we still do not know how to actually communicate with the coroutine from the caller to the callee. Concrete example: we want to resume the coroutine passing some value to it. With the knowledge from the previous posts, we cannot achieve that.

We cannot because we do not know about the co_await and Awaiter in detail, but first, we need to know about

Awaitable

So there is something magical about the co_await operator. Right now we know, that you can call it with the suspend_never and suspend_always operands, but how this works is actually a bit of magic. We will be explaining the magic in this section.

As I have mentioned in the previous posts, the suspend_always and suspend_never are types, that fulfill the Awaitable concept. The awaitable is needed for us only to get the object, that fulfills the Awaiter concept.

We can get the Awaitable object in two ways:

  • Direct creation of Awaitable,
  • Transformation of the object into Awaitable because of await_transform function.

The first way of acquiring the object of Awaitable concept is already known. We simply create the object of type, that fulfills the Awaitable concept just as we did by creating the object of the suspend_always or supend_never types.

The other way is a little bit different and goes for the following rule: if the promise type has await_transform function member, then the mentioned function is called with the expression given, as the argument of the co_await operator, as its argument and the result of the function await_expression becomes the argument of the co_await operator. In other words this:

co_await expression;

gets transformed to this:

co_await p.await_trasform(expression);

where p is the object of the Promise type.

Also the await_transform function lookup and transformation will not take place if the expression is initial suspend, final suspend and also if expression is produced by the co_yield statement.

so now the question what is actually co_await operator doing

co_await operator and Awaiter

The co_await operator is actually responsible for two things:

  • Forcing compiler to generate some coroutine boilerplate code
  • Creating the Awaiter object.

So first let’s have a look at how is the awaiter object is created. The co_await operator is responsible for the creation of the awaiter object. The co_await operator declaration is looked upon in the awaitable object and if it’s found this co_await operator is executed to obtain awaiter object. Otherwise, if the appropriate function is not found, then awaitable becomes the awaiter. We will talk about this in detail later.

Now let’s see what kind of the code is generated by the compiler. Whenever compiler encounters occurrence of the co_await operator it generates the more or less following code:

std::exception_ptr exception = nullptr;
if (not a.await_ready()) {
  suspend_coroutine();

  //if await_suspend returns void
  try {
    a.await_suspend(coroutine_handle);
    return_to_the_caller();
  } catch (...) {
    exception = std::current_exception();
    goto resume_point;
  }
  //endif


  //if await_suspend returns bool
  bool await_suspend_result;
  try {
    await_suspend_result = a.await_suspend(coroutine_handle);
  } catch (...) {
    exception = std::current_exception();
    goto resume_point;
  }
  if (not await_suspend_result)
    goto resume_point;
  return_to_the_caller();
  //endif

  //if await_suspend returns another coroutine_handle
  decltype(a.await_suspend(std::declval<coro_handle_t>())) another_coro_handle;
  try {
    another_coro_handle = a.await_suspend(coroutine_handle);
  } catch (...) {
    exception = std::current_exception();
    goto resume_point;
  }
  another_coro_handle.resume();
  return_to_the_caller();
  //endif
}

resume_point:
if(exception)
  std::rethrow_exception(exception);
"return" a.await_resume();

Even though the code is quite complex we will manage to figure out what is happening in it.

So first of all the Awaiter’s await_ready function member indicates whether the “thing” we are awaiting is ready or not. If it is (await_ready returns true), then there is not much that the program does. We just jump to the resume_point and the result of the co_await expression is whatever await_resume returns.

When the await_ready function evaluates to false, however, there are three things that need to happen:

  • coroutine gets suspended
  • await_suspend is executed
  • based on the await_suspend returned value, another coroutine is executed or program returns control flow to the caller.

Coroutine suspension

Immediately after evaluating await_ready expression the coroutine is suspended (on condition, that await_ready evaluated to false). It doesn’t mean, that control flow returns to the caller. It means that the needed local data is saved (possibly on the heap) to be restored when coroutine is suspended. The point of resumption of the coroutine is the resume_point label, that can be seen in the code example above.

Just after the coroutine suspension, await_suspend is evaluated to decide where the control flow should go to.

await_suspend expression

The await_suspend expression is of the following form:

a.await_suspend(coroutine_handle);

The await_suspend function can return:

  • void
  • bool by a value
  • coroutine_handle by value

The await_suspend function exists to decide where the control flow should go after the coroutine is suspended. If our coroutine returns void, then coroutine remains suspended and the control flow is returned to the caller or the resumer of the coroutine.

We might also decide inside the await_suspend function to continue execution of the coroutine. For this exact purpose, there is the option for the await_suspend function to return a boolean value. If the value evaluates to true, then the result is the same as if the void returning version of the function was called. Otherwise (if the function returns false) the coroutine is immediately resumed and control flow is not returned to the caller. The resume point is the resume_point label from the code example above.

The last thing we can do is to give control flow to a different coroutine. To do that we should implement the version of the function, that returns coroutine handle to another coroutine. This option, at first sight, is missing the possibility to give control back to the caller, or resume the coroutine immediately and go to the resume_point label. You can achieve the return to the caller behavior by returning std::noop_coroutine_handle (it can be created by the call to the std::noop_coroutine function). Similarly, you can achieve the resumption to the current coroutine you are in by returning the handle to that coroutine (in other words return the argument of the await_suspend).

await_suspend and exception

So there is always this issue with fancy features like that – what happens if an exception is thrown. You could have already figured this out by analyzing the code snippet above, generally, once the exception is thrown from the await_suspend, the exception is caught and saved, coroutine gets resumed and the exception is immediately rethrown. Please note, that await_resume will not be called.

You might be curious what will happen when an exception is thrown in the await_ready or await_resume functions. In those cases, the functions are called as a part of the active coroutine so the exception would be thrown as in any other function called inside the coroutine.

If the exception is thrown in the await_ready function, then the coroutine is not suspended at all. Neither the await_suspend nor await_resume functions will be evaluated.

On the other hand when the exception would be thrown inside the await_resume function, then both await_ready and await_suspend functions would be evaluated (await_suspend would be evaluated only if await_ready evaluates to false) and all the consequences of their returned value would be visible. The co_await expression, on the other hand, wouldn’t have a result value, speaking of which…

co_await result

The value, that is finally returned from the co_await expression is the result of the await_resume function.

custom co_await operator

The legendary flexibility of the C++ language is confirmed also in the coroutine feature. Not only you can implement your own coroutine on the low level. You can also customize the creation of the Awaiter on two levels:

  • custom await_transform function in the promise_type.
  • custom co_await operator in the Awaitable type.

We were already talking about the await_transfrom function, so now let’s talk a little bit about the co_await operator.

First, let me explain. You cannot change what kind of code does compiler generate when it encounters a co_await operator, defining custom co_await operator will only modify the way, awaiter object is obtained.

Custom co_await operator can be defined as a freestanding co_await function, that takes the awaitable as the argument or as a function member of the Awaitable type.

In case the co_await operator is defined the Awaiter is obtained as follows:

Awaiter&& a = awaitable.operator co_await();

if there is no operator co_await inside the awaitable, then the freestanding function is being searched and awaiter is acquired like:

Awaiter&& a = operator co_await(awaitable);

If there is no applicable co_await operator found, then awaitable becomes our awaiter object in following way:

auto&& awaiter = awaitable;

So as you can see there is a lot of things that are going on. Please read it twice or more if you need to understand it. Once you get the clue of what does co_await operator do, let’s see how we can implement some awaitables.

Examples

So now once we know, what does co_await operator do, we can implement some Awaitables and co_await operators ourselves. Let’s start with some easy examples.

suspend_always and suspend_never

The most primitive Awaitables are ones provided by the standard library – suspend_always and suspend_never. Suspend_always is the type, that makes co_await operator always suspend the coroutine and return back to the caller. Suspend_never on the other hand never suspends the coroutine.

Let’s start with the simpler type.

suspend_never

Let’s have a look at the source code, that implements the suspend_never type:

struct suspend_never {
  constexpr bool await_ready() const noexcept { return true; }
  constexpr void await_suspend(coroutine_handle<>) const noexcept {}
  constexpr void await_resume() const noexcept {}
};

Implementations of the suspend_always and never are actually specified by the standard. The way suspend_never works is, that await_ready always returns true. If we say, that awaitable is ready, then there is no need to perform any more computations, thus the implementation of the await_suspend will have no impact since the function will never be called (it must be defined though for the compilation process to succeed). The await_resume function returns void, so the result of the co_await expression will also be void.

suspend_always

We implemented the most simple awaitable, now it’s time for something a little bit more complex and we will implement suspend_always.

Suspend_always always suspends the coroutine, thus the await_ready() function cannot return true. If it cannot return true, it must return false :).
The await_suspend could be implemented as returning void, boolean with value true, or can return noop_coroutine_handle. Any of those three values will cause coroutine to suspend and return to the caller.

The co_await operator called on the suspend_always object will also result in void return type, so the await_resume needs to return void.

Summing this all together, we might get following implementation:

struct suspend_always {
  constexpr bool await_ready() const noexcept { return false; }
  constexpr void await_suspend(coroutine_handle<>) const noexcept {}
  constexpr void await_resume() const noexcept {}
};

This is actually how standard says, the suspend_always should be defined. If it’s about the await_suspend, its simplest form is used.

State machines

We all know about the state machines. Usually, they consist of some states connected with each other with some kind of transitions. The transitions from one state to another happen when the signal with some value is sent to the state machine. We can implement this using the coroutines. I am not claiming, that this is the best way to actually implement the state machines, but I think it is a good exercise. You can actually try to do it yourself before reading the solution below.

So the state machine I implemented was actually a state machine, which was waiting for two clicks of the mouse button and then the file was opened. The coroutine looked like the following:

The coroutine body

StateMachine<button_press, std::FILE *> open_file(const char *file_name) 
  using this_coroutine = StateMachine<button_press, std::FILE *>;
  button_press first_button = co_await this_coroutine::signal{};
  while (true) {
    button_press second_button = co_await this_coroutine::signal{};
    if (first_button == button_press::LEFT_MOUSE and
        second_button == button_press::LEFT_MOUSE)
      co_return std::fopen(file_name, "r");

    first_button = second_button;
  }
}

What can we see in the implementation:

We do co_await on some new type. We hope that the result of the co_await this_coroutine::signal{} will be the signal passed to the coroutine. We will see in a second what implementation is needed to achieve this kind of behavior.

We can also see, that two subsequent signals of LEFT_MOUSE clicks will result in co_return returning a pointer to the file.

At the same time, we can see, that there is no need to suspend the coroutine at the beginning of the function since this will make StateMachine API more complicated.

Having said that we can start implementing our StateMachine type and the promise_type.

Coroutine interface type

template <typename Signal, typename Result> class StateMachine {
public:
  using coro_handle = stde::coroutine_handle<promise_type>;
  StateMachine(coro_handle coro_handle) : coroutine_handle(coro_handle) {}
  StateMachine(StateMachine &&) = default;
  StateMachine(const StateMachine &) = delete;
  ~StateMachine() { coroutine_handle.destroy(); }

  void send_signal(Signal signal) {
    coroutine_handle.promise().recent_signal = signal;
    if (not coroutine_handle.done())
      coroutine_handle.resume();
  }

  std::optional<Result> get_result() {
    return coroutine_handle.promise().returned_value;
  }

private:
  coro_handle coroutine_handle;
};

Ok, so we have got an API and its implementation of the StateMachine type. This type is templated with two other types. The first type will be the type of the signal, that is sent to the state machine, the second is the type returned from the state machine once it ends.

Of course, in order to have some connection with the coroutine state, the machine must hold coroutine_handle inside and also to initialize it, it takes the coroutine_handle as an argument in the constructor.

We make the copy of the StateMachine impossible, but move is allowed.

The most important is the send_signal function, that sends the signal to the coroutine by assigning one of the promise_types data members with a value and resumes the coroutine. There is also the get_result function. The get result function returns the result of the coroutine. The result is optional because the coroutine might not have ended its execution yet (thus it might be empty).

What is left to implement is the Awaiter, Awaitable and the promise_type

Promise type and Awaitables

Let’s start with the Awaiter and Awaitable

As we have mentioned Awaitable is the expression, that is an argument of the co_await operator. From the coroutine body we can see, that, the signal{} is our awaitable. Its implementation couldn’t be simpler as it’s just:

struct signal {};

Of course, it is not enough, because we also need to have some object, that fulfills the Awaiter concept. In our case the type looks like the following:

struct SignalAwaiter {
  std::optional<Signal> &recent_signal;
  SignalAwaiter(std::optional<Signal> &signal) : recent_signal(signal) {}

  bool await_ready() { return recent_signal.has_value(); }
  void await_suspend(stde::coroutine_handle<promise_type> coro_handle) {}

  Signal await_resume() {
    assert(recent_signal.has_value());
    Signal tmp = *recent_signal;
    recent_signal.reset();
    return *recent_signal;
  }
};

Ok, what we have here is the SignalAwaiter, that stores the reference to the optional signal. We assume that the reference to the optional is pointing to the value stored inside the StateMachine.

The await_ready function returns true, when we have some value inside the optional, since then it would be pointless for us to resume the coroutine. Otherwise, we suspend the coroutine waiting for the next value. When the coroutine is resumed, the value is fetched from the optional (we assume, that now the optional has a value) and reset the optional.

In this case, the SignalAwaiter is part of the promise_type, which implementation is as follows:

struct promise_type {
    std::optional<Signal> recent_signal;
    std::optional<Result> returned_value;

    StateMachine get_return_object() {
      return stde::coroutine_handle<promise_type>::from_promise(*this);
    }
    stde::suspend_never initial_suspend() { return {}; }
    stde::suspend_always final_suspend() { return {}; }

    void unhandled_exception() {
      auto exceptionPtr = std::current_exception();
      if(exceptionPtr)
        std::rethrow_exception(exceptionPtr);
    }

    void return_value(Result value) { returned_value.emplace(value); };

    struct SignalAwaiter {
      /* implementation */
    };

    SignalAwaiter await_transform(signal) {
      return SignalAwaiter(recent_signal);
    }
  };

The promise type contains the recent_signal, that has been sent through the StateMachine object. and the optional returned_value, to store the result of the coroutine once one goes to its end.

We do not want to suspend coroutine at it’s beginning, but we want to suspend it on it’s finish, to be able to hold the result value in the promise_type (once the coroutine goes out of the body, the promise is destroyed).

In case of the unhandled_exception, we are just throwing the exception further to the caller/resumer of the coroutine.

On the co_return we need to save the returned value into our promise type, to be able to refer to it from the StateMachine type.

And the key thing is the await_transform, which creates the object, that fulfills the Awaiter concept. The await_transform creates the SignalAwaiter object with the optional signal as its argument. The Awaiter will later on know based on the value of the optional whether the suspension of the coroutine should be performed.

Simple test of the StateMachine

We can test the behavior of the coroutine with following code:

int main() {
  auto machine = open_file("test");
  machine.send_signal(button_press::LEFT_MOUSE);
  machine.send_signal(button_press::RIGHT_MOUSE);
  machine.send_signal(button_press::LEFT_MOUSE);
  machine.send_signal(button_press::LEFT_MOUSE);

  auto result = machine.get_result();
  milli::raii close_guard ([&result] {
    if (result.has_value())
      std::fclose(*result);
  });

  std::cout << result.value() << std::endl;

  return 0;
}

The test is pretty straightforward and you can play with it. If you wonder about the milli::raii, then it’s the type defined in the library of mine – milli. Right now it’s not production ready yet. The raii thing makes one thing – invokes the given lambda once the variable goes out of scope. This causes, that file will be properly closed once the main function finishes.

The thing that is to be noticed is, that the StateMachine is a very simplified example of what coroutines are capable of. I.e. we cannot execute co_routine in a different thread and send signals from another thread, since the implementation is not thread-safe and execution starts immediately after the send_signal function is executed.

Summary

That’s it. Right now you should be able to implement any coroutine you imagine as you know all the tools for that. Of course, it will not be easy at first it would be good to get some experience with the coroutines.

Since in this post we were only looking into some easy examples of the coroutines and Awaitables, in the next post I am planning to have a look into more complex but also more useful examples of the coroutines for this we will probably use the cppcoro library.

When looking into the coroutines you will definitely feel (or should feel) that this is not something you want to write as your daily work and you are right! In the future standard library will come with predefined sets of implemented Awaitables, and coroutine types. Once C++20 comes out, make yourself and your friend the pleasure and use cppcoro instead writing your own implementations.

I hope that you have learned something with those posts and this will make your understanding of the coroutines better once they are available in C++20 🙂


Bibliography


6 responses to “co_awaiting coroutines”

  1. The sentence:

    > The value, that is finally returned from the co_await expression is the result of the await_ready function.

    should be changed to the following, I suppose:

    > The value, that is finally returned from the co_await expression is the result of the await_resume function.

      • I think he means:
        result of the **await_resume** function
        instead of the **await_ready** function.

        I have another question about the logic of your state-machine example. Maybe, I didn’t get it right.

        The send_signal function will change the “recent_signal” member of the promise object. And resume the suspended routine til next co_await point. At this point, a new awaiter is created with the await_transform function from promise. This will use “recent_signal” member set from last “send_signal” call.

        So the value change looks like this:
        first_button = Signal{}
        second_button = button_press::LEFT_MOUSE
        first_button = button_press::LEFT_MOUSE
        second_button = button_press::RIGHT_MOUSE
        co_return()

        Is it the intended behavior?

          • Thanks for question! Actually it’s been already a bit of time since I wrote that examples, so I had to reread them from scratch 😉

            So, the first time, we do co_await, no value is produced – when await_ready returns false, the coroutine gets suspended. When we send_signal, the recent_signal gets updated the coroutine gets resumed and only then the value is produced. co_await can be understood intuitivelly as waiting for something to become available. In this case it’s value of the signal, so we fetch it once it’s available (after the first send_signal call).

            Feel free to ask further, if you still have some doubts. Coroutines are not easy to understand

            You also reminded me about milli, that I have abondoned quite a long time ago xD

Leave a Reply to [email protected] Cancel reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.