Your first coroutine

Once you got familiar with the introduction to the coroutines, then I think it’s high time to actually implement your first co-routine object. This whole post is all about understanding how to implement co-routines and related entities (especially promise_type). Before we start the adventure, please ensure that your compiler does support coroutines. (as of this writing gcc does not support coroutines at all, my propositions of compilers are clang and msvc. The examples in this post should all compile with clang 9). This is going to be a long post, so reserve yourself an hour or two.

First let me ask you a question. Given following code snippet:

std::future<int> foo();

Is this a function or co-routine declaration? How do you think?

Well, at first sight, it does not differ at all from the regular function. It actually looks exactly as a function declaration. But whether this is a function or co-routine is actually just an implementation detail, or to rephrase it, whether the function is co-routine or not depends on its body. The co-routines, as they are going to get into the C++20, define three new keywords.

• co_await
• co_return
• co_yield

If any of those keywords occur in the function, then it becomes a coroutine.

Why co_ at the beginning of the keywords? For backward compatibility, we do not want to make programs, that for example defined function yield() for their implementations of the stackful coroutines, ill-formed.

Let’s then try to write our first co-routine.

#include <experimental/coroutine>

void foo(){
  std::cout << "Hello" << std::endl;
  co_await std::experimental::suspend_always();
  std::cout << "World" << std::endl;
}

2 new things spotted in the wild:

• previously mentioned co_await,
• suspend_always object.

So the operator co_await is a unary operator, which takes the Awaitable object as its argument. We will talk about the Awaitable concept in detail in the next post about the coroutines. Right now, all you need to know is suspend_always is the Awaitable object (defined in experimental/coroutine), and calling co_await operator on it will result in suspending the coroutine. [Note: there is also suspend_never defined in the standard library, that will not cause a suspend when co_await is performed on this object.]

We do not know how to call this kind of a subroutine yet, so let’s skip this for now. But even if we try to compile this code we will get the following error:

Compilation error in case of msvc (manually translated to english)

“promise_type”: is not member of “std::experimental::coroutine_traits<void>

This is not particularly useful information to us without a deeper understanding of coroutines, but we already see, that there are some additional types, that need to be defined.

Why do we need to define additional types?

You probably know, the C++ language really is a flexible one. There is one good meme about it, I was shown during a recent discussion with the colleagues of mine:

Well, in the case of the coroutines, we need to implement most of their behaviour by ourselves. The keywords only generate some boilerplate code for the coroutines, which we need to customise via customisation points. So let’s try to understand what is wrong with our coroutine.

Implementing a coroutine

The thing is, that once we have our coroutine, there needs to be some way to communicate with it. If our coroutine suspends, there must be some way to resume the coroutine, and for this particular task, we need to have a dedicated object.

So the object used to communicate with the coroutine is the object of the coroutine’s return type. Let’s start slowly implementing this type. If our coroutine is able to suspend, we need to be able to somehow resume it, so our return type needs to have some resume() method. Let’s create this type.

class resumable{
public:
  bool resume();
};

The call to the resume method will resume the coroutine if it hasn’t finished yet it’s execution, and the return value will indicate whether after resumption there still is something to be executed inside the coroutine.

so our coroutine definition needs to change as well (return type updated):

#include <experimental/coroutine>

resumable foo(){
  std::cout << "Hello" << std::endl;
  co_await std::experimental::suspend_always();
  std::cout << "Coroutine" << std::endl;
}

So the question now arises. How will the resumable object be created (there is no return statement)? This is a little bit of the coroutines magic, that happens under the hood. Basically, the compiler generates some additional code for the coroutines. Every coroutine function is transformed by the compiler to the form similar to this:

Promise promise;
co_await promise.initial_suspend();
try {
  // co-routine body
}
catch(...) {
  promise.unhandled_exception(); 
}
final_suspend:
  co_await promise.final_suspend();

What is more, the resumable object will be created before the initial_suspend through a call to the:

promise.get_return_object();

So what we need to do is to create the Promise type. We can do it in two ways:

• create promise_type as a member of the resumable type (or create an alias of the same name)
• specialize coroutine_traits<resumable> type and define promise_type in it (you can even specialize coroutine_traits<resumable, Args…> to differentiate coroutines) or create an alias of the same name.

And if it’s not enough, then if your coroutine does not return anything (through co_return) and flow inside coroutine possibly reaches the functions end, then we also need to have additional member function defined – return_void.

Let’s see the draft of our type:

class resumable{
public:
  struct promise_type;
  bool resume();
};

struct resumable::promise_type{
  resumable get_return_object() {/**/}
  auto initial_suspend() {/**/}
  auto final_suspend() {/**/}
  void return_void() {}
  void unhandled_exception();
};

Unfortunately, we are not yet ready to define functions of our first promise type. To operate on the coroutine, we need to have some sort of handle to the coroutine, which we will manage. There already is a built-in object for this exact purpose.

The coroutine_handle

The coroutine_handle is an object, that refers to the coroutine’s dynamically allocated state. Thanks to this object you can, for example, resume the coroutine. The coroutine_handle is a templated type, where Promise type is its template argument. Let’s have a look at its definition:

template <typename Promise = void>
struct coroutine_handle;

template <typename Promise>
struct coroutine_handle : coroutine_handle<> {
  using coroutine_handle<>::coroutine_handle;
  static coroutine_handle from_promise(Promise&);
  coroutine_handle& operator=(nullptr_t) noexcept;
  constexpr static coroutine_handle from_address(void* addr);
  Promise& promise() const;
}

so now we need to have a look at the specialization:

template <> struct coroutine_handle<void>{
  constexpr coroutine_handle() noexcept;
  constexpr coroutine_handle(nullptr_t) noexcept;
  coroutine_handle& operator=(nullptr_t) noexcept;
  constexpr void* address() const noexcept;
  constexpr static coroutine_handle from_address(void* addr);
  constexpr explicit operator bool() const noexcept;
  bool done() const;
  void operator()();
  void resume();
  void destroy();
private:
  void* ptr;// exposition only
};

Coroutine_handle emptiness

Ok, so first of all, the coroutine_handle is default constructible. The effect of the parameterless constructor call is the same as calling the constructor with the nullptr_t parameter. When such constructor is invoked, the coroutine_handle will be empty. Its emptiness can be checked with the operator bool or by comparison of the result of the address member function with nullptr literal. [Note: the bool operator might easily be mistaken with the done member function. It’s worth to pay attention to it.]

Valid coroutine_handle

If we want to create non-empty coroutine, we need to use its static member function from_address. This is not something we should really pay attention to since the creation of the coroutine_handle lays usually on the compiler’s responsibility. Once the coroutine is created we can use bool operator and address member function to check its validity.

Coroutine resumption

There are two ways to resume the coroutine. First one by the call to the resume member function of the coroutine_handle and another via the call to the function call operator of the same object.

Coroutine destruction

Coroutine destruction happens whichever of the following events occurs first. A first possible event is a call to the destroy member function of the coroutine_handle type. Coroutine destruction also occurs when the flow goes off the coroutine function (after the final suspend point).

Checking the state of the execution

The coroutine can be suspended many times during its lifetime. If (and only if) the coroutine is suspended at it’s final suspend point, the method done returns true.

Coroutine_handle and Promise type

Coroutine_handle specialisations make it possible to create coroutine_handle from the promise object and allow to get the promise object of a coroutine.[Note: manual specialisations of the coroutine_handles are possible, but the behaviour of a program with manual specialization of coroutine_handle is undefined.]

Implementing promise_type

Now we have got all the needed knowledge to finish the implementation of the resumable and promise_type.

Let’s begin with the resumable type.

class resumable {
public:
  struct promise_type;
  using coro_handle = std::experimental::coroutine_handle<promise_type>;

  resumable(coro_handle handle) : handle_(handle) { assert(handle); }
  resumable(resumable&) = delete;
  resumable(resumable&&) = delete;

  bool resume() {
    if (not handle_.done())
      handle_.resume();
    return not handle_.done();
  }
  ~resumable() { handle_.destroy(); }
private:
  coro_handle handle_;
};

So, first of all, we do not want to copy this object, since we can call destroy function member only once. We also do not allow moving the object around for the code’s simplicity. The resume logic is as follows: if our coroutine didn’t finish yet, then resume it, otherwise, don’t. The value returned tells whether, after the call to resume, a coroutine can again be continued or not.

Another thing is, that resumable uses done function member. For this method to work correctly, coroutine must suspend itself on the final suspend point.

Also, a side note: the coroutine_handle object is not thread-safe. Simultaneous access to the destroy and resume methods might result in a data race. Multiple calls to destroy function is an error just like calling free two times on the same pointer, so it’s essential to be careful when destroying coroutines [Note: coroutine destroys itself also after reaching its very end (after final suspend point)].

Now we need to define the promise_type, so that co_routine behaves according to our expectations.

struct resumable::promise_type {
  using coro_handle = std::experimental::coroutine_handle<promise_type>;
  auto get_return_object() {
    return coro_handle::from_promise(*this);
  }
  auto initial_suspend() { return std::experimental::suspend_always(); }
  auto final_suspend() { return std::experimental::suspend_always(); }
  void return_void() {}
  void unhandled_exception() {
    std::terminate();
  }
};

So first the get_return_object is called to create the resumable type. The resumable type needs the coroutine_handle which can be created from the *this promise object.

Right after this, the initial_suspend member function will be called. For our example it makes no difference whether or not the coroutine will suspend its execution at the beginning or not, so by the dice roll suspend_always was chosen. This will result in a fact, that we will need to call resume for the coroutine to start its execution.

The unhandled_exception must be defined, but for our case, it makes no difference what it’s implementation will be, because we won’t throw an exception.

The final_suspend must, in our case, return always_suspend since only then the done method of the coroutine_handle will work adequately (return true when function suspends on final_suspend).

The last method return_void even though it’s empty is obligatory for our coroutine. In case of the coroutines which never end (often generators), neither return_void nor return_value functions need to be defined. But in our case, the coroutine eventually runs till the end of its body, and if there was no return_void, then we would meet undefined behaviour on coroutine return. If coroutines return some values via co_return, then instead of return_void, the return_value function needs to be defined.

Using our coroutine

Once we have defined our own coroutine, let’s see how should we use it.

int main(){
  resumable res = foo();
  while (res.resume());
}

And the expected result should be

Hello
Coroutine

Deeper into promise type.

So besides what we have shown, the promise type can have more methods than those we implemented. In this section, we will take a closer look into that.

Memory Allocation

When the coroutine state gets allocated, then it might happen, that allocation will take place on the heap (we should assume it will happen, but the compiler is free to optimize this out). If such allocation will happen and the promise_type has get_return_object_on_allocation_failure declared, then non-throwing version of the operator new will be searched for. If the allocation fails, we will not have the exception thrown in the face immediately, what compiler will do instead, is, call the get_return_object_on_allocation_failure static function member on the caller side. We can extend our promise type:

struct resumable::promise_type {
  // ...
  static resumable get_return_object_on_allocation_failure(){
    throw std::bad_alloc();   
  }
  // ...
};

In our case, we should throw the exception because our resumable does not support empty coroutine_handles (there is an assertion in the constructor + we do not check if handle casts to true before doing operations). But if our implementation could somehow support the case of failed allocation of the memory, then the behavior could be appropriately adjusted. In our case, the behaviour will be the same as if the function get_return_object_on_allocation_failure was not declared.

We can check this behaviour, by providing custom operator new for this coroutine inside the promise_type. Let’s define one.

struct resumable::promise_type {
  using coro_handle = std::experimental::coroutine_handle<promise_type>;
  auto get_return_object() {
    return coro_handle::from_promise(*this);
  }
  void* operator new(std::size_t) noexcept {
    return nullptr;   
  }
  static resumable get_return_object_on_allocation_failure(){
    throw std::bad_alloc();   
  }
  // ...
};

This should be enough to see the result of our changes. In case of the clang the following error during the program execution pops up:

terminating with uncaught exception of type std::bad_alloc: std::bad_alloc

Handling co_return

Previously we had a coroutine, that could only suspend itself. We can easily imagine the coroutine, that can also eventually return some value. As mentioned, returning the value from the coroutine is done by co_return keyword, but this requires some additional support from the developer’s side.

I think the best way to understand how co_return works, is to see, what kind of code is generated by the compiler when the co_return keyword is encountered.

First of all, if you use co_return keyword without any expression on its right side (or void expression), the compiler generates:

promise.return_void(); goto final_suspend;

In this case, our coroutine and promise_type already support those operations, so there is no action required.

But if there is some non-void expression, the compiler generates a slightly different code:

promise.return_value(expression); goto final_suspend;

In which case, we need to define the return_value member function of the promise_type. Let’s do this.

struct resumable::promise_type {
  const char* string_;
  // ...
  void return_value(const char* string) {string_ = string;}
  // ...
};

So there are two changes. First, we defined additional member string_ at the top of the promise_type, and return_void had to be changed to the return_value, which takes a const char * as its parameter. The argument is later on saved into the string_ member. Now you might wonder, how to actually get the value on the caller side. And the key to the answer is the coroutine_handle object.

Just to remind you, coroutine_handle knows about the promise object and is able to return the reference to it through the promise member function.

Let’s now extend the example, so that our coroutine can return a value.

resumable foo(){
  std::cout << "Hello" << std::endl;
  co_await std::experimental::suspend_always();
  co_return "Coroutine";
}

int main(){
  resumable res = foo();
  while(res.resume());
  std::cout << res.return_val() << std::endl;
}

class resumable{
  // ...
  const char* return_val();
  // ...
};

const char* resumable::return_val(){
  return handle_.promise().string_;
}

It is crucial in such cases to remember to suspend and not finish the coroutine till the end of the resumable object. The reason for that is, that promise object is created inside the coroutine, so after it’s destroyed access to it will result in the undefined behaviour.

Making the use of co_yield operator

We haven’t yet spoken about the co_yield keyword in detail and this one is crucial for some of the use cases. What co_yield is useful for is to return some value from the coroutine, but without finishing it. Usually, if you are going to implement some kind of the generator you will need to use this keyword.

So let’s write a generator coroutine.

resumable foo(){
  while(true){
    co_yield "Hello";
    co_yeild "Coroutine";
  }
}

Now to implement our promise_type correctly, we need to know what kind of code compiler generates when it encounters the co_yield keyword. And the following snippet shows precisely this.

co_await promise.yield_value(expression);

So what is missing is the yield_value member function. Also worth to note is, that no co_await keyword appears, but we will be talking about the co_await keyword later on. For now, the knowledge, that co_await + suspend_always suspends the coroutine is enough.

Let’s now modify our promise_type again.

struct resumable::promise_type {
  const char* string_ = nullptr;
  // ...
  auto yield_value(const char* string){
    string_=string;
    return std::experimental::suspend_always();
  }
  // ...
};

So we added the yield_value and at the same time, we no longer need return_void.

The way we modify the resumable type will be following.

class resumable{
// ...
  const char* recent_val();
// ...
};

//...

const char* resumable::recent_val(){return handle_.promise().string_;}

Now let’s use our coroutine !

int main() {
  resumable res = foo();
  int i=10;  
  while (i--){
    res.resume();
    std::cout << res.recent_val() << std::endl;  
  }
}

After execution of our program, the output should be following:

Hello
Coroutine
// x5

Summary

So as you can see, the coroutines are hard to learn, after all, it’s still not everything about the coroutines since we only touched Awaitables and Awaiter objects. Fortunately because of the level of difficulty the coroutines the coroutines are very flexible.

The good thing is that the whole complicated part of this feature is not to be known by the regular C++ developer. In fact, the regular C++ developer should know how to write the body of the coroutine, but not coroutine objects themselves.

Developers definitely should use already defined coroutines objects from the standard library (which later will come into the standard) or third-party libraries (like cppcoro).

---

Bibliography

• https://lewissbaker.github.io/2017/11/17/understanding-operator-co-await
• http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2018/n4775.pdf
• https://www.youtube.com/watch?v=ZTqHjjm86Bw (cppcon McNeils introduction into coroutines)
• https://github.com/lewissbaker/cppcoro