Your first coroutine

Once you got familiar with the introduction to the coroutines, then I think it’s high time to actually implement your first co-routine object. This whole post is all about understanding how to implement co-routines and related entities (especially promise_type). Before we start the adventure, please ensure that your compiler does support coroutines. (as of this writing gcc does not support coroutines at all, my propositions of compilers are clang and msvc. The examples in this post should all compile with clang 9). This is going to be a long post, so reserve yourself an hour or two.

First let me ask you a question. Given following code snippet:

std::future<int> foo();

Is this a function or co-routine declaration? How do you think?

Well, at first sight, it does not differ at all from the regular function. It actually looks exactly as a function declaration. But whether this is a function or co-routine is actually just an implementation detail, or to rephrase it, whether the function is co-routine or not depends on its body. The co-routines, as they are going to get into the C++20, define three new keywords.

  • co_await
  • co_return
  • co_yield

If any of those keywords occur in the function, then it becomes a coroutine.

Why co_ at the beginning of the keywords? For backward compatibility, we do not want to make programs, that for example defined function yield() for their implementations of the stackful coroutines, ill-formed.

Let’s then try to write our first co-routine.

#include <experimental/coroutine>

void foo(){
  std::cout << "Hello" << std::endl;
  co_await std::experimental::suspend_always();
  std::cout << "World" << std::endl;
}

2 new things spotted in the wild:

  • previously mentioned co_await,
  • suspend_always object.

So the operator co_await is a unary operator, which takes the Awaitable object as its argument. We will talk about the Awaitable concept in detail in the next post about the coroutines. Right now, all you need to know is suspend_always is the Awaitable object (defined in experimental/coroutine), and calling co_await operator on it will result in suspending the coroutine. [Note: there is also suspend_never defined in the standard library, that will not cause a suspend when co_await is performed on this object.]

We do not know how to call this kind of a subroutine yet, so let’s skip this for now. But even if we try to compile this code we will get the following error:

Compilation error in case of msvc (manually translated to english)

“promise_type”: is not member of “std::experimental::coroutine_traits<void>

This is not particularly useful information to us without a deeper understanding of coroutines, but we already see, that there are some additional types, that need to be defined.

Why do we need to define additional types?

You probably know, the C++ language really is a flexible one. There is one good meme about it, I was shown during a recent discussion with the colleagues of mine:

C++ flexibility

Well, in the case of the coroutines, we need to implement most of their behaviour by ourselves. The keywords only generate some boilerplate code for the coroutines, which we need to customise via customisation points. So let’s try to understand what is wrong with our coroutine.

Implementing a coroutine

The thing is, that once we have our coroutine, there needs to be some way to communicate with it. If our coroutine suspends, there must be some way to resume the coroutine, and for this particular task, we need to have a dedicated object.

So the object used to communicate with the coroutine is the object of the coroutine’s return type. Let’s start slowly implementing this type. If our coroutine is able to suspend, we need to be able to somehow resume it, so our return type needs to have some resume() method. Let’s create this type.

class resumable{
public:
  bool resume();
};

The call to the resume method will resume the coroutine if it hasn’t finished yet it’s execution, and the return value will indicate whether after resumption there still is something to be executed inside the coroutine.

so our coroutine definition needs to change as well (return type updated):

#include <experimental/coroutine>

resumable foo(){
  std::cout << "Hello" << std::endl;
  co_await std::experimental::suspend_always();
  std::cout << "Coroutine" << std::endl;
}

So the question now arises. How will the resumable object be created (there is no return statement)? This is a little bit of the coroutines magic, that happens under the hood. Basically, the compiler generates some additional code for the coroutines. Every coroutine function is transformed by the compiler to the form similar to this:

Promise promise;
co_await promise.initial_suspend();
try {
  // co-routine body
}
catch(...) {
  promise.unhandled_exception(); 
}
final_suspend:
  co_await promise.final_suspend();

What is more, the resumable object will be created before the initial_suspend through a call to the:

promise.get_return_object();

So what we need to do is to create the Promise type. We can do it in two ways:

  • create promise_type as a member of the resumable type (or create an alias of the same name)
  • specialize coroutine_traits<resumable> type and define promise_type in it (you can even specialize coroutine_traits<resumable, Args…> to differentiate coroutines) or create an alias of the same name.

And if it’s not enough, then if your coroutine does not return anything (through co_return) and flow inside coroutine possibly reaches the functions end, then we also need to have additional member function defined – return_void.

Let’s see the draft of our type:

class resumable{
public:
  struct promise_type;
  bool resume();
};

struct resumable::promise_type{
  resumable get_return_object() {/**/}
  auto initial_suspend() {/**/}
  auto final_suspend() {/**/}
  void return_void() {}
  void unhandled_exception();
};

Unfortunately, we are not yet ready to define functions of our first promise type. To operate on the coroutine, we need to have some sort of handle to the coroutine, which we will manage. There already is a built-in object for this exact purpose.

The coroutine_handle

The coroutine_handle is an object, that refers to the coroutine’s dynamically allocated state. Thanks to this object you can, for example, resume the coroutine. The coroutine_handle is a templated type, where Promise type is its template argument. Let’s have a look at its definition:

template <typename Promise = void>
struct coroutine_handle;

template <typename Promise>
struct coroutine_handle : coroutine_handle<> {
  using coroutine_handle<>::coroutine_handle;
  static coroutine_handle from_promise(Promise&);
  coroutine_handle& operator=(nullptr_t) noexcept;
  constexpr static coroutine_handle from_address(void* addr);
  Promise& promise() const;
}

so now we need to have a look at the specialization:

template <> struct coroutine_handle<void>{
  constexpr coroutine_handle() noexcept;
  constexpr coroutine_handle(nullptr_t) noexcept;
  coroutine_handle& operator=(nullptr_t) noexcept;
  constexpr void* address() const noexcept;
  constexpr static coroutine_handle from_address(void* addr);
  constexpr explicit operator bool() const noexcept;
  bool done() const;
  void operator()();
  void resume();
  void destroy();
private:
  void* ptr;// exposition only
};
Coroutine_handle emptiness

Ok, so first of all, the coroutine_handle is default constructible. The effect of the parameterless constructor call is the same as calling the constructor with the nullptr_t parameter. When such constructor is invoked, the coroutine_handle will be empty. Its emptiness can be checked with the operator bool or by comparison of the result of the address member function with nullptr literal. [Note: the bool operator might easily be mistaken with the done member function. It’s worth to pay attention to it.]

Valid coroutine_handle

If we want to create non-empty coroutine, we need to use its static member function from_address. This is not something we should really pay attention to since the creation of the coroutine_handle lays usually on the compiler’s responsibility. Once the coroutine is created we can use bool operator and address member function to check its validity.

Coroutine resumption

There are two ways to resume the coroutine. First one by the call to the resume member function of the coroutine_handle and another via the call to the function call operator of the same object.

Coroutine destruction

Coroutine destruction happens whichever of the following events occurs first. A first possible event is a call to the destroy member function of the coroutine_handle type. Coroutine destruction also occurs when the flow goes off the coroutine function (after the final suspend point).

Checking the state of the execution

The coroutine can be suspended many times during its lifetime. If (and only if) the coroutine is suspended at it’s final suspend point, the method done returns true.

Coroutine_handle and Promise type

Coroutine_handle specialisations make it possible to create coroutine_handle from the promise object and allow to get the promise object of a coroutine.[Note: manual specialisations of the coroutine_handles are possible, but the behaviour of a program with manual specialization of coroutine_handle is undefined.]

Implementing promise_type

Now we have got all the needed knowledge to finish the implementation of the resumable and promise_type.

Let’s begin with the resumable type.

class resumable {
public:
  struct promise_type;
  using coro_handle = std::experimental::coroutine_handle<promise_type>;

  resumable(coro_handle handle) : handle_(handle) { assert(handle); }
  resumable(resumable&) = delete;
  resumable(resumable&&) = delete;

  bool resume() {
    if (not handle_.done())
      handle_.resume();
    return not handle_.done();
  }
  ~resumable() { handle_.destroy(); }
private:
  coro_handle handle_;
};

So, first of all, we do not want to copy this object, since we can call destroy function member only once. We also do not allow moving the object around for the code’s simplicity. The resume logic is as follows: if our coroutine didn’t finish yet, then resume it, otherwise, don’t. The value returned tells whether, after the call to resume, a coroutine can again be continued or not.

Another thing is, that resumable uses done function member. For this method to work correctly, coroutine must suspend itself on the final suspend point.

Also, a side note: the coroutine_handle object is not thread-safe. Simultaneous access to the destroy and resume methods might result in a data race. Multiple calls to destroy function is an error just like calling free two times on the same pointer, so it’s essential to be careful when destroying coroutines [Note: coroutine destroys itself also after reaching its very end (after final suspend point)].

Now we need to define the promise_type, so that co_routine behaves according to our expectations.

struct resumable::promise_type {
  using coro_handle = std::experimental::coroutine_handle<promise_type>;
  auto get_return_object() {
    return coro_handle::from_promise(*this);
  }
  auto initial_suspend() { return std::experimental::suspend_always(); }
  auto final_suspend() { return std::experimental::suspend_always(); }
  void return_void() {}
  void unhandled_exception() {
    std::terminate();
  }
};

So first the get_return_object is called to create the resumable type. The resumable type needs the coroutine_handle which can be created from the *this promise object.

Right after this, the initial_suspend member function will be called. For our example it makes no difference whether or not the coroutine will suspend its execution at the beginning or not, so by the dice roll suspend_always was chosen. This will result in a fact, that we will need to call resume for the coroutine to start its execution.

The unhandled_exception must be defined, but for our case, it makes no difference what it’s implementation will be, because we won’t throw an exception.

The final_suspend must, in our case, return always_suspend since only then the done method of the coroutine_handle will work adequately (return true when function suspends on final_suspend).

The last method return_void even though it’s empty is obligatory for our coroutine. In case of the coroutines which never end (often generators), neither return_void nor return_value functions need to be defined. But in our case, the coroutine eventually runs till the end of its body, and if there was no return_void, then we would meet undefined behaviour on coroutine return. If coroutines return some values via co_return, then instead of return_void, the return_value function needs to be defined.

Using our coroutine

Once we have defined our own coroutine, let’s see how should we use it.

int main(){
  resumable res = foo();
  while (res.resume());
}

And the expected result should be

Hello
Coroutine

Deeper into promise type.

So besides what we have shown, the promise type can have more methods than those we implemented. In this section, we will take a closer look into that.

Memory Allocation

When the coroutine state gets allocated, then it might happen, that allocation will take place on the heap (we should assume it will happen, but the compiler is free to optimize this out). If such allocation will happen and the promise_type has get_return_object_on_allocation_failure declared, then non-throwing version of the operator new will be searched for. If the allocation fails, we will not have the exception thrown in the face immediately, what compiler will do instead, is, call the get_return_object_on_allocation_failure static function member on the caller side. We can extend our promise type:

struct resumable::promise_type {
  // ...
  static resumable get_return_object_on_allocation_failure(){
    throw std::bad_alloc();   
  }
  // ...
};

In our case, we should throw the exception because our resumable does not support empty coroutine_handles (there is an assertion in the constructor + we do not check if handle casts to true before doing operations). But if our implementation could somehow support the case of failed allocation of the memory, then the behavior could be appropriately adjusted. In our case, the behaviour will be the same as if the function get_return_object_on_allocation_failure was not declared.

We can check this behaviour, by providing custom operator new for this coroutine inside the promise_type. Let’s define one.

struct resumable::promise_type {
  using coro_handle = std::experimental::coroutine_handle<promise_type>;
  auto get_return_object() {
    return coro_handle::from_promise(*this);
  }
  void* operator new(std::size_t) noexcept {
    return nullptr;   
  }
  static resumable get_return_object_on_allocation_failure(){
    throw std::bad_alloc();   
  }
  // ...
};

This should be enough to see the result of our changes. In case of the clang the following error during the program execution pops up:

terminating with uncaught exception of type std::bad_alloc: std::bad_alloc

Handling co_return

Previously we had a coroutine, that could only suspend itself. We can easily imagine the coroutine, that can also eventually return some value. As mentioned, returning the value from the coroutine is done by co_return keyword, but this requires some additional support from the developer’s side.

I think the best way to understand how co_return works, is to see, what kind of code is generated by the compiler when the co_return keyword is encountered.

First of all, if you use co_return keyword without any expression on its right side (or void expression), the compiler generates:

promise.return_void(); goto final_suspend;

In this case, our coroutine and promise_type already support those operations, so there is no action required.

But if there is some non-void expression, the compiler generates a slightly different code:

promise.return_value(expression); goto final_suspend;

In which case, we need to define the return_value member function of the promise_type. Let’s do this.

struct resumable::promise_type {
  const char* string_;
  // ...
  void return_value(const char* string) {string_ = string;}
  // ...
};

So there are two changes. First, we defined additional member string_ at the top of the promise_type, and return_void had to be changed to the return_value, which takes a const char * as its parameter. The argument is later on saved into the string_ member. Now you might wonder, how to actually get the value on the caller side. And the key to the answer is the coroutine_handle object.

Just to remind you, coroutine_handle knows about the promise object and is able to return the reference to it through the promise member function.

Let’s now extend the example, so that our coroutine can return a value.

resumable foo(){
  std::cout << "Hello" << std::endl;
  co_await std::experimental::suspend_always();
  co_return "Coroutine";
}
int main(){
  resumable res = foo();
  while(res.resume());
  std::cout << res.return_val() << std::endl;
}
class resumable{
  // ...
  const char* return_val();
  // ...
};
const char* resumable::return_val(){
  return handle_.promise().string_;
}

It is crucial in such cases to remember to suspend and not finish the coroutine till the end of the resumable object. The reason for that is, that promise object is created inside the coroutine, so after it’s destroyed access to it will result in the undefined behaviour.

Making the use of co_yield operator

We haven’t yet spoken about the co_yield keyword in detail and this one is crucial for some of the use cases. What co_yield is useful for is to return some value from the coroutine, but without finishing it. Usually, if you are going to implement some kind of the generator you will need to use this keyword.

So let’s write a generator coroutine.

resumable foo(){
  while(true){
    co_yield "Hello";
    co_yeild "Coroutine";
  }
}

Now to implement our promise_type correctly, we need to know what kind of code compiler generates when it encounters the co_yield keyword. And the following snippet shows precisely this.

co_await promise.yield_value(expression);

So what is missing is the yield_value member function. Also worth to note is, that no co_await keyword appears, but we will be talking about the co_await keyword later on. For now, the knowledge, that co_await + suspend_always suspends the coroutine is enough.

Let’s now modify our promise_type again.

struct resumable::promise_type {
  const char* string_ = nullptr;
  // ...
  auto yield_value(const char* string){
    string_=string;
    return std::experimental::suspend_always();
  }
  // ...
};

So we added the yield_value and at the same time, we no longer need return_void.

The way we modify the resumable type will be following.

class resumable{
// ...
  const char* recent_val();
// ...
};

//...

const char* resumable::recent_val(){return handle_.promise().string_;}

Now let’s use our coroutine !

int main() {
  resumable res = foo();
  int i=10;  
  while (i--){
    res.resume();
    std::cout << res.recent_val() << std::endl;  
  }
}

After execution of our program, the output should be following:

Hello
Coroutine
// x5

Summary

So as you can see, the coroutines are hard to learn, after all, it’s still not everything about the coroutines since we only touched Awaitables and Awaiter objects. Fortunately because of the level of difficulty the coroutines the coroutines are very flexible.

The good thing is that the whole complicated part of this feature is not to be known by the regular C++ developer. In fact, the regular C++ developer should know how to write the body of the coroutine, but not coroutine objects themselves.

Developers definitely should use already defined coroutines objects from the standard library (which later will come into the standard) or third-party libraries (like cppcoro).


Bibliography


21 responses to “Your first coroutine”

  1. Hi,

    Why is this line under co_return section : “And the key to the answer is the coroutine_handle object.”
    Shouldn’t it be :”And the key to answer is resumable object” … as outside world(rest of program) interacts with the resumable object.
    Also is it right to say coroutine_handle is the basic functionality that the language provides(like a ptr to coroutine frame) and resumable object is a higher level abstraction.

    • @Rishabh

      The program deals with the coroutine_handle to resume the coroutine, this is why I stated it that way. The caller actually does not see anything behind the created abstraction (like resumable type), but still if it wasn’t for coroutine_handle, the program couldn’t resume the coroutine at all.

      It’s important for the coroutine implementer to know about the coroutine_handle, this is why I emphasized it’s existence.

      And yes, it’s totally right to say, that coroutine_handle is a language level basic functionality. You should never deal directly with coroutine_handle, but always create some higher lever abstraction like resumable type.

  2. Thank you for the great article.
    I have one question.
    Is there any way to call another function that returns resumable from the function that returns resumable.

    For example, call `bar()` from `foo()` as follows:

    “`
    resumable bar() {
    std::cout << "Coroutine" << std::endl;
    co_await std::experimental::suspend_always();
    std::cout << "World" << std::endl;
    }

    resumable foo() {
    std::cout << "Hello" << std::endl;
    co_await std::experimental::suspend_always();
    bar();
    }
    “`

    • Thanks for the interest!

      So, yes you can call bar inside the foo function, but what will happen is not, that body of bar is going to be executed. Instead it will give you a resumable object, which you can use ๐Ÿ˜‰

  3. Many thx.
    I’ve visited a local tech talk inside a company I’m working for. And got nothing.
    Here I’ve read the 2 articles and started understanding.

  4. Hi,
    So the stack-less coroutine cpp20 is about to introduce is a basic utility. To fully utilize coroutines, e.g. make coroutines multi-threaded, programmers implement a genius scheduler themselves, is that right? Are there libraries to provide scheduler implementation or greatly simplify the implementation?

  5. Hi Dawid,
    Great post! Thanks for writing it. I learned a lot.

    However, I found a small inaccuracy while testing your code with clang++ 10.0.0. You write: “What is more, the resumable object will be created before the call to the initial_suspend through a call to the: promise.get_return_object();” In fact, the resumable object doesn’t get created until after the call to the initial_suspend() before returning from the subroutine to the caller.

    This is what I see: 1) foo(), the coroutine, is called from main(). 2) The promise is created. 3) promise.get_return_object() is called, but it returns the coroutine handle, not the resumable object. 4) promise.initial_suspend() is called. 5) The resumable object is created (the constructor to the resumable object is called). 6) Control flow is returned to main() right after the initial call to foo() and before the while loop. etc.

    • Thanks!

      call to the get_return_object must return resumable object, because that’s its signature: resumable resumable::promise_type::get_return_object();.
      Did you try to add some logging to the operations?

      Could you send a piece of code, that shows the behavior you are describing? It would be easier for me to reason about it ๐Ÿ™‚

      Cheers!

      • Hi Dawid,

        It was precisely the get_return_object() in your example code that got me curious: “auto get_return_object() { return coro_handle::from_promise(*this); }”. It uses “auto” to deduce its return type, and it returns what “coro_handle::from_promise(*this);” returns, which is “coroutine_handle.”

        If you change get_return_object to: “auto get_return_object() { return resumable(coro_handle::from_promise(*this)); }”, then the resumable object will be created before the call to the initial_suspend through a call to the: promise.get_return_object(), and your statement will be correct, of course, you’ll have to provide a copy constructor or the compiler will complain.

        As to reproducing the described behavior, adding a “cout” statement to the constructor of resumable, to get_return_object(), and to initial_suspend() should be enough since you’ll see the order in which they run. The rest of the code I copy and pasted from your post pretty much as is. I’ll append the code I used for these three functions below. If you can’t reproduce what I describe, please let me know, and I’ll send you all of the code that I used.


        resumable(coro_handle handle) : handle_(handle) {
        assert(handle);
        cout << "resumable()" << endl;
        }

        auto get_return_object() {
        cout << "promise::get_return_object()" << endl;
        return coro_handle::from_promise(*this);
        }

        auto initial_suspend() {
        cout << "promise::initial_suspend()" << endl;
        return std::experimental::suspend_always();
        }

        This is a portion of the output (I also added a couple of messages to main() before and after the call to foo()):
        $ a.out
        main(): before call to foo()
        promise::promise_type()
        promise::get_return_object()
        promise::initial_suspend()
        resumable()
        main(): after call to foo()

  6. Just to summarize… The behavior that I observe is due to the signature of get_return_object in your examples:
    auto resumable::promise_type::get_return_object();
    which, because of the type it returns, ends up being:
    coro_handle resumable::promise_type::get_return_object();
    instead of:
    resumable resumable::promise_type::get_return_object();
    However, the coroutine, foo(), still works because it is implicitly converting “coro_handle” to “resumable” when foo() returns.

  7. Ah right! That is something, that I did not intend ๐Ÿ˜‰ I in fact wanted to return resumable explicitly, but because of the rules you are mentioning it still works!

    Thanks for spotting that Luis! I will fix the description then.

  8. int main() {
    resumable res = foo();
    int i=10;
    while (i–){
    res.resume();
    std::cout << res.recent_val() << std::endl;
    }
    }
    ———————————————————–
    nice post, but i have a question, if i want move res object to other scope so that other function can call it ,how can i do that?(not use global variable)

    • The examples were very simplified, so that we can focus on the “meat”, this is why resumable cannot be moved nor copied.

      If you want to move it, you need to properly implement move constructor. Of course even now you can pass by reference.

      So in case you want to implement move constructor:

      resumable(resumable&& rhs) : handle_(rhs.handle_){
      rhs.handle_ = nullptr; // make sure to not destroy handle_ in the destructor if it’s null, because otherwise you will have problem.
      }

      And I think that’s all you really need to know to implement correctly. Tell me if that helps.

  9. Hi Dawid, many thanks for the write-up!

    Yet I’ve get an error when compiling with VS2017.

    I’ve implemented what you’ve written up until “Using our coroutine”,
    i.e. (in order)
    class resumable and
    struct resumable::promise_type from section “Implementing promise_type”,
    function foo() from “Implementing a coroutine”, and the
    main() function from “Using our coroutine”.

    The error I get from VS2017 (15.9.22, /await, /std:c++latest) are

    Error C2139 ‘resumable::promise_type’: an undefined class is not allowed as an argument to compiler intrinsic type trait ‘__is_empty’ mytest d:\program files (x86)\microsoft visual studio\2017\professional\vc\tools\msvc\14.16.27023\include\type_traits 719

    Error C2027 use of undefined type ‘resumable::promise_type’ mytest d:\program files (x86)\microsoft visual studio\2017\professional\vc\tools\msvc\14.16.27023\include\experimental\resumable 176

    Do you have any idea what’s missing here?
    Many thanks in advance!

  10. Looks like I’be found the problem.
    It seems I have to implement struct promise_type
    right into class resumable instead of doing it outside
    like struct resumable::promise_type{} which doesn’t work
    on my end/compiler, anyone?

    • Hi Rene!

      I cannot be sure without the source code. Did you remember to forward declare the class promise_type?

      like so:

      class resumable{
      //… stuff

      struct/class promise_type;
      };

      If you could provide more data on the issue I could have a look ๐Ÿ˜‰

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.