What is Asyncio

Introduction

Python is a powerful scripting language. The success of the python programming language is probably coming from the rather intuitive syntax which addresses shortcomings of many other languages. A standard python distribution contains powerful libraries for a lot of "every day" problems. In addition python can be interfaced to more efficient C- or C++-libraries which then can be used in a "pythonic way" (i.e. using python syntax) but anyway profiting from the much higher efficiency of these compiled languages. Very popular examples for this are the numpy and the scipy libraries for complex mathematical operations which would take "ages" if programmed in pure python.

However, python also has a very anachronistic feature which goes completely against the trends we see in the development of modern computing hardware. Processors and computers become more powerful by supporting parallelism. The days were computing power was increased by increasing the clock frequency of the processor are over. Instead processors implement more and more "cores" which are processing units which operate simultaneously and execute different pieces of code in parallel. Multiple processing "threads" are executed in parallel on the various cores. Programming in such an environment is difficult and there are a lot of pitfalls but it can be extremely powerful when done correctly. However, Python does NOT support multiple threads of execution and parallelism. Every python program contains the so called GIL (Global Interpreter Lock) which makes sure that the python interpreter only executes one processing thread at a time, and this is also valid on a multi core machine: Only one core can be used by a python program. The other cores cannot do work in the same program at the same time (they can execute other programs of course.)

Of course the developers of python knew about this feature and anyway they chose to develop the language around this GIL. Why this? In python the memory management is done for the user automatically. The programmer does not need to do bookkeeping of the requested memory and does not need to give memory explicitly back to the system in case it is not needed anymore. In short: There is not danger to create memory leaks (...well if you try hard you can manage anyway...). Memory management would become much more complex without the GIL and single threaded programs would become less efficient than they are now in python. In addition the GIL makes it easily possible to interface python to C-libraries which are itself not thread-safe (thread-safe means a library is safe to be used in a multi-threaded environment where are library function can be called by various threads at the same time). This opens up a huge code base for inclusion into python. These arguments were strong enough to choose this implementation option.

There are existing python interpreters which do not have a GIL and which do allow for a real multithreaded program flow. However, these python interpreters cannot easily interface to so many C and C++ libraries.

What can you do to make use of the many cores in your computer? Well, you can use multiprocessing. There is a dedicated module for multiprocessing in python which allows you to start many python processes in parallel. These processes do not share any memory with each other. If they need to communicate with each other they can use networking or the filesystem but these are, of course, heavy techniques if you have to exchange just a few numbers.

There is one situation in which multi-threading works in python: a program can execute a second processing thread in parallel to the first thread in case the first thread is waiting for input (or output). If you are waiting for a packet to arrive on the network or for a keystroke on the keyboard your python program actually CAN process another thread of your program during that time. This helps for simple programs where you have IO (input/output) and some processing to be done on some data. It does not help you when you want to do image processing on many tiles of an image in parallel.

ASYNCIO: a convenience for the programmer in some situations

Now you know some essential features of how the python interpreter works. However, we also have recurring programming patterns which would need some multi-processing, especially when you consider io-based programs (see last paragraph of the previous section)

Imagine you want to program a webserver. A webserver is clearly an io-driven program. It is waiting for requests coming over the internet. If a request comes, in the simplest case, it sends back a static page. In a more complex web-application the request contains some data and some processing with this data needs to be done to prepare the contents which needs to be sent back (e.g. some database has to be read and the data has to be prepared for presentation on a web-page.) A webserver typically should accept multiple requests at (almost) the same time. And this is where mutlithreaded programming seems very adequate: a first request comes in and the server spawns a thread processing this request and sending back the data. This might take some time when the quantity of data to be processed is large. But the server should be able to satisfy another request during this time (e.g. another user wants to read a simple static web page. The user would be highly dissappointed if he had to wait for a minute for a simple static page only because someone else shortly before clicked on another page of the server which requires complex processing.) In this situation the serving of the second request should continue immediately (on another core of the processor) while the first request is processed on a different core. And this is exactly the typical situation which asyncio makes easy to program. And in python it makes sure that if a thread is waiting for io, useful work can continue in another thread. However, it cannot execute processing code magically on many cores in parallel due to the GIL as we have explained above. Asyncio programs will also work on processors with only one core (like many microcontrollers). This is because asyncio under the hood is not really executing multiple threads in parallel on multiple cores but it just tries to intelligently use the one and only core used by the Python program to do other things when one thread is just waiting for something to happen. Asyncio gives you the tools at hand with which you can program your processing thread in a way that it can be interrupted if need be and for example a simple request to the webserver can be satisfied in a short interval during which the complex processing is interrupted.

And this is already something: you can make relatively easily responsive programs, which need to react to user input, network input and in addition need to do some constant processing in the background.

Basics of asyncio

Tasks and Event Loops

A "task" in asyncio is a thread of execution. It has its own stack to save local variables and multiple functions or routines are called one after the other depending on how the program is written. It is equivalent to one "normal" program.

In asyncio programming you can have multiple tasks simultaneously. (Always remember that each task has its own stack!) However be careful: This does not mean that the tasks by some magic can be executed at the same time. Also when using asyncio there is the GIL and at any point in time only one task can be executed. So what is the advantage then? In an asyncio-program there is another object which implements a more or less clever algorithm to decide which task to run at a given moment in time. This object is called the Event Loop.

WHEN the switching from one task to the other should occur is in the hand of the programmer. In the code which corresponds to a task you insert statements which tell the Event-Loop that you accept to have a break and some other task should get its turn to continue processing. These statements you typically insert in your program when you wait for user input or for data coming from the network or for another task to deliver a result you need in order to continue your own job.

The Event Loop might also decide to continue with your task immediately if no other task needs to run. But it makes sure that every task which needs to run get's its fair share of processing time.

Functions and Coroutines

In a "normal" program the things which need to be done are programmed in mutiple functions (also if you do not have functions but one long spaghetti-program you can consider this long program as one big function). If a function is called (from the main program or from another function) it is executed until a statement tells the processor to return from this function, possibly giving back a return value to the calling function. The calling function resumes operation at the statement following the function call.

As we have discussed in the section of the memory handling, the variables which are only created and used in a function are created on the stack. Also the address where the processor has to jump back after the function is finished is stored on the stack. When returning from a function, all variables created on the stack in this function are not known to the system anymore and the memory is given back to the stack as ununsed memory ready to be used in new stack operations. There is one single stack in the entire program.

A coroutine is a different "animal". It is also called from another function (or coroutine) and it runs in the context of a "Task", but it runs until it decides to have a break. In this moment something radically different happens than in the traditional "function" model: When a coroutine decides to "have a break" (this is called it "yields" the execution control) the Event Loop looks at all the tasks in the program to decide which task should now continue processing. A coroutine is simply a function which contains statements which yield the process control to give other tasks a change to run. And each task, if selected to run, resumes running in the coroutine and at the statement following the point where it decided to have a break previously.

Of course co-routines can (and do) finish like "normal" functions and they can return results. And other co-routines can wait for them to finish (and give results).

Technical part

In order to use asyncio you have to import the corresponding module:

import asyncio

A coroutine is defined by putting the keyword "async" in front of the definition:

async def mycoroutine( var ):
    while True:
        print( var )
        await async.sleep(1)

To make use of the features of asyncio programming a coroutine needs to yield the process control at some point. This is done with the satement "await" (see above). The object following the await statement needs to be an "awaitable" object (this is python slang). Awaitable objects are other coroutines or other tasks. If you await a coroutine or a task you get also back the return value:

async def mycoroutine(param):
    print("in mycoroutine")
    result = 0
    for i in range(0,param):
        result += param
        await asyncio.sleep(1)
    return result

async def main():
   await res = mycoroutine( 5 )
   print("result was ", res)

asyncio.sleep is actually a coroutine. It sleeps for the number of seconds given as a paramter like the normal time.sleep function in python. However, and this is the big difference to the "normal" time.sleep, during this time the EventLoop can give execution control to another task in the program. It is possible to put the parameter of asyncio.sleep(0) to 0: this means that you are ready to let another task being scheduled but you are also ready to immediately proceed (no waiting required).

Finally we need to get our asyncio program going. For this we need to create an event loop and start it and tell the programme where to start the processing. The most often used and most convenient method for this is to run the command

asyncio.run( main() )

This command will create an EventLoop for asyncio and starts execution with the coroutine main() which we defined above. (The function given to the run() command must be a co-routine!).

Asyncio in practice

If you really want to make use of asyncio you need compatible libraries containing coroutines you can use in your code to do something useful. For example you need coroutines to wait for input from the network or some other Input/Output devices (keyboard, mouse, ...).

Fortunately these libraries and modules exist. In the asyncio module for PCs (NOT for microcontrollers!) you have already useful asyncio compatible objects like Streams to work with network connections, Queues to pass data from one coroutine to another waiting in a different task, you have synchronization primitives like Locks, Events, Conditions which can be used similar to corresponding objects in real multithread programming in C or C++. They help to program situations where a coroutine should wait for something to happen (which is ussually happening in a different Task). There is also a asyncio compatible version of the subprocess module allowing you to spawn processes on the PC (really running in parallel to your python program but in a separate process) and wait for them to finish.

In micropython the possibilities are much more limited. But the main usecase is probably network programming and here we have powerful methods which support the asyncio programming paradigma. Events, Flags and Locks exist to synchronise coroutines with each other. And for TCP/IP connections there are Streams and high level functions. These allow you to "await" for incoming network traffic (i.e. you can wait for data coming on the network but while you are waiting the EventLoop executes other threads) or to implement TCP/IP servers or clients. In a few lines you can set up a Web Server as we will see in the exercices.

Summary

A short summary of this lengthy section:
THERE IS NO BLACK MAGIC IN ASYNCIO: programs written for asyncio are still only doing one thing at a time due to the python GIL. Asyncio cannot change this. However, it can use the CPU core used by the program efficiently by pausing tasks which are just waiting for something to happen. The following table summarise the most important objects used in asyncio:

Item	Definition	Syntax
Task	A sequence of functions (or one main function) running on the computer. A task has it's own stack. A "normal program" is a task. An asyncio program typically has multiple tasks running. To be meaningful, in asyncio programs the task should run coroutines.	asyncio.create_task(myco(params))
Event Loop	A simple "scheduler" which decides which task to run next when the running task is going to pause execution.	asyncio.run(myco(params)) creates an event loop and starts executing the coroutine myco in the current task.
Coroutine	A function which every now and then decides to pause. The heart of asyncio programs.	async def myco(params) ATTENTION: if you call a coroutine like a "normal" function: myco(params) this call returns just an instance of a coroutine. IT DOES NOT RUN the coroutine. In order to run the coroutine you have to either create a task for it, or you have to use `await myco(params) to run the coroutine in the current task and to wait for it to end.
Awaitable	An object which you can wait for to finish or to change. Coroutines are awaitable. There are synchronisation objects like Locks, semaphores, Queues where you can wait for, too.
Give up process control	Tell the EventLoop that now another task can run and we have pause with our task	await myco(params) executes myco in the current task and waits for it to to finsh. await asyncio.sleep(1) waits at least 1 second before it can be chosen again by the Event Loop to run. asyncio compatible libraries contain functions which can be waited for (i.e. coroutines).

References

[1] An excellent introduction can be found here:
https://bbc.github.io/cloudfit-public-docs/asyncio/asyncio-part-1.html