I’m sorting out that there are three coroutine flavors in Python:

• Native coroutines — the async def / await style.

    async def fn(x: str) -> str:
        foo = await some_fn(x)
        return foo
  

  Key points:

  • fn("bar") doesn’t start running the body — it returns a coroutine object.

  • The event loop is what actually drives execution.

  • Execution proceeds until the first await, then suspends.

  • While suspended, the loop can run something else (another coroutine).

  • When the awaited thing finishes, control returns and execution continues.

• Classic coroutines — the “old style” with .send(...) and yield.
  You manually push data in, get yields out, etc. More low-level.

• Generator-based coroutines — you can wrap a generator and turn it into something awaitable using @types.coroutine.
  So you get something of a hybrid: generator under the hood, but usable with await.

Also: asynchronous generators — async def with yield inside. Useful when you want to yield values over time, with awaits in between, and consume via async for.

How Native Coroutines Work (Step by Step)

Let me walk through the lifecycle of:

async def fn(x: str) -> str:
    foo = await some_fn(x)
    return foo

1. Call fn("bar") → you get a coroutine object. Nothing inside runs yet.

2. You hand it off to the event loop (directly or via a Task).

3. Event loop starts it — runs until await some_fn(x).

4. At await some_fn(x), it calls some_fn(x), which must itself return something awaitable.

5. The current coroutine (fn("bar")) suspends, yielding control back to the event loop.

6. The loop can pick up other coroutines and run them in the meantime.

7. When some_fn("bar") completes (e.g. after I/O), the loop resumes fn("bar") right after the await.

8. That expression yields a result (or raises), which gets assigned to foo.

9. Execution continues until the function returns (or raises).

This model is what gives concurrency (but cooperatively).

Awaitables, Tasks & Scheduling

Here’s where I had to pause and think:

• await some_coro → you wait for the result. You cannot proceed until that awaitable is done (or errors).

• asyncio.create_task(some_coro) → you wrap that coroutine into a Task and hand it to the loop. It will run in the background (interleaved with others).

Important nuance:

• You don’t have to await a Task if you don’t care about its result.

• If all you want is “run this concurrently, don’t block me here”, task is your friend.

• If you need the result or must wait for it, use await.

Because tasks are just wrappers around coroutines: when one task is suspended (on await), the loop can go run another.

Running Multiple Coroutines Together

I see two main choices:

asyncio.gather

results = await asyncio.gather(
    fn("bar"),
    fn("baz"),
    fn("qux"),
)

• It schedules all given coroutines concurrently.

• You await gather(...) to get the list of results.

• Problem: error handling is awkward. If one task fails, the others keep running unless you explicitly cancel them.

asyncio.TaskGroup

async with asyncio.TaskGroup() as tg:
    tg.create_task(fn("bar"))
    tg.create_task(fn("baz"))
    tg.create_task(fn("qux"))
    # when you exit the `async with`, it waits on all, and cancels on failure

• More “structured concurrency” style.

• If one fails, the rest are cancelled.

• Cleaner, safer than gather for many use cases.

Key Takeaways

• Calling an async function doesn’t run it → it just returns a coroutine object.

• The event loop drives execution, stepping through until await points, suspending and resuming as needed.

• Use await when you need the result.

• Use asyncio.create_task when you want to schedule concurrent work without blocking.

• For running multiple coroutines, TaskGroup is safer and more structured than gather.

Sources & Further Reading

• Fluent Python, Chapter 21: Asynchronous Programming — the chapter gives a deep, practical dive into async def, await, async iterators, and more. O'Reilly Media

• “Asyncio in Python – Full Tutorial” (YouTube video) — a solid visual walkthrough of how async IO works in Python. YouTube

To handle this, I used a fixed-size buffer. Each time I read from the connection, new data gets appended at buf[bufLen:]. Then I hand the “filled” portion, buf[:bufLen], to the parser.

For example, say the connection sends me:

GET /index.html HTTP/1.1\r\nHost: example.com\r\n\r\n

That’s 48 bytes total. The parser only cares about the request line, so it consumes the first 26 bytes (up to the first \r\n). That leaves 22 bytes still in the buffer (Host: example.com\r\n\r\n) waiting for the next stage.

After parsing, I do something called buffer compaction:

copy(buf, buf[readNumbers:bufferLength])
bufferLength -= readNumbers

This removes the 26 bytes that were already parsed and slides the 22 leftover bytes down to the start of the buffer. The buffer now always contains only the data that still needs parsing — kind of like a rolling window.

What I found interesting:

• Reader vs Parser → The reader just delivers arbitrary byte chunks from the OS or TCP socket; it doesn’t care about HTTP. The parser is the one that understands structure and decides how many bytes to consume. That’s why the reader might give me 48 bytes, but the parser only consumes 26 and the rest belongs to the next stage.

• Why compaction matters → Without compaction, I’d have to track “start” and “end” indexes manually inside the buffer. Copying leftovers to the front makes the logic simple: buf[:bufLen] always contains valid, unparsed data.

• State machine design → Right now, my parser only has two states: init (parse the request line) and done (stop). That means the leftover 22 bytes (headers + body) aren’t being parsed yet. A real HTTP parser would add states like ParseHeaders and ParseBody to handle the rest.

The separation between raw data handling (reader) and protocol logic (parser) really clicked for me here. Before, I never thought much about what happens between the TCP stream and the parsed request — now it makes sense.