Authority lesson · 880 words
Concurrency, Asyncio, and Cooperative Cancellation
Concurrency is a workload decision
Asyncio is most useful when one process spends substantial time waiting on many independent I/O operations. It does not make CPU-heavy Python code automatically faster, and it does not remove remote-service limits. Begin by measuring where time is spent. If a crawler waits on twenty sites, asynchronous I/O may reduce idle time. If a report spends ninety per cent of its time compressing images, process-level parallelism, a queue, or a different algorithm may be the relevant choice.
Define the maximum useful concurrency from external rate limits, local file descriptors, memory, database connections, and the amount of work that may need cancellation. Unlimited task creation converts a slow provider into a local outage. Use bounded queues, semaphores, worker pools, and explicit admission control. A system that can start ten thousand tasks but cannot finish or cancel them is not scalable.
Structured ownership prevents orphaned work
Every task should have an owner, lifetime, and error path. Prefer task groups or scopes where child tasks finish, fail, or are cancelled before the parent exits. Fire-and-forget tasks can disappear after an exception, outlive request context, or be destroyed during shutdown. If work must survive a request or process restart, persist a job and let a managed worker own it rather than hiding it in an in-process task.
Collect results deliberately. Decide whether one child failure cancels siblings, whether partial results are useful, and how exceptions are aggregated. Logging an exception is not the same as handling it. A batch importer may continue independent rows while a financial settlement should stop on the first inconsistency. The policy belongs to the business operation, not to whichever concurrency helper was easiest to call.
Cancellation must be cooperative and safe
Cancellation is a request for code to stop at an await point; it is not an instant rollback. Use try/finally or async context managers to release locks, close responses, return connections, and mark job state. Do not broadly catch and suppress cancellation exceptions. If cleanup itself can block, bound it and record incomplete cleanup for recovery. Design idempotent checkpoints so a cancelled job can restart without duplicating external effects.
Timeouts should describe a deadline for useful work. A timeout around one await is different from a deadline for the whole operation. Propagate remaining time through nested calls and stop starting new work when the deadline cannot be met. Record whether an operation timed out before sending, after sending, or while reading; those states have different retry consequences.
Test schedules, shutdown, and pressure
Concurrency defects often appear only under unusual ordering. Use deterministic fakes for clocks and remote calls, barriers to force interleavings, and tests that cancel at each important await. Verify that permits are returned, transactions are closed, job states are accurate, and repeated retries do not duplicate effects. Add load tests that measure queue growth and latency rather than only successful throughput.
Operationally, expose active tasks, queue depth, oldest job age, timeout counts, cancellation counts, retry reasons, and event-loop lag. Graceful shutdown should stop accepting work, allow a bounded drain, cancel remaining tasks, persist recoverable state, and exit with evidence. Without those signals, asynchronous code can look healthy while work silently accumulates.
Scenario
A fast crawler overwhelms its own database
A team converts a sequential audit crawler to create one task per URL. Network time falls, but hundreds of completed tasks attempt database writes at once, exhausting the connection pool and causing retries. The correct design bounds both HTTP concurrency and persistence concurrency, streams results through a queue, and stops accepting new URLs when the deadline or queue limit is reached.
- Measure each constrained resource
- Bound task creation and database writers separately
- Propagate cancellation through the pipeline
- Persist resumable progress
Worked example
Build a bounded asynchronous fetch pipeline
- Create a queue with a fixed maximum size and a semaphore for outbound requests.
- Start a small number of fetch workers and one persistence worker.
- Give the overall audit a deadline and pass remaining time to each request.
- On cancellation, close responses, mark unfinished URLs, and persist the checkpoint.
- Test shutdown while requests, parsing, and database writes are each in progress.
Checklist
Use this before acting
- Measure whether the workload is I/O-bound
- Set explicit concurrency and queue limits
- Give every task an owner and lifetime
- Persist work that must survive process exit
- Propagate deadlines and cancellation
- Make checkpoints idempotent
- Test forced interleavings and shutdown
- Monitor queue age, timeouts, cancellations, and event-loop lag
Common mistakes
Failure patterns to avoid
- Creating a task for every item
- Using async for CPU-heavy work without measurement
- Starting background tasks inside web requests
- Suppressing cancellation exceptions
- Retrying timed-out non-idempotent operations blindly
- Testing only the happy scheduling order
Practical exercise
Add bounded concurrency to one I/O workflow
Choose a network or file workflow. Measure the sequential baseline, add an explicit limit, define cancellation checkpoints, and test shutdown at three await points. Document the remote rate limit and local resource limit that justify the chosen concurrency.
Deliverable: A benchmark, bounded implementation, cancellation tests, and operational signal list.
Sources and further reading
Primary and official references
Topic-specific takeaway
Safe concurrency is controlled ownership of waiting work: bounded admission, cooperative cancellation, idempotent checkpoints, and observable pressure.
