It was an ordinary Wednesday afternoon, and the sun was shining outside the window, and only the keyboard typing and my occasional sighs were left in the office. The project is about to go online, everything looks so smooth, until a series of mysterious error messages begin to appear in the production environment logs
RuntimeError: coroutine ignored GeneratorExit
"What the hell is this?" I frowned and stared at the screen, thinking that this was definitely not the kind of abnormality I was familiar with. What's worse is that not only is the API containing this exception unavailable, but other seemingly unrelated interfaces also start to report errors. I began to panic and wanted to quickly find out what was causing this.
It wasn't until that moment that I realized that my journey with Python coroutines was just beginning.
GeneratorExit: Death notice from the coroutine world
What is GeneratorExit
Before we dive into the problem, let’s understand the nature of this exception.GeneratorExitis a built-in exception to Python. When the generator or coroutine is forced to be closed, the Python interpreter will send this exception to it. It is essentially a kind of "please end with decent" notification.
Python will causeGeneratorExitabnormal:
- Calling the generator
close()method - The generator is garbage collected
- In asynchronous programming, coroutines are cancelled execution
Under normal circumstances, after receiving this exception, the generator or coroutine should stop generating values, perform necessary cleanup work, and then exit normally. If it tries to continue to generate a value or ignore this exception, it will causeRuntimeError: generator ignored GeneratorExitorRuntimeError: coroutine ignored GeneratorExit。
Actual problem cases
Let me illustrate this with a practical example. Suppose we have a payment API that needs to process other business logic when we receive a payment message:
async def handle_payment_notification(self):
print("Received Payment Notice")
#..Receive parameters, parse parameters, etc..
#handle other logic, including 20 second delay await self.other_handler() # This contains await (20)
# Return a successful response ({"code": "SUCCESS"})
This code looks OK, but it hides a serious flaw. When a third-party payment service waits for a response to timeout and closes the connection, the Tornado framework tries to cancel the coroutine being processed. However, since the coroutine is(20)is suspended and it cannot respond to the cancel request immediately.
whensleepFinally, when the coroutine attempts to continue executing the remaining code, Python finds that the coroutine has been asked to close, but it is still executing, so it is throwing it angrilyRuntimeError: coroutine ignored GeneratorExitabnormal.
Why is this abnormality so dangerous?
One that was not properly processedGeneratorExitExceptions not only cause the current API to fail, but also produce the following chain reaction:
1. Event cycle pollution
Python's asynchronous system manages all coroutines based on a single event loop. When a coroutine does not handle the closing request correctly, it may cause the event loop to enter an unstable state, affecting all other coroutines.
2. Resource leakage
The most common catastrophic consequence is a database connection leak:
async def handle_with_transaction():
db_transaction = DbTransaction()
try:
await db_transaction.begin()
# Business logic... await db_transaction.commit() # If the coroutine is cancelled, it will not be executed here except Exception as e:
await db_transaction.rollback() # If the coroutine is cancelled, it will not be executed here.When the coroutine is cancelled, neither commit nor rollback will be executed, resulting in the database connection being returned to the connection pool. Over time, the connection pool is exhausted and all APIs that require database operations fail.
3. The sharing status is inconsistent
Unexpected termination of coroutines may cause the global shared state to be inconsistent, affecting the normal execution of other coroutines.
How to correctly handle coroutine cancellation?
Now that we understand the severity of the problem, let’s take a look at the solution:
Solution 1: Use background tasks to separate long-term operations
The best practice is to separate long-running operations from request processing:
async def handle_payment_notification(self):
# Analyze payment information...
# Create background task processing business logic instead of waiting for it to complete asyncio.create_task(self.other_handler())
# Return to successful response immediately ({"code": "SUCCESS"})
This method allows HTTP requests to be completed quickly, while background tasks can handle time-consuming operations.
Solution 2: Correctly catch and handle cancel exceptions
If you cannot use background tasks, make sure to handle them correctly：
async def process_with_cancellation():
try:
await (20)
#Other operations... except :
# Perform necessary cleaning print("The operation was cancelled")
# Important: Re-raise exception, telling Python that we have handled cancellation correctly raise
Solution 3: Use an asynchronous context manager to manage resources securely
For resources such as database transactions that need to be closed correctly, use an asynchronous context manager:
class DbTransaction:
async def __aenter__(self):
await ()
return self
async def __aexit__(self, exc_type, exc_val, exc_tb):
if exc_type: # Include CancelledError await ()
else:
await ()
# How to useasync def safe_transaction():
async with DbTransaction() as tran:
# Business logic... # After exiting the context, the connection will be closed correctly whether it is completed normally or cancelled.Solution 4: Implement distributed locks to prevent duplicate processing
For scenarios such as payment callbacks that may be triggered multiple times, use distributed locks to ensure idempotence:
async def process_payment_notification(payment_id):
lock_key = f"payment_processing:{payment_id}"
# Try to acquire the lock if not await acquire_lock(lock_key, timeout=5):
return # There are already processes that are processing
try:
# Process payment logic... finally:
# Make sure to release the lock await release_lock(lock_key)
Prevent problems before they happen: system-level protection measures
In addition to fixing specific codes, the following system-level protection measures should also be considered:
1. Connection pool monitoring and automatic recovery
async def monitor_connection_pool():
"""Regularly monitor database connection pool status"""
async def monitor_pool:
stats = get_pool_stats()
if stats['used'] / stats['total'] > 0.8: # More than 80% usage rate (f"Database connection pool approaches capacity upper limit: {stats}")
if stats['used'] > stats['total'] * 0.9: # More than 90% usage rate ("Connection pool may leak, try resetting")
await reset_connection_pool()
2. Coroutine Audit and Timeout Control
Apply timeout control to all API interfaces to prevent a single operation from blocking the system:
async def api_with_timeout(request_handler):
"""Decorator: Add timeout control to the API"""
async def wrapper(self, *args, **kwargs):
try:
return await asyncio.wait_for(
request_handler(self, *args, **kwargs),
timeout=5.0 #5 seconds timeout )
except :
self.set_status(504) # Gateway Timeout
return ({"error": "Request processing timeout"})
return wrapper
3. Global exception handling middleware
Catch all unhandled exceptions at the framework level:
def setup_global_exception_handler(app):
"""Setting up the global exception handler"""
async def exception_middleware(request, handler):
try:
return await handler(request)
except Exception as e:
(f"Uncaught exception: {e}", exc_info=True)
# Try to reset key resources await emergency_resource_cleanup()
# Return an error response return error_response(500, "Internal Server Error")
app.add_middleware(exception_middleware)
Conclusion: Handle the Generator gracefully
The birth and end of coroutines are like the cycle of life and need to be treated respectfully and gracefully.GeneratorExitIt is not an enemy, but a natural ending signal, reminding us to follow the rules of order in the digital world.
Correctly understanding and handling the life cycle of coroutines can not only avoid headache-inducing system crashes, but also build more robust and efficient asynchronous applications. Just as every farewell in life deserves to be treated gracefully, the exit of the coroutine should be decent without regrets.
When I meet it next timeGeneratorExitDon't panic when related exceptions. Recalling the suggestions of this article, you will find that this is just a normal conversation in the asynchronous world - the system is saying "It's time to say goodbye", and all we need to do is just respond politely "I'm ready".
This is the article about this article that will introduce you to the GeneratorExit exception handling in Python. For more related content on Python GeneratorExit exception handling, please search for my previous article or continue browsing the related articles below. I hope everyone will support me in the future!