-
Notifications
You must be signed in to change notification settings - Fork 1.2k
Description
We encountered an issue in some rare and unpredictable situations, primarily during long-running iterations over MongoEngine querysets. Occasionally, for loops over these querysets would enter an infinite loop, repeatedly iterating over the same items.
Upon investigation, we discovered that this behavior was caused by repr(queryset) being called in the background by Sentry. Specifically, when a logger.error (or similar function) was triggered, Sentry would attempt to send context data to its server in a separate thread. As part of this process, it called QuerySetNoCache.repr. However, the current implementation of this method inadvertently iterates over the cursor and subsequently rewinds it using rewind().
This interaction caused the active loop to restart from the beginning, ultimately leading to an infinite cycle where logger.error triggered Sentry, which rewound the cursor again, and so on.
To resolve this, we applied the following monkey-patch to QuerySetNoCache.repr:
def repr_queryset(qs):
"""Custom __repr__ implementation for QuerySetNoCache."""
data = str({
'document': qs._document,
'mongo_query': qs._mongo_query,
'ordering': qs._ordering,
'limit': qs._limit,
'skip': qs._skip,
'batch_size': qs._batch_size,
'iter': qs._iter,
'cursor': qs._cursor,
'auto_dereference': qs._auto_dereference,
})
return f'{qs.__class__.__qualname__}({data})'
QuerySetNoCache.__repr__ = repr_queryset
In our opinion, the repr method should be read-only and should not perform operations that iterate over or modify the cursor state. Such behavior leads to subtle and hard-to-diagnose issues like this one, which took us years to reproduce and identify. Unfortunately, due to time constraints, we are unable to submit a formal pull request, but we hope this explanation and workaround will be helpful to others facing similar issues.