Skip to content

Infinite loop over QuerySetNoCache #2870

@quique0194

Description

@quique0194

We encountered an issue in some rare and unpredictable situations, primarily during long-running iterations over MongoEngine querysets. Occasionally, for loops over these querysets would enter an infinite loop, repeatedly iterating over the same items.

Upon investigation, we discovered that this behavior was caused by repr(queryset) being called in the background by Sentry. Specifically, when a logger.error (or similar function) was triggered, Sentry would attempt to send context data to its server in a separate thread. As part of this process, it called QuerySetNoCache.repr. However, the current implementation of this method inadvertently iterates over the cursor and subsequently rewinds it using rewind().

This interaction caused the active loop to restart from the beginning, ultimately leading to an infinite cycle where logger.error triggered Sentry, which rewound the cursor again, and so on.

To resolve this, we applied the following monkey-patch to QuerySetNoCache.repr:

def repr_queryset(qs):
    """Custom __repr__ implementation for QuerySetNoCache."""
    data = str({
        'document': qs._document,
        'mongo_query': qs._mongo_query,
        'ordering': qs._ordering,
        'limit': qs._limit,
        'skip': qs._skip,
        'batch_size': qs._batch_size,
        'iter': qs._iter,
        'cursor': qs._cursor,
        'auto_dereference': qs._auto_dereference,
    })
    return f'{qs.__class__.__qualname__}({data})'

QuerySetNoCache.__repr__ = repr_queryset

In our opinion, the repr method should be read-only and should not perform operations that iterate over or modify the cursor state. Such behavior leads to subtle and hard-to-diagnose issues like this one, which took us years to reproduce and identify. Unfortunately, due to time constraints, we are unable to submit a formal pull request, but we hope this explanation and workaround will be helpful to others facing similar issues.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions