Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Limit event fetches to prevent stack size from growing too large #17914

Open
wants to merge 3 commits into
base: develop
Choose a base branch
from

Conversation

devonh
Copy link
Member

@devonh devonh commented Nov 7, 2024

Pull Request Checklist

  • Pull request is based on the develop branch
  • Pull request includes a changelog file. The entry should:
    • Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from EventStore to EventWorkerStore.".
    • Use markdown where necessary, mostly for code blocks.
    • End with either a period (.) or an exclamation mark (!).
    • Start with a capital letter.
    • Feel free to credit yourself, by adding a sentence "Contributed by @github_username." or "Contributed by [Your Name]." to the end of the entry.
  • Code style is correct
    (run the linters)

@devonh devonh requested a review from a team as a code owner November 7, 2024 22:10
@devonh
Copy link
Member Author

devonh commented Nov 7, 2024

We might want to lower the upper range even further.
The stack size depends on the path and can easily have some variation in it between systems/python versions.

With the full list of events (~20) the full stack message of the exception on my laptop was 73839 bytes whereas the maximum allowed size is 65535 bytes.
Fetching only 10 should reduce the size to around 36k bytes which is much below the max. But this will differ by environment.

@erikjohnston
Copy link
Member

Ok: so I guess the change is that in Python 3.12 and latest Twisted something has changed with the stack traces for twisted failures? I tried with older py3.12 and twisted and couldn't reproduce even if I increased the number of events we sent. I think Twisted does try and do stuff to correctly keep track of stack traces, so could believe that something has gone wibbly there. What does confuse me though is that the stack traces you shared didn't seem to be one long stack trace but multiple copies of the same thing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants