Support multithreaded profiling #353
Conversation
|
Hey 👋 Good work! It seems that you still need to insert the |
| @dataclass | ||
| class StackSamplerSubscriberTarget: | ||
| call_stack: SubscriberCallstackFn | ||
| event: SubscriberEventFn |
There was a problem hiding this comment.
I wonder if we can combine this event and async_state change concepts into one interface. It seems that they're pretty similar to me.
| def record_thread_start(self, thread_id: str, time: float) -> None: | ||
| if not self.thread_start_times: | ||
| self.first_start_time = time | ||
| self.thread_start_times[thread_id] = time - self.first_start_time |
There was a problem hiding this comment.
Why not store the raw start time in each, rather than the offset from the first_start_time? Generally I find it better to store truth over computed values.
There was a problem hiding this comment.
I initially tried raw time but it had some complexity when trying to view the traces on the same view, i'd be happy to look at that again.
| self._active_session.record_thread_start(thread_id, time) | ||
|
|
||
| # pylint: disable=W0613 | ||
| def _sampler_saw_call_stack( |
There was a problem hiding this comment.
I'm confused why this method _sampler_saw_call_stack doesn't need to change? Don't we need to separate out the storage of the different threads?
There was a problem hiding this comment.
The frame records get a root thread id added which allows us to reconstruct what is separate - this is done in build_call_stack
There was a problem hiding this comment.
(different threads will never clash in the pythons code due to the GIL)
|
Done!
Done! |
|
Hi @lucamuscat 😄 Is there anything stopping this being merged? |
Hey 👋 @joerick is the right person to ask. |
|
There were some unresolved comments that I can help with (though a little snowed under work wise). I don't think it's super far off |
4 participants
As per #352 I have modified pyinstrument to support multiple threads.
It does this by separating out both timing and frames for the different threads - as such it modifies a substantial amount of the data model (usually in the form of
List[frames]->Dict[str, List[Frames]]and the equivalents in typescript.I don't do much typescript so it's possible there are prettier ways of doing the modifications on that side.
I'd be interested in discussing how this looks to you