I believe that the way we encode frames we create the possibility of a drift between the real time and the resulting video times. This is exacerbated on large captures.

In Mp4EventHandler's init:

        self.delta = 1000 // fps  # ms per frame

then:

        nframes = (dt // self.delta)
        if nframes > 0:
            for _ in range(nframes):
                self.writeFrame()

However:

In [18]: 1000//30
Out[18]: 33

In [19]: 1000/30
Out[19]: 33.333333333333336

There's a 0.3ms per 1000ms drift opportunity. On a long capture this amount to a very large maximum theoretical drift:

In [26]: ((1000/30-1000//30) * 12*60*60) / 60
Out[26]: 240.0000000000017

Further investigation: I will record an RDP session playing a high precision clock and timer on a screen for a long time and compare after video encoding.