Skip to content

SedRegex: Cache ircdb.checkIgnored during a request - #1652

Merged
progval merged 1 commit into
masterfrom
sedregex-cache
Mar 8, 2026
Merged

progval merged 1 commit into
masterfrom
sedregex-cache

Conversation

@progval

@progval progval commented Mar 8, 2026

Copy link
Copy Markdown
Owner

Benchmarking shows a 2 to 4 seconds improvement for all multiprocessing methods:

method cached target check far middle last
fork no before checkIgnored 17.0 17.1 13.3
fork yes before checkIgnored 12.6 13.5 11.0
fork no after checkIgnored 16.7 17.1 13.4
fork yes after checkIgnored 12.6 13.4 11.2
list fork no before checkIgnored
list fork yes before checkIgnored 10.2 10.8 10.7
list fork no after checkIgnored 14.7 15.3 14.9
list fork yes after checkIgnored 10.2 10.7 10.5
forkserver no before checkIgnored 171.9 171.9 172.3
forkserver yes before checkIgnored 168.5 167.8 169.6
forkserver no after checkIgnored
forkserver yes after checkIgnored
    def testPerfLast(self):
        for i in range(10000):
            self.feedMsg(f'xxx {i} yyy')
        self.assertResponse('ping', 'pong')

        before = time.time()
        for _ in range(1000):
            self.feedMsg('s/xxx 9999 yyy/foo/')  # no ^ or $, so it matches itself
        after = time.time()
        print(after - before)
        assert False

    def testPerfMiddle(self):
        for i in range(10000):
            self.feedMsg(f'xxx {i} yyy')
        self.assertResponse('ping', 'pong')

        before = time.time()
        for _ in range(1000):
            self.feedMsg('s/^xxx 1234 yyy$/foo/')  # I meant to use 9876 here. so it's actually the same test as testPerfFar. too bad.
        after = time.time()
        print(after - before)
        assert False

    def testPerfFar(self):
        for i in range(10000):
            self.feedMsg(f'xxx {i} yyy')
        self.assertResponse('ping', 'pong')

        before = time.time()
        for _ in range(1000):
            self.feedMsg('s/^xxx 1234 yyy$/foo/')
        after = time.time()
        print(after - before)
        assert False

"list fork" means I used fork with this patch:

diff --git a/plugins/SedRegex/plugin.py b/plugins/SedRegex/plugin.py
@@ -242,8 +246,7 @@ def doPrivmsg(self, irc, msg):
         if self.registryValue('boldReplacementText', msg.channel, irc.network):
             replacement = ircutils.bold(replacement)
         try:
-            if isinstance(world.SUPYPROCESS_MULTIPROCESSING_CONTEXT,
-                          multiprocessing.context.ForkContext):
+            if False:
                 # global state is shared with child processes, so the child
                 # process has access to history and can lazily filter it
                 message = process(self._replacer_process, irc, msg,

Benchmarking shows a 2 to 4 seconds improvement for all multiprocessing methods:

| method     | cached | target check        | far  | middle | last |
|  --------- | ------ | ------------------- | ---- | ------ | ---- |
| fork       | no     | before checkIgnored | 17.0 | 17.1 | 13.3 |
| fork       | yes    | before checkIgnored | 12.6 | 13.5 | 11.0 |
| fork       | no     | after checkIgnored  | 16.7 | 17.1 | 13.4 |
| fork       | yes    | after checkIgnored  | 12.6 | 13.4 | 11.2 |
| list fork  | no     | before checkIgnored |
| list fork  | yes    | before checkIgnored | 10.2 | 10.8 | 10.7 |
| list fork  | no     | after checkIgnored  | 14.7 | 15.3 | 14.9 |
| list fork  | yes    | after checkIgnored  | 10.2 | 10.7 | 10.5 |
| forkserver | no     | before checkIgnored | 171.9 | 171.9 | 172.3 |
| forkserver | yes    | before checkIgnored | 168.5 | 167.8 | 169.6 |
| forkserver | no     | after checkIgnored  |
| forkserver | yes    | after checkIgnored  |

```py
    def testPerfLast(self):
        for i in range(10000):
            self.feedMsg(f'xxx {i} yyy')
        self.assertResponse('ping', 'pong')

        before = time.time()
        for _ in range(1000):
            self.feedMsg('s/xxx 9999 yyy/foo/')  # no ^ or $, so it matches itself
        after = time.time()
        print(after - before)
        assert False

    def testPerfMiddle(self):
        for i in range(10000):
            self.feedMsg(f'xxx {i} yyy')
        self.assertResponse('ping', 'pong')

        before = time.time()
        for _ in range(1000):
            self.feedMsg('s/^xxx 1234 yyy$/foo/')  # I meant to use 9876 here. so it's actually the same test as testPerfFar. too bad.
        after = time.time()
        print(after - before)
        assert False

    def testPerfFar(self):
        for i in range(10000):
            self.feedMsg(f'xxx {i} yyy')
        self.assertResponse('ping', 'pong')

        before = time.time()
        for _ in range(1000):
            self.feedMsg('s/^xxx 1234 yyy$/foo/')
        after = time.time()
        print(after - before)
        assert False
```

"list fork" means I used fork with this patch:

```diff
diff --git a/plugins/SedRegex/plugin.py b/plugins/SedRegex/plugin.py
@@ -242,8 +246,7 @@ def doPrivmsg(self, irc, msg):
         if self.registryValue('boldReplacementText', msg.channel, irc.network):
             replacement = ircutils.bold(replacement)
         try:
-            if isinstance(world.SUPYPROCESS_MULTIPROCESSING_CONTEXT,
-                          multiprocessing.context.ForkContext):
+            if False:
                 # global state is shared with child processes, so the child
                 # process has access to history and can lazily filter it
                 message = process(self._replacer_process, irc, msg,
```
@progval
progval merged commit d43edf7 into master Mar 8, 2026
23 of 30 checks passed
@progval
progval deleted the sedregex-cache branch March 8, 2026 20:12
@progval progval added this to the Forkserver support milestone Mar 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants