ConnectionResetError on Redis DB

[benoit74]

Task: https://farm.openzim.org/pipeline/cd36db85-6748-4510-a3b6-b4a94425fa0a/debug

[sotoki::T-0::2025-12-22 23:42:44,395] INFO:PROGRESS: 6.7% – Step 3/8: Questions_Meta -- 24198099/24198178
[sotoki::T-1::2025-12-22 23:42:44,514] INFO:PROGRESS: 6.7% – Step 3/8: Questions_Meta -- 24198149/24198178
[sotoki::MainThread::2025-12-22 23:42:44,565] DEBUG:Done parsing PostFirstPasser, collecting workers…
[sotoki::MainThread::2025-12-22 23:42:44,565] DEBUG:joining all threads for T-
[sotoki::MainThread::2025-12-22 23:42:44,565] DEBUG:Giving T-0 60s to join
[sotoki::MainThread::2025-12-22 23:42:47,566] DEBUG:Thread T-0 joined
[sotoki::MainThread::2025-12-22 23:42:47,566] DEBUG:Giving T-1 60s to join
[sotoki::MainThread::2025-12-22 23:42:47,566] DEBUG:Thread T-1 joined
[sotoki::MainThread::2025-12-22 23:42:47,566] DEBUG:Giving T-2 60s to join
[sotoki::MainThread::2025-12-22 23:42:47,567] DEBUG:Thread T-2 joined
[sotoki::MainThread::2025-12-22 23:42:47,567] DEBUG:all threads joined for T-
[sotoki::MainThread::2025-12-22 23:42:47,567] DEBUG:PostFirstPasser Workers collected.
[sotoki::MainThread::2025-12-22 23:48:42,874] ERROR:Interrupting process due to error: Error 104 while writing to socket. Connection reset by peer.
[sotoki::MainThread::2025-12-22 23:48:42,874] ERROR:Error 104 while writing to socket. Connection reset by peer.
Traceback (most recent call last):
  File "/usr/local/lib/python3.14/site-packages/redis/connection.py", line 1074, in send_packed_command
    self._sock.sendall(item)
    ~~~~~~~~~~~~~~~~~~^^^^^^
ConnectionResetError: [Errno 104] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.14/site-packages/sotoki/scraper.py", line 432, in start
    self.process_questions_metadata()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/usr/local/lib/python3.14/site-packages/sotoki/scraper.py", line 507, in process_questions_metadata
    PostFirstPasser().run()
    ~~~~~~~~~~~~~~~~~~~~~^^
  File "/usr/local/lib/python3.14/site-packages/sotoki/posts.py", line 98, in run
    super().run()
    ~~~~~~~~~~~^^
  File "/usr/local/lib/python3.14/site-packages/sotoki/utils/generator.py", line 50, in run
    shared.database.commit(done=True)
    ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^
  File "/usr/local/lib/python3.14/site-packages/sotoki/utils/database/redisdb.py", line 121, in commit
    pipe.execute()
    ~~~~~~~~~~~~^^
  File "/usr/local/lib/python3.14/site-packages/redis/client.py", line 1693, in execute
    return conn.retry.call_with_retry(
           ~~~~~~~~~~~~~~~~~~~~~~~~~~^
        lambda: execute(conn, stack, raise_on_error),
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        lambda error: self._disconnect_raise_on_watching(conn, error),
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/usr/local/lib/python3.14/site-packages/redis/retry.py", line 123, in call_with_retry
    raise error
  File "/usr/local/lib/python3.14/site-packages/redis/retry.py", line 116, in call_with_retry
    return do()
  File "/usr/local/lib/python3.14/site-packages/redis/client.py", line 1694, in <lambda>
    lambda: execute(conn, stack, raise_on_error),
            ~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.14/site-packages/redis/client.py", line 1533, in _execute_transaction
    connection.send_packed_command(all_cmds)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/usr/local/lib/python3.14/site-packages/redis/connection.py", line 1085, in send_packed_command
    raise ConnectionError(f"Error {errno} while writing to socket. {errmsg}.")
redis.exceptions.ConnectionError: Error 104 while writing to socket. Connection reset by peer.
[sotoki::MainThread::2025-12-22 23:48:42,887] DEBUG:shutting down executor T- with wait=False
[sotoki::MainThread::2025-12-22 23:48:42,888] DEBUG:shutting down executor IMG-T- with wait=False
[sotoki::MainThread::2025-12-22 23:49:25,064] INFO:PROGRESS: 6.7% – Step 3/8: Questions_Meta -- 24198177/24198178
[sotoki::MainThread::2025-12-22 23:49:25,069] DEBUG:Removing /build/stackoverflow.com_y3cpvpt2

To be investigated:

• does scraper retry also on these errors?
• could we harden that differently?
• or is this just a very rare edge case where Redis completely died / hung forever?
• kinda noticeable it happened at the very end of Questions_Meta phase, is there something special happening at that moment?