KeyError when accessing model.i_to_sid[i]

[StackOverflowExcept1on]

$ for i in `seq 12001 12005`; do python3 participant.py $i & done
$ python3 coordinator.py localhost 12001 3 5 0 0 10 # malicious = 0
INFO:root:Participant 3: Received sign_round request, run_id = 1, is_malicious = False
INFO:root:Participant 4: Received sign_round request, run_id = 1, is_malicious = False
INFO:root:Participant 5: Received sign_round request, run_id = 1, is_malicious = False
INFO:root:Participant 4: Sent sign_round response and next pre_i value in 0.0014 seconds
INFO:root:Participant 3: Sent sign_round response and next pre_i value in 0.0015 seconds
INFO:root:Participant 5: Sent sign_round response and next pre_i value in 0.0015 seconds
INFO:root:Participant 3: Received sign_round request, run_id = 1, is_malicious = False
INFO:root:Participant 1: Received sign_round request, run_id = 1, is_malicious = False
INFO:root:Participant 2: Received sign_round request, run_id = 1, is_malicious = False
INFO:root:Participant 2: Sent sign_round response and next pre_i value in 0.0017 seconds
INFO:root:Participant 1: Sent sign_round response and next pre_i value in 0.0019 seconds
3,5,0,AttackerLevel.STATIC,0.014068126678466797,11,8,1
INFO:root:Participant 3: Sent sign_round response and next pre_i value in 0.0014 seconds
Traceback (most recent call last):
  File "roast/coordinator.py", line 200, in <module>
    elapsed, send_count, recv_count, sid = coordinator.run(i_to_sk, model, attacker_strategy)
  File "roast/coordinator.py", line 136, in run
    logging.debug(f'Incoming message from participant {i} in session {model.i_to_sid[i]}')
KeyError: 3

[StackOverflowExcept1on]

I also tried with runs=1 and malicious=0 and now everything always runs fine. Maybe there is some bug in cleaning sessions?

[real-or-random]

I think this is a race condition in the coordinator between queue_incoming_loop and the main thread. They use a shared variable run_id. But queue_incoming_loop should simply write to the queue and let the main thread care about ignoring messages from old runs...

[StackOverflowExcept1on]

@real-or-random Yes, you are right.

queue_incoming_loop should simply write to the queue and let the main thread care about ignoring messages from old runs...

def queue_incoming_loop(self, sock, cached_ctx_queue):
    while True:
        # ...
+       print(f"added INCOMING data, run_id = {run_id}")
        self.queue_action(ActionType.INCOMING, data)

DEBUG:root:Established connection to participant 1 at ('localhost', 12001)
DEBUG:root:Established connection to participant 2 at ('localhost', 12002)
DEBUG:root:Established connection to participant 3 at ('localhost', 12003)
============================================================
added INCOMING data, run_id = 1
added INCOMING data, run_id = 1
added INCOMING data, run_id = 1
DEBUG:root:Initial incoming message from participant 3
DEBUG:root:Initial incoming message from participant 1
DEBUG:root:Initial incoming message from participant 2
DEBUG:root:Enough participants are ready, starting new session with sid 1
added INCOMING data, run_id = 1
DEBUG:root:Incoming message from participant 1 in session 1
DEBUG:root:Enough participants are ready, starting new session with sid 2
added INCOMING data, run_id = 1
DEBUG:root:Incoming message from participant 3 in session 1
2,3,0,AttackerLevel.STATIC,0.01569223403930664,7,5,1
============================================================
added INCOMING data, run_id = 1
added INCOMING data, run_id = 2
added INCOMING data, run_id = 1
added INCOMING data, run_id = 2
added INCOMING data, run_id = 2
Traceback (most recent call last):
  File "roast/coordinator.py", line 203, in <module>
    elapsed, send_count, recv_count, sid = coordinator.run(i_to_sk, model, attacker_strategy)
  File "/roast/coordinator.py", line 137, in run
    logging.debug(f'Incoming message from participant {i} in session {model.i_to_sid[i]}')
KeyError: 1

[StackOverflowExcept1on]

I duplicated this code before calling self.queue_action(ActionType.INCOMING, data) and it seems to work most of the time, but not always. I still get KeyError, but less frequently.

def queue_incoming_loop(self, sock, cached_ctx_queue):
    while True:
        # ...
+       # Ignore incoming messages from wrong run_id
+       with self.run_id.get_lock():
+           if run_id != self.run_id.value:
+               logging.debug(f'Ignoring incoming message from previous run (message run_id = {run_id}, my run_id = {self.run_id.value})')
+               continue
        self.queue_action(ActionType.INCOMING, data)

[real-or-random]

Should be fixed by #13