Add configurable timeout for evaluation functions to avoid inefficient solutions running for too long.
Proposal
Grader implementation must be able to configure a timeout attribute like evaluation_timeout. Calls to to eval_solution must be executed in a different thread and controlled by the main process.
Python's built-in signal lib might allow us to achieve this.
Add configurable timeout for evaluation functions to avoid inefficient solutions running for too long.
Proposal
Grader implementation must be able to configure a timeout attribute like
evaluation_timeout. Calls to toeval_solutionmust be executed in a different thread and controlled by the main process.Python's built-in
signallib might allow us to achieve this.