Hello, during my research on this project, I noticed that OP_CALL handler does not check the bounds when saving programCounter into image[programStack + 4] (https://github.com/jnz/q3vm/blob/master/src/vm/vm.c#L1098). Combining this with OP_ENTER (to move programStack), OP_LEAVE (to restore programStack) and #38 gives an attacker the ability to corrupt the interpreter's memory and call arbitrary function.

Please see PoC and more details here:
https://github.com/TeamAustria/writeups/blob/main/2024-justctf/q3vm.md
https://enzo.run/posts/justctf2024/