Combining sixel_encoder_encode_bytes() with any option that requires an image modification, e.g. libsixel image scaling, results in a memory allocation crash.
The problem is this:
- The external buffer supplied to
sixel_encoder_encode_bytes() is placed into a sixel_frame_t.
- Eventually
sixel_frame_convert_to_rgb888() or sixel_frame_resize() is called to modify the frame.
- This will allocate a new pixel buffer with
sixel_allocator_malloc().
- The old pixel buffer in the
sixel_frame_t is freed with sixel_allocator_free().
- The existing pixel buffer in the
sixel_frame_t is replaced with the newly allocated buffer.
If the original pixel buffer supplied to sixel_encoder_encode_bytes() was not allocated by sixel_allocator_malloc(), then step 4 above will crash or corrupt the heap.
The example examples/opengl/main.c suffers from this problem. The buffer is from opengl, not allocated with sixel_allocator_malloc(), and will crash with image scaling or anything else that requires normalizing the image is done. The libpixel Python bindings also have this problem, as the image data is a Python object.
While this could be fixed by requiring the data passed to sixel_encoder_encode_bytes() to be a buffer allocated with sixel_allocator_malloc(), this is not very convenient for library users. The image will already be in some other kind of buffer, e.g. OpenGL or a Python PIL image, and the user would need to allocate another image buffer and copy the image into it.
Another solution, more efficient, would be to mark with a flag in sixel_frame_t whether the pixel data is "internally managed" or not. Internally managed data would be freed when the pixel data in the frame is replaced. External pixel data would not be freed. This would also fix existing users of sixel_encoder_encode_bytes() without needing a change to the API.
Combining
sixel_encoder_encode_bytes()with any option that requires an image modification, e.g. libsixel image scaling, results in a memory allocation crash.The problem is this:
sixel_encoder_encode_bytes()is placed into asixel_frame_t.sixel_frame_convert_to_rgb888()orsixel_frame_resize()is called to modify the frame.sixel_allocator_malloc().sixel_frame_tis freed withsixel_allocator_free().sixel_frame_tis replaced with the newly allocated buffer.If the original pixel buffer supplied to
sixel_encoder_encode_bytes()was not allocated bysixel_allocator_malloc(), then step 4 above will crash or corrupt the heap.The example
examples/opengl/main.csuffers from this problem. The buffer is from opengl, not allocated withsixel_allocator_malloc(), and will crash with image scaling or anything else that requires normalizing the image is done. The libpixel Python bindings also have this problem, as the image data is a Python object.While this could be fixed by requiring the data passed to
sixel_encoder_encode_bytes()to be a buffer allocated withsixel_allocator_malloc(), this is not very convenient for library users. The image will already be in some other kind of buffer, e.g. OpenGL or a Python PIL image, and the user would need to allocate another image buffer and copy the image into it.Another solution, more efficient, would be to mark with a flag in
sixel_frame_twhether the pixel data is "internally managed" or not. Internally managed data would be freed when the pixel data in the frame is replaced. External pixel data would not be freed. This would also fix existing users ofsixel_encoder_encode_bytes()without needing a change to the API.