rs2_extract_frame documentation unclear or inaccurate

[GrubbyHalo]

The rs2_extract_frame API documentation says:

If you wish to keep this frame after the composite is released, you need to call acquire_ref Otherwise the resulting frame lifetime is bound by owning composite frame

The example code and sources seems to contradict this as they specifically release the sub frames in addition to releasing the composite frame.
Also I have found if i stick to the above recommendation of acquiring the subframes after releasing the composite frame i get a memory leak as a consequences of effectively having acquired the sub frames twice but only releasing once.

[MartyG-RealSense]

Hi @GrubbyHalo rs2_extract_frame is usually only used in RealSense 'C' language code rather than C++ so the number of code references available that demonstrate this instruction is limited, unfortunately.

If you are using C code and experiencing a memory leak due to unreleased frames, is your program releasing resources like in the rs-distance C example?

https://github.com/IntelRealSense/librealsense/blob/master/examples/C/distance/rs-distance.c#L125-L131

[GrubbyHalo]

@MartyG-RealSense i have two functions that get called in a loop:

static void grab_frame(){

    rs2_error* e = NULL;
    rs2_frame* frames = rs2_pipeline_wait_for_frames(priv->pipe, RS2_DEFAULT_TIMEOUT, &e);
    if(e)
    {
        g_warning("Error waiting for frames: %s", rs2_get_error_message(e));
        rs2_free_error(e);
        return;
    }
    int num_of_frames = rs2_embedded_frames_count(frames, &e);
    if(e)
    {
        g_warning("Error getting number of frames: %s", rs2_get_error_message(e));
        rs2_free_error(e);
        rs2_release_frame(frames);
        return;
        
    }

    if(num_of_frames<2){
        g_warning("Not enough frames: %d", num_of_frames);
        rs2_release_frame(frames);
        return;
    }

    rs2_frame* depth_frame = NULL;
    rs2_frame* ir_frame = NULL;

    int i;
    for(i=0;i<num_of_frames;i++){
        rs2_frame* frame = rs2_extract_frame(frames, i, &e);
        if(e){
            g_warning("Error extracting frame: %s", rs2_get_error_message(e));
            break;
        }
        const rs2_stream_profile* p=rs2_get_frame_stream_profile(frame, &e);
        if(e){
            g_warning("Error getting stream profile: %s", rs2_get_error_message(e));
            break;
        }
        rs2_stream stream;
        rs2_format format;
        int index;
        int unique_id;
        int fps;
        rs2_get_stream_profile_data(p, &stream, &format, &index, &unique_id, &fps, &e);
        if(e){
            g_warning("Error getting stream profile data: %s", rs2_get_error_message(e));
            break;
        }
        if(stream==RS2_STREAM_DEPTH){
            depth_frame=frame;
        }
        else if(stream==RS2_STREAM_INFRARED){
            ir_frame=frame;
        }
    }

    g_assert(depth_frame && ir_frame);

    priv->current_depth_frame=depth_frame;
    priv->current_ir_frame=ir_frame;

    // Documention for rs2_extract_frame says that we must add-ref to the two subframes otherwise their lifetime is bound by the composite frame. So add ref so that we still have these frames around in process_frame
    
    rs2_frame_add_ref(depth_frame, NULL);
    rs2_frame_add_ref(ir_frame, NULL);
    

    if(frames)
        rs2_release_frame(frames);
    if(e)
        rs2_free_error(e);

    
}

gboolean rs_sensor_process_frame(){


    g_assert(priv->current_depth_frame && priv->current_ir_frame);

    // for now just release them
    rs2_release_frame(priv->current_depth_frame);
    rs2_release_frame(priv->current_ir_frame);
    priv->current_depth_frame=NULL;
    priv->current_ir_frame=NULL;


    return TRUE;
}

The above causes a memory leak even though the documentation implies this is the correct thing to do (Read the only comment in grab_frame )

If i modify the grab_frame function so that it doesn't increasr the reference count on the to sub frame...i.e./ by removing:

    //rs2_frame_add_ref(depth_frame, NULL);
    //rs2_frame_add_ref(ir_frame, NULL);

Then I have no more memory leaks.
This suggests that the documentation for rs2_extract_frame is incorrect

[MartyG-RealSense]

Thanks very much for sharing your findings, @GrubbyHalo

I added a Documentation link to this issue. Let's keep it open and see if any other C users report a similar experience to yours.