Exposure control

[yoyoooooooo]

Question
Hi. Nice work, well done. I was wondering whether it is possible with this package to control of fix the exposure of the passthrough camera.

Context
I need to stitch shots and auto exposure makes them difficult to blend.

Additional Information
None

Thanks!

[Uralstech]

Hi! Thank you for your question and kind words.

I did some testing to see if this was possible, and have published my changes to the autoexposure-experiment branch. It adds a new ModifyRequest callback to all capture session's proxies. If a listener is added to the callback, it will return the CaptureRequest.Builder for modification. I ran an experiment like so:

        ...

        ContinuousCaptureSession session = _camera.CreateContinuousSession(highestResolution, CaptureTemplate.Preview, useCase);
        session.NativeProxy.ModifyRequest += static (builder, _) =>
        {
            using AndroidJavaClass captureRequest = new("android.hardware.camera2.CaptureRequest");
            using AndroidJavaObject aeKey = captureRequest.GetStatic<AndroidJavaObject>("CONTROL_AE_MODE");
            using AndroidJavaObject expTimeKey = captureRequest.GetStatic<AndroidJavaObject>("SENSOR_EXPOSURE_TIME");
            using AndroidJavaObject sensitivityKey = captureRequest.GetStatic<AndroidJavaObject>("SENSOR_SENSITIVITY");
            using AndroidJavaObject frameDurKey = captureRequest.GetStatic<AndroidJavaObject>("SENSOR_FRAME_DURATION");

            using AndroidJavaObject aeOff = new("java.lang.Integer", captureRequest.GetStatic<int>("CONTROL_AE_MODE_OFF"));
            using AndroidJavaObject exposure = new("java.lang.Long", 10_000_000L);
            using AndroidJavaObject iso = new("java.lang.Integer", 3200);
            using AndroidJavaObject frameDur = new("java.lang.Long", 33_333_333L);

            builder.Call("set", aeKey, aeOff);
            builder.Call("set", expTimeKey, exposure);
            builder.Call("set", sensitivityKey, iso);
            builder.Call("set", frameDurKey, frameDur);

            Debug.Log("Builder keys set.");
        };

        if (!await session.WaitForInitializationAsync())
            ...

And, from the logs, while the C# callback executes in time and does set the correct keys, the system seems to ignore them.

The logs from UXRQC.SessionManager were added for debugging and are included in the built AAR in the branch.

Please feel free to clone the branch and explore the request builder settings, and do let me know if you find a way to make it work!

[Uralstech]

With the latest preview release, you can now inspect all keys exposed through CameraCharacteristics directly from the C# API. From my testing, the only available auto-exposure mode currently reported by the device/runtime is 1, i.e. CONTROL_AE_MODE_ON:

        CameraInfo currentCamera = ...;
        currentCamera.TryGet("CONTROL_AE_AVAILABLE_MODES", out int[] aeModes);
        Debug.Log($"AE MODES: {string.Join(',', aeModes ?? Array.Empty<int>())}");

The latest preview release also changes how capture requests can be modified, compared to the experimental branch:

        ContinuousCaptureSession session = ...
        session.NativeProxy.ModifyRequestBuilder += static (builder, _) =>
        {
            builder.Set("CONTROL_AE_MODE", builder.GetConstant("CONTROL_AE_MODE_OFF"));
            builder.Set("SENSOR_EXPOSURE_TIME", 10_000_000L);
            builder.Set("SENSOR_SENSITIVITY", 3200);
            builder.Set("SENSOR_FRAME_DURATION", 33_333_333L);
            Debug.Log("Builder keys set.");
        };

From my experiments, the request values are applied to the builder correctly, but the camera pipeline appears to ignore manual exposure controls and keeps AE enabled.

At the moment, this means manual exposure control does not appear to be supported by the underlying passthrough camera implementation/runtime. Since there's no actionable change remaining on the package side for now, I'll close this issue, but if Meta exposes additional camera controls in future runtime updates, I'd be happy to revisit it.

[yoyoooooooo]

OK. Many thanks for the time you spent investigating!