Enhance NV12 and NV21 Support in sensor_msgs::image_encodings #264

zycczy · 2025-01-14T08:49:56Z

Description

This pull request introduces significant improvements to the sensor_msgs::image_encodings module in ROS2 Rolling, specifically enhancing support for NV12 and NV21 image encodings. The changes aim to provide more accurate channel definitions, facilitate planar encoding detection, and offer height scaling factors necessary for proper image buffer management.

Changes Made

Redefine numChannels for NV12 and NV21:
- Updated the numChannels function to return 1 for NV12 and NV21 instead of 2. This change accurately represents their planar YUV 4:2:0 structure, where the image data is split into Y and UV planes.
Add isPlanar Function:
- Introduced a new utility function isPlanar to check if a given encoding is planar. This function currently identifies NV12, NV21, and NV24 as planar encodings.
- Usage:
```
bool is_planar = sensor_msgs::image_encodings::isPlanar(encoding);
```
Introduce getHeightScaling Function:
- Added getHeightScaling to provide the height scaling factor for planar encodings. For example, NV21 images with height H will have an actual buffer height of H * 1.5 to accommodate the UV planes.
- Usage:
```
float scaling_factor = sensor_msgs::image_encodings::getHeightScaling(encoding);
```
Correct bitDepth Function:
- Fixed the bitDepth function to accurately parse and return bit depths for complex encoding strings such as "8UC10".
Code Consistency and Documentation:
- Unified the handling of planar formats to ensure consistency across different encoding types.
- Added detailed documentation comments for the newly introduced functions to aid developers in understanding and utilizing them effectively.

- Redefine numChannels for NV12 and NV21 from 2 to 1 to accurately reflect their planar YUV 4:2:0 format. - Add isPlanar function to determine if an encoding is planar (e.g., NV12, NV21, NV24). - Introduce getHeightScaling function to provide height scaling factors for planar encodings. - Improve code consistency and add documentation for new utility functions. Signed-off-by: Zhaoyuan Cheng <[email protected]>

sloretz

Thank you for the PR!

sloretz · 2025-01-24T00:44:36Z

sensor_msgs/include/sensor_msgs/image_encodings.hpp

 static inline int numChannels(const std::string & encoding)
 {
  // First do the common-case encodings
  if (encoding == MONO8 ||
-    encoding == MONO16)
+      encoding == MONO16)


Please remove the whitespace changes. It makes the diff smaller, and a little easier to review

sloretz · 2025-01-24T00:45:12Z

sensor_msgs/include/sensor_msgs/image_encodings.hpp

@@ -241,23 +249,34 @@ static inline int bitDepth(const std::string & encoding)
  std::cmatch m;
  // ex. 8UC -> 8, 8UC10 -> 10
  if (std::regex_match(encoding.c_str(), m, cv_type_regex)) {
-    return std::atoi(m[0].str().c_str());
+    return std::atoi(m[1].str().c_str());


What does this change do? Mind also adding a bounds check to make sure m[1] exists?

Because m[0] will return the full match and m[1] return the depth, which may more make senses, although after atio it return the same value

But you need to be sure that m.size() is at least 2, isn't it?

Your right, ahcorde, I will modify my code, the origin code seems more concise

sloretz · 2025-01-24T00:46:05Z

sensor_msgs/include/sensor_msgs/image_encodings.hpp

  {
    return 8;
  }

  throw std::runtime_error("Unknown encoding " + encoding);
  return -1;
 }
+
+static inline float getHeightScaling(const std::string & encoding)


Please add tests for the new functions in https://github.com/ros2/common_interfaces/blob/rolling/sensor_msgs/test/test_image_encodings.cpp

Yadunund · 2025-02-06T21:23:51Z

@zycczy could you please follow up on @sloretz's feedback?

Signed-off-by: Zhaoyuan Cheng <[email protected]>

zycczy · 2025-02-10T07:43:14Z

@zycczy could you please follow up on @sloretz's feedback?

Sorry for the late reply, Has pushed new changes, these days are Chinese New Year, so I have some days vacation, back to work now

Signed-off-by: Zhaoyuan Cheng <[email protected]>

zycczy · 2025-04-01T07:44:08Z

@sloretz @Yadunund @ahcorde could you please have a review for this PR?

ahcorde · 2025-04-01T21:08:31Z

sensor_msgs/include/sensor_msgs/image_encodings.hpp

@@ -241,23 +249,34 @@ static inline int bitDepth(const std::string & encoding)
  std::cmatch m;
  // ex. 8UC -> 8, 8UC10 -> 10
  if (std::regex_match(encoding.c_str(), m, cv_type_regex)) {
-    return std::atoi(m[0].str().c_str());
+    return std::atoi(m[1].str().c_str());


But you need to be sure that m.size() is at least 2, isn't it?

Signed-off-by: Zhaoyuan Cheng <[email protected]>

ahcorde

@sloretz you already review this PR, do you mind to take another look ?

ahcorde · 2025-04-03T11:14:53Z

Pulls: #264
Gist: https://gist.githubusercontent.com/ahcorde/51a66c824e3b09acaacf3666af403e5d/raw/4932642bb4790d1a78b8b61e483d915659b07b64/ros2.repos
BUILD args: --packages-above-and-dependencies sensor_msgs
TEST args: --packages-above sensor_msgs
ROS Distro: rolling
Job: ci_launcher
ci_launcher ran: https://ci.ros2.org/job/ci_launcher/15568

Linux
Linux-aarch64
Linux-rhel
Windows

zycczy · 2025-05-13T02:56:56Z

Hi @ahcorde @sloretz
Can we merge this PR? So that next I will work on RQT and image transport NV12 support, the local change has almost done, depends on this PR merge

Thanks.
Zhaoyuan

ahcorde · 2025-05-15T12:04:11Z

@sloretz you already review this PR, do you mind to take another look ?

ahcorde · 2025-05-15T12:04:28Z

Linux
Linux-aarch64
Linux-rhel
Windows

sloretz requested changes Jan 24, 2025

View reviewed changes

Yadunund assigned sloretz Feb 6, 2025

Yadunund added the more-information-needed Further information is required label Feb 6, 2025

remove the whitespace changes and add getHeightScaling test function

3214e9f

Signed-off-by: Zhaoyuan Cheng <[email protected]>

zycczy mentioned this pull request Feb 10, 2025

Add support for NV12 encoding in cv_bridge ros-perception/vision_opencv#544

Open

zycczy added 2 commits February 12, 2025 17:33

Delete common_interfaces.code-workspace

a96b186

Signed-off-by: Zhaoyuan Cheng <[email protected]>

Update image_encodings.hpp

b9ba7ac

Signed-off-by: Zhaoyuan Cheng <[email protected]>

zycczy requested a review from sloretz February 17, 2025 03:31

Update image_encodings.hpp

20c3b95

Signed-off-by: Zhaoyuan Cheng <[email protected]>

ahcorde requested changes Apr 1, 2025

View reviewed changes

Update image_encodings.hpp

9b80f76

Signed-off-by: Zhaoyuan Cheng <[email protected]>

ahcorde approved these changes Apr 3, 2025

View reviewed changes

ahcorde merged commit fa74e95 into ros2:rolling May 20, 2025
1 check passed

Enhance NV12 and NV21 Support in sensor_msgs::image_encodings #264

Enhance NV12 and NV21 Support in sensor_msgs::image_encodings #264

Uh oh!

Conversation

zycczy commented Jan 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Changes Made

Uh oh!

sloretz left a comment

Choose a reason for hiding this comment

Uh oh!

sloretz Jan 24, 2025

Choose a reason for hiding this comment

Uh oh!

zycczy Feb 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sloretz Jan 24, 2025

Choose a reason for hiding this comment

Uh oh!

zycczy Feb 10, 2025

Choose a reason for hiding this comment

Uh oh!

ahcorde Apr 1, 2025

Choose a reason for hiding this comment

Uh oh!

zycczy Apr 3, 2025

Choose a reason for hiding this comment

Uh oh!

sloretz Jan 24, 2025

Choose a reason for hiding this comment

Uh oh!

zycczy Feb 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Yadunund commented Feb 6, 2025

Uh oh!

zycczy commented Feb 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zycczy commented Apr 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ahcorde Apr 1, 2025

Choose a reason for hiding this comment

Uh oh!

ahcorde left a comment

Choose a reason for hiding this comment

Uh oh!

ahcorde commented Apr 3, 2025

Uh oh!

zycczy commented May 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ahcorde commented May 15, 2025

Uh oh!

ahcorde commented May 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

zycczy commented Jan 14, 2025 •

edited

Loading

zycczy Feb 10, 2025 •

edited

Loading

zycczy Feb 10, 2025 •

edited

Loading

zycczy commented Feb 10, 2025 •

edited

Loading

zycczy commented Apr 1, 2025 •

edited

Loading

zycczy commented May 13, 2025 •

edited

Loading

ahcorde commented May 15, 2025 •

edited

Loading