Skip to content

Enhance NV12 and NV21 Support in sensor_msgs::image_encodings #264

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: rolling
Choose a base branch
from

Conversation

zycczy
Copy link

@zycczy zycczy commented Jan 14, 2025

Description

This pull request introduces significant improvements to the sensor_msgs::image_encodings module in ROS2 Rolling, specifically enhancing support for NV12 and NV21 image encodings. The changes aim to provide more accurate channel definitions, facilitate planar encoding detection, and offer height scaling factors necessary for proper image buffer management.

Changes Made

  1. Redefine numChannels for NV12 and NV21:

    • Updated the numChannels function to return 1 for NV12 and NV21 instead of 2. This change accurately represents their planar YUV 4:2:0 structure, where the image data is split into Y and UV planes.
  2. Add isPlanar Function:

    • Introduced a new utility function isPlanar to check if a given encoding is planar. This function currently identifies NV12, NV21, and NV24 as planar encodings.
    • Usage:
      bool is_planar = sensor_msgs::image_encodings::isPlanar(encoding);
  3. Introduce getHeightScaling Function:

    • Added getHeightScaling to provide the height scaling factor for planar encodings. For example, NV21 images with height H will have an actual buffer height of H * 1.5 to accommodate the UV planes.
    • Usage:
      float scaling_factor = sensor_msgs::image_encodings::getHeightScaling(encoding);
  4. Correct bitDepth Function:

    • Fixed the bitDepth function to accurately parse and return bit depths for complex encoding strings such as "8UC10".
  5. Code Consistency and Documentation:

    • Unified the handling of planar formats to ensure consistency across different encoding types.
    • Added detailed documentation comments for the newly introduced functions to aid developers in understanding and utilizing them effectively.

- Redefine numChannels for NV12 and NV21 from 2 to 1 to accurately reflect their planar YUV 4:2:0 format.
- Add isPlanar function to determine if an encoding is planar (e.g., NV12, NV21, NV24).
- Introduce getHeightScaling function to provide height scaling factors for planar encodings.
- Improve code consistency and add documentation for new utility functions.

Signed-off-by: Zhaoyuan Cheng <[email protected]>
Copy link
Contributor

@sloretz sloretz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the PR!

static inline int numChannels(const std::string & encoding)
{
// First do the common-case encodings
if (encoding == MONO8 ||
encoding == MONO16)
encoding == MONO16)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove the whitespace changes. It makes the diff smaller, and a little easier to review

Copy link
Author

@zycczy zycczy Feb 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@@ -241,23 +249,34 @@ static inline int bitDepth(const std::string & encoding)
std::cmatch m;
// ex. 8UC -> 8, 8UC10 -> 10
if (std::regex_match(encoding.c_str(), m, cv_type_regex)) {
return std::atoi(m[0].str().c_str());
return std::atoi(m[1].str().c_str());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does this change do? Mind also adding a bounds check to make sure m[1] exists?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because m[0] will return the full match and m[1] return the depth, which may more make senses, although after atio it return the same value

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But you need to be sure that m.size() is at least 2, isn't it?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your right, ahcorde, I will modify my code, the origin code seems more concise

{
return 8;
}

throw std::runtime_error("Unknown encoding " + encoding);
return -1;
}

static inline float getHeightScaling(const std::string & encoding)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Author

@zycczy zycczy Feb 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@Yadunund
Copy link
Member

Yadunund commented Feb 6, 2025

@zycczy could you please follow up on @sloretz's feedback?

@Yadunund Yadunund added the more-information-needed Further information is required label Feb 6, 2025
@zycczy
Copy link
Author

zycczy commented Feb 10, 2025

@zycczy could you please follow up on @sloretz's feedback?

Sorry for the late reply, Has pushed new changes, these days are Chinese New Year, so I have some days vacation, back to work now

@zycczy zycczy requested a review from sloretz February 17, 2025 03:31
Signed-off-by: Zhaoyuan Cheng <[email protected]>
@zycczy
Copy link
Author

zycczy commented Apr 1, 2025

@sloretz @Yadunund @ahcorde could you please have a review for this PR?

@@ -241,23 +249,34 @@ static inline int bitDepth(const std::string & encoding)
std::cmatch m;
// ex. 8UC -> 8, 8UC10 -> 10
if (std::regex_match(encoding.c_str(), m, cv_type_regex)) {
return std::atoi(m[0].str().c_str());
return std::atoi(m[1].str().c_str());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But you need to be sure that m.size() is at least 2, isn't it?

Signed-off-by: Zhaoyuan Cheng <[email protected]>
Copy link
Contributor

@ahcorde ahcorde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sloretz you already review this PR, do you mind to take another look ?

@ahcorde
Copy link
Contributor

ahcorde commented Apr 3, 2025

Pulls: #264
Gist: https://gist.githubusercontent.com/ahcorde/51a66c824e3b09acaacf3666af403e5d/raw/4932642bb4790d1a78b8b61e483d915659b07b64/ros2.repos
BUILD args: --packages-above-and-dependencies sensor_msgs
TEST args: --packages-above sensor_msgs
ROS Distro: rolling
Job: ci_launcher
ci_launcher ran: https://ci.ros2.org/job/ci_launcher/15568

  • Linux Build Status
  • Linux-aarch64 Build Status
  • Linux-rhel Build Status
  • Windows Build Status

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
more-information-needed Further information is required
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants