Avoiding integer overflow in pixel values#173
Open
maneeshrj wants to merge 3 commits intohaesleinhuepf:mainfrom
Open
Avoiding integer overflow in pixel values#173maneeshrj wants to merge 3 commits intohaesleinhuepf:mainfrom
maneeshrj wants to merge 3 commits intohaesleinhuepf:mainfrom
Conversation
Casting to uint8 will cause overflow if the pixel value is beyond the display range. Common behavior (e.g. in MATLAB, matplotlib) is that any values above the display range will be displayed as 255 so to match this in stackview the casting should be done after the clipping to [0,255].
Owner
|
Hi @maneeshrj interesting, thanka for this suggestion! Could you please provide a scenario / example code / example image where I could see the impact of this code modification? Thanks! Best, |
Author
Trying to implement a tool for python notebooks that matches the functionality of Yi Wang Lab's in-house MATLAB "vis" tool
This reverts commit 6900755.
Author
|
Sorry I didn't realize that any further commits I made to my fork would show up in this pull request, I've reverted that commit. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


Casting to uint8 will cause integer overflow if the pixel value is beyond the display range, e.g. a value of 257 will overflow to 1. The convention (e.g. in MATLAB, matplotlib) seems to be that values above the display range will be displayed as 255 and values below the display range will be displayed as 0, so to match this behavior in stackview, the casting should be done after clipping the pixel value to the range [0,255].
If the way it's implemented now is intentional, feel free to ignore this, but I figured it's probably a bug.