-
Notifications
You must be signed in to change notification settings - Fork 16
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Image segmentation for ios #113
base: main
Are you sure you want to change the base?
Conversation
5ef4c4d
to
0eb1cdd
Compare
ios/RnExecutorch/models/image_segmentation/ImageSegmentationModel.mm
Outdated
Show resolved
Hide resolved
ios/RnExecutorch/models/image_segmentation/ImageSegmentationModel.mm
Outdated
Show resolved
Hide resolved
ios/RnExecutorch/models/image_segmentation/ImageSegmentationModel.h
Outdated
Show resolved
Hide resolved
ios/RnExecutorch/models/image_segmentation/ImageSegmentationModel.mm
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall, LGTM 🚀
@@ -16,8 +17,8 @@ export const useImageSegmentation = ({ | |||
downloadProgress: number; | |||
forward: ( | |||
input: string, | |||
classesOfInterest?: string[] | |||
) => Promise<{ [category: string]: number[] }>; | |||
classesOfInterest?: DeeplabLabel[] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wondering if we should use this or keyof typeof DeeplabLabel. Using keyof typeof would make it consistent, since it is used in object detection but I think i like this one better. WDYT?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I haven't notice the use in object detection, but I would still make the case for this. In object detection the enum is used as a return value, so we're just interested in the string content, whereas here the enum is supplied by the user (in both cases, either as the filter for classes in a list, or as the key to query results in the dict), so we care more about correctness. I think that autocompletion for the enum values when writing is worth more than consistency in this case.
Description
Add semantic image segmentation with iOS native code. The result of a model run is a map containing masks of per-pixel probability for specified classes and an 'argmax' mask containing the indices of max value labels for each pixel.
Type of change
Tested on
Checklist