Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Image segmentation for ios #113

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

JakubGonera
Copy link
Contributor

@JakubGonera JakubGonera commented Mar 5, 2025

Description

Add semantic image segmentation with iOS native code. The result of a model run is a map containing masks of per-pixel probability for specified classes and an 'argmax' mask containing the indices of max value labels for each pixel.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update (improves or adds clarity to existing documentation)

Tested on

  • iOS
  • Android

Checklist

  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have updated the documentation accordingly
  • My changes generate no new warnings

@JakubGonera JakubGonera force-pushed the @jakubgonera/image-segmentation branch from 5ef4c4d to 0eb1cdd Compare March 5, 2025 11:57
@JakubGonera JakubGonera requested a review from chmjkb March 5, 2025 12:31
@JakubGonera JakubGonera requested a review from chmjkb March 10, 2025 11:32
Copy link
Contributor

@chmjkb chmjkb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, LGTM 🚀

@@ -16,8 +17,8 @@ export const useImageSegmentation = ({
downloadProgress: number;
forward: (
input: string,
classesOfInterest?: string[]
) => Promise<{ [category: string]: number[] }>;
classesOfInterest?: DeeplabLabel[]
Copy link
Contributor

@chmjkb chmjkb Mar 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wondering if we should use this or keyof typeof DeeplabLabel. Using keyof typeof would make it consistent, since it is used in object detection but I think i like this one better. WDYT?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't notice the use in object detection, but I would still make the case for this. In object detection the enum is used as a return value, so we're just interested in the string content, whereas here the enum is supplied by the user (in both cases, either as the filter for classes in a list, or as the key to query results in the dict), so we care more about correctness. I think that autocompletion for the enum values when writing is worth more than consistency in this case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants