Can I remove everything but faces?

[alkinnon]

Hi, as part of a student games dev project, I'm looking for a linux solution which will remove everything but a face from the image, leaving only the face, and transparent alpha channel ideally.

Can backscrub extract just faces?

The virtual cam output can then be read within a browser 2D platform game for further processing (using players face on their actual player character) .

[floe]

Interesting question - this won't work out-of-the-box, because all the ML models are trained to segment persons and not faces. But you could try a two-step approach, where you take the person outline from backscrub, and then run a dedicated face detector such as described in https://docs.opencv.org/4.2.0/db/d28/tutorial_cascade_classifier.html (and then just take the intersection of the two regions).

[BenBE]

Also if you take a compatible model that only marks faces in the image this might even work with backscrub directly. But depending on the model you will likely need to adapt some stuff to properly integrate the model with the internal processing pipeline we use.

[alkinnon]

Thanks for the comments. interesting things to look into!

[floe]

Ah wait, now I remember that body-pix actually does separate classes for body parts, including head. That might work almost out of the box.