Add OpenVINO iGPU pipeline + forkserver guard + meson install fix#1124
Open
dys0re wants to merge 3 commits into
Open
Add OpenVINO iGPU pipeline + forkserver guard + meson install fix#1124dys0re wants to merge 3 commits into
dys0re wants to merge 3 commits into
Conversation
Replace the CPU-bound dlib HOG/CNN face detection and ResNet encoding with OpenVINO inference on Intel integrated GPUs (Iris Xe / UHD / Arc). New module `openvino_face.py` exposes: - FaceDetector: face-detection-adas-0001 (SSD-based) on iGPU - FaceEncoder: face-reidentification-retail-0095 (256-dim embedding) on iGPU Detection results are converted to dlib.rectangle so the rest of the howdy pipeline (PAM integration, model storage, UI) works unchanged. Design: - OpenVINO is imported lazily (not at module load time) to avoid interfering with OpenCV's V4L2 camera backend in PAM environments. - Compiled GPU models are cached under /var/cache/howdy/openvino/ for fast startup (~10s first run, <500ms after). - Camera is opened BEFORE OpenVINO initializes to prevent backend conflicts observed on some integrated webcams. - Automatic fallback to dlib if OpenVINO is unavailable or the model files are missing. Files: - howdy/src/openvino_face.py (new) - shared pipeline module - howdy/src/openvino-models/install.sh (new) - model downloader script - howdy/src/compare.py mod - use OpenVINO for PAM auth - howdy/src/cli/add.py mod - use OpenVINO for enrollment - howdy/src/cli/test.py mod - use OpenVINO for test mode - howdy/src/recorders/video_capture.py mod - V4L2 open fallback for PAM - howdy/src/meson.build mod - install openvino_face.py Benchmark on Dell Latitude 7420 (i7-1185G7, Iris Xe): - dlib HOG: ~15 FPS, ~3s auth time - dlib CNN: <1 FPS, times out - OpenVINO: ~30 FPS, ~1.6s auth time
OpenVINO >= 2026.x uses multiprocessing.forkserver to spawn GPU worker
processes. forkserver starts each worker by calling
spawn.import_main_path(main_path), which re-executes the entry script
as __mp_main__.
howdy's entry scripts (compare.py for PAM, cli.py for the `howdy`
command) have all logic at module top level with no `if __name__ ==
"__main__"` guard, so every forkserver worker re-ran the entire
script:
worker reads config
worker calls VideoCapture(config) <-- open /dev/video<id>
open fails with EBUSY <-- UVC camera is exclusive
"Failed to read camera specified..."
worker sys.exit(14)
The forkserver worker crashes during import, which destabilises the
OpenVINO inference path in the parent process, causing PAM howdy to
exceed the configured timeout. End-user symptom: every sudo / login
that goes through pam_howdy.so fails with one of:
- "Failure, unknown error 120"
- "Failure, timeout reached"
- "Failure, not possible to open camera at configured path"
Reproduced via strace: a single `compare.py` run spawned 131 PIDs;
forkserver workers repeatedly attempted openat("/dev/video2") and
hit EBUSY.
Fix: exit early when __name__ != "__main__" (i.e. when imported by
forkserver as __mp_main__). Normal invocation (`python compare.py
<user>` or `python cli.py <cmd>`) has __name__ == "__main__" and is
unaffected.
Verified on Dell Latitude 7420 (i7-1185G7, Iris Xe, IR camera),
Fedora 44, OpenVINO 2026.2.1, Python 3.14.6:
- Before patch: 4/4 consecutive pam_howdy runs FAILED (timeout)
- After patch: 4/4 consecutive pam_howdy runs "Login approved"
This is the standard Python multiprocessing "guard your __main__"
pattern; no behavioural change for normal execution.
The previous commit added howdy/src/openvino-models/install.sh as the
user-facing model downloader, and the README instructs users to run:
sudo bash /usr/share/howdy/openvino-models/install.sh
But meson.build only added `openvino_face.py` to py_sources; the
install.sh script was never installed, so a fresh `ninja install`
silently produced no install.sh at the documented path.
Fix: install_data the script to <datadir>/howdy/openvino-models/
matching the pattern already used for dlib-data/install.sh.
Verified: `meson setup builddir --prefix=/tmp/x && ninja -C builddir
install` now produces /tmp/x/share/howdy/openvino-models/install.sh
with mode rwxr-xr-x.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an optional OpenVINO inference pipeline that runs face detection and encoding on Intel integrated GPUs (Iris Xe / UHD / Arc), replacing the CPU-bound dlib HOG/CNN + ResNet path. Also includes two follow-up fixes that are required for the OpenVINO path to actually work end-to-end.
Three commits, each independently reviewable:
1.
feat: add OpenVINO iGPU face detection and encoding pipelinehowdy/src/openvino_face.pyexposingFaceDetectorandFaceEncoderbuilt on top of OpenVINO'sCore:face-detection-adas-0001(SSD-based)face-reidentification-retail-0095(256-dim embedding)/var/cache/howdy/openvino/for fast startup.dlib.rectangleso the rest of the pipeline is unchanged.use_cnn = trueconfig; automatic fallback to dlib if OpenVINO or the model files are unavailable.howdy/src/openvino-models/install.shdownloads the FP16 IR files on demand.2.
fix: guard compare.py and cli.py against forkserver re-importThis is the critical bug fix. Without it, PAM howdy fails on every authentication attempt when OpenVINO ≥ 2026.x is installed.
OpenVINO 2026.x's GPU runtime uses
multiprocessing.forkserverto spawn worker processes. forkserver starts each worker by callingspawn.import_main_path(main_path), which re-executes the entry script under the name__mp_main__.howdy's entry scripts (
compare.pyfor PAM,cli.pyfor thehowdycommand) have all logic at module top level with noif __name__ == "__main__"guard, so every forkserver worker re-ran the entire script and tried to reopen the camera:The forkserver worker crashing during import destabilises the parent's OpenVINO inference path, causing pam_howdy to exceed the configured timeout. End-user symptom under
sudo/ login:Reproduced via
strace -f: a singlecompare.py dryrun spawned 131 PIDs, with forkserver workers repeatedly attemptingopenat("/dev/video2")and hittingEBUSY.Fix: exit early when
__name__ != "__main__". Normal invocation has__name__ == "__main__"and is unaffected. This is the standard Python multiprocessing "guard your__main__" pattern.3.
fix(meson): install openvino-models/install.sh to datadirCommit 1 added
openvino-models/install.shand the README instructs users to runsudo bash /usr/share/howdy/openvino-models/install.sh, butmeson.buildonly addedopenvino_face.pytopy_sources— the script was never installed. Now installed to<datadir>/howdy/openvino-models/matching the existingdlib-data/install.shpattern.Benchmark
Dell Latitude 7420, i7-1185G7, Iris Xe, IR camera, Fedora 44, OpenVINO 2026.2.1, Python 3.14.6:
Measured
pam_howdyoutcomes (journalctl), 4 consecutivesudoinvocations, same hardware:Login approvedFailure, ...Cosine-distance certainty under OpenVINO (30-frame sample, IR camera, single subject): median 0.22–0.28, min 0.15, max 0.36; current default threshold
certainty = 4.5(= 0.45) has comfortable margin.Compatibility notes
howdy clear && howdy add). The two paths use different distance metrics (L2 vs cosine) so mixing them is not safe.use_cnn = trueis reused as the activation switch. If a future maintainer prefers a dedicateduse_openvinooption I'm happy to refactor — kept the reuse to minimise config surface.compare.py,cli.py) need the guard. Subcommand modules undercli/are imported viaimport cli.addetc., not re-executed by forkserver, so they don't need it.face-detection-adas-0001was trained on visible-light data and shows reduced recall (~30% per-frame) on pure IR frames processed via grayscale→BGR; in practice this is fine because the PAM timeout budget allows ~60 frames per authentication. Adding landmark-based face alignment (the model card forface-reidentification-retail-0095recommends it) would further improve precision and is left as future work.Files changed
howdy/src/openvino_face.pyhowdy/src/openvino-models/install.shhowdy/src/compare.pyhowdy/src/cli.pyhowdy/src/cli/add.pyhowdy/src/cli/test.pyhowdy/src/recorders/video_capture.pyhowdy/src/meson.buildopenvino_face.pyandopenvino-models/install.shHappy to split this into smaller PRs (e.g. land commit 2 standalone first since it's an unconditional correctness fix for any user on OpenVINO ≥ 2026.x) if the maintainer prefers.