Skip to content

Add OpenVINO iGPU pipeline + forkserver guard + meson install fix#1124

Open
dys0re wants to merge 3 commits into
boltgolt:masterfrom
dys0re:openvino-igpu-support
Open

Add OpenVINO iGPU pipeline + forkserver guard + meson install fix#1124
dys0re wants to merge 3 commits into
boltgolt:masterfrom
dys0re:openvino-igpu-support

Conversation

@dys0re

@dys0re dys0re commented Jun 28, 2026

Copy link
Copy Markdown

Summary

Adds an optional OpenVINO inference pipeline that runs face detection and encoding on Intel integrated GPUs (Iris Xe / UHD / Arc), replacing the CPU-bound dlib HOG/CNN + ResNet path. Also includes two follow-up fixes that are required for the OpenVINO path to actually work end-to-end.

Three commits, each independently reviewable:

1. feat: add OpenVINO iGPU face detection and encoding pipeline

  • New module howdy/src/openvino_face.py exposing FaceDetector and FaceEncoder built on top of OpenVINO's Core:
    • Detection: face-detection-adas-0001 (SSD-based)
    • Encoding: face-reidentification-retail-0095 (256-dim embedding)
  • Compiled GPU models are cached under /var/cache/howdy/openvino/ for fast startup.
  • Detection results are converted to dlib.rectangle so the rest of the pipeline is unchanged.
  • Activated by the existing use_cnn = true config; automatic fallback to dlib if OpenVINO or the model files are unavailable.
  • OpenVINO is imported lazily (not at module load time) to avoid interfering with OpenCV's V4L2 backend in PAM environments.
  • New howdy/src/openvino-models/install.sh downloads the FP16 IR files on demand.

2. fix: guard compare.py and cli.py against forkserver re-import

This is the critical bug fix. Without it, PAM howdy fails on every authentication attempt when OpenVINO ≥ 2026.x is installed.

OpenVINO 2026.x's GPU runtime uses multiprocessing.forkserver to spawn worker processes. forkserver starts each worker by calling spawn.import_main_path(main_path), which re-executes the entry script under the name __mp_main__.

howdy's entry scripts (compare.py for PAM, cli.py for the howdy command) have all logic at module top level with no if __name__ == "__main__" guard, so every forkserver worker re-ran the entire script and tried to reopen the camera:

worker reads config
worker calls VideoCapture(config)        <-- open /dev/video<id>
open fails with EBUSY                    <-- UVC camera is exclusive
"Failed to read camera specified..."
worker sys.exit(14)

The forkserver worker crashing during import destabilises the parent's OpenVINO inference path, causing pam_howdy to exceed the configured timeout. End-user symptom under sudo / login:

pam_howdy[...]: Failure, unknown error 120
pam_howdy[...]: Failure, timeout reached
pam_howdy[...]: Failure, not possible to open camera at configured path

Reproduced via strace -f: a single compare.py dry run spawned 131 PIDs, with forkserver workers repeatedly attempting openat("/dev/video2") and hitting EBUSY.

Fix: exit early when __name__ != "__main__". Normal invocation has __name__ == "__main__" and is unaffected. This is the standard Python multiprocessing "guard your __main__" pattern.

3. fix(meson): install openvino-models/install.sh to datadir

Commit 1 added openvino-models/install.sh and the README instructs users to run sudo bash /usr/share/howdy/openvino-models/install.sh, but meson.build only added openvino_face.py to py_sources — the script was never installed. Now installed to <datadir>/howdy/openvino-models/ matching the existing dlib-data/install.sh pattern.


Benchmark

Dell Latitude 7420, i7-1185G7, Iris Xe, IR camera, Fedora 44, OpenVINO 2026.2.1, Python 3.14.6:

Pipeline FPS Auth time CPU usage
dlib HOG (CPU) ~15 ~3 s 1 core 100%
dlib CNN (CPU) <1 timeout 1 core 100%
OpenVINO (iGPU) ~30 ~1.6 s minimal

Measured pam_howdy outcomes (journalctl), 4 consecutive sudo invocations, same hardware:

Before forkserver guard After guard
Login approved 0 / 4 4 / 4
Failure, ... 4 / 4 0 / 4

Cosine-distance certainty under OpenVINO (30-frame sample, IR camera, single subject): median 0.22–0.28, min 0.15, max 0.36; current default threshold certainty = 4.5 (= 0.45) has comfortable margin.


Compatibility notes

  • Encoding dimensions change from 128 (dlib) to 256 (OpenVINO). Existing user models encoded by dlib are not comparable with OpenVINO embeddings. Users switching to the OpenVINO path must re-enroll (howdy clear && howdy add). The two paths use different distance metrics (L2 vs cosine) so mixing them is not safe.
  • use_cnn = true is reused as the activation switch. If a future maintainer prefers a dedicated use_openvino option I'm happy to refactor — kept the reuse to minimise config surface.
  • OpenVINO ≥ 2026.x triggers the forkserver behaviour that commit 2 guards against. Earlier OpenVINO versions don't use forkserver and work without the guard, but the guard is a no-op for them, so it's safe to apply unconditionally.
  • Only the entry scripts (compare.py, cli.py) need the guard. Subcommand modules under cli/ are imported via import cli.add etc., not re-executed by forkserver, so they don't need it.
  • face-detection-adas-0001 was trained on visible-light data and shows reduced recall (~30% per-frame) on pure IR frames processed via grayscale→BGR; in practice this is fine because the PAM timeout budget allows ~60 frames per authentication. Adding landmark-based face alignment (the model card for face-reidentification-retail-0095 recommends it) would further improve precision and is left as future work.

Files changed

File Change
howdy/src/openvino_face.py new — OpenVINO detector/encoder module
howdy/src/openvino-models/install.sh new — model downloader script
howdy/src/compare.py mod — use OpenVINO pipeline + forkserver guard
howdy/src/cli.py mod — forkserver guard
howdy/src/cli/add.py mod — use OpenVINO pipeline for enrollment
howdy/src/cli/test.py mod — use OpenVINO pipeline for test mode
howdy/src/recorders/video_capture.py mod — V4L2 open fallback for PAM
howdy/src/meson.build mod — install openvino_face.py and openvino-models/install.sh

Happy to split this into smaller PRs (e.g. land commit 2 standalone first since it's an unconditional correctness fix for any user on OpenVINO ≥ 2026.x) if the maintainer prefers.

dys0re added 3 commits June 28, 2026 22:40
Replace the CPU-bound dlib HOG/CNN face detection and ResNet encoding
with OpenVINO inference on Intel integrated GPUs (Iris Xe / UHD / Arc).

New module `openvino_face.py` exposes:
- FaceDetector: face-detection-adas-0001 (SSD-based) on iGPU
- FaceEncoder:  face-reidentification-retail-0095 (256-dim embedding) on iGPU

Detection results are converted to dlib.rectangle so the rest of the
howdy pipeline (PAM integration, model storage, UI) works unchanged.

Design:
- OpenVINO is imported lazily (not at module load time) to avoid
  interfering with OpenCV's V4L2 camera backend in PAM environments.
- Compiled GPU models are cached under /var/cache/howdy/openvino/
  for fast startup (~10s first run, <500ms after).
- Camera is opened BEFORE OpenVINO initializes to prevent backend
  conflicts observed on some integrated webcams.
- Automatic fallback to dlib if OpenVINO is unavailable or the model
  files are missing.

Files:
- howdy/src/openvino_face.py            (new) - shared pipeline module
- howdy/src/openvino-models/install.sh  (new) - model downloader script
- howdy/src/compare.py                    mod  - use OpenVINO for PAM auth
- howdy/src/cli/add.py                    mod  - use OpenVINO for enrollment
- howdy/src/cli/test.py                   mod  - use OpenVINO for test mode
- howdy/src/recorders/video_capture.py    mod  - V4L2 open fallback for PAM
- howdy/src/meson.build                   mod  - install openvino_face.py

Benchmark on Dell Latitude 7420 (i7-1185G7, Iris Xe):
- dlib HOG:    ~15 FPS, ~3s auth time
- dlib CNN:    <1 FPS, times out
- OpenVINO:   ~30 FPS, ~1.6s auth time
OpenVINO >= 2026.x uses multiprocessing.forkserver to spawn GPU worker
processes. forkserver starts each worker by calling
spawn.import_main_path(main_path), which re-executes the entry script
as __mp_main__.

howdy's entry scripts (compare.py for PAM, cli.py for the `howdy`
command) have all logic at module top level with no `if __name__ ==
"__main__"` guard, so every forkserver worker re-ran the entire
script:

  worker reads config
  worker calls VideoCapture(config)        <-- open /dev/video<id>
  open fails with EBUSY                    <-- UVC camera is exclusive
  "Failed to read camera specified..."
  worker sys.exit(14)

The forkserver worker crashes during import, which destabilises the
OpenVINO inference path in the parent process, causing PAM howdy to
exceed the configured timeout. End-user symptom: every sudo / login
that goes through pam_howdy.so fails with one of:

  - "Failure, unknown error 120"
  - "Failure, timeout reached"
  - "Failure, not possible to open camera at configured path"

Reproduced via strace: a single `compare.py` run spawned 131 PIDs;
forkserver workers repeatedly attempted openat("/dev/video2") and
hit EBUSY.

Fix: exit early when __name__ != "__main__" (i.e. when imported by
forkserver as __mp_main__). Normal invocation (`python compare.py
<user>` or `python cli.py <cmd>`) has __name__ == "__main__" and is
unaffected.

Verified on Dell Latitude 7420 (i7-1185G7, Iris Xe, IR camera),
Fedora 44, OpenVINO 2026.2.1, Python 3.14.6:
- Before patch: 4/4 consecutive pam_howdy runs FAILED (timeout)
- After patch:  4/4 consecutive pam_howdy runs "Login approved"

This is the standard Python multiprocessing "guard your __main__"
pattern; no behavioural change for normal execution.
The previous commit added howdy/src/openvino-models/install.sh as the
user-facing model downloader, and the README instructs users to run:

    sudo bash /usr/share/howdy/openvino-models/install.sh

But meson.build only added `openvino_face.py` to py_sources; the
install.sh script was never installed, so a fresh `ninja install`
silently produced no install.sh at the documented path.

Fix: install_data the script to <datadir>/howdy/openvino-models/
matching the pattern already used for dlib-data/install.sh.

Verified: `meson setup builddir --prefix=/tmp/x && ninja -C builddir
install` now produces /tmp/x/share/howdy/openvino-models/install.sh
with mode rwxr-xr-x.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant