Skip to content

Misc bugfixes#162

Merged
TheTechromancer merged 5 commits into
devfrom
misc-bugfixes
Apr 30, 2026
Merged

Misc bugfixes#162
TheTechromancer merged 5 commits into
devfrom
misc-bugfixes

Conversation

@TheTechromancer
Copy link
Copy Markdown
Contributor

bbot-server bug fixes

Fixes blacklanternsecurity/bbot-enterprise-frontend#58 — confidence always stored as UNKNOWN.
Backs blacklanternsecurity/bbot-enterprise-frontend#81 — server-side regression test for the asset query builder is not empty fix (the UI fix lives in the frontend PR).
Backs blacklanternsecurity/bbot-enterprise-frontend#84CONFIDENCE_LEVELS rename so the API and the UI agree on the middle label.


#58confidence always stored as UNKNOWN

Every finding rendered Confidence: UNKNOWN, regardless of the value the bbot module emitted. Trace: bbot core emits, validates, and serializes confidence: "HIGH" correctly through scan → event.json() → Pydantic Event → Redis → worker. event.data_json["confidence"] arrives at the findings handler intact. But:

Finding(name="X", host="x.com", description="d", severity="HIGH", confidence="HIGH")
# severity:   HIGH    score: 4   ✓
# confidence: UNKNOWN score: 1   ✗

Root cause

confidence_score was declared with alias="confidence" and default=1. In Pydantic v2 without populate_by_name=True (which BBOTBaseModel does not set), aliased fields can only be populated via the alias — passing the field name is silently ignored. The custom __init__ rewrites kwargs["confidence_score"] = N and calls super().__init__(**kwargs), which Pydantic ignores (alias mismatch) and applies default=1. Reads from Mongo via Finding(**db_dict) had the same problem — even loads couldn't recover the right value.

severity_score works correctly because it has neither alias nor default; the __init__ rewrite to severity_score=N lands as expected.

Change

  • bbot_server/modules/findings/findings_models.py — drop alias="confidence" and default=1 from confidence_score. The field now mirrors severity_score's shape; the existing __init__ already handles the symmetric confidence= kwarg conversion. The handler-level default at findings_api.py:220 (event.data_json.get("confidence", 1)) covers the missing-field case the same way severity is at line 221.

Existing UNKNOWNs heal on re-scan

Old findings stay UNKNOWN until re-emitted. The update branch in _insert_or_update_finding $sets both confidence and confidence_score on every match, and findings are matched by id = sha1(description + ":" + netloc), so the next scan that re-emits a known finding rewrites its row with the corrected score. No backfill needed.

Tests

  • tests/test_applets/test_applet_findings.py — three assertion sites flipped from "UNKNOWN"/1 to "HIGH"/4 (the fixture in gen_scan_data.py has always emitted "confidence": "HIGH" — the prior assertions documented the bug). Plus a new module-level test_finding_confidence_kwarg that constructs Finding(...) directly with both string and integer confidence, locking in the contract at the model layer so a future Pydantic-alias regression can't slip through the scan-pipeline tests again.
  • tests/test_cli/test_cli_findingctl.py — same assertion flip.

Bug-fix workflow per AGENTS.md: assertions flipped first, confirmed failing (AssertionError: 'UNKNOWN' == 'HIGH'), then the model fix applied, then re-run green.


#81 — Asset query: $in: [None, []] regression test

The actual UI fix (new IS_NOT_EMPTY_OPERATOR, symmetric isEmpty rewrite) is in the bbot-enterprise-frontend PR. The pass-through MongoDB pipeline (backend/robotics/api/assets.pyBaseQuery.build) already handled $in/$nin; this PR just locks in the contract.

  • tests/test_applets/test_applet_assets.py — new test_applet_assets_empty_query_open_ports seeds four shapes (field absent / explicit null / [] / [80]) via the raw motor collection and verifies $in: [None, []] returns the first three and $nin: [None, []] returns only the populated row.
  • bbot_server/compose.yml, compose.yml — drive-by: raised mongo nofile ulimits to 64000 (matches the test runner expectations from AGENTS.md).

The HTTP parametrization is a no-op early-return, matching the existing test_applet_custom_attributes pattern, since direct collection mutation isn't available over HTTP.


#84CONFIDENCE_LEVELS: MODERATEMEDIUM

The frontend used MODERATE interchangeably with MEDIUM in severity contexts (the actual bug is in the frontend PR). Confidence is a separate vocabulary, but it also used MODERATE as the middle label, which kept the inconsistency alive on the API side. Renaming here lets the frontend drop the MODERATE → MEDIUM runtime translation hack and unify on MEDIUM end to end.

  • bbot_server/modules/findings/findings_models.pyCONFIDENCE_LEVELS = {"UNKNOWN": 1, "LOW": 2, "MODERATE": 3, "HIGH": 4, "CONFIRMED": 5} becomes {"UNKNOWN": 1, "LOW": 2, "MEDIUM": 3, "HIGH": 4, "CONFIRMED": 5}. Confirmed bbot core has no confidence references, so there's no upstream emission of "MODERATE" to break.
  • tests/test_applets/test_applet_findings.py — extended test_finding_confidence_kwarg with a confidence="MEDIUM"/confidence_score == 3 assertion as a regression for the rename.

Existing Mongo docs containing confidence: "MODERATE" heal on re-scan via the same $set branch as #58 — no backfill.


Test plan

# Test deps (if not already running)
docker run -d --name bbot-mongo --ulimit nofile=64000:64000 -p 127.0.0.1:27017:27017 mongo
docker run -d --name bbot-redis -p 127.0.0.1:6379:6379 redis

uv run pytest tests/test_applets/test_applet_findings.py tests/test_applets/test_applet_assets.py tests/test_cli/test_cli_findingctl.py

test_applet_findings.py: 3 passed (the new test_finding_confidence_kwarg with both string-form and the MEDIUM regression assertion, plus both python and http parametrizations of TestAppletFindings::test_applet_run).

test_applet_assets.py::test_applet_assets_empty_query_open_ports: passes both parametrizations.

test_cli_findingctl.py: passes (assertion-flip from "UNKNOWN""HIGH").

@ausmaster ausmaster self-requested a review April 29, 2026 22:09
@TheTechromancer TheTechromancer merged commit d9ac1b9 into dev Apr 30, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants