Skip to content

Commit db3fa2d

Browse files
committed
fix: correct adapter dispatch and safe data completeness
1 parent 67a517b commit db3fa2d

6 files changed

Lines changed: 267 additions & 55 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@ All notable changes to this project will be documented in this file.
4242
- `LIMIT_ADAPTER_ERROR`, `LIMIT_UNSUPPORTED_TYPE`, `LIMIT_RECURSIVE` constants
4343
exported from `logprivacy.internal.traversal` for stable limitation identifiers.
4444

45-
## 0.5.1 - Unreleased
45+
## 0.5.1 - 2026-06-04
4646

4747
### Changed
4848

‎docs/data/structured-data.md‎

Lines changed: 110 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,8 @@ LogPrivacy has two structured-data APIs:
77
- `to_safe_data()` converts supported Python values into sanitized JSON-safe
88
data: `None`, booleans, finite numbers, strings, lists, and dictionaries with
99
string keys.
10+
- `to_safe_data_with_result()` does the same and additionally returns completeness
11+
metadata, limitation identifiers, and per-call statistics.
1012

1113
Use `to_safe_data()` before structured logging, JSON responses, JSON files, or
1214
anywhere arbitrary Python objects must not escape into output.
@@ -84,6 +86,52 @@ to_safe_data(Request("req-1", {"token": "abc123456789"}), adapters=adapters)
8486
`AdapterRegistry.default()` returns a fresh registry. Registering a converter in
8587
one registry does not mutate another registry or global process state.
8688

89+
### Dispatch Order
90+
91+
Adapter resolution runs **before** the built-in structural handlers for
92+
`Mapping`, `list`/`tuple`, and `set`/`frozenset`. This means a custom subclass
93+
registered in the adapter registry always takes precedence over the native
94+
fallback, even if the subclass inherits from a built-in container type.
95+
96+
Dispatch order within `_normalize`:
97+
98+
1. Exact primitive types: `None`, `bool`, `int`, `float`, `str`, `bytes`,
99+
`bytearray`, `memoryview` — always handled by the core pipeline, never
100+
intercepted by adapters.
101+
2. Adapter registry — MRO lookup first (no `isinstance`), then virtual/abstract
102+
types (protected `isinstance` calls).
103+
3. Structural fallbacks — `str` subclasses, `Mapping`, `list`/`tuple`,
104+
`set`/`frozenset`, dataclasses, `Enum`, `Decimal`, date/time types, `UUID`,
105+
`Path`, exceptions.
106+
4. Unsupported — becomes `[UNSUPPORTED:TypeName]`.
107+
108+
### Reserved Types
109+
110+
The following types are reserved for the core pipeline and cannot be used as
111+
adapter targets. Attempting to register them raises `ValueError`:
112+
113+
`object`, `str`, `int`, `float`, `bool`, `bytes`, `bytearray`, `memoryview`,
114+
`dict`, `list`, `tuple`, `set`, `frozenset`.
115+
116+
Custom subclasses are fully supported:
117+
118+
```python
119+
class ExternalList(list):
120+
pass
121+
122+
adapters.register(ExternalList, lambda v: {"items": list(v), "source": "external"})
123+
```
124+
125+
### Adapter Failures
126+
127+
If a converter raises, returns the original object, or if `__instancecheck__`
128+
raises during virtual-type resolution, the value becomes
129+
`[UNSUPPORTED:TypeName]` and `adapter_error` is added to limitations. The
130+
original value is never exposed.
131+
132+
Converters are never called on reserved primitive types regardless of how the
133+
registry was configured.
134+
87135
## Field Rules
88136

89137
`CleanerPolicy.sensitive_keys` is still supported. For more control, add
@@ -115,13 +163,70 @@ Field rules support four actions:
115163

116164
- `mask`: replace the value using the policy masking strategy;
117165
- `remove`: keep the field but replace the value with `[REMOVED]`;
118-
- `truncate`: keep only `max_chars` from textual values, then sanitize that
119-
preserved text;
166+
- `truncate`: sanitize the **full** text first, then cut to `max_chars` and
167+
append `[TRUNCATED]` if needed. This order ensures secrets that fall after
168+
the cut point are never exposed in the output.
120169
- `block`: raise `LogBlockedError` without including the blocked value.
121170

122-
In this phase, `truncate` is intentionally limited to textual values
123-
(`str`, `bytes`, `bytearray`, and `memoryview`). Non-text values become
124-
`[TRUNCATED]`.
171+
`truncate` is limited to textual values (`str`, `bytes`, `bytearray`, and
172+
`memoryview`). Non-text values become `[TRUNCATED]` without sanitization
173+
(there is no text content to redact).
174+
175+
### Field Rules on Exceptions
176+
177+
Field rules apply to the `type` and `message` fields produced when an exception
178+
is sanitized, giving the same protection as mappings and dataclasses:
179+
180+
```python
181+
from logprivacy import CleanerPolicy, FieldRule, to_safe_data
182+
183+
policy = CleanerPolicy.default().add_field_rules(
184+
FieldRule.exact("message", action="remove"),
185+
)
186+
187+
to_safe_data(ValueError("secret token: abc123"), policy=policy)
188+
# {"type": "ValueError", "message": "[REMOVED]"}
189+
```
190+
191+
## Rich Output with SafeDataResult
192+
193+
Use `to_safe_data_with_result()` when you need to know whether sanitization was
194+
complete, which limits were reached, or how many fields were redacted:
195+
196+
```python
197+
from logprivacy import to_safe_data_with_result
198+
199+
result = to_safe_data_with_result(payload)
200+
201+
if not result.complete:
202+
logger.warning("partial sanitization: %s", result.limitations)
203+
204+
log_event(result.cleaned)
205+
```
206+
207+
`SafeDataResult` fields:
208+
209+
- `cleaned` — the sanitized value, equivalent to `to_safe_data(payload)`;
210+
- `complete` — `False` when any traversal limit was reached or any branch could
211+
not be fully inspected;
212+
- `limitations` — a tuple of stable string identifiers for each limit hit:
213+
`max_depth`, `max_items`, `iteration_error`, `representation_error`,
214+
`adapter_error`, `unsupported_type`, `recursive_value`;
215+
- `stats` — a `SafeDataStats` dataclass with aggregate counters.
216+
217+
`SafeDataStats` fields:
218+
219+
| Field | Counts |
220+
|---|---|
221+
| `masked` | text findings redacted by the text pipeline plus structural mask actions and sensitive-key masking |
222+
| `removed` | fields replaced with `[REMOVED]` by a `remove` rule |
223+
| `truncated` | fields truncated by a `truncate` rule |
224+
| `unsupported` | values replaced with `[UNSUPPORTED:TypeName]` |
225+
| `adapter_errors` | converters that raised, returned self, or had `__instancecheck__` raise during resolution |
226+
| `field_rule_matches` | total `FieldRule` matches across all fields |
227+
228+
Both `SafeDataResult` and `SafeDataStats` are frozen dataclasses. Neither retains
229+
any reference to the original value or raw converter output.
125230

126231
## Safe JSON
127232

‎src/logprivacy/adapters/registry.py‎

Lines changed: 53 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -31,10 +31,18 @@
3131
)
3232

3333

34-
def _safe_adapter_type_name(value: object) -> str:
35-
name = type(value).__name__ or "object"
36-
safe = _SAFE_TYPE_NAME_PATTERN.sub("_", name)[:80]
37-
return safe or "object"
34+
@dataclass(frozen=True, slots=True)
35+
class _AdapterResolution:
36+
"""Internal result of an adapter resolution attempt.
37+
38+
``converter`` is the callable to use, or ``None`` when no adapter matched.
39+
``resolution_failed`` is ``True`` when ``__instancecheck__`` raised during
40+
the virtual-type scan; in that case the resolution is not fully reliable and
41+
the caller must mark ``adapter_error`` and not execute the converter.
42+
"""
43+
44+
converter: AdapterConverter | None = None
45+
resolution_failed: bool = False
3846

3947

4048
@dataclass(slots=True)
@@ -47,6 +55,10 @@ class AdapterRegistry:
4755
registered. A custom subclass such as ``class ExternalList(list)`` *can*
4856
be registered; ``list`` itself cannot.
4957
58+
Adapter dispatch for custom subclasses happens **before** the built-in
59+
structural handlers (Mapping, list/tuple, set/frozenset), so a registered
60+
converter always takes precedence over the native fallback.
61+
5062
``default()`` returns a fresh registry so applications and tests can add
5163
converters without mutating shared global state.
5264
"""
@@ -88,24 +100,52 @@ def resolve(self, value: object) -> AdapterConverter | None:
88100
"""Return the best converter for ``value`` or ``None``.
89101
90102
Resolution order:
91-
1. Concrete type registered directly.
92-
2. Base classes in MRO order, most specific first.
93-
3. Abstract / virtual base classes in registration order.
103+
1. Concrete type registered directly (MRO walk, no ``isinstance``).
104+
2. Abstract / virtual base classes in registration order.
94105
95-
``__instancecheck__`` calls are protected; a metaclass that raises
96-
during ``isinstance`` is treated as non-matching.
106+
``__instancecheck__`` calls are protected. A metaclass that raises
107+
during ``isinstance`` is treated as non-matching. Use
108+
``_resolve_result()`` when you need to distinguish "no match" from
109+
"resolution failed".
110+
"""
111+
return self._resolve_result(value).converter
112+
113+
def _resolve_result(self, value: object) -> _AdapterResolution:
114+
"""Return a resolution result distinguishing success, failure, and no-match.
115+
116+
Resolution order:
117+
1. Concrete type and MRO bases (direct dict lookup — never raises).
118+
2. Virtual / abstract base classes in registration order (protected
119+
``isinstance`` calls — a metaclass that raises sets
120+
``resolution_failed=True`` and the normalizer marks ``adapter_error``).
121+
122+
When ``resolution_failed`` is ``True`` the caller must mark
123+
``adapter_error`` and not execute any converter.
97124
"""
98125
value_type = type(value)
126+
127+
# Phase 1: MRO-based lookup (safe — pure dict lookup, no user code).
99128
for candidate in value_type.__mro__:
100129
converter = self._converters.get(candidate)
101130
if converter is not None:
102-
return converter
131+
return _AdapterResolution(converter=converter)
103132

133+
# Phase 2: virtual / abstract base class scan (may call __instancecheck__).
134+
resolution_failed = False
104135
for registered_type, converter in self._converters.items():
105136
try:
106137
matches = isinstance(value, registered_type)
107138
except Exception:
108-
matches = False
139+
# __instancecheck__ raised — resolution is not fully reliable.
140+
# Record the failure; do NOT use this or any subsequent converter.
141+
resolution_failed = True
142+
continue
109143
if matches:
110-
return converter
111-
return None
144+
if resolution_failed:
145+
# A prior failure makes the overall resolution unreliable.
146+
break
147+
return _AdapterResolution(converter=converter)
148+
149+
if resolution_failed:
150+
return _AdapterResolution(resolution_failed=True)
151+
return _AdapterResolution()

‎src/logprivacy/cleaner/engine.py‎

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -44,13 +44,19 @@ class Cleaner:
4444
_scanner: TextScanner = field(init=False, repr=False)
4545
_resolver: FindingResolver = field(init=False, repr=False)
4646
_redactor: TextRedactor = field(init=False, repr=False)
47+
_placeholder_redactor: TextRedactor = field(init=False, repr=False)
4748

4849
def __post_init__(self) -> None:
4950
"""Build the text pipeline once and validate the rule set eagerly."""
5051
rule_set = RuleSet(self.policy.rules)
5152
object.__setattr__(self, "_scanner", TextScanner(rule_set))
5253
object.__setattr__(self, "_resolver", FindingResolver())
5354
object.__setattr__(self, "_redactor", TextRedactor(rule_set, self.policy.masking))
55+
object.__setattr__(
56+
self,
57+
"_placeholder_redactor",
58+
TextRedactor(rule_set, PlaceholderMaskingStrategy()),
59+
)
5460

5561
# ------------------------------------------------------------------
5662
# Public API
@@ -249,9 +255,7 @@ def _sanitize_location_text(self, text: str) -> str:
249255
resolved = self._resolver.resolve(raw_matches)
250256
if not resolved:
251257
return text
252-
rule_set = RuleSet(self.policy.rules)
253-
placeholder_redactor = TextRedactor(rule_set, PlaceholderMaskingStrategy())
254-
cleaned, _ = placeholder_redactor.redact(text, resolved)
258+
cleaned, _ = self._placeholder_redactor.redact(text, resolved)
255259
return cleaned
256260

257261
def _bounded_findings(

0 commit comments

Comments
 (0)