Release 2.0.3a1 - #98
Open
github-actions[bot] wants to merge 58 commits into
Open
Conversation
* fix: thread safety * fix: thread safety
…ecture, and theory (#62) * fix: thread safety * fix: thread safety * docs: add documentation * Apply suggestions from code review Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> --------- Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
…e suite (#67) * feat(test): ovoscope end-to-end tests for PadatiousPipeline Adds an end-to-end test suite for PadatiousPipeline using ovoscope's E2EPipelineHarness. Covers: - inline-samples intent registration via padatious:register_intent - entity registration + slot capture via padatious:register_entity - no-match scenarios and best-of-multiple selection - detach_intent and detach_skill isolation - session blacklisted_intents / blacklisted_skills The harness sets instant_train: true on the plugin config so each register_intent triggers a synchronous train, removing the need to listen for mycroft.skills.trained or sleep. CI job uses the shared OpenVoiceOS/gh-automations reusable workflow with require_padatious: true (workflow auto-installs swig + libfann-dev) and pre_release: true (temporary until ovoscope ships a release containing the E2EPipelineHarness). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix: clear _calc_padatious_intent LRU cache on train/detach `_calc_padatious_intent` is decorated with `@lru_cache(maxsize=3)`. The intent container is mutable and passed by reference, so the cache key is stable even when intents are added or removed. This caused stale matches to be returned after `detach_intent` / `detach_skill` (and after retraining), breaking the ovoscope E2E tests: - `test_no_match_when_no_intents_registered` — setUp calls detach_skill but the cached result from a prior test was served. - `test_detach_skill_removes_all_its_intents` — same root cause. Fix: call `_calc_padatious_intent.cache_clear()` after every call to `train()`, `handle_detach_intent()`, and `handle_detach_skill()`. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: retrain on detach in instant_train mode When `instant_train: true`, every register_intent already triggers a synchronous train. Detach paths were inconsistent: the registered_intents list and the container's internal state were updated, but the model itself kept the detached intents trained, so the next calc_intent still matched them once the lru_cache was evicted. Fix: in handle_detach_intent and handle_detach_skill, call self.train() after detaching when instant_train is enabled. The model now reflects the post-detach intent roster. This is what the new ovoscope e2e suite was hitting in test_no_match_when_no_intents_registered and test_detach_skill_removes_all_its_intents. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * ci+test: pin license-check action; supply skill_id at register time Two fixes for residual CI failures: 1. tests/test_ovoscope_e2e.py — the ovoscope register_padatious_intent helper doesn't pass skill_id, but padatious's register_intent handler defaults missing skill_id to 'anonymous_skill' and stores the registered intent under that key in _skill2intent. When the harness's per-test setUp emits detach_skill('test_skill_padatious'), nothing is matched and the prior test's intent stays trained — the next test then matches "hello" instead of getting complete_intent_failure. Override _register_intent to derive skill_id from the intent name prefix and include it in the bus message. 2. .github/workflows/license_tests.yml — pilosus/action-pip-license-checker v0.5.0 misclassifies modern PEP 639 license metadata as 'Error', and the action fail-list included 'Error'. Bump to v3.1.0 (current classifier) and drop 'Error' from the fail list so transient unrecognised metadata doesn't break CI. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * ci: migrate to pyproject.toml + apply shared gh-automations workflows Drop the legacy setup.py / setup.cfg / requirements.txt / MANIFEST.in in favour of a single pyproject.toml. Append the __version__ assignment to ovos_padatious/version.py so setuptools' attr-based dynamic versioning can read it. Replace the legacy mycroft-style PR workflows with the standard OpenVoiceOS/gh-automations reusable workflows: build-tests.yml coverage.yml license_check.yml lint.yml pip_audit.yml release-preview.yml repo-health.yml opm-check.yml conventional-label.yml publish_stable.yml release_workflow.yml Removed: unit_tests.yml, license_tests.yml, build_tests.yml, install_tests.yml — all replaced by the gh-automations equivalents. opm-check.yml uses plugin_type=pipeline (and drops the buggy nested- quote entry_point input). build-tests.yml uses test_path=tests (this repo's test dir is plural). ovoscope.yml uses install_extras=test now that pyproject.toml declares a [test] optional-dependencies group. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * ci: pass system_deps=swig libfann-dev to every workflow; bound matrix Padatious depends on fann2 (a swig-bound C extension for libfann), so every workflow that installs the package must apt-install swig and libfann-dev before pip-installing. Pass the system_deps input to: - build-tests.yml, coverage.yml, opm-check.yml, license_check.yml, pip_audit.yml Also: - coverage.yml: fix test_path 'test/' -> 'tests/' and install_extras '' -> 'test' to match the actual test directory and the new [test] extras in pyproject.toml. - build-tests.yml: bound the python matrix to 3.10-3.12 (fann2 1.0.7 has no wheel and no working build on Python 3.13+). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * ci+test: fix coverage install_extras, exclude fann2 LGPL, tighten conf_low Three targeted fixes for residual CI failures after the shared-workflows migration: 1. coverage.yml — coverage reusable workflow treats install_extras as raw pip args (unlike build-tests.yml which appends them as bracket extras). Pass '.[test]' explicitly so the package is installed with its test extras. 2. license_check.yml — fann2 is LGPLv2+ (WeakCopyleft), used here as a library dependency only with no modifications. Exclude it from the license check; the existing exclude_packages regex covers it. 3. tests/test_ovoscope_e2e.py — padatious's default conf_low is permissive enough that 'set a timer for five minutes' fuzzy-matched a trained 'hello' intent. Bump conf_low to 0.6 in PLUGIN_CONFIG so the `test_no_match_unrelated_utterance` test reflects a sensible cutoff. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
* feat!: migrate to ovos-spec-tools
Replace local reinventions with ovos-spec-tools equivalents:
- bracket_expansion.SentenceTreeParser/Fragment/Word/Sentence/Options
become deprecation shims around ovos_spec_tools.expand
- ovos_utils.bracket_expansion.expand_template -> ovos_spec_tools.expand
in util.py and opm.py
- ovos_utils.lang.standardize_lang_tag -> ovos_spec_tools.standardize_lang
- langcodes.closest_match + score<10 -> ovos_spec_tools.closest_lang
in PadatiousPipeline._get_closest_lang and Stemmer
Lang policy switches to the full normalized BCP-47 tag everywhere;
the previous macro bucketing is dropped. Containers are keyed by the
full tag and reconciled at match time via closest_lang. This is a
behaviour change for callers that relied on macro-tag bucketing.
Public symbols kept as deprecation shims with removal in 2.0.0.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: note ovos-spec-tools delegation for expansion and language matching (OVOS-INTENT-1)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: preserve slot/vocab braces in utterance normalization (OVOS-INTENT-1)
normalize_utterances stripped trailing punctuation with str.rstrip(string.punctuation),
which removed the closing brace of a template ending in a slot ({name}) — the
strict ovos_spec_tools.expand then raised MalformedTemplate on re-expansion of the
corrupted '...{name' template. Exclude the slot/vocabulary metacharacters {} <> from
the trailing-punctuation strip.
Also make the entity-recognition test spec-compliant: a literal word must separate
two slots (INTENT-1 §3.6), so 'weather for {place} {time}' -> '... {place} at {time}'.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: lowercase-normalize input for matching, preserve original case in sent (OVOS-INTENT-1 §2)
ovos_spec_tools.expand preserves case where the prior expander lowercased, so
matched slot values regressed to original case ({'thing': 'Mycroft'}). Restore
case-insensitive matching: normalize_utterances lowercases samples, and
_calc_padatious_intent matches against utt.lower() while reporting the original
utterance as sent.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(deps): allow ovos-workshop 9.x (widen <9.0.0 -> <10.0.0) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(deps): drop unused ovos-workshop dependency --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…#75) ovos-bus-client 2.x makes Session unhashable (Session.__hash__ is None), so passing it as an @lru_cache argument to _calc_padatious_intent raised TypeError: unhashable type 'Session' on every match attempt — padatious silently matched nothing under bus-client 2.x. Pass the two blacklist sets the matcher reads as frozensets (hashable) instead. cache_clear on register/detach is already present, so this is the only change needed. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…72) * feat: consume OVOS-INTENT-4 template registration (alongside legacy) Add bus handlers on the OVOS-INTENT-4 spec registration topics (ovos.intent.register.template, ovos.entity.register, deregister, skill.deregister, enable, disable) that parse the §6/§7/§8 payloads and register into the same internal padatious container as the legacy padatious:register_intent contract, which is kept unchanged. Padatious is a TEMPLATE engine, so register.template is its primary new topic; keyword registrations are ignored by design (§11). The spec (skill_id, intent_name) pair maps to padatious's internal <skill_id>:<intent_name> name; §6 'blacklist' maps to padatious 'blacklisted_words'. Enable/disable detach and re-register from a retained definition since padatious has no native suppression flag. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: bump ovos-spec-tools floor to 0.11.0a1 * test: add INTENT-4 consumer e2e proving spec-topic register + match Adds tests/end2end/test_intent4_consume_e2e.py: boots a real MiniCroft on the padatious pipeline (ovoscope E2EPipelineHarness, instant_train) and asserts padatious CONSUMES the OVOS-INTENT-4 spec registration topics, not just legacy: - §6 ovos.intent.register.template -> utterance matches (two intents) - legacy padatious:register_intent still matches (back-compat) - §8.2 ovos.intent.deregister / §8.4 ovos.skill.deregister remove the intent - §8.5 ovos.intent.disable suppresses + ovos.intent.enable re-arms - §11 the template engine does NOT match a keyword-topic registration Training is async even under instant_train, so the suite polls the engine's own calc_intent for readiness (_wait_trained / _wait_gone) instead of racing fixed sleeps, and uses one long-lived MiniCroft with per-test detach + unique intent names to stay deterministic. Also fixes a latent crash exposed by the pinned stack: _calc_padatious_intent is @lru_cache'd over its args, but ovos-bus-client>=2.4.0a1 makes Session unhashable -> "unhashable type: 'Session'" at the cache key. Pass the blacklists as hashable frozensets instead (also None-safe). Wires tests/end2end/ into the ovoscope workflow and floors the e2e stack (ovos-bus-client>=2.4.0a1, ovos-spec-tools>=0.16.1a2). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(deps): pin opm prerelease floor so the bus-client 2.x test stack resolves build_tests installs .[test] with pip and allow-prereleases:false. The stable ovos-plugin-manager releases all cap ovos-bus-client<2.0.0, so the ovos-bus-client>=2.4.0a1 test floor was unsatisfiable. Pin the opm prerelease floor (>=2.3.0a1) so pip selects opm 2.x without --pre. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(deps): bump INTENT-4 e2e bus-client floor to >=2.5.1a1 Raise the test-stack ovos-bus-client floor to >=2.5.1a1,<3.0.0. The opm prerelease floor already in the runtime deps keeps .[test] resolvable without --pre (stable opm caps ovos-bus-client<2). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
spec-tools crossed to 1.x; bus-client 2.6.0a1 and the rest of the stack now require ovos-spec-tools>=1.1.0a1. Drop the <1.0.0 cap (keep the floor) so the stack resolves. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…h time (#80) Registration MAY carry optional requires_context / excludes_context declarations (bare-string keys or {key, scope} mappings, default private). They arrive on the INTENT-4 register.template payload and the legacy padatious:register_intent, and are stored per registered intent keyed by the internal <skill_id>:<name>, retained across the disable/enable lifecycle and dropped on deregister. At match time each produced candidate is dropped when the shared gate_satisfied helper (ovos_spec_tools.context) reports its gate is not satisfied against the session's intent_context (owner_id = the intent's skill_id). The helper handles liveness / scope / decay. Intents without a gate are unaffected. Raises the ovos-spec-tools floor to the alpha that publishes the gating helpers. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
register_intent passed blacklisted_words positionally to IntentContainer.add_intent, where the third parameter is reload_cache. The blacklist landed in reload_cache and was silently dropped, so engine-level voc_blacklist/blacklisted_words never took effect. Pass it (and the DomainIntentContainer variant) by keyword so it reaches the per-intent blacklist and blacklisted utterances are dropped at match time. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ist (#82) Fill every declared template slot of a matched intent from a live session.intent_context entry (private <skill_id>:name over shared bare name) whenever the utterance leaves it unresolved, independent of requires_context. requires_context / excludes_context now gate only the presence flags via gate_satisfied, never the fill. Replace any hardcoded pronoun handling with a per-slot value blacklist carried in the registration payload (slot_blacklist, keyed by slot name): a slot the utterance binds to a blacklisted value, matched as a whole-word sequence, is treated as unresolved so the context candidate fills it. Pronouns become a locale/payload resource, not code. Consume the uniform context_slot_candidates(intent_context, slot_names, owner_id) helper and floor-pin ovos-spec-tools>=1.5.0a1. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Annotate the entity, positional-intent and match-data modules with type hints and docstrings. Modules already reworked on dev keep their current form to avoid regressing their behavior. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat: pure numpy neural network backend, drop fann2 dependency Replace the fann2/libfann bindings with a numpy implementation of the same networks (ovos_padatious.fann): fully connected feed-forward nets trained with batch iRPROP- and FANN's exact stepwise sigmoid activations. Models are read and written in the FANN_FLO_2.1 text format, so networks trained by earlier libfann-based releases keep loading unchanged and saved models remain loadable by libfann. This removes the last native dependency: installation no longer needs libfann, swig, or a compiler, the LGPL license exception no longer applies, and a C-library crash can no longer take down the host process. Fixture .net files under tests/fixtures were generated with genuine libfann and pin the loader and forward pass to its numeric behavior. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: match FANN training semantics for reliable convergence Apply FANN's TANH error transform, symmetric-activation error halving, and saturation-clipped activation derivatives during backprop. Without these the optimizer could stall on saturated targets and leave some intents (notably entity-heavy ones) unconverged after the restart loop. Also cache the stepwise-activation interpolation points and slim the single-sample inference path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: replicate FANN's iRPROP- update and seed training from the data Mirror fann_update_weights_irpropm exactly: slopes carry the negative gradient, a zero sign-product still grows the step, a sign flip zeroes the slope before the direction test, and weights clip at 1500. Training convergence now matches libfann's, keeping match confidences inside the range libfann produced. Seed weight initialization from a checksum of the training data plus the retry count, so identical training data always yields identical models while the caller's restart loop still explores fresh initializations. libfann seeded from the clock, which made trained models irreproducible and confidences vary run to run. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Root cause of the session blacklist bypass: this plugin subscribes to both the legacy registration topic (name suffixed `<skill_id>:<file>.intent`) and the OVOS-INTENT-4 spec topic (`<skill_id>:<file>`, suffix-less), so one logical skill intent lands as TWO engine entries. Blacklisting one alias left the other matchable. This plugin owns its own back-compat for the legacy topic, so it now collapses the alias at REGISTRATION time instead of at compare time: - `register_intent` (legacy topic handler) and `handle_register_template` (spec topic handler) both canonicalize the intent name — strip a trailing `.intent` suffix — before indexing/training, so both wire messages land as a single engine entry. - `__detach_intent` canonicalizes too, so detaching by either alias works. - The underlying engine (`training_manager.IntentManager.add`) is made idempotent on re-registration of the same name: it now removes any existing object under that name before adding, so the second arrival replaces instead of stacking a duplicate matchable entry. Container-level `blacklisted_words` bookkeeping was also switched from accumulating (`+=`) to replacing (`=`) for the same reason. - The session blacklist filter (`_calc_padatious_intent`) no longer dealiases `m.name` — engine matches are canonical by construction now. It still canonicalizes the blacklist entries themselves, since old sessions/mycroft.conf may still list an intent by its legacy `.intent`-suffixed id. A one-time-per-entry LOG.warning flags any such legacy blacklist entry as deprecated, naming the offending entry and its canonical replacement, so operators can update their config. Tests: TestSessionBlacklistAlias (compare-time suppression, either alias) kept/updated; new TestRegistrationCollapse + TestSessionBlacklistCanonicalization in tests/test_registration_collapse.py cover the registration-time collapse (single manifest entry, detach-by-legacy-name, deprecation warning). Known gap surfaced by this rework: ovos-core's end2end test_padatious.py::test_padatious_match now fails against a real skill, because ovos-workshop's register_intent_file() binds the skill's dispatch handler only to the legacy `<skill_id>:<file>.intent` topic, not the spec canonical one. Since PIPELINE-1 dispatches on the matched intent's name verbatim, an orchestrator dispatching the now-canonical (suffix-stripped) name no longer reaches that handler. This is an ovos-workshop dispatch- binding gap, not fixable in this plugin; test_intent_blacklist (the regression this PR targets) and test_skill_blacklist remain green. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
…95) ovos-workshop's register_entity_file() names an entity <skill_id>:<basename>_<md5(entity_file)> and puts that name on padatious:register_entity verbatim. Slot lookup (EntityManager.find) builds its key from the matching intent's skill_id plus the RAW slot token from the template, so a hash-suffixed entry is never found and the slot silently degrades to an unconstrained wildcard - every skill using entity files was affected. Collapse the name at registration time, where this plugin owns its own lookup contract, so every emitter vintage is repaired (deployed workshop releases keep emitting the munged name forever). The legacy twin of a workshop >= 9.3 dual-emit now lands on the same canonical name as its ovos.entity.register twin, so also make entity registration exactly-once in the manifest. Also fix EntityManager.remove: entities are stored as <skill_id>:{<entity>} (Entity.wrap_name), but removal built {<skill_id>:<entity>}, which matched nothing. Entities were therefore unremovable, and TrainingManager.add's replace-on-re-register stacked duplicate objects instead of collapsing them.
…ary (#97) Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Human review requested!