Describe the feature or idea you want to propose
both are built on scikit and could be tweaked to be faster without logical changes. After that, parameter changes could yield improvement also. All this is prior to introducing or at least experimentally testing algorithmic changes, good to optimise what we do first for easy wins
Note the regressor and classifier are currently independent and reproduce code, will move that into the base package to mirror pattern with interval based
Describe your proposed solution
Optimisation plan, ranked by promise
-
Replace sklearn PCA with a minimal in-house PCA (fit and transform). Groups are 3 columns wide by default; a mean + 3×3 covariance + np.linalg.eigh, storing just mean_ and
components_ arrays, measures 8× faster per group than sklearn's PCA.fit + transform (0.052 ms vs 0.426 ms). This removes the PCA fit cost, all transform-time check_array/validate_data
overhead in both fit and predict (thousands of calls), and lets the odd NaN-retry loop become a rare fallback (eigh on a covariance can't produce the SVD failure mode; keep the
random-padding fallback for degenerate input). Bonus: models shrink a lot on disk — 200 trees × ~270 sklearn PCA objects is heavy for STC/HC2 pickles. Expected: ~1.3–1.5× fit, 3–5×
predict.
-
Skip redundant validation on the base tree: check_input=False and direct construction. When base_estimator is the default tree, pass check_input=False to
tree.fit/predict_proba/predict (X_t is already a finite float32 array we just built), and construct the tree directly instead of going through _clone_estimator —
clone/get_params/inspect.signature alone are ~4% of fit. Guard on the default so user-supplied estimators keep the safe path. Expected: another ~5–10% on fit, more on predict.
-
Vectorise per-group training-data assembly (classifier _fit_estimator). The inner loop grows X_t with repeated np.concatenate per class (O(k²) copying) and fancy-indexes columns
group by group. Precompute per-class row indices once in _fit_rotf, build the row selection first, then slice columns in one operation; similarly gather all groups' columns for the
final transform with a single fancy index instead of ~270 small ones. _fit_estimator self-time was ~5% at 800 features and grows with width. Expected: ~5% on fit, more on wide data.
-
Fix the O(n²) out-of-bag computation in fit_predict(_proba). oob = [n for n in indices if n not in subsample] scans a numpy array per element — quadratic in n_cases. At n=5,000 it's
already ~4s across 200 estimators and grows quadratically (n=20,000 → ~a minute of pure waste). Replace with a boolean mask (mask[subsample] = True; oob = np.flatnonzero(~mask)). Only
affects fit_predict, but that's the path STC/HC2 use for train estimates, where n can be large. Also X_t[idx][subsample] double-indexes; minor.
-
Parallelism hygiene. Items 1–3 remove most GIL-holding Python from the workers, so the existing prefer="threads" will actually scale (tree fit/predict release the GIL). On top: use
joblib batch_size/chunking to cut dispatch overhead for 200 small tasks, and the contract-time loop's "exactly n_jobs tasks per round" pattern could dispatch larger rounds. No
Describe alternatives you've considered, if relevant
No response
Additional context
No response
Describe the feature or idea you want to propose
both are built on scikit and could be tweaked to be faster without logical changes. After that, parameter changes could yield improvement also. All this is prior to introducing or at least experimentally testing algorithmic changes, good to optimise what we do first for easy wins
Note the regressor and classifier are currently independent and reproduce code, will move that into the base package to mirror pattern with interval based
Describe your proposed solution
Optimisation plan, ranked by promise
Replace sklearn PCA with a minimal in-house PCA (fit and transform). Groups are 3 columns wide by default; a mean + 3×3 covariance + np.linalg.eigh, storing just mean_ and
components_ arrays, measures 8× faster per group than sklearn's PCA.fit + transform (0.052 ms vs 0.426 ms). This removes the PCA fit cost, all transform-time check_array/validate_data
overhead in both fit and predict (thousands of calls), and lets the odd NaN-retry loop become a rare fallback (eigh on a covariance can't produce the SVD failure mode; keep the
random-padding fallback for degenerate input). Bonus: models shrink a lot on disk — 200 trees × ~270 sklearn PCA objects is heavy for STC/HC2 pickles. Expected: ~1.3–1.5× fit, 3–5×
predict.
Skip redundant validation on the base tree: check_input=False and direct construction. When base_estimator is the default tree, pass check_input=False to
tree.fit/predict_proba/predict (X_t is already a finite float32 array we just built), and construct the tree directly instead of going through _clone_estimator —
clone/get_params/inspect.signature alone are ~4% of fit. Guard on the default so user-supplied estimators keep the safe path. Expected: another ~5–10% on fit, more on predict.
Vectorise per-group training-data assembly (classifier _fit_estimator). The inner loop grows X_t with repeated np.concatenate per class (O(k²) copying) and fancy-indexes columns
group by group. Precompute per-class row indices once in _fit_rotf, build the row selection first, then slice columns in one operation; similarly gather all groups' columns for the
final transform with a single fancy index instead of ~270 small ones. _fit_estimator self-time was ~5% at 800 features and grows with width. Expected: ~5% on fit, more on wide data.
Fix the O(n²) out-of-bag computation in fit_predict(_proba). oob = [n for n in indices if n not in subsample] scans a numpy array per element — quadratic in n_cases. At n=5,000 it's
already ~4s across 200 estimators and grows quadratically (n=20,000 → ~a minute of pure waste). Replace with a boolean mask (mask[subsample] = True; oob = np.flatnonzero(~mask)). Only
affects fit_predict, but that's the path STC/HC2 use for train estimates, where n can be large. Also X_t[idx][subsample] double-indexes; minor.
Parallelism hygiene. Items 1–3 remove most GIL-holding Python from the workers, so the existing prefer="threads" will actually scale (tree fit/predict release the GIL). On top: use
joblib batch_size/chunking to cut dispatch overhead for 200 small tasks, and the contract-time loop's "exactly n_jobs tasks per round" pattern could dispatch larger rounds. No
Describe alternatives you've considered, if relevant
No response
Additional context
No response