Speed up the Solr test suite ~8.5x: 7:27 -> 0:53 - #86
Open
reekitconcept wants to merge 1 commit into
Open
Conversation
Two fixes for the runtime regression since the unittest to pytest migration: 1. Keep the Plone test layers alive for the whole session. zope.pytestlayer only preserves layers across test classes for zope.testrunner style tests; for pytest style tests its class-scoped layer fixture tore the whole layer stack down after every test class, rebuilding the Plone site many times per run. An autouse session fixture depending on the generated *_session layer fixtures marks the layers keep-for-whole-session. (tests/services: 7:27 -> 4:18) 2. Class-scoped content and query setup for the service tests. The services fixture chain (portal, users/contents, content creation, API sessions) and the per-class _init fixtures (which perform the Solr query) move from function to class scope, on a new functional_class_bracket fixture that runs the layer's testSetUp/testTearDown once per test class. A parametrized 'one query, N assertions' class now creates its content and runs its query exactly once (verified with --setup-show) instead of once per assertion. All mutations live in fixtures (verified by AST scan), so sharing one ZODB/Solr state per class is safe. (tests/services: 4:18 -> 0:53) Per-test isolation for tests using the function-scoped fixtures is unaffected. 449 tests, green.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #81.
The Solr-based test suite became much slower after the unittest → pytest migration. Two verified causes, two fixes —
tests/servicesgoes from 7:27 to 0:53 (449 tests, green):1. Keep the Plone test layers alive for the whole session (7:27 → 4:18)
zope.pytestlayer only preserves layers across test classes for zope.testrunner-style tests; for pytest-style tests its class-scoped layer fixture tore the whole layer stack down after every test class, rebuilding the Plone site many times per run (empirically: 8 layer setups in a two-directory run). An autouse session fixture depending on the generated
*_sessionlayer fixtures marks the layers keep-for-whole-session. Per-test isolation is unaffected.2. Class-scoped content and query setup for the service tests (4:18 → 0:53)
With parametrized tests, a single Solr query followed by N assertions ran as N separate tests — each repeating the full setup (registry, index clear, content creation, commit/reindex) and the same query. The services fixture chain and the per-class
_initfixtures move to class scope, on a newfunctional_class_bracketfixture that runs the layer'stestSetUp/testTearDownonce per test class (plone.testing layer resources stack LIFO, so per-function brackets still nest inside where used).Verified with
--setup-show: a parametrized class with 4 tests runsportal_with_content(content creation) and_init(the query) exactly once each — previously 4× each. Safety verified by AST scan: no test body mutates content or registry (all mutations live in fixtures), so sharing one ZODB/Solr state per class is sound.Note: expect a tiny overlap with #85 on the
solr_servicefixture scope — resolves trivially whichever merges second.