Skip to content

Speed up the Solr test suite ~8.5x: 7:27 -> 0:53 - #86

Open
reekitconcept wants to merge 1 commit into
mainfrom
gh-81/solr-test-performance
Open

Speed up the Solr test suite ~8.5x: 7:27 -> 0:53#86
reekitconcept wants to merge 1 commit into
mainfrom
gh-81/solr-test-performance

Conversation

@reekitconcept

Copy link
Copy Markdown
Member

Fixes #81.

The Solr-based test suite became much slower after the unittest → pytest migration. Two verified causes, two fixes — tests/services goes from 7:27 to 0:53 (449 tests, green):

1. Keep the Plone test layers alive for the whole session (7:27 → 4:18)

zope.pytestlayer only preserves layers across test classes for zope.testrunner-style tests; for pytest-style tests its class-scoped layer fixture tore the whole layer stack down after every test class, rebuilding the Plone site many times per run (empirically: 8 layer setups in a two-directory run). An autouse session fixture depending on the generated *_session layer fixtures marks the layers keep-for-whole-session. Per-test isolation is unaffected.

2. Class-scoped content and query setup for the service tests (4:18 → 0:53)

With parametrized tests, a single Solr query followed by N assertions ran as N separate tests — each repeating the full setup (registry, index clear, content creation, commit/reindex) and the same query. The services fixture chain and the per-class _init fixtures move to class scope, on a new functional_class_bracket fixture that runs the layer's testSetUp/testTearDown once per test class (plone.testing layer resources stack LIFO, so per-function brackets still nest inside where used).

Verified with --setup-show: a parametrized class with 4 tests runs portal_with_content (content creation) and _init (the query) exactly once each — previously 4× each. Safety verified by AST scan: no test body mutates content or registry (all mutations live in fixtures), so sharing one ZODB/Solr state per class is sound.

Note: expect a tiny overlap with #85 on the solr_service fixture scope — resolves trivially whichever merges second.

Two fixes for the runtime regression since the unittest to pytest
migration:

1. Keep the Plone test layers alive for the whole session.
   zope.pytestlayer only preserves layers across test classes for
   zope.testrunner style tests; for pytest style tests its
   class-scoped layer fixture tore the whole layer stack down after
   every test class, rebuilding the Plone site many times per run.
   An autouse session fixture depending on the generated *_session
   layer fixtures marks the layers keep-for-whole-session.
   (tests/services: 7:27 -> 4:18)

2. Class-scoped content and query setup for the service tests.
   The services fixture chain (portal, users/contents, content
   creation, API sessions) and the per-class _init fixtures (which
   perform the Solr query) move from function to class scope, on a
   new functional_class_bracket fixture that runs the layer's
   testSetUp/testTearDown once per test class. A parametrized
   'one query, N assertions' class now creates its content and runs
   its query exactly once (verified with --setup-show) instead of
   once per assertion. All mutations live in fixtures (verified by
   AST scan), so sharing one ZODB/Solr state per class is safe.
   (tests/services: 4:18 -> 0:53)

Per-test isolation for tests using the function-scoped fixtures is
unaffected. 449 tests, green.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Solr test suite: runtime regression since the unittest to pytest migration

1 participant