Skip to content

🎨 Add Reusable Lifespan Contexts for RabbitMQ and Redis in servicelib.fastapi #7547

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Apr 17, 2025

Conversation

pcrespov
Copy link
Member

@pcrespov pcrespov commented Apr 16, 2025

What do these changes do?

Enhances the servicelib.fastapi module by extending its lifespan management functionality using a context manager pattern (on_startup/on_shutdown). This update introduces reusable lifespan contexts for managing RabbitMQ (both connection and RPC clients) and Redis resources (only clients).

All these new functions are reusable and should drastically reduce the boilerplate to create simcore-service. Please check tests to see usage

Related issue/s

How to test

cd packages/service-library
make "install-dev[all]"
pytest -vv tests/fastapi/test_*lifespan*.py

Dev-ops

None

@pcrespov pcrespov self-assigned this Apr 16, 2025
@pcrespov pcrespov added this to the Pauwel Kwak milestone Apr 16, 2025
@pcrespov pcrespov added the a:services-library issues on packages/service-libs label Apr 16, 2025
Copy link

codecov bot commented Apr 16, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 87.40%. Comparing base (d7e9263) to head (3370053).
Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #7547      +/-   ##
==========================================
- Coverage   87.52%   87.40%   -0.12%     
==========================================
  Files        1745     1412     -333     
  Lines       67554    57856    -9698     
  Branches     1144      638     -506     
==========================================
- Hits        59125    50571    -8554     
+ Misses       8108     7089    -1019     
+ Partials      321      196     -125     
Flag Coverage Δ
integrationtests 65.17% <ø> (-0.09%) ⬇️
unittests 86.45% <100.00%> (-0.26%) ⬇️
Components Coverage Δ
api ∅ <ø> (∅)
pkg_aws_library ∅ <ø> (∅)
pkg_dask_task_models_library ∅ <ø> (∅)
pkg_models_library ∅ <ø> (∅)
pkg_notifications_library ∅ <ø> (∅)
pkg_postgres_database ∅ <ø> (∅)
pkg_service_integration ∅ <ø> (∅)
pkg_service_library 72.96% <100.00%> (+0.43%) ⬆️
pkg_settings_library 90.84% <100.00%> (+0.05%) ⬆️
pkg_simcore_sdk 85.40% <ø> (ø)
agent 96.46% <ø> (ø)
api_server 91.23% <ø> (ø)
autoscaling 96.08% <ø> (ø)
catalog 92.52% <ø> (ø)
clusters_keeper 99.24% <ø> (ø)
dask_sidecar 91.29% <ø> (ø)
datcore_adapter 98.12% <ø> (ø)
director 76.78% <ø> (ø)
director_v2 91.27% <ø> (-0.03%) ⬇️
dynamic_scheduler 97.40% <ø> (ø)
dynamic_sidecar 90.11% <ø> (ø)
efs_guardian 89.79% <ø> (ø)
invitations 93.28% <ø> (ø)
payments 92.66% <ø> (ø)
resource_usage_tracker 89.23% <ø> (+0.05%) ⬆️
storage 87.91% <ø> (+0.25%) ⬆️
webclient ∅ <ø> (∅)
webserver 85.97% <ø> (+<0.01%) ⬆️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d7e9263...3370053. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@pcrespov pcrespov force-pushed the mai/extends-lifespan branch from b2290e9 to 035f475 Compare April 16, 2025 19:29
@pcrespov pcrespov changed the title WIP: Mai/extends lifespan 🎨 Extends lifespan functionality in servicelib.fastapi Apr 16, 2025
@pcrespov pcrespov added the t:maintenance Some planned maintenance work label Apr 16, 2025
@pcrespov pcrespov marked this pull request as ready for review April 16, 2025 22:57
@pcrespov pcrespov requested a review from sanderegg as a code owner April 16, 2025 22:57
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR extends the lifespan functionality within servicelib.fastapi to support redis, rabbitmq, and postgres by introducing new lifespan contexts and updating associated utilities and tests.

  • Added model configuration examples in the settings modules for redis and rabbitmq.
  • Introduced new lifespan functions for redis, rabbitmq, and postgres with enhanced error handling and state management.
  • Updated lifespan utilities and tests to support stricter lifespan state tracking.

Reviewed Changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated no comments.

Show a summary per file
File Description
packages/settings-library/src/settings_library/redis.py Added examples via model_config for redis connection settings.
packages/settings-library/src/settings_library/rabbit.py Added examples via model_config for rabbitmq connection settings.
packages/service-library/tests/fastapi/test_redis_lifespan.py Added tests to verify the redis lifespan behavior and error handling.
packages/service-library/tests/fastapi/test_rabbitmq_lifespan.py Added tests for rabbitmq connectivity and RPC client/server lifespans.
packages/service-library/tests/fastapi/test_lifespan_utils.py Updated tests for lifespan utility functions and error message changes.
packages/service-library/src/servicelib/rabbitmq/_client_rpc.py Introduced rabbitmq_rpc_client_context for managing RPC client context.
packages/service-library/src/servicelib/rabbitmq/init.py Exported the new rabbitmq_rpc_client_context for external usage.
packages/service-library/src/servicelib/fastapi/redis_lifespan.py Implemented redis lifespan context with proper client shutdown logic.
packages/service-library/src/servicelib/fastapi/rabbitmq_lifespan.py Implemented rabbitmq connectivity lifespan with error handling.
packages/service-library/src/servicelib/fastapi/postgres_lifespan.py Enhanced postgres lifespan to integrate lifespan_context and called_state.
packages/service-library/src/servicelib/fastapi/lifespan_utils.py Updated error message templates and lifespan state management utilities.
Comments suppressed due to low confidence (1)

packages/service-library/src/servicelib/fastapi/lifespan_utils.py:42

  • [nitpick] The function name 'mark_lifespace_called' appears to have a typo. Consider renaming it to 'mark_lifespan_called' for consistency with other lifespan-related function names.
def mark_lifespace_called(state: State, lifespan_name: str) -> State:

@pcrespov pcrespov changed the title 🎨 Extends lifespan functionality in servicelib.fastapi 🎨 Add Reusable Lifespan Contexts for RabbitMQ and Redis in servicelib.fastapi Apr 16, 2025
@pcrespov pcrespov enabled auto-merge (squash) April 16, 2025 23:13
Copy link
Contributor

@bisgaard-itis bisgaard-itis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very cool! Thanks a lot. Left a few suggestions

@pcrespov pcrespov force-pushed the mai/extends-lifespan branch from 1559ef7 to f7f3b24 Compare April 17, 2025 12:45
@pcrespov pcrespov disabled auto-merge April 17, 2025 14:58
@pcrespov pcrespov merged commit ed1c032 into ITISFoundation:master Apr 17, 2025
3 of 4 checks passed
@pcrespov pcrespov deleted the mai/extends-lifespan branch April 17, 2025 14:59
Copy link

@matusdrobuliak66 matusdrobuliak66 mentioned this pull request May 8, 2025
34 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a:services-library issues on packages/service-libs t:maintenance Some planned maintenance work
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants