Skip to content

🎨 add Postgres connection to dynamic-scheduler #7600

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 18 commits into
base: master
Choose a base branch
from

Conversation

GitHK
Copy link
Contributor

@GitHK GitHK commented Apr 28, 2025

What do these changes do?

Connects Postgres to dynamic-scheduler and also adds the ProjectNetowrks repository.

Related issue/s

How to test

Dev-ops checklist

@GitHK GitHK added this to the Pauwel Kwak milestone Apr 28, 2025
@GitHK GitHK self-assigned this Apr 28, 2025
Copy link

codecov bot commented Apr 28, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 87.78%. Comparing base (a21d245) to head (3058318).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #7600      +/-   ##
==========================================
+ Coverage   87.73%   87.78%   +0.04%     
==========================================
  Files        1784     1779       -5     
  Lines       68854    68720     -134     
  Branches     1133     1133              
==========================================
- Hits        60411    60327      -84     
+ Misses       8131     8081      -50     
  Partials      312      312              
Flag Coverage Δ
integrationtests 64.97% <ø> (-0.06%) ⬇️
unittests 86.98% <100.00%> (+0.03%) ⬆️
Components Coverage Δ
api ∅ <ø> (∅)
pkg_aws_library 93.92% <ø> (ø)
pkg_dask_task_models_library 97.38% <ø> (ø)
pkg_models_library 92.79% <ø> (ø)
pkg_notifications_library 85.26% <ø> (ø)
pkg_postgres_database 88.41% <ø> (ø)
pkg_service_integration 69.92% <ø> (ø)
pkg_service_library 73.02% <100.00%> (+<0.01%) ⬆️
pkg_settings_library 90.90% <ø> (ø)
pkg_simcore_sdk 85.66% <ø> (-0.06%) ⬇️
agent 96.46% <ø> (ø)
api_server 92.77% <ø> (ø)
autoscaling 96.08% <ø> (ø)
catalog 92.64% <ø> (ø)
clusters_keeper 99.25% <ø> (ø)
dask_sidecar 91.30% <ø> (ø)
datcore_adapter 98.12% <ø> (ø)
director 76.80% <ø> (ø)
director_v2 91.10% <ø> (ø)
dynamic_scheduler 96.76% <100.00%> (-0.65%) ⬇️
dynamic_sidecar 90.15% <ø> (ø)
efs_guardian 89.79% <ø> (ø)
invitations 93.28% <ø> (ø)
payments 92.66% <ø> (ø)
resource_usage_tracker 89.12% <ø> (ø)
storage 87.56% <ø> (+0.07%) ⬆️
webclient ∅ <ø> (∅)
webserver 86.10% <ø> (+0.07%) ⬆️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a21d245...3058318. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@GitHK GitHK marked this pull request as ready for review April 29, 2025 07:50
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR connects Postgres to the dynamic-scheduler service and introduces the ProjectNetworks repository. Key changes include:

  • Injecting Postgres lifespan and connection details in tests and configuration.
  • Adding repository logic for handling project networks.
  • Updating CLI and docker-compose files to support new Postgres settings.

Reviewed Changes

Copilot reviewed 24 out of 28 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
services/dynamic-scheduler/tests/unit/service_tracker/test__api.py Added disable_postgres_lifespan parameter to the fixture
services/dynamic-scheduler/tests/unit/api_rpc/test_api_rpc__services.py Added disable_postgres_lifespan parameter in RPC client fixture
services/dynamic-scheduler/tests/unit/api_rest/test_api_rest__meta.py Renamed test function from test_health to test_meta for clarity
services/dynamic-scheduler/tests/unit/api_rest/conftest.py Added Postgres lifespan disable parameter
services/dynamic-scheduler/tests/unit/api_frontend/*.py Added "postgres" to core services selection lists
services/dynamic-scheduler/tests/unit/api_frontend/conftest.py Added Postgres configuration fixtures for frontend tests
services/dynamic-scheduler/tests/conftest.py Added Postgres service and updated event patching configuration (note duplicate fixture entry)
services/dynamic-scheduler/src/simcore_service_dynamic_scheduler/repository/project_networks.py Introduced repository and error classes for project networks (typo in error class name)
services/dynamic-scheduler/src/simcore_service_dynamic_scheduler/repository/events.py Registered the new repository lifespan manager
services/dynamic-scheduler/src/simcore_service_dynamic_scheduler/core/settings.py Added Postgres settings configuration
services/dynamic-scheduler/src/simcore_service_dynamic_scheduler/core/events.py Included repository lifespan in app startup
services/dynamic-scheduler/src/simcore_service_dynamic_scheduler/cli.py Updated CLI to handle new Postgres configuration
services/docker-compose.yml Added Postgres environment variables to the service configuration
packages/service-library/src/servicelib/logging_utils.py Updated logging calls with a stacklevel parameter
Files not reviewed (4)
  • services/dynamic-scheduler/requirements/_test.in: Language not supported
  • services/dynamic-scheduler/requirements/_test.txt: Language not supported
  • services/dynamic-scheduler/requirements/_tools.txt: Language not supported
  • services/dynamic-scheduler/setup.cfg: Language not supported
Comments suppressed due to low confidence (1)

services/dynamic-scheduler/src/simcore_service_dynamic_scheduler/repository/project_networks.py:14

  • There is a typo in the class name 'BaseProjectNetwroksError'. It should be 'BaseProjectNetworksError' to maintain consistent naming.
class BaseProjectNetwroksError(OsparcErrorMixin, RuntimeError):

Copy link
Member

@sanderegg sanderegg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Please check my comments regarding ProjectNetworks.

Copy link
Collaborator

@matusdrobuliak66 matusdrobuliak66 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks look good 👍 I just started a conversation to discuss project networks and where they should live.

Copy link
Member

@pcrespov pcrespov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thx. I left some suggestions

Andrei Neagu added 2 commits April 30, 2025 07:31
:wq
Merge remote-tracking branch 'upstream/master' into pr-osparc-add-posgres-tables-to-dyamic-scheduler
@GitHK
Copy link
Contributor Author

GitHK commented Apr 30, 2025

@Mergifyio queue

Copy link
Contributor

mergify bot commented Apr 30, 2025

queue

🟠 Waiting for conditions to match

  • any of: [🔀 queue conditions]
    • all of: [📌 queue conditions of queue default]
      • label=🤖-automerge
      • any of: [🛡 GitHub branch protection]
        • check-neutral = system-tests
        • check-skipped = system-tests
        • check-success = system-tests
      • any of: [🛡 GitHub branch protection]
        • check-neutral = unit-tests
        • check-skipped = unit-tests
        • check-success = unit-tests
      • #approved-reviews-by >= 2 [🛡 GitHub branch protection]
      • #approved-reviews-by>=2
      • #changes-requested-reviews-by = 0 [🛡 GitHub branch protection]
      • #changes-requested-reviews-by=0
      • #review-threads-unresolved = 0 [🛡 GitHub branch protection]
      • #review-threads-unresolved=0
      • -conflict
      • -draft
      • base=master
      • branch-protection-review-decision = APPROVED [🛡 GitHub branch protection]
      • label!=🤖-do-not-merge
      • any of: [🛡 GitHub branch protection]
        • check-skipped = deploy to dockerhub
        • check-neutral = deploy to dockerhub
        • check-success = deploy to dockerhub
      • any of: [🛡 GitHub branch protection]
        • check-success = check OAS' are up to date
        • check-neutral = check OAS' are up to date
        • check-skipped = check OAS' are up to date
      • any of: [🛡 GitHub branch protection]
        • check-success = integration-tests
        • check-neutral = integration-tests
        • check-skipped = integration-tests
      • any of: [🛡 GitHub branch protection]
        • check-success = build-test-images (frontend) / build-test-images
        • check-neutral = build-test-images (frontend) / build-test-images
        • check-skipped = build-test-images (frontend) / build-test-images
  • -closed [📌 queue requirement]
  • -conflict [📌 queue requirement]
  • -draft [📌 queue requirement]
  • any of: [📌 queue -> configuration change requirements]
    • -mergify-configuration-changed
    • check-success = Configuration changed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

requirement: add postgres database client to dynamic-scheduler
6 participants