-
Notifications
You must be signed in to change notification settings - Fork 63
Add multicloud federated learning parttern #570
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from 13 commits
5aa4742
d9c2d35
2a0966e
976f151
144cd7b
385d606
904653f
0a75f38
82b4a1b
e96714c
f86ef9c
744d3a8
e40cadf
ac50e61
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,79 @@ | ||||||||||
| --- | ||||||||||
| title: Multicloud Federated Learning | ||||||||||
| date: 2025-05-23 | ||||||||||
| summary: This pattern helps you develop and deploy federated learning applications on an open hybrid cloud via Open Cluster Management. | ||||||||||
| rh_products: | ||||||||||
| - Red Hat Advanced Cluster Management | ||||||||||
| - Red Hat OpenShift Container Platform | ||||||||||
| industries: | ||||||||||
| - General | ||||||||||
| aliases: /multicloud-federated-learning/ | ||||||||||
| links: | ||||||||||
| install: getting-started | ||||||||||
| feedback: https://docs.google.com/forms/d/e/1FAIpQLScI76b6tD1WyPu2-d_9CCVDr3Fu5jYERthqLKJDUGwqBg7Vcg/viewform | ||||||||||
| bugs: https://github.com/open-cluster-management-io/addon-contrib/issues | ||||||||||
| ci: multicloudfederatedlearning | ||||||||||
| --- | ||||||||||
|
|
||||||||||
| :toc: | ||||||||||
| :imagesdir: /images | ||||||||||
| :_content-type: ASSEMBLY | ||||||||||
| include::modules/comm-attributes.adoc[] | ||||||||||
|
|
||||||||||
| == Multicloud Federated Learning | ||||||||||
|
|
||||||||||
| === Background | ||||||||||
|
|
||||||||||
| As machine learning (ML) evolves, protecting data privacy becomes increasingly important. Since ML depends on large volumes of data, it is essential to secure that data without disrupting the learning process. | ||||||||||
|
|
||||||||||
| Federated learning addresses this by allowing multiple clusters or organizations to collaboratively train models without sharing sensitive data. Computation happens where the data lives, ensuring privacy, regulatory compliance, and efficiency. | ||||||||||
|
|
||||||||||
| By integrating federated learning with {rh-rhacm-first}, this pattern provides an automated and scalable solution for deploying FL workloads across hybrid and multicluster environments. | ||||||||||
|
|
||||||||||
| ==== Technologies | ||||||||||
| * Open Cluster Management (OCM) | ||||||||||
| - ManagedCluster | ||||||||||
| - ManifestWork | ||||||||||
| - Placement | ||||||||||
| ... | ||||||||||
| * Federated Learning frameworks | ||||||||||
| - Flower | ||||||||||
| - OpenFL | ||||||||||
| ... | ||||||||||
| * Grafana | ||||||||||
| * OpenTelemetry | ||||||||||
|
|
||||||||||
| === Why Use Advanced Cluster Management for Federated Learning? | ||||||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
Suggested change
|
||||||||||
|
|
||||||||||
| **Advanced Cluster Management (ACM)** simplifies and automates the deployment and orchestration of Federated Learning (FL) workloads across clusters: | ||||||||||
|
||||||||||
| **Advanced Cluster Management (ACM)** simplifies and automates the deployment and orchestration of Federated Learning (FL) workloads across clusters: | |
| **{rh-rhacm-first}** simplifies and automates the deployment and orchestration of federated learning workloads across clusters: |
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the bullet got mistakenly deleted from my earlier comment
| **Advanced Cluster Management (ACM)** simplifies and automates the deployment and orchestration of Federated Learning (FL) workloads across clusters: | |
| - **Advanced Cluster Management (ACM)** simplifies and automates the deployment and orchestration of Federated Learning (FL) workloads across clusters: |
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use sentence style capitalization
| - **Automatic Deployment & Simplified Operations**: ACM provides a unified and automated approach to running FL workflows across different runtimes (e.g., Flower, OpenFL). Its controller manages the entire FL lifecycle—including setup, coordination, status tracking, and teardown—across multiple clusters in a multicloud environment. This eliminates repetitive manual configurations, significantly reduces operational overhead, and ensures consistent, scalable FL deployments. | |
| - **Automatic deployment and simplified operations**: ACM provides a unified and automated approach to running FL workflows across different runtimes (e.g., Flower, OpenFL). Its controller manages the entire FL lifecycle—including setup, coordination, status tracking, and teardown—across multiple clusters in a multicloud environment. This eliminates repetitive manual configurations, significantly reduces operational overhead, and ensures consistent, scalable FL deployments. |
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| - **Dynamic Client Selection**: ACM's scheduling capabilities allow FL clients to be selected not only based on where the data resides, but also dynamically based on cluster labels, resource availability, and governance criteria. This enables a more adaptive and intelligent approach to client participation. | |
| - **Dynamic client selection**: {rh-rhacm} scheduling capabilities allow federated learning clients to be selected not only based on where the data resides, but also dynamically based on cluster labels, resource availability, and governance criteria. This enables a more adaptive and intelligent approach to client participation. |
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| Together, these capabilities support a **flexible FL client model**, where clusters can join or exit the training process dynamically, without requiring static or manual configuration. | |
| Together, these capabilities support a flexible federated learning client model, where clusters can join or exit the training process dynamically, without requiring static or manual configuration. |
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| - 🔒 Privacy-preserving training without moving sensitive data | |
| - Privacy-preserving training without moving sensitive data |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please address all the comments about removing these small icons/images
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| - ⚙️ Automated dynamic FL client orchestration across distributed clusters | |
| - Automated dynamic FL client orchestration across distributed clusters |
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| - 🧩 Adaptable to different FL frameworks, such as OpenFL and Flower | |
| - Adaptable to different federated learning frameworks, such as OpenFL and Flower |
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| - 🌍 Scalability across hybrid and edge clusters | |
| - Scalability across hybrid and edge clusters |
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| - 📉 Lower infrastructure and operational costs | |
| - Lower infrastructure and operational costs |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Global comment: Avoid anthropomorphism
Do not assign human characteristics to inanimate objects, which are incapable of human behaviors and emotions. As much as possible, focus technical information on users and their actions, not on a product and its actions. Some examples are
enables,allows,lets,expects,permits
| This approach empowers organizations to build smarter, privacy-first AI solutions with less complexity and more flexibility. | |
| Organizations can use this approach to build smarter, privacy-first AI solutions with less complexity and more flexibility. |
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| - In this architecture, a central **Hub Cluster** acts as the aggregator, running the Federated Learning (FL) controller and scheduling workloads using ACM APIs like `Placement` and `ManifestWork`. | |
| - In this architecture, a central hub cluster acts as the aggregator, running the federated learning controller and scheduling workloads using {rh-rhacm} APIs like `Placement` and `ManifestWork`. |
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use emphasis sparingly so that it does not lose its impact.
| - Multiple **Managed Clusters**, potentially across different clouds, serve as FL clients—each holding private data. These clusters pull the global model from the hub, train it locally, and push model updates back. | |
| - Multiple managed clusters, potentially across different clouds, serve as FL clients—each holding private data. These clusters pull the global model from the hub, train it locally, and push model updates back. |
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| - The controller manages this lifecycle using custom resources and supports runtimes like Flower and OpenFL. This setup enables scalable, multi-cloud model training with **data privacy preserved by design**, requiring no changes to existing FL training code. | |
| - The controller manages this lifecycle using custom resources and supports runtimes like Flower and OpenFL. This setup enables scalable, multi-cloud model training with data privacy preserved by design, requiring no changes to existing federated learning training code. |
| Original file line number | Diff line number | Diff line change | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,233 @@ | ||||||||||||
| --- | ||||||||||||
| title: Getting Started | ||||||||||||
| weight: 10 | ||||||||||||
| aliases: /multicloud-federated-learning/getting-started/ | ||||||||||||
| --- | ||||||||||||
|
|
||||||||||||
| :toc: | ||||||||||||
| :imagesdir: /images | ||||||||||||
| :_content-type: ASSEMBLY | ||||||||||||
| include::modules/comm-attributes.adoc[] | ||||||||||||
|
|
||||||||||||
| == Deploying the Multicloud Federated Learning Pattern | ||||||||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||
|
|
||||||||||||
| === Prerequisites | ||||||||||||
|
|
||||||||||||
| ==== Ensure the following tools are installed: | ||||||||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||
|
|
||||||||||||
| - link:https://kubernetes.io/docs/reference/kubectl/[`kubectl`] | ||||||||||||
| - link:https://kubectl.docs.kubernetes.io/installation/kustomize/[`kustomize`] | ||||||||||||
| - link:https://kind.sigs.k8s.io/[`kind`] (recommended version > v0.9.0) | ||||||||||||
| - link:https://www.gnu.org/software/make/[`make`] (for build automation) | ||||||||||||
|
|
||||||||||||
| ==== Optional (for container image building): | ||||||||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||
|
|
||||||||||||
| - link:https://podman.io/[Podman] or link:https://www.docker.com/[Docker] | ||||||||||||
| - link:https://go.dev/doc/install[Go] (version 1.19 or higher) | ||||||||||||
|
|
||||||||||||
| ===== Advanced Cluster Management Environment | ||||||||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||
|
|
||||||||||||
| Prepare at least three clusters: one hub cluster and two managed clusters. | ||||||||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||
|
|
||||||||||||
| Verify the managed clusters are registered on the hub by running: | ||||||||||||
|
|
||||||||||||
| [source,bash] | ||||||||||||
|
||||||||||||
| [source,bash] | |
| [source,terminal] |
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| [source,bash] | |
| [source,terminal] |
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| [source,bash] | |
| [source,terminal] |
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| [source,bash] | |
| [source,terminal] |
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| [source,bash] | |
| [source,terminal] |
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| [source,bash] | |
| [source,terminal] |
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| [source,bash] | |
| [source,terminal] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| === Deploy the Federated Learning Instance | |
| === Deploy the federated learning instance |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| . Build the Application Image | |
| . Build the application image |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| *Note*: You can skip this step by using the pre-built image `quay.io/myan/flower-app-torch:latest`. | |
| [NOTE] | |
| ==== | |
| You can skip this step by using the pre-built image `quay.io/myan/flower-app-torch:latest`. | |
| ==== |
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| [source,bash] | |
| [source,terminal] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| . Deploy a Federated Learning Instance | |
| . Deploy a federated learning instance |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| In this example, both the server and clients use the same image—either the one built above or the pre-built `quay.io/myan/flower-app-torch:latest`. Once the resource is created, the server is deployed to the hub cluster, and the clients are prepared for deployment to the managed clusters. | |
| In this example, both the server and clients use the same image — either the one built above or the prebuilt `quay.io/myan/flower-app-torch:latest`. After the resource is created, the server is deployed to the hub cluster, and the clients are prepared for deployment to the managed clusters. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| . Schedule the Federated Learning Clients into Managed Clusters | |
| . Schedule the federated learning clients into managed clusters |
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| [source,bash] | |
| [source,terminal] |
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| [source,bash] | |
| [source,terminal] |
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| [source,bash] | |
| [source,terminal] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| .. Download and Verify the Trained Model | |
| .. Download and verify the trained model |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure if we have CI set up for this. Kindly check this with the Validated Patterns engineering team
cc: @day0hero
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@mlabonte-rh @yukinchan - what do we need to do to get this into 'testing' ?