-
Notifications
You must be signed in to change notification settings - Fork 775
feat: Add package registry to eck #8800
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
🔍 Preview links for changed docs |
jsoriano
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Took a quick look from the side of the team maintaining Package Registry.
It looks great, thanks for adding support for package registry in ECK, this will help many users.
Added some comments, please let us know if you need a more in-depth review from our side.
| AgentImage Image = "elastic-agent/elastic-agent" | ||
| MapsImage Image = "elastic-maps-service/elastic-maps-server" | ||
| LogstashImage Image = "logstash/logstash" | ||
| PackageRegistryImage Image = "package-registry/distribution" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there an image used by default, or setting the image is required?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes the default version is package-registry/distribution:<version> where the user specifies the version. Version is a required field so if they do not specify one it will fail. Something I thought about was adding a epr_type or something like that where the user could specify the different EPR versions we publish. Like 9.1.2, lite-9.1.2, production, and lite. However I think its just as easy to specify the image in the template if you want something other than package-registry/distribution:<version>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While doing a first test I just realized that the default image is around ~5Gb 😅 (I was wondering why my container was not starting 🙃 ). I'm also wondering if it would not make sense to have short flag to select the image "type".
Edit: I just had a Pod that failed to start with the following error on GKE:
Warning Failed 90s kubelet Failed to pull image "docker.elastic.co/package-registry/distribution:9.1.0": failed to pull and unpack image "docker.elastic.co/package-registry/distribution:9.1.0": failed to extract layer sha256:0f0888ef6ac576c67e3a9acf8ec7216533b7f3144aeb14c9b93d0db9469830cd: write /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/889/fs/packages/package-storage/security_detection_engine-9.0.9-beta.1.zip: no space left on device: unknown
I also had disk pressure conditions. I think the image size should be highlighted in the documentation so that K8s nodes can handle it.
Edit 2: This maybe also means that we need to check the disk size on the nodes used for our e2e tests
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jsoriano do you think we should change the default image to be something smaller? I selected this image because it is what we recommend.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, the lite images were added as smaller images for this kind of use cases, but even this image is starting to be too big.
There are vanilla images that don't contain any package. They fail to start if no directory with packages is configured, but they can also be configured in proxy mode, to forward requests for example to the public EPR.
Latest of these images is docker.elastic.co/package-registry/package-registry:v1.31.1.
There is an open issue about allowing to start even if no package is available yet: elastic/package-registry#1351.
More about the proxy mode in https://github.com/elastic/package-registry/?tab=readme-ov-file#proxy-mode.
We also have a WIP to create custom distributions, and smaller images elastic/package-registry#1335.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR adds Elastic Package Registry (EPR) support to ECK, introducing a new CRD for deploying EPR instances and enabling Kibana to reference EPR instances for Fleet package management.
- Adds
ElasticPackageRegistryCRD with controller to manage EPR deployments - Enables Kibana to associate with EPR instances via
packageRegistryReffield - Implements TLS certificate handling and CA mounting for secure communication between Kibana and EPR
Reviewed Changes
Copilot reviewed 60 out of 61 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
pkg/apis/epr/v1alpha1/ |
New API definitions for ElasticPackageRegistry CRD |
pkg/controller/packageregistry/ |
Controller implementation for managing EPR resources |
pkg/controller/association/controller/kibana_epr.go |
Association controller for Kibana-EPR relationships |
pkg/apis/kibana/v1/kibana_types.go |
Adds packageRegistryRef field and EPR association support |
pkg/controller/kibana/ |
Updates Kibana controller to handle EPR associations and CA certificates |
test/e2e/ |
E2E tests for EPR functionality and associations |
Comments suppressed due to low confidence (1)
pkg/controller/kibana/pod_test.go:1
- The comment on line 67 says 'readinessProbe is the readiness probe for the maps container' but this function is in the packageregistry controller and should refer to the package registry container.
// Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
This blocker has been addressed in elastic/package-registry#1451, which is now merged. |
In this PR we are using the Package Registry distribution images. To support UBI there we would also need to update https://github.com/elastic/package-storage-infra/blob/13bf4e9ba03c028b16ed37772cd0d1afaa45af4f/.buildkite/scripts/build_distributions.sh. |
naemono
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My understanding is that we will take ownership of this PR and move it towards a merging state. Review notes are primarily for our own purposes.
| name: registry | ||
| spec: | ||
| version: 9.1.2 | ||
| count: 1 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we note here links to the lite container instead?
pkg/apis/epr/v1alpha1/epr_types.go
Outdated
| EPRContainerName = "package-registry" | ||
| // Kind is inferred from the struct name using reflection in SchemeBuilder.Register() | ||
| // we duplicate it as a constant here for practical purposes. | ||
| Kind = "ElasticPackageRegistry" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
#8905 is in place for tracking of these additions.
| func (m *ElasticPackageRegistry) GetIdentityLabels() map[string]string { | ||
| return map[string]string{ | ||
| commonv1.TypeLabelName: "epr", | ||
| "packageregistry.k8s.elastic.co/name": m.Name, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is the groupVersion epr.k8s.elastic.co, but the label is packageregistry.k8s.elastic.co?
| ) | ||
|
|
||
| const ( | ||
| HTTPPort = 8080 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note to self: Can specifying the spec.http settings cause issues because this is const?
Remove credentials label from secret. review comments. Signed-off-by: Michael Montgomery <[email protected]>
|
Signed-off-by: Michael Montgomery <[email protected]>
Signed-off-by: Michael Montgomery <[email protected]>
Signed-off-by: Michael Montgomery <[email protected]>
|
Definitely seeing some issues testing on openshift, but it doesn't seem like it's ocp specific: |
|
buildkite test this -f p=gke,E2E_TAGS=epr |
Nope, it's ocp specific: 49b1e56. (was missing |
And more fun on ocp: |
Elastic Package Registry (EPR) has been highly requested to be added to ECK.
EPR does not have any references since it does not require a license nor any other application.
The following was implemented for EPR
xpack.fleet.registryUrland set the environment variableNODE_EXTRA_CA_CERTSto the path of EPR's CA which is mountedNODE_EXTRA_CA_CERTSwith a mount the controller will combine the certs appending the EPR's CA to the users specified CAThis was tested with and without setting
NODE_EXTRA_CA_CERTSusing the below manifest