Operator watching resources outside K8s Cluster

[cviktorov]

Hi Colleagues,

Is it possible to use Java Operator SDK for watching and updating resources from some special API server, which is not K8S client (fabric8). For example, some extension of K8s, but the http client which watches and updates the resources is custom. So, is it possible to change the K8S API client with my own client?

Thank you!

Best regards,
Tsvetislav

[metacosm]

Hi, the mysql-schema sample operator shows how to check external resources that live outside of the cluster and still trigger your primary resource reconciler based on events that happen on non-k8s resources.
I'm not sure if that's what you mean / want, though.

[csviri]

see also here out of the box event sources supporting that: https://javaoperatorsdk.io/docs/documentation/eventing/#perresourcepollingeventsource

[cviktorov]

Thank you for the sample! From it I understand that operator watch for K8S custom resource and it will prompt the operator to create a schema in DB.

My question is can operator watch K8S custom resource but not from K8S, but from external API for example? In my case, I do not have K8s cluster. We have server which is based on K8s cluster, but the API is different.

When I try the example it tries to fetch resources from https://kubernetes.default.svc/apis/mysql.sample.javaoperatorsdk/v1/mysqlschemas?resourceVersion=0
So, I need this fetching to be from my custom API with my custom client. Is it possible?

[metacosm]

There's no reason why you couldn't create an event source that would call to your API using your own client, as long as you can associate whatever you're calling to with a primary resource to trigger your operator. Very often this is done by polling but if your API is able to stream, your event source could react to events and use them to identify the associated primary resource and then trigger the associated reconciler. It's difficult to tell without more details but this generic idea should be doable.

[csviri]

@cviktorov I you mean to replace primary resource (central resource that is passed to the reconciler) that is not possible ATM.

However having a custom resource just to identify your target resource on the other api and have an EventSource for that is almost that good.

You need just a secondary job maybe to create the simple custom resources for each resource of your type.

This is rather workaround but a feasible one.

[cviktorov]

I tried with second job and is working. But I need to duplicate each custom resource from external server in my k8s cluster.

I see that there is an option to create my own KubernetesClient. If create a new class which implements KubernetesClient.java and create the operator with:
Operator operator = new Operator(override -> override.withKubernetesClient(new MyKubernetesClient)).

So, is it possible to add my http client in MyKubernetesClient which will communicate with the external server?

[csviri]

So, is it possible to add my http client in MyKubernetesClient which will communicate with the external server?

You mean a non Kubernetes Cluster? that is not possible

But we might consider this for v6, although probably won't happen.

[cviktorov]

It would be great if it is possible to use custom non Kubernetes client, or to use PerResourcePollingEventSource, but with option reconciling to be trigger without Kubernetes resource, just with external resource.

Can you consider some of these options for next version, please?

[csviri]

Can you consider some of these options for next version, please?

That would be only possible for next major release, what will probably take some time. But event for that it is highly questionable. The project's scope is Kubernetes Operators, which by definition work on top of custom resources, and mainly we realy on some features on the fact that the resource has certain structure, for generation in metedata; how we handling finalizers, etc. Generalizing these would make a significant effort. So we will check if we can support it, but unfortunatelly I cannot promise it will happen.

[cviktorov]

OK, thank you all for the help!