diff --git a/other-examples/collector/otel-kubernetes-infra/README.md b/other-examples/collector/otel-kubernetes-infra/README.md new file mode 100644 index 00000000..3db7bd78 --- /dev/null +++ b/other-examples/collector/otel-kubernetes-infra/README.md @@ -0,0 +1,81 @@ +# Monitor Kubernetes with OpenTelemetry collector and correlate with OpenTelemetry APM services + +This example demonstrates correlation between kubernetes container monitored with the [OpenTelemetry collector](https://opentelemetry.io/docs/collector/) and OpenTelemetry APM services. + + +## Requirements + +* You need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. Docker desktop [includes a standalone Kubernetes server and client](https://docs.docker.com/desktop/kubernetes/) which is useful for local testing. +* [A New Relic account](https://one.newrelic.com/) +* [A New Relic license key](https://docs.newrelic.com/docs/apis/intro-apis/new-relic-api-keys/#license-key) + +## Running the example + + 1. Update the `NEW_RELIC_API_KEY` value in [config.yaml](./k8s/config.yml) to your New Relic license key. + + ```yaml + # ...omitted for brevity + otlphttp: + endpoint: https://otlp.nr-data.net + headers: + api-key: + # New Relic API key to authenticate the export requests. + # docs: https://docs.newrelic.com/docs/apis/intro-apis/new-relic-api-keys/#license-key + ``` +2. Update the `NEW_RELIC_API_KEY` value in [values.yaml](.values.yml) to your New Relic license key. + + ```yaml + # ...omitted for brevity + licenseKey: "" + # New Relic API key to authenticate the export requests. + # + ``` + + 3. Install and Configure nr-k8s-otel-collector: + + https://docs.newrelic.com/docs/kubernetes-pixie/kubernetes-integration/installation/k8s-otel/#install + To get OpenTelemetry up and running in your cluster, follow these steps: + + * Download the [Helm chart values file](https://github.com/newrelic/helm-charts/tree/master/charts/nr-k8s-otel-collector/values.yaml#L20-L24) adapt it to meet your specific requirements. + + * Cluster name and are mandatory. + + * Check the entire list of [configuration parameters](https://github.com/newrelic/helm-charts/tree/master/charts/nr-k8s-otel-collector#values). + + * Install the [Helm chart](https://github.com/newrelic/helm-charts/tree/master/charts/nr-k8s-otel-collector) together with the values file. + + ```shell + helm repo add newrelic https://helm-charts.newrelic.com + helm upgrade nr-k8s-otel-collector newrelic/nr-k8s-otel-collector -f values.yaml -n newrelic --create-namespace --install + ``` + 4. Run the application with the following command. + + ```shell + kubectl create namespace opentelemetry-demo + + kubectl apply -n opentelemetry-demo -f k8s/ + + ``` + + * When finished, cleanup resources with the following command. This is also useful to reset if modifying configuration. + + ```shell + kubectl delete -n opentelemetry-demo -f k8s/ + ``` +## Viewing your data + +To review your kubernetes container data in New Relic, navigate to "New Relic -> All Entities -> Containers". You should see entities named `adservice` as defined in `name` property of the respective services in [deployment.yaml](k8s/deployement.yaml). Click to view the container summary. + +To review your OpenTelemetry APM data in New Relic, navigate to "New Relic -> All Entities -> OpenTelemetry" and You should see an entity named `adservice` as defined in `OTEL_SERVICE_NAME` in `deployment.yaml`. Click to view the OpenTelemetry summary. Click "Service Map" in the left navigation, and notice the relationship to the `adservice` container entity. + +## Additional notes + +This example deploys an instance of the opentelemetry demo [AdService](https://opentelemetry.io/docs/demo/services/ad/), defined in `deployement.yaml` `.services.adservice`. The AdService is instrumented with the [OpenTelemetry Java Agent](https://opentelemetry.io/docs/languages/java/instrumentation/#zero-code-java-agent) and is configured to export data via OTLP to New Relic. + +See [get started with querying](https://docs.newrelic.com/docs/query-your-data/explore-query-data/get-started/introduction-querying-new-relic-data/) for additional details on querying data in New Relic. + + + + + + \ No newline at end of file diff --git a/other-examples/collector/otel-kubernetes-infra/k8s/clusterrole.yml b/other-examples/collector/otel-kubernetes-infra/k8s/clusterrole.yml new file mode 100644 index 00000000..442a588d --- /dev/null +++ b/other-examples/collector/otel-kubernetes-infra/k8s/clusterrole.yml @@ -0,0 +1,21 @@ +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: opentelemetry-demo-otelcol + labels: + app.kubernetes.io/name: otelcol + app.kubernetes.io/instance: opentelemetry-demo + app.kubernetes.io/version: "0.82.0" +rules: + - apiGroups: [""] + resources: ["pods", "events", "namespaces", "nodes", "nodes/metrics", "pods/status", "services", "endpoints"] + verbs: ["get", "watch", "list"] + - apiGroups: [""] + resources: ["nodes/spec", "nodes/stats", "nodes/proxy", "pods/logs" ] + verbs: ["get"] + - apiGroups: ["apps"] + resources: ["replicasets"] + verbs: ["get", "list", "watch"] + - apiGroups: ["extensions"] + resources: ["replicasets"] + verbs: ["get", "list", "watch"] \ No newline at end of file diff --git a/other-examples/collector/otel-kubernetes-infra/k8s/clusterrolebinding.yml b/other-examples/collector/otel-kubernetes-infra/k8s/clusterrolebinding.yml new file mode 100644 index 00000000..1cddc8d8 --- /dev/null +++ b/other-examples/collector/otel-kubernetes-infra/k8s/clusterrolebinding.yml @@ -0,0 +1,16 @@ +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: opentelemetry-demo-otelcol + labels: + app.kubernetes.io/name: otelcol + app.kubernetes.io/instance: opentelemetry-demo + app.kubernetes.io/version: "0.82.0" +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: opentelemetry-demo-otelcol +subjects: +- kind: ServiceAccount + name: default + namespace: opentelemetry-demo \ No newline at end of file diff --git a/other-examples/collector/otel-kubernetes-infra/k8s/config.yaml b/other-examples/collector/otel-kubernetes-infra/k8s/config.yaml new file mode 100644 index 00000000..aa891bf9 --- /dev/null +++ b/other-examples/collector/otel-kubernetes-infra/k8s/config.yaml @@ -0,0 +1,82 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: otel-collector-config + namespace: opentelemetry-demo +data: + otel-collector-config.yaml: | + receivers: + otlp: + protocols: + grpc: + http: + processors: + k8sattributes: + k8sattributes/2: + auth_type: "serviceAccount" + passthrough: false + filter: + # only retrieve pods running on the same node as the collector + node_from_env_var: KUBE_NODE_NAME + extract: + # The attributes provided in 'metadata' will be added to associated resources + metadata: + - k8s.namespace.name + - k8s.deployment.name + - k8s.statefulset.name + - k8s.daemonset.name + - k8s.cronjob.name + - k8s.job.name + - k8s.node.name + - k8s.pod.name + - k8s.pod.uid + - k8s.pod.start_time + - k8s.container.name + labels: + # This label extraction rule takes the value 'app.kubernetes.io/component' label and maps it to the 'app.label.component' attribute which will be added to the associated resources + - tag_name: app.label.component + key: app.kubernetes.io/component + from: pod + pod_association: + - sources: + # This rule associates all resources containing the 'k8s.pod.ip' attribute with the matching pods. If this attribute is not present in the resource, this rule will not be able to find the matching pod. + - from: resource_attribute + name: k8s.pod.ip + - sources: + # This rule associates all resources containing the 'k8s.pod.uid' attribute with the matching pods. If this attribute is not present in the resource, this rule will not be able to find the matching pod. + - from: resource_attribute + name: k8s.pod.uid + - sources: + # This rule will use the IP from the incoming connection from which the resource is received, and find the matching pod, based on the 'pod.status.podIP' of the observed pods + - from: connection + resource: + attributes: + - key: host.id + from_attribute: host.name + action: upsert + # TODO (chris): Upsert only when cluster name not found (resource detection override: true) + - key: k8s.cluster.name + action: upsert + value: "opentelemetry-demo" + - key: k8s.container.name + action: upsert + from_attribute: service.name + + exporters: + logging: + loglevel: debug + otlphttp: + endpoint: "https://otlp.nr-data.net" + headers: + api-key: "" + + service: + pipelines: + traces: + receivers: [otlp] + processors: [k8sattributes,resource] + exporters: [logging, otlphttp] + metrics: + receivers: [otlp] + processors: [k8sattributes,resource] + exporters: [logging, otlphttp] \ No newline at end of file diff --git a/other-examples/collector/otel-kubernetes-infra/k8s/deployment.yaml b/other-examples/collector/otel-kubernetes-infra/k8s/deployment.yaml new file mode 100644 index 00000000..07014637 --- /dev/null +++ b/other-examples/collector/otel-kubernetes-infra/k8s/deployment.yaml @@ -0,0 +1,76 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: otel-collector + namespace: opentelemetry-demo +spec: + replicas: 1 + selector: + matchLabels: + app: otel-collector + template: + metadata: + labels: + app: otel-collector + spec: + containers: + - name: otel-collector + image: otel/opentelemetry-collector-contrib:0.98.0 + command: + - "/otelcol-contrib" + args: + - "--config=/etc/otel/otel-collector-config.yaml" + volumeMounts: + - name: collector-config + mountPath: /etc/otel/otel-collector-config.yaml + subPath: otel-collector-config.yaml # Using subPath to map single file + volumes: + - name: collector-config + configMap: + name: otel-collector-config + +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: adservice + namespace: opentelemetry-demo +spec: + replicas: 1 + selector: + matchLabels: + app: adservice + template: + metadata: + labels: + app: adservice + spec: + containers: + - name: adservice + image: otel/demo:1.10.0-adservice # Use the specific image or the latest available version + ports: + - containerPort: 8080 + env: + - name: OTEL_EXPORTER_OTLP_ENDPOINT + value: "http://otel-collector:4318" + - name: OTEL_SERVICE_NAME + value: "adservice" + - name: AD_SERVICE_PORT + value: "8080" + - name: OTEL_CLUSTER_NAME + value: "opentelemetry-demo" + +--- +apiVersion: v1 +kind: Service +metadata: + name: otel-collector + namespace: opentelemetry-demo +spec: + ports: + - port: 4317 + name: otlp-grpc + - port: 4318 + name: otlp-http + selector: + app: otel-collector \ No newline at end of file diff --git a/other-examples/collector/otel-kubernetes-infra/k8s/sa.yml b/other-examples/collector/otel-kubernetes-infra/k8s/sa.yml new file mode 100644 index 00000000..d0697f3b --- /dev/null +++ b/other-examples/collector/otel-kubernetes-infra/k8s/sa.yml @@ -0,0 +1,8 @@ +apiVersion: v1 +kind: ServiceAccount +metadata: + name: default + labels: + app.kubernetes.io/name: otelcol + app.kubernetes.io/instance: opentelemetry-demo + app.kubernetes.io/version: "0.82.0" \ No newline at end of file diff --git a/other-examples/collector/otel-kubernetes-infra/values.yaml b/other-examples/collector/otel-kubernetes-infra/values.yaml new file mode 100644 index 00000000..4bce6d3a --- /dev/null +++ b/other-examples/collector/otel-kubernetes-infra/values.yaml @@ -0,0 +1,164 @@ +# Default values for nr-k8s-otel-collector. +# This is a YAML-formatted file. +# Declare variables to be passed into your templates. + +kube-state-metrics: + # kube-state-metrics.enabled -- Install the [`kube-state-metrics` chart](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-state-metrics) from the stable helm charts repository. + # This is mandatory if `infrastructure.enabled` is set to `true` and the user does not provide its own instance of KSM version >=1.8 and <=2.0. Note, kube-state-metrics v2+ disables labels/annotations + # metrics by default. You can enable the target labels/annotations metrics to be monitored by using the metricLabelsAllowlist/metricAnnotationsAllowList options described [here](https://github.com/prometheus-community/helm-charts/blob/159cd8e4fb89b8b107dcc100287504bb91bf30e0/charts/kube-state-metrics/values.yaml#L274) in + # your Kubernetes clusters. + enabled: true + # -- Disable prometheus from auto-discovering KSM and potentially scraping duplicated data + prometheusScrape: false + +image: + # -- OTel collector image to be deployed. You can use your own collector as long it accomplish the following requirements mentioned below. + repository: newrelic/nr-otel-collector + # -- The pull policy is defaulted to IfNotPresent, which skips pulling an image if it already exists. If pullPolicy is defined without a specific value, it is also set to Always. + pullPolicy: IfNotPresent + # -- Overrides the image tag whose default is the chart appVersion. + tag: "0.8.3" + +# -- Name of the Kubernetes cluster monitored. Mandatory. Can be configured also with `global.cluster` +cluster: "opentelemetry-demo" +# -- This set this license key to use. Can be configured also with `global.licenseKey` +licenseKey: "" +# -- In case you don't want to have the license key in you values, this allows you to point to a user created secret to get the key from there. Can be configured also with `global.customSecretName` +customSecretName: "" +# -- In case you don't want to have the license key in you values, this allows you to point to which secret key is the license key located. Can be configured also with `global.customSecretLicenseKey` +customSecretLicenseKey: "" + +# -- Additional labels for chart pods +podLabels: {} +# -- Additional labels for chart objects +labels: {} + +# -- Sets pod's priorityClassName. Can be configured also with `global.priorityClassName` +priorityClassName: "" + +# -- Sets pod's dnsConfig. Can be configured also with `global.dnsConfig` +dnsConfig: {} + +# -- If deploying to a GKE autopilot cluster, set to true +# @default -- `false` +gkeAutopilot: false + +daemonset: + # -- Sets daemonset pod node selector. Overrides `nodeSelector` and `global.nodeSelector` + nodeSelector: {} + # -- Sets daemonset pod tolerations. Overrides `tolerations` and `global.tolerations` + tolerations: [] + # -- Sets daemonset pod affinities. Overrides `affinity` and `global.affinity` + affinity: {} + # -- Annotations to be added to the daemonset. + podAnnotations: {} + # -- Sets security context (at pod level) for the daemonset. Overrides `podSecurityContext` and `global.podSecurityContext` + podSecurityContext: {} + # -- Sets security context (at container level) for the daemonset. Overrides `containerSecurityContext` and `global.containerSecurityContext` + containerSecurityContext: + privileged: true + # -- Sets resources for the daemonset. + resources: {} + # -- Sets additional environment variables for the daemonset. + envs: [] + # -- Sets additional environment variable sources for the daemonset. + envsFrom: [] + # -- Settings for daemonset configmap + # @default -- See `values.yaml` + configMap: + # -- OpenTelemetry config for the daemonset. If set, overrides default config and disables configuration parameters for the daemonset. + config: {} + +deployment: + # -- Sets deployment pod node selector. Overrides `nodeSelector` and `global.nodeSelector` + nodeSelector: {} + # -- Sets deployment pod tolerations. Overrides `tolerations` and `global.tolerations` + tolerations: [] + # -- Sets deployment pod affinities. Overrides `affinity` and `global.affinity` + affinity: {} + # -- Annotations to be added to the deployment. + podAnnotations: {} + # -- Sets security context (at pod level) for the deployment. Overrides `podSecurityContext` and `global.podSecurityContext` + podSecurityContext: {} + # -- Sets security context (at container level) for the deployment. Overrides `containerSecurityContext` and `global.containerSecurityContext` + containerSecurityContext: {} + # -- Sets resources for the deployment. + resources: {} + # -- Sets additional environment variables for the deployment. + envs: [] + # -- Sets additional environment variable sources for the deployment. + envsFrom: [] + # -- Settings for deployment configmap + # @default -- See `values.yaml` + configMap: + # -- OpenTelemetry config for the deployment. If set, overrides default config and disables configuration parameters for the deployment. + config: {} + +# -- Sets all pods' node selector. Can be configured also with `global.nodeSelector` +nodeSelector: {} +# -- Sets all pods' tolerations to node taints. Can be configured also with `global.tolerations` +tolerations: [] +# -- Sets all pods' affinities. Can be configured also with `global.affinity` +affinity: {} +# -- Sets all security contexts (at pod level). Can be configured also with `global.securityContext.pod` +podSecurityContext: {} +# -- Sets all security context (at container level). Can be configured also with `global.securityContext.container` +containerSecurityContext: {} + +rbac: + # -- Specifies whether RBAC resources should be created + create: true + +# -- Settings controlling ServiceAccount creation +# @default -- See `values.yaml` +serviceAccount: + # serviceAccount.create -- (bool) Specifies whether a ServiceAccount should be created + # @default -- `true` + create: + # If not set and create is true, a name is generated using the fullname template + name: "" + # Specify any annotations to add to the ServiceAccount + annotations: + +# -- (bool) Sets the debug logs to this integration or all integrations if it is set globally. Can be configured also with `global.verboseLog` +# @default -- `false` +verboseLog: + +# -- (bool) Send the metrics to the staging backend. Requires a valid staging license key. Can be configured also with `global.nrStaging` +# @default -- `false` +nrStaging: + +receivers: + prometheus: + # -- (bool) Specifies whether the `prometheus` receiver is enabled + # @default -- `true` + enabled: true + # -- Sets the scrape interval for the `prometheus` receiver + # @default -- `1m` + scrapeInterval: 1m + k8sEvents: + # -- (bool) Specifies whether the `k8s_events` receiver is enabled + # @default -- `true` + enabled: true + hostmetrics: + # -- (bool) Specifies whether the `hostmetrics` receiver is enabled + # @default -- `true` + enabled: true + # -- Sets the scrape interval for the `hostmetrics` receiver + # @default -- `1m` + scrapeInterval: 1m + kubeletstats: + # -- (bool) Specifies whether the `kubeletstats` receiver is enabled + # @default -- `true` + enabled: true + # -- Sets the scrape interval for the `kubeletstats` receiver + # @default -- `1m` + scrapeInterval: 1m + filelog: + # -- (bool) Specifies whether the `filelog` receiver is enabled + # @default -- `true` + enabled: true + +# -- (bool) Send only the [metrics required](https://github.com/newrelic/helm-charts/tree/master/charts/nr-k8s-otel-collector/docs/metrics-lowDataMode.md) to light up the NR kubernetes UI +# @default -- `true` +lowDataMode: