This is Splunk's fork of the OpenTelemetry Demo, maintained by the Splunk Observability field/demo team as a Kubernetes-only distribution with deep Splunk Observability Cloud-specific instrumentation and demo scenarios, built for field demos and workshops.
This repository contains the OpenTelemetry Astronomy Shop, a microservice-based distributed system intended to illustrate the implementation of OpenTelemetry in a near real-world environment — extended with Splunk Observability Cloud integrations, additional demo services, and fault-injection scenarios built for live demos and hands-on workshops.
| Aspect | Upstream OTel Demo | This fork |
|---|---|---|
| Deployment | Docker Compose or Kubernetes | Kubernetes only — Docker Compose is inherited from upstream, unmaintained, and known-broken (issues #298/#299) |
| Collector | Bundled OpenTelemetry Collector | Splunk OTel Collector Helm chart, swapped in entirely |
| Backends | Jaeger, Prometheus, OpenSearch (generic OTLP fan-out) | Splunk Observability Cloud (APM, IM, RUM, Log Observer) via signalfx/otlp_http/splunk_hec exporters |
| Load generation | Locust-based | Custom Puppeteer-based load generator — chosen for better RUM compatibility and easier scripted fault conditions |
| CI/CD | GitHub Actions targeting Docker Hub images | Splunk-specific build/promote/release pipeline publishing to ghcr.io/splunk/opentelemetry-demo |
None of the Splunk-specific functionality below ships in the upstream demo.
- Splunk backends — signalfx, otlp_http, and splunk_hec (×2, for metrics/traces and logs) exporters
- Database Monitoring (DBMon) — Postgres/MySQL/Oracle/SQL Server auto-discovery, query-sample and top-query collection
- Kafka metrics scraper
- SecureApp log routing — dedicated event pipeline for attack-simulation telemetry
- Cardinality control —
strip_verbose(enforces SignalFx's 36-dimension cap),add_environment - Noise reduction — strips flagd's noisy EventStream/Resolve spans; reclassifies Envoy DC/canceled as not-error
- Kubernetes infra telemetry — kubelet stats, cluster events, syslog/auth_log tagged with
com.splunk.* - Async trace continuity across Kafka — span-link-based propagation into accounting and planning, and trace correlation into fraud-detection (solves the classic broken-trace-across-async-messaging problem)
- Pipeline tuning — signalfx traces disabled where redundant, cluster-receiver memory limits (500Mi/1Gi)
- Splunk RUM — full Real User Monitoring with sourcemap upload for de-minified browser stack traces
- Continuous profiling — CPU + memory, across all supported languages, ~14 services
- SecureApp — dedicated attack load generator + telemetry for security-focused demos
- Custom Puppeteer load generator — replaces upstream's generator for better RUM fidelity and scriptable fault conditions
- Database Monitoring (slow query detection, query-to-trace correlation)
- RUM (panic-mode incident, blue/green deploy comparison)
- Continuous profiling (CPU hotspot investigation)
- Order-validation CPU throttling
- GenAI failures (inaccurate LLM output, rate limiting)
- SecureApp attack simulation
- Payment unreachable / A-B testing
- AWS Lambda (planning service, serverless)
- Hybrid on-prem datacenter (
shop-dc-shim, dual AppDynamics + Splunk instrumentation)
Note: a couple of additional scenarios are in active development and not yet part of a release — a logging-workshop scenario (checkout promo-discount bug) and an improved Database Monitoring "slow query" scenario. They'll be added here once merged.
This distribution deploys exclusively via Kubernetes manifests (no Helm chart for the demo app itself; the collector uses the official Splunk Helm chart). See ARCHITECTURE.md for the full service inventory and DEPLOYMENT.md for cluster/account prerequisites.
If you're forking this Splunk-specific repository, run the setup script after cloning:
./setup-fork.shThis script will:
- Configure your development registry (
dev-repo.yaml) - Prevent accidental production version file commits
- Set up your fork for test builds
See PRODUCTION-WORKFLOW-GUIDE.md for workflow documentation.
This Splunk fork deploys on Kubernetes using released manifests. See HOW-TO-DEPLOY-AND-RUN.md for the short path and DEPLOYMENT.md for the full Splunk Observability Cloud setup.
Note: Docker Compose (
make start/docker-compose*.yml) is inherited from upstream, is not maintained in this fork, and is known-broken on a fresh clone (issues #298/#299). Use the Kubernetes manifests instead.
Upstream (generic, non-Splunk) deployment docs:
- ARCHITECTURE.md — service inventory, deployment topology, telemetry patterns
- DEPLOYMENT.md — Splunk Observability Cloud + collector setup
- HOW-TO-DEPLOY-AND-RUN.md — shortest path to a running demo
- DEVELOPING.md — local dev, fork setup, contributing a service
- PRODUCTION-WORKFLOW-GUIDE.md — release/promotion workflow
For upstream OpenTelemetry Demo documentation not specific to this fork, see the Demo Documentation.
We welcome any vendor to fork the project to demonstrate their services and adding a link below. The community is committed to maintaining the project and keeping it up to date for you.
To get involved with this fork, see CONTRIBUTING and DEVELOPING.md. For the upstream community project, our SIG Calls are every other Wednesday at 8:30 AM PST and anyone is welcome.
- Cyrille Le Clerc, Grafana Labs
- Juliano Costa, Datadog
- Pierre Tessier, Honeycomb
- Roger Coll, Elastic
For more information about the maintainer role, see the community repository.
- Cedric Ziel, Grafana Labs
- Mikko Viitanen, Dynatrace
- Piotr Kiełkowicz, Splunk
- Shenoy Pratik, AWS OpenSearch
For more information about the approver role, see the community repository.
For more information about the emeritus role, see the community repository.