An example which shows how to integrate Camel with Kafka offsetRepository to use in order to locally store the offset of each partition of the topic
Start the kafka broker:
docker run -d -e ALLOW_PLAINTEXT_LISTENER=yes -e KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://localhost:9092 -e KAFKA_CFG_AUTO_CREATE_TOPICS_ENABLE=true -p 9092:9092 bitnami/kafka:latest
You can run this example using
mvn spring-boot:run
You can then examine the newly created file offsetStateRepo.dat
to see the topic offset:
offset-repo-example/0=0 offset-repo-example/0=1 offset-repo-example/0=2 offset-repo-example/0=3 offset-repo-example/0=4 offset-repo-example/0=5
First, start with creating a new OpenShift project:
oc new-project csb-example-kafka-offset
Deploy Kafka using OperatorHub:
oc process -f src/main/resources/openshift/kafka-template.yml -p STARTING_CSV=$(oc get packagemanifests strimzi-kafka-operator -o jsonpath='{.status.channels[?(@.name=="stable")].currentCSV}') | oc create -f -
The application is deployed using the openshift-maven-plugin
that takes care of creating all the necessary OpenShift resources.
Simply use the following command to deploy the application:
mvn clean package -Popenshift
and wait until application pod reaches the Ready
state. You can then access the offsetStateRepo.dat
file for example with:
oc exec dc/camel-example-spring-boot-kafka-offsetrepository -- cat /deployments/offsetStateRepo.dat
If you hit any problem using Camel or have some feedback, then please let us know.
We also love contributors, so get involved :-)
The Camel riders!