pulsar-4.0.0

Apache Pulsar Helm Chart 4.0.0 Release Notes

Breaking Changes

• Pulsar Proxy Service Configuration Change: Default service type changed from LoadBalancer to ClusterIP for more secure defaults.

  • Review upgrade instructions before updating existing deployments

• Monitoring Stack Replacement: Replaced kube-prometheus-stack with victoria-metrics-k8s-stack

  • Victoria Metrics provides full Prometheus compatibility under Apache 2.0 license
  • Required Action: Follow upgrade instructions when migrating from 3.x.x to 4.0.0

Enhanced Monitoring Experience: Usable Metrics and Grafana Dashboards pre-installed

• Pre-configured and thoroughly tested dashboards now included
• Dashboards maintained in dedicated repository: https://github.com/lhotari/pulsar-grafana-dashboards
• Sourced from various Apache 2.0 licensed projects, then optimized and updated for Pulsar 4.0.0 compatibility
• Resolves long-standing dashboard compatibility issues with recent Pulsar and Bookkeeper
  • Many metrics names have changed since Pulsar 2.10 and previously available dashboards were broken
• While testing, some bugs were faced in Prometheus 3.2.1 were metrics scraping failed without proper errors even with debug log level and that triggered a switch from kube-prometheus-stack to victoriametrics-k8s-stack and VictoriaMetrics.
  • VictoriaMetrics is more efficient than Prometheus and it doesn't have issues parsing Pulsar metrics.

What's Changed

• Fix OOM issue on broker wait-zookeeper-ready initContainer by @got-root in #568
• Increase defaults for ensemble size, write quorum, and ack quorum to 2 by @pdolif in #570
• Fix pulsar-cluster-initialize / pulsar-init rendering with kustomize by @lhotari in #572
• [fix] Fix broker service annotations issue, add annotations support to autorecovery service and make handling consistent by @lhotari in #574
• Upgrade to Pulsar 4.0.3 by @lhotari in #575
• Upgrade to kube-prometheus-stack 69.x including prometheus-operator 0.80.0 defaulting to Prometheus 3.x by @lhotari in #578
• Workaround issue with Prometheus 3.0 and metrics by @lhotari in #577
• Improve kube-prometheus-stack config in values.yaml by adding missing key and some basic comments by @lhotari in #579
• Restore support for dbStorage_rocksDB_* settings defined in bookkeeper.configData by @lhotari in #580
• Upgrade default cert-manager version from 1.12.13 to 1.12.16 by @lhotari in #581
• Replace ">" with "|" to avoid Go Yaml issue go-yaml/yaml#789 by @lhotari in #582
• Revisit solution to configure Bookkeeper RocksDB settings using individual RocksDB config files by @lhotari in #583
• Remove the dependency to pulsarctl when generating JWT tokens by @lhotari in #584
• Use PIP-45 metadata store config to replace deprecated ZK config and make PulsarMetadataBookieDriver configurable in BK by @lhotari in #576
• Use BookKeeper BP-29 metadataServiceUri to configure bookie metadata store, also when using Zookeeper by @lhotari in #585
• Add Broker Cache and Sockets dashboards by @lhotari in #586
• Test with k8s 1.32.2 and upgrade tool versions used in CI by @lhotari in #587
• Change Pulsar Proxy service load balancer type to ClusterIP by @lhotari in #588
• Use bookkeeperMetadataServiceUri in broker and make PulsarMetadataClientDriver configurable by @lhotari in #589
• Remove Prometheus scrape annotations when podmonitors are enabled by @lhotari in #590
• Remove PodSecurityPolicy support which was removed in Kubernetes 1.25 and bump minimum version to 1.25 by @lhotari in #591
• Migrate from kube-prometheus-metrics to victoria-metrics-k8s-stack by @lhotari in #592
• Fix Grafana dashboards for Broker with honorLabels, remove unnecessary *_created metrics and improve docs by @lhotari in #593

New Contributors

• @got-root made their first contribution in #568
• @pdolif made their first contribution in #570

Full Changelog: pulsar-3.9.0...pulsar-4.0.0

pulsar-4.0.0-candidate-3

Apache Pulsar Helm Chart 4.0.0 Release Notes

Breaking Changes

• Pulsar Proxy Service Configuration Change: Default service type changed from LoadBalancer to ClusterIP for more secure defaults.

  • Review upgrade instructions before updating existing deployments

• Monitoring Stack Replacement: Replaced kube-prometheus-stack with victoria-metrics-k8s-stack

  • Victoria Metrics provides full Prometheus compatibility under Apache 2.0 license
  • Required Action: Follow upgrade instructions when migrating from 3.x.x to 4.0.0

Enhanced Monitoring Experience: Usable Metrics and Grafana Dashboards pre-installed

• Pre-configured and thoroughly tested dashboards now included
• Dashboards maintained in dedicated repository: https://github.com/lhotari/pulsar-grafana-dashboards
• Sourced from various Apache 2.0 licensed projects, then optimized and updated for Pulsar 4.0.0 compatibility
• Resolves long-standing dashboard compatibility issues with recent Pulsar and Bookkeeper
  • Many metrics names have changed since Pulsar 2.10 and previously available dashboards were broken
• While testing, some bugs were faced in Prometheus 3.2.1 were metrics scraping failed without proper errors even with debug log level and that triggered a switch from kube-prometheus-stack to victoriametrics-k8s-stack and VictoriaMetrics.
  • VictoriaMetrics is more efficient than Prometheus and it doesn't have issues with Pulsar metrics.

What's Changed

• Fix OOM issue on broker wait-zookeeper-ready initContainer by @got-root in #568
• Increase defaults for ensemble size, write quorum, and ack quorum to 2 by @pdolif in #570
• Fix pulsar-cluster-initialize / pulsar-init rendering with kustomize by @lhotari in #572
• [fix] Fix broker service annotations issue, add annotations support to autorecovery service and make handling consistent by @lhotari in #574
• Upgrade to Pulsar 4.0.3 by @lhotari in #575
• Upgrade to kube-prometheus-stack 69.x including prometheus-operator 0.80.0 defaulting to Prometheus 3.x by @lhotari in #578
• Workaround issue with Prometheus 3.0 and metrics by @lhotari in #577
• Improve kube-prometheus-stack config in values.yaml by adding missing key and some basic comments by @lhotari in #579
• Restore support for dbStorage_rocksDB_* settings defined in bookkeeper.configData by @lhotari in #580
• Upgrade default cert-manager version from 1.12.13 to 1.12.16 by @lhotari in #581
• Replace ">" with "|" to avoid Go Yaml issue go-yaml/yaml#789 by @lhotari in #582
• Revisit solution to configure Bookkeeper RocksDB settings using individual RocksDB config files by @lhotari in #583
• Remove the dependency to pulsarctl when generating JWT tokens by @lhotari in #584
• Use PIP-45 metadata store config to replace deprecated ZK config and make PulsarMetadataBookieDriver configurable in BK by @lhotari in #576
• Use BookKeeper BP-29 metadataServiceUri to configure bookie metadata store, also when using Zookeeper by @lhotari in #585
• Add Broker Cache and Sockets dashboards by @lhotari in #586
• Test with k8s 1.32.2 and upgrade tool versions used in CI by @lhotari in #587
• Change Pulsar Proxy service load balancer type to ClusterIP by @lhotari in #588
• Use bookkeeperMetadataServiceUri in broker and make PulsarMetadataClientDriver configurable by @lhotari in #589
• Remove Prometheus scrape annotations when podmonitors are enabled by @lhotari in #590
• Remove PodSecurityPolicy support which was removed in Kubernetes 1.25 and bump minimum version to 1.25 by @lhotari in #591
• Migrate from kube-prometheus-metrics to victoria-metrics-k8s-stack by @lhotari in #592
• Fix Grafana dashboards for Broker with honorLabels, remove unnecessary *_created metrics and improve docs by @lhotari in #593

New Contributors

• @got-root made their first contribution in #568
• @pdolif made their first contribution in #570

Full Changelog: pulsar-3.9.0...pulsar-4.0.0-candidate-3

pulsar-3.9.0

What's Changed

• Fix helm chart to allow configurable ingress pathType by @rawmind0 in #558
• feat(tls): support ca type issuer and v1alpha* version cert-manager api by @ericsyh in #561
• Use Pulsar 4.0.2 image by default by @lhotari in #563
• Use Pulsar 3.0.9 as previous LTS version in CI by @lhotari in #564

New Contributors

• @rawmind0 made their first contribution in #558
• @ericsyh made their first contribution in #561

Full Changelog: pulsar-3.8.0...pulsar-3.9.0

pulsar-3.9.0-candidate-1

What's Changed

• Fix helm chart to allow configurable ingress pathType by @rawmind0 in #558
• feat(tls): support ca type issuer and v1alpha* version cert-manager api by @ericsyh in #561
• Use Pulsar 4.0.2 image by default by @lhotari in #563
• Use Pulsar 3.0.9 as previous LTS version in CI by @lhotari in #564

New Contributors

• @rawmind0 made their first contribution in #558
• @ericsyh made their first contribution in #561

Full Changelog: pulsar-3.8.0...pulsar-3.9.0-candidate-1

pulsar-3.8.0

What's Changed

• feat: add loadBalancerClass for proxy and pulsar-manager by @lenglet-k in #546
• Correct pulsar proxy prometheus.io/port annotation by @doug-ba in #548
• Add support for admin port on ZooKeeper by @liamgib in #550
• Add support for using Oxia as the metadata store for Pulsar and BookKeeper by @yuweisung in #544
• Fix invalid internal server name in Oxia config by @lhotari in #552
• Fix Oxia config so that it includes a list of all pods in the statefulset by @lhotari in #553
• Revert "Wrap Zookeeper probe script with timeout command (#214)" by @lhotari in #556
• Upgrade to Pulsar 4.0.1 image by @lhotari in #557

New Contributors

• @liamgib made their first contribution in #550

Full Changelog: pulsar-3.7.0...pulsar-3.8.0

pulsar-3.8.0-candidate-1

What's Changed

• feat: add loadBalancerClass for proxy and pulsar-manager by @lenglet-k in #546
• Correct pulsar proxy prometheus.io/port annotation by @doug-ba in #548
• Add support for admin port on ZooKeeper by @liamgib in #550
• Add support for using Oxia as the metadata store for Pulsar and BookKeeper by @yuweisung in #544
• Fix invalid internal server name in Oxia config by @lhotari in #552
• Fix Oxia config so that it includes a list of all pods in the statefulset by @lhotari in #553
• Revert "Wrap Zookeeper probe script with timeout command (#214)" by @lhotari in #556
• Upgrade to Pulsar 4.0.1 image by @lhotari in #557

New Contributors

• @liamgib made their first contribution in #550

Full Changelog: pulsar-3.7.0...pulsar-3.8.0-candidate-1

pulsar-3.7.0

This is the first Apache Pulsar Helm chart release to support Pulsar 4.0.0 release.

Breaking changes

• The kube-prometheus-stack version has been upgraded to 65.x.x in Pulsar Helm Chart version 3.7.0. It is necessary to follow the upgrade instructions when upgrading from Helm Chart version 3.0.0-3.6.0 to 3.7.0 version and above.
• The typo in the configuration key extraEnvs has been fixed (previously misspelled as extreEnvs). Configurations using .Values.broker.extreEnvs or .Values.pulsar.extreEnvs should be updated accordingly (#540).

What's Changed

• feat: add extraVolumes and Mounts for pulsar-manager by @lenglet-k in #535
• Test with 3.3.2 image by @lhotari in #541
• fix: broker extraEnv variable by @lenglet-k in #540
• fix: update role: deployments and statefulsets belong to apiGroups "apps" by @yccharles in #543
• Use Pulsar 4.0.0 image, bump chart version to 3.7.0, kube-prometheus-stack to 65.x by @lhotari in #542

New Contributors

• @yccharles made their first contribution in #543

Full Changelog: pulsar-3.6.0...pulsar-3.7.0

pulsar-3.7.0-candidate-1

Breaking changes

• The kube-prometheus-stack version has been upgraded to 65.x.x in Pulsar Helm Chart version 3.7.0. It is necessary to follow the upgrade instructions when upgrading from Helm Chart version 3.0.0-3.6.0 to 3.7.0 version and above.
• The typo in the configuration key extraEnvs has been fixed (previously misspelled as extreEnvs). Configurations using .Values.broker.extreEnvs or .Values.pulsar.extreEnvs should be updated accordingly (#540).

What's Changed

• feat: add extraVolumes and Mounts for pulsar-manager by @lenglet-k in #535
• Test with 3.3.2 image by @lhotari in #541
• fix: broker extraEnv variable by @lenglet-k in #540
• fix: update role: deployments and statefulsets belong to apiGroups "apps" by @yccharles in #543
• Use Pulsar 4.0.0 image, bump chart version to 3.7.0, kube-prometheus-stack to 65.x by @lhotari in #542

New Contributors

• @yccharles made their first contribution in #543

Full Changelog: pulsar-3.6.0...pulsar-3.7.0-candidate-1

pulsar-3.6.0

It is necessary to follow the upgrade instructions when upgrading from Helm Chart version 3.0.0-3.4.x to 3.5.0 version and above.

What's Changed

• Add initContainers to templates by @CodePrometheus in #516
• Bump minimum k8s version to 1.23.0 by @lhotari in #518
• Upgrade cert-manager to v1.12.13 by @lhotari in #517
• Support using self generated certificates by @duncaan in #523
• feat: Support to customize broker podManagementPolicy from values.yaml by @ludmanl in #525
• Add topologyspreadconstraint to deploy pods in sts cross different az evenly by @wangshu3000 in #526
• Fix compatibility with Pulsar 3.3.x+ docker images where /pulsar isn't writable by @lhotari in #531
• feat: add imagepullsecrets on pulsar-manager-initialize job by @lenglet-k in #533
• Use Pulsar 3.0.7 image by default to address CVE-2024-47561 by @lhotari in #536

New Contributors

• @CodePrometheus made their first contribution in #516
• @duncaan made their first contribution in #523
• @ludmanl made their first contribution in #525

Full Changelog: pulsar-3.5.0...pulsar-3.6.0

pulsar-3.6.0-candidate-1

It is necessary to follow the upgrade instructions when upgrading from Helm Chart version 3.0.0-3.4.x to 3.5.0 version and above.

What's Changed

• Add initContainers to templates by @CodePrometheus in #516
• Bump minimum k8s version to 1.23.0 by @lhotari in #518
• Upgrade cert-manager to v1.12.13 by @lhotari in #517
• Support using self generated certificates by @duncaan in #523
• feat: Support to customize broker podManagementPolicy from values.yaml by @ludmanl in #525
• Add topologyspreadconstraint to deploy pods in sts cross different az evenly by @wangshu3000 in #526
• Fix compatibility with Pulsar 3.3.x+ docker images where /pulsar isn't writable by @lhotari in #531
• feat: add imagepullsecrets on pulsar-manager-initialize job by @lenglet-k in #533
• Use Pulsar 3.0.7 image by default to address CVE-2024-47561 by @lhotari in #536

New Contributors

• @CodePrometheus made their first contribution in #516
• @duncaan made their first contribution in #523
• @ludmanl made their first contribution in #525

Full Changelog: pulsar-3.5.0...pulsar-3.6.0-candidate-1