Skip to content

Support specifying number of partitions for Kafka pubsub component #3110

Description

@olitomlinson

Describe the feature

As discussed with Bernd on discord

It would be highly convenient in dev/test scenarios, if dapr provided a metadata configuration to specify the default number of partitions for a given kafka pubsub component (The default would apply to all Topics created within the component)

The kafka server env var to configure the default partition count is KAFKA_NUM_PARTITIONS so maybe a familiar metadata property name could be numPartitions

apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
  name: kafka-pubsub
spec:
  type: pubsub.kafka
  version: v1
  metadata:
  - name: brokers # Required. Kafka broker connection setting
    value: "kafka:29092"
  - name: consumerGroup
    value: "{namespace}"
  - name: clientID 
    value: "workflow-client"
  - name: numPartitions
  - value: [int, greater than 0] [if not specified, assume default behaviour of sarama client]

Release Note

RELEASE NOTE:

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions