How to Achieve Equal Distribution of Consumers on Partitions in Datahub 0.12.1

Original Slack Thread

Hi Team,

I am observing that a single consumer is getting assigned to multiple partitions, and thus the consumption of messages is slow and also multiple consumers are getting starved.
Is there any configuration to assign consumers with equal distribution?

Consumer: generic-mae-consumer-job-client
Datahub version: 0.12.1
Number of consumer threads: 10
Partitions: 7

Hey there! :wave: Make sure your message includes the following information if relevant, so we can help more effectively!

  1. Which DataHub version are you using? (e.g. 0.12.0)
  2. Please post any relevant error logs on the thread!

Any help would be greatly appreciated.