Representative Docs vs. Random Sample for Cluster Label

[annarrzz]

Hi BERTopic Team, I encountered a discrepancy when generating cluster labels using LLMs, depending on the quotes provided as examples. For context, I'm analyzing some user feedback about our app.

• When I use the top 10 Representative Docs, the generated label is along the lines of "Returning to the app"
• When I use 300 randomly sampled quotes (sample size determined by power calculation), the label is quite different, e.g. "Leaving the app"

Upon reviewing the 300 randomly selected quotes, I noticed that "Returning to the app (after technical issues were resolved)" is indeed one of the most "coherent" sub-theme as it constituted one of the highest %s of quotes. However, that was still only 20% of the quotes. The remaining quotes cover a long tail of topics, including various technical issues and users mentioning taking a break from the app.

My questions:

1. Are Representative Docs selected to maximize theme "coherence", or do they represent the most frequent theme within the cluster? (read the wiki on Representative Docs but would appreciate a more intuitive understanding)

2. Any advice on which method to choose, such that I generate label that best reflects the overall distribution of themes within a cluster?

Thanks for your help and for developing such a great tool!

[MaartenGr]

Are Representative Docs selected to maximize theme "coherence", or do they represent the most frequent theme within the cluster? (read the wiki on Representative Docs but would appreciate a more intuitive understanding)

They are extracted based on their c-TF-IDF similarity to the topic's c-TF-IDF values. Then, they can be further reranked using MMR to select the most appropriate ones (for example to remove duplicates). The code can be found here:

https://github.com/MaartenGr/BERTopic/blob/c6157fe84cda43046448a01ced5389f7f65ef497/bertopic/_bertopic.py#L4226C9-L4226C37

Any advice on which method to choose, such that I generate label that best reflects the overall distribution of themes within a cluster?

I personally had great success with the one that is already in BERTopic. Selecting the top n and reducing it to k.