Trying to replicate a visualisation in the documentation

[Quadrophonia]

Hi all

Apologies if this is not the right place for this: I don't think it's a bug - rather, my programming ability!

I have two related issues. First, I'm trying to replicate the plot on this page of the documentation:

I'm specifically interested in being able to control the label font size: I like that the plot is "zoomed out", and the labels are clear. However, when I run both versions of the accompanying code, the resulting plot doesn't resemble the one on the page:

Second, I'm guessing that these labels can be achieved by controlling the font size in datamapplot, as discussed here.

However, when I try and adapt the code to my own case (for example, by passing arguments in datamap_kwds), I get a persistent error "cannot access local variable "font_scale_factor" where it is not associated with a value". I've tried declaring a value for "font_scale_factor" in my datamap_kwds dict, but to no avail. Here is a minimum sample code:

from sklearn.datasets import fetch_20newsgroups
from sentence_transformers import SentenceTransformer
from bertopic import BERTopic
from umap import UMAP

# Prepare embeddings
docs = fetch_20newsgroups(subset='all',  remove=('headers', 'footers', 'quotes'))['data']
sentence_model = SentenceTransformer("all-MiniLM-L6-v2")
embeddings = sentence_model.encode(docs, show_progress_bar=False)

# Train BERTopic
topic_model = BERTopic().fit(docs, embeddings)

datamap_kwds = {"label_font_size" : 11, 
                "label_over_points" : True}

# Run the visualization with the original embeddings
topic_model.visualize_document_datamap(datamap_kwds,
                                        docs, 
                                        embeddings=embeddings, 
                                        width=2000, 
                                        height = 2000 
                                        )

...and here is the full stack trace showing the error:

{
	"name": "UnboundLocalError",
	"message": "cannot access local variable 'font_scale_factor' where it is not associated with a value",
	"stack": "---------------------------------------------------------------------------
UnboundLocalError                         Traceback (most recent call last)
Cell In[1], line 32
     26 datamap_kwds = {\"label_font_size\" : 11, 
     27                 \"label_over_points\" : True}
     31 # Run the visualization with the original embeddings
---> 32 topic_model.visualize_document_datamap(datamap_kwds,
     33                                         docs, 
     34                                         embeddings=embeddings, 
     35                                         width=2000, 
     36                                         height = 2000 
     37                                         )
     44 # Run the visualization with the original embeddings
     45 ##topic_model.visualize_document_datamap(docs, embeddings=embeddings)
     46 
     47 # Or, if you have reduced the original embeddings already:
     48 ##topic_model.visualize_document_datamap(docs, reduced_embeddings=reduced_embeddings)

File c:\\Users\\Main\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\bertopic\\_bertopic.py:2645, in BERTopic.visualize_document_datamap(self, docs, topics, embeddings, reduced_embeddings, custom_labels, title, sub_title, width, height, interactive, enable_search, topic_prefix, datamap_kwds, int_datamap_kwds)
   2569 \"\"\"Visualize documents and their topics in 2D as a static plot for publication using
   2570 DataMapPlot. This works best if there are between 5 and 60 topics. It is therefore best
   2571 to use a sufficiently large `min_topic_size` or set `nr_topics` when building the model.
   (...)   2642 ```
   2643 \"\"\"
   2644 check_is_fitted(self)
-> 2645 return plotting.visualize_document_datamap(
   2646     self,
   2647     docs,
   2648     topics,
   2649     embeddings,
   2650     reduced_embeddings,
   2651     custom_labels,
   2652     title,
   2653     sub_title,
   2654     width,
   2655     height,
   2656     interactive,
   2657     enable_search,
   2658     topic_prefix,
   2659     datamap_kwds,
   2660     int_datamap_kwds,
   2661 )

File c:\\Users\\Main\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\bertopic\\plotting\\_datamap.py:178, in visualize_document_datamap(topic_model, docs, topics, embeddings, reduced_embeddings, custom_labels, title, sub_title, width, height, interactive, enable_search, topic_prefix, datamap_kwds, int_datamap_kwds)
    168     figure = datamapplot.create_interactive_plot(
    169         embeddings_2d,
    170         named_topic_per_doc,
   (...)    175         **int_datamap_kwds,
    176     )
    177 else:
--> 178     figure, _ = datamapplot.create_plot(
    179         embeddings_2d,
    180         named_topic_per_doc,
    181         figsize=(width / 100, height / 100),
    182         dpi=100,
    183         title=title,
    184         sub_title=sub_title,
    185         **datamap_kwds,
    186     )
    188 return figure

File c:\\Users\\Main\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\datamapplot\\config.py:126, in ConfigManager.complete.<locals>.decorator.<locals>.fn_with_config(*args, **kwargs)
    119             raise ConfigError(
    120                 f\"Parameter {param.name} is deliberately listed as \"
    121                 \"forbidden from being defined through the DataMapPlot \"
    122                 \"configuration file.\",
    123                 param
    124             )
    125         from_config[name] = self[name]
--> 126 return fn(*bound_args.args, **(bound_args.kwargs | from_config))

File c:\\Users\\Main\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\datamapplot\\create_plots.py:308, in create_plot(data_map_coords, labels, title, sub_title, noise_label, noise_color, color_label_text, color_label_arrows, label_wrap_width, label_color_map, figsize, dynamic_label_size, dpi, force_matplotlib, darkmode, highlight_labels, palette_hue_shift, palette_hue_radius_dependence, palette_theta_range, palette_min_lightness, use_medoids, cmap, cvd_safer, marker_color_array, **render_plot_kwds)
    305 if \"alpha\" in render_plot_kwds:
    306     alpha = render_plot_kwds.pop(\"alpha\")
--> 308 fig, ax = render_plot(
    309     data_map_coords,
    310     color_list,
    311     label_text,
    312     label_locations,
    313     label_cluster_sizes,
    314     title=title,
    315     sub_title=sub_title,
    316     point_size=point_size,
    317     alpha=alpha,
    318     label_text_colors=None if not color_label_text else label_text_colors,
    319     label_arrow_colors=None if not color_label_arrows else label_arrow_colors,
    320     highlight_colors=[label_color_map[x] for x in unique_non_noise_labels],
    321     figsize=figsize,
    322     noise_color=noise_color,
    323     dynamic_label_size=dynamic_label_size,
    324     dpi=dpi,
    325     force_matplotlib=force_matplotlib,
    326     darkmode=darkmode,
    327     highlight_labels=highlight_labels,
    328     **render_plot_kwds,
    329 )
    331 return fig, ax

File c:\\Users\\Main\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\datamapplot\\config.py:126, in ConfigManager.complete.<locals>.decorator.<locals>.fn_with_config(*args, **kwargs)
    119             raise ConfigError(
    120                 f\"Parameter {param.name} is deliberately listed as \"
    121                 \"forbidden from being defined through the DataMapPlot \"
    122                 \"configuration file.\",
    123                 param
    124             )
    125         from_config[name] = self[name]
--> 126 return fn(*bound_args.args, **(bound_args.kwargs | from_config))

File c:\\Users\\Main\\AppData\\Local\\Programs\\Python\\Python312\\Lib\\site-packages\\datamapplot\\plot_rendering.py:827, in render_plot(data_map_coords, color_list, label_text, label_locations, label_cluster_sizes, title, sub_title, figsize, dynamic_label_size, dynamic_label_size_scaling_factor, font_family, font_weight, label_linespacing, label_font_size, label_text_colors, label_arrow_colors, highlight_colors, point_size, alpha, dpi, label_over_points, label_base_radius, label_margin_factor, min_font_size, max_font_size, min_font_weight, max_font_weight, highlight_labels, highlight_label_keywords, add_glow, noise_color, glow_keywords, darkmode, logo, logo_width, force_matplotlib, label_direction_bias, marker_type, marker_size_array, arrowprops, title_keywords, sub_title_keywords, pylabeladjust_speed, pylabeladjust_max_iterations, pylabeladjust_adjust_by_size, pylabeladjust_margin_percentage, pylabeladjust_radius_scale, label_font_stroke_width, label_font_outline_alpha, ax, verbose)
    812         keyword_args = {
    813             \"color\": \"white\" if darkmode else \"black\",
    814             \"ha\": \"left\",
   (...)    819             **title_keywords,
    820         }
    821     else:
    822         keyword_args = {
    823             \"color\": \"white\" if darkmode else \"black\",
    824             \"ha\": \"left\",
    825             \"va\": \"bottom\",
    826             \"fontweight\": 900,
--> 827             \"fontsize\": int(3.2 * font_scale_factor),
    828             \"fontfamily\": font_family,
    829         }
    830     fig.suptitle(
    831         title, x=0.0, y=sup_title_y_value, transform=ax.transAxes, **keyword_args
    832     )
    834 ax_x_min, ax_x_max = ax.get_xlim()

UnboundLocalError: cannot access local variable 'font_scale_factor' where it is not associated with a value"
}

[MaartenGr]

Could you share the full stack trace? Important information seems to be missing.

I'm not sure, but it might be bug with datamapplot, so depending on the full stack trace it might be helpful to also open up an issue there.

[Quadrophonia]

Thank you for your comment, Maarten! I have (hopefully) uploaded the stack trace to the original post.

[MaartenGr]

Thank you for the update. The stack trace is your error. In your original post it is truncated (those three dots). Clicking on those will expand the stack trace and show the full error. I only need the full error.

[Quadrophonia]

Thanks very much for your patience here, Maarten! I've updated the post with what I think you need.

[MaartenGr]

Hmmm, have you tried doing datamap_kwds=datamap_kwds. I believe there is a specific parameter that you should use for those keyword arguments. Have you checked the docstrings?

[Quadrophonia]

Aha, Maarten you have solved it! Following your comment, I tried this declaration:

topic_model.visualize_document_datamap(docs, 
                                        embeddings=embeddings, 
                                        width=2000, 
                                        height = 2000,
                                        datamap_kwds=datamap_kwds
                                        )

Here, I declared datamap_kwds=datamap_kwds, and moved the datamap_kwds dict to the end of the declaration. This change produced the following plot which, while not the same as in the documentation, is a lot better than what I originally received: