Skip to content

Automate alt text generation #679

@maxitg

Description

@maxitg

The problem

A lot of documentation contains images of outputs in case the outputs are graphs, hypergraphs, plots, etc. However, the images also need alt text. Currently, the alt text needs to be written manually (e.g., #673), which is error prone and takes time.

Possible solution

We could try to generate alt text automatically. This has to depend on the head of the output and maybe needs additional hints beyond the head (e.g., it might be necessary to know that a Graph is actually a token-event graph). Further, this can never be perfect as some parts of the output will often need to be omitted and potentially described with words instead (e.g., if a graph has 10k edges). However, we can still generate the starting point automatically. This likely needs to be implemented in Rasterization.m which already generates code for including images.

Alternative solutions

We can also try to just use LLMs to generate alt text, however, it doesn't look like LLMs (at least OpenAI o1 and 4o) are any good at understanding SetReplace at this point.

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureNew functionality, or change in existing functionalityinfrastructureHas to do with changes to the development process, e.g., build scripts, CI, testing utilitieswolfram languageRequires Wolfram Language implementation

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions