feat!: tag categories

[CyanVoxel]

Replaces "Tag Fields" with Tag Categories.

Instead of tags needing to be added to a Tag field type such as "Meta Tags", "Content Tags", or simply the "Tags" field, tags are now added directly to file entries with no intermediary step. While tag field types offered a method to further organize tags, it was cumbersome, inflexible, and simply not fully fleshed out. Tag Categories offer all of the previous (intentional) functionality while greatly increasing the ease of use and customization.

How it works

Tags now have an additional boolean property that states whether or not a tag is marked as a "category" or not. Tags marked as categories have no functional difference in the backend but are treated specially by frontend views. Category tags take on the appearance similar to the previous tag fields while automatically showing all tags inheriting from that tag underneath its own group. This means that duplicates of tags can appear on entries if they inherit from multiple categories, however this is by design. Any tags not inheriting from a category tag will simply show under a default "Tag" field-like section.

The previous built-in tags, "Favorite" and "Archived", now inherit from a new built-in "Meta Tags" tag that's marked as a category by default. This means that the appearance of these tags is effectively unchanged from how they're added via the badge buttons - however the organization is now fully automatic, universal across all tag adding sources, and fully customizable by the user.

This does mean that existing user tag field configurations will be changed. Instead of tags being organized into field types on an per-entry basis, tags themselves determine their organizational layout via the new category flag. Any tags (not currently inheriting from either the "Favorite" or "Archived" tags) will be shown under the default "Tags" header upon migrating from v9.4 libraries. If the user wishes to permanently mark some of their own tags as "Meta Tags" they can easily do that by having these tags inherit from the new built-in "Meta Tags" tag. And if they'd rather set some of their own existing tags as categories (i.e. "People", "Places", "Characters") then they are free to do so as well!

Related Changes

A major component of this PR has been a comprehensive refactor of the preview panel. The panel has been broken up in the following ways:

• PreviewPanel has been made a lightweight container widget
• Image/Media Previews have been broken off into PreviewThumb
• File Attributes have been broken off into FileAttributes
• Field Containers have been broken off into FieldContainers
• The "Recent Libraries" panel has been converted into a submenu under "File -> Open Recent"

The following features have not been re-implemented for the following reasons:

• The dedicated "Recent Libraries" panel: I have opted to move this to the File menu to free up valuable screen real estate used by tags and fields in the preview panel, and to convert this feature into a more mainstream implementation. I wouldn't mind if someone re-implemented the panel widget (with the option to disable) if they wished, however I have no interest to refactor this myself.
• Mixed Selection updating and deleting: While adding tags and fields to multiple entries at once is still possible, editing and updating is currently not in this PR. Since these features already have numerous issues with them I've opted to disable them here while saving a more comprehensive rework of these (including addressing [Feature Request]: Making adding tags to many files with different tags less tedious #337) for a future PR.

---

As a bonus with this approach, this PR fixes numerous issues present with field editing and rendering, both documented and undocumented since the SQL switch. Currently this PR is set to close #92, close #249, close #18, close #321, and close #416.

[Computerdores]

While you are already changing the DB, you might want to also fix the tag_subtags table.
(I would rename it to something like tag_inheritance so that subtag isn't in the name anymore and most importantly swap the column names so they are actually correct)

[CyanVoxel]

While you are already changing the DB, you might want to also fix the tag_subtags table. (I would rename it to something like tag_inheritance so that subtag isn't in the name anymore and most importantly swap the column names so they are actually correct)

I would love to do this 👍

[Computerdores]

Had some free time and wanted to get ahead of the curve a little bit as this PR seems to be shaping up to be a big one.
Feel free to ignore for now, the PR is still a draft after all.

[Tishj]

Couldn't resist so I tested it a little, when I have a tag A that is a category, and I have a tag B that has A as parent, it will show up under the A category

Then when I create a tag C that has B as parent, C will not show up under the A category, is that expected behavior?

A
└── B      // shows up under A
    └── C  // does not show up under A

[Computerdores]

Couldn't resist so I tested it a little, when I have a tag A that is a category, and I have a tag B that has A as parent, it will show up under the A category

Then when I create a tag C that has B as parent, C will not show up under the A category, is that expected behavior?

A
└── B      // shows up under A
    └── C  // does not show up under A

I believe Cyan is currently working on fixing that

[Tishj]

Awesome! 👍

[CyanVoxel]

Couldn't resist so I tested it a little, when I have a tag A that is a category, and I have a tag B that has A as parent, it will show up under the A category

Then when I create a tag C that has B as parent, C will not show up under the A category, is that expected behavior?

A
└── B      // shows up under A
    └── C  // does not show up under A

You just happened to catch me as I (hopefully) finally just got this working in the latest commit! 😁

[Tishj]

I checked out the last commit and it works as expected!
But now I'm finding situations where I wouldn't really want this to happen :/ So it's slightly more nuanced than this.

Problematic Case

Lets say I have a library of books, and these books have characteristics, the themes are a category, the setting is a category, the time period as well
And then some authors are tags, because some authors are known for having the same recurring theme or setting in their books

As a result, the Theme, Setting and Time Period categories now show John Doe because John Doe is known for featuring specific themes and all their books feature a specific time period, so they are bundled under John Doe

Might be a bad example, but it's related to bundling of tags under one tag, and the tags that are bundled are part of several categories

I also don't have an immediate answer as to how those should be displayed instead, but I just wanted to point out a problem with displaying sub tags under the categories of their parent tags

Ideas

One easy-ish(?) solution would be to not display tags under a category if the tag itself is a category
Then the author tags could be marked as categories to avoid showing them under the categories of their parent tags

Another idea would be to add another checkbox that disables categorization for a tag, so they only show under the Tags section.

A third idea could be to disassemble the tag, the category would only contain the direct child, letting the further descendants be displayed under Tags
Then the further descendant can explicitly inherit from the grandparent (category) tag to be included under that category (so C from the earlier example would need to add A as parent)

[CyanVoxel]

I checked out the last commit and it works as expected! But now I'm finding situations where I wouldn't really want this to happen :/ So it's slightly more nuanced than this.

Problematic Case

Lets say I have a library of books, and these books have characteristics, the themes are a category, the setting is a category, the time period as well And then some authors are tags, because some authors are known for having the same recurring theme or setting in their books

As a result, the Theme, Setting and Time Period categories now show John Doe because John Doe is known for featuring specific themes and all their books feature a specific time period, so they are bundled under John Doe

Might be a bad example, but it's related to bundling of tags under one tag, and the tags that are bundled are part of several categories

I think this is more so an issue that arises with that approach to tagging rather than the category system. Parent Tags are intended to be used in an inheritance type of relationship with other tags. This works well for books that are certain genres, but not for describing someone like "John Doe" as "Science Fiction" because he writes in that genre often. "John Doe" may have some association with the genre, but he isn't that genre in the same way that a book may be a genre, or how another genre such as "Hard Sci-Fi" may also be and inherit from "Science Fiction". Having genre parent tags on authors themselves would also break as soon as you came across a book by them that no longer fell under that genre - there would no longer be a way to get rid of that genre from the file entry.

Note: For 9.6 I'm planning to introduce a new type of subtag in addition to Parent Tags called "Component Tags" which will use a composition/HAS relationship instead of the inheritance/IS relationship that Parent Tags use. I'm not sure how I want this to work with the tag category system yet, so I'm keeping my eyes out for situations that may help inform this behavior. Upon some playing around I don't think Component/HAS tags on other tags should be allowed to have any categories in their hierarchies be used in tag categories. This will allow you to use these future types of subtags to solve this issue, with the other 9.6 feature being Tag Overrides letting you take care of cases where the author has a book that isn't in the genre they're typically associated with.

One easy-ish(?) solution would be to not display tags under a category if the tag itself is a category Then the author tags could be marked as categories to avoid showing them under the categories of their parent tags

In this case wouldn't you be marking "John Doe" and every other author as categories? That wouldn't be great from a visual perspective with each author displaying under their own self-titled category, and also wouldn't work from an organizational perspective as those authors are not categories themselves but rather entities who fall under one or more categories such as, well, authors

Another idea would be to add another checkbox that disables categorization for a tag, so they only show under the Tags section.

This would work, however I (perhaps stubbornly) believe that if a tag inherits from a tag marked as a category then it should belong under that category. If a case such as the one you described with the genres and authors arise, I would consider that an issue with the tagging approach rather than the category system itself. Same if there was an instance where a separate category didn't make sense for a group of tags - in that case, just don't display that tag as a category and instead rethink the nature of how you would like certain tags to be organized. I'm open to hearing more cases against this if they arise, though. Life is full of more edge cases than I can ever encounter just by my own experiences, after all.

A third idea could be to disassemble the tag, the category would only contain the direct child, letting the further descendants be displayed under Tags Then the further descendant can explicitly inherit from the grandparent (category) tag to be included under that category (so C from the earlier example would need to add A as parent)

This would also unfortunately undermine much of the automatic inheritance provided by the tag category system. For example I have several tags such as "Media Type" for photos vs illustrations which I keep as a child of "Meta Tags". This allows me to have C tags like "Photo", "Video", "Drawing", "3D Render", etc be organized under as many B-like tags as I want or need, while still appearing under the category for A (Meta Tags) without me needing to manually mark each of these C tags.

[Tishj]

Thanks for the detailed reply!

I understand that these issues arise from the way I'm treating these tag hierarchies, I think the described component tags would solve my problem, because essentially I want to bundle an assortment of tags under a new name, so I don't have to assign all of the bundled tags separately and manually to an entry.

Perhaps a more clear example would be something like a Generation category, where the author (John Doe) gets assigned a tag like Millenial/Boomer that inherits from Generation.
As a result John Doe would show up under the Generation category, which is weird

[Tishj]

In this case wouldn't you be marking "John Doe" and every other author as categories? That wouldn't be great from a visual perspective with each author displaying under their own self-titled category

At least in the current state of the branch, I have found that the tag that is marked as the category does not show up under its own category when assigned to an entry, they display under Tags, only the tags that inherit from them are shown under the category.
Which is why I proposed that idea, since there isn't a tag that inherits from John Doe, the John Doe category would never appear

[CyanVoxel]

In this case wouldn't you be marking "John Doe" and every other author as categories? That wouldn't be great from a visual perspective with each author displaying under their own self-titled category

At least in the current state of the branch, I have found that the tag that is marked as the category does not show up under its own category when assigned to an entry, they display under Tags, only the tags that inherit from them are shown under the category. Which is why I proposed that idea, since there isn't a tag that inherits from John Doe, the John Doe category would never appear

I see, I just happened to see and fix that bug (88b886c) after your message haha. It slipped past me while I was working on this latest approach to recursive categories, but it should be fixed now where category tags themselves show under their own categories.

[CyanVoxel]

Thanks for the detailed reply!

I understand that these issues arise from the way I'm treating these tag hierarchies, I think the described component tags would solve my problem, because essentially I want to bundle an assortment of tags under a new name, so I don't have to assign all of the bundled tags separately and manually to an entry.

Perhaps a more clear example would be something like a Generation category, where the author (John Doe) gets assigned a tag like Millenial/Boomer that inherits from Generation. As a result John Doe would show up under the Generation category, which is weird

This... might a good example of an issue with the category system. Another one I realized could be having a "Movie" tag and a "Character" tag, with a tag "Shrek" (Character) inheriting from "Shrek" (Movie) and "Character". If "Character" is marked as a category, all is fine. But if you also wanted "Movie" as a category, then "Shrek" (Character) would also show up as a "Movie" because it is a Shrek Movie franchise entity. Not ideal. But that also comes with a couple quirks of the tagging method:

1. In that library, we're using the Character tags such as Shrek to contain movie information, not movie tags on entries themselves. We don't need to add tags for the Shrek movie since Shrek the character tag holds that inheritance relationship.
   Although, this falls apart if you have other types of entries in your library that don't use characters and need to inherit straight from "Movie", while making that a category - which I think is a valid use case that breaks the tag category system as it stands... The same would be true for your "Generation" tag, if you wanted to use that for other non-author types of file entries.

2. While having Shrek (Movie) be a component tag on Shrek (Character) would properly prevent the character from showing up under the "Movie" category, it would also break what is intended to be an inheritance relationship with the Movie. If a tag such as, oh I really don't know, "Shrek Backpack" included the Shrek (Character) tag, it would not show up in searches for Shrek (Movie) due to component tags not being planned to be passed down like inherited tags (i.e. Shrek (Character) can not substitute for its component tag, Shrek (Movie); Only for its parent tags like "Character").
   A user-side fix to this would be to add the Shrek (Movie) tag to every entity that's included in that movie franchise, but I feel that's going against what makes TagStudio easy to use with automatic tag inheritance.
   Another "fix" would be to have a Shrek (Franchise) tag from which both Shrek (Movie) and Shrek (Character) inherit from - which would allow both Movie and Character to properly show their own children. Actually I feel this is the proper way to describe this relationship regardless of tag categories. Buuuuut, the same issue arises as soon as you want "Franchise" to be its own category, too...

I'll continue to give this some thought in the hopes that there's a simple and general solution here either in tagging methods themselves, with 9.6 component tags, or simply with a tweak to how tag categories operate. My worry is that there won't be a way to fix this without doing something much more tailored and involved to category tags, such as having an extra subtag box where users can list which potential child tags this category should not show. That may function but I would consider that a very messy fix that would also have to extend as an option across all types of tags. Of course in the meantime, I'm very open to other proposals or scenarios that may help shed some more light on this.

[CyanVoxel]

@Computerdores I've reworked the migration of edited built-in JSON tags in a way that addresses all of the comments regarding the passes on the built-in range as well as the default behavior of trying to add all built-in tags every time a library is opened. The only thing I haven't done and am struggling with is trying to get JSON tag color overrides for built-in tags applying to the SQL tags. The update_tags() method doesn't do anything specific for this, and in fact just calls add_tags() in the background which is the same thing that the buld_tag panel uses to update colors, so I'm not sure where I'm going wrong here. This is also responsible for the failing test.

For the rest of the comments on library.py I believe those all hinge on the verdict of this: #655 (comment).

completely commenting out some of the tests is a bit iffy, but I think it'll be fine

And as far as this goes, I agree that it's not ideal but I certainly wasn't expecting to have to refactor the item grid selection system to behave more like how it did in v9.4 as a part of this feature (whose purpose was to add a future v9.6 feature in order to avoid bugs stemming from flaws in the how fields are connected to entries in the DB). So I guess I'd consider rewriting all of those tests to be out of the scope of this PR, especially since they'll likely all need to be rewritten again at some point. In my opinion it was just too early for those tests to exist.

[Tishj]

Do thumbnails appear in the thumbnail grid? Does the video still play in the preview panel? Does it let you open the file itself in its default program, as well as in your file explorer? Are you trying to select more than one file at once?

I've just made a reproduction with some stock videos.
With 9.4.2 I make a library, the folder contains 4 videos (.mp4 files)

I've added some tags to them to help simulate my real library, but I doubt that matters?

Environment
- Forest (Environment is it's parent tag)
Season
- Winter (Season is it's parent tag)

I close the old version, reopen with a build from this PR (haven't pulled new changes since our last conversation though)
migrate successfully, the library works correctly, thumbnails and previews are working.
What's even stranger is that I just discovered that when I right click on the preview and "Open file" from there it does work.
But when I right click on an entry in the main panel (search results I guess you'd call that?) and select "Open file" it doesn't work.

I just pulled main and build that, there both methods of "Open file" work
(I went into .TagStudio of the library and deleted the .sqlite file to let it remigrate from json)

In this test I am selecting a single file.
The file opens in the default application when I double click it in the file explorer.
The file opens in the default application when I "Open file" from the preview panel that opens to the right after selecting the entry.
The file does not open in the default application (it does nothing) when I select "Open file" in the menu that opens when you rightclick on an entry in the search result panel. I also believe "show in file explorer" doesn't work from there, but that's going off of memory

Just for completeness I'm going to try to see if this problem also happens on a fresh library with this build
Yes, even with a fresh library I have the same problem (should have checked this earlier..)

"Show in File Explorer" does not work from the rightclick menu on the entry either, but both options work from rightclick on the preview window
@CyanVoxel

[CyanVoxel]

Do thumbnails appear in the thumbnail grid? Does the video still play in the preview panel? Does it let you open the file itself in its default program, as well as in your file explorer? Are you trying to select more than one file at once?

I've just made a reproduction with some stock videos. With 9.4.2 I make a library, the folder contains 4 videos (.mp4 files)

I've added some tags to them to help simulate my real library, but I doubt that matters?

Environment
- Forest (Environment is it's parent tag)
Season
- Winter (Season is it's parent tag)

I close the old version, reopen with a build from this PR (haven't pulled new changes since our last conversation though) migrate successfully, the library works correctly, thumbnails and previews are working. What's even stranger is that I just discovered that when I right click on the preview and "Open file" from there it does work. But when I right click on an entry in the main panel (search results I guess you'd call that?) and select "Open file" it doesn't work.

I just pulled main and build that, there both methods of "Open file" work (I went into .TagStudio of the library and deleted the .sqlite file to let it remigrate from json)

In this test I am selecting a single file. The file opens in the default application when I double click it in the file explorer. The file opens in the default application when I "Open file" from the preview panel that opens to the right after selecting the entry. The file does not open in the default application (it does nothing) when I select "Open file" in the menu that opens when you rightclick on an entry in the search result panel. I also believe "show in file explorer" doesn't work from there, but that's going off of memory

I've just replicated this, however it's not limited to just videos - the "open file" right-click options just no longer work in the thumbnail grid for me. That's definitely an area I touched in this PR, so I'll give it a look. Thank you for your thorough response!
OH actually in the logs I can see what's happening, it's not getting passed the full filepath. Should be quick fix!

2025-01-11 08:26:41 [info     ] Opening file                   path=PosixPath('dice.jxl')
2025-01-11 08:26:41 [error    ] File not found                 path=PosixPath('dice.jxl')

[Computerdores]

Has my approval as all comments have been addressed!

[Computerdores]

I did just notice that the tests are failing, do you know what is up with that?

[CyanVoxel]

I did just notice that the tests are failing, do you know what is up with that?

Mentioned here: #655 (comment)

The only thing I haven't done and am struggling with is trying to get JSON tag color overrides for built-in tags applying to the SQL tags. The update_tags() method doesn't do anything specific for this, and in fact just calls add_tags() in the background which is the same thing that the buld_tag panel uses to update colors, so I'm not sure where I'm going wrong here. This is also responsible for the failing test.

[Computerdores]

I did just notice that the tests are failing, do you know what is up with that?

Mentioned here: #655 (comment)

The only thing I haven't done and am struggling with is trying to get JSON tag color overrides for built-in tags applying to the SQL tags. The update_tags() method doesn't do anything specific for this, and in fact just calls add_tags() in the background which is the same thing that the buld_tag panel uses to update colors, so I'm not sure where I'm going wrong here. This is also responsible for the failing test.

I have narrowed down the problem to this line: https://github.com/TagStudioDev/TagStudio/blob/tag-categories-feat/tagstudio/src/core/library/alchemy/library.py#L897
In the test that fails because of the unique constraint on tags.id because it is trying to insert the modified tag. What I don't understand is why that works when the UI does the same thing

[CyanVoxel]

I did just notice that the tests are failing, do you know what is up with that?

Mentioned here: #655 (comment)

The only thing I haven't done and am struggling with is trying to get JSON tag color overrides for built-in tags applying to the SQL tags. The update_tags() method doesn't do anything specific for this, and in fact just calls add_tags() in the background which is the same thing that the buld_tag panel uses to update colors, so I'm not sure where I'm going wrong here. This is also responsible for the failing test.

I have narrowed down the problem to this line: https://github.com/TagStudioDev/TagStudio/blob/tag-categories-feat/tagstudio/src/core/library/alchemy/library.py#L897 In the test that fails because of the unique constraint on tags.id because it is trying to insert the modified tag. What I don't understand is why that works when the UI does the same thing

I managed to "fix" it by grabbing the built-in tags from the DB, editing the color, and then re-adding/"updating" the modified tags to the DB. Why this works, and why I couldn't do it in one step, I don't know...

# Tags
for tag in json_lib.tags:
    self.add_tag(
        Tag(
            id=tag.id,
            name=tag.name,
            shorthand=tag.shorthand,
            color=TagColor.get_color_from_str(tag.color),
        )
    )
    # Apply user edits to built-in JSON tags.
    if tag.id in range(RESERVED_TAG_START, RESERVED_TAG_END + 1):
        updated_tag = self.get_tag(tag.id)
        updated_tag.color = TagColor.get_color_from_str(tag.color)
        self.update_tag(updated_tag)  # NOTE: This just calls add_tag?

[Computerdores]

I did just notice that the tests are failing, do you know what is up with that?

Mentioned here: #655 (comment)

The only thing I haven't done and am struggling with is trying to get JSON tag color overrides for built-in tags applying to the SQL tags. The update_tags() method doesn't do anything specific for this, and in fact just calls add_tags() in the background which is the same thing that the buld_tag panel uses to update colors, so I'm not sure where I'm going wrong here. This is also responsible for the failing test.

I have narrowed down the problem to this line: https://github.com/TagStudioDev/TagStudio/blob/tag-categories-feat/tagstudio/src/core/library/alchemy/library.py#L897 In the test that fails because of the unique constraint on tags.id because it is trying to insert the modified tag. What I don't understand is why that works when the UI does the same thing

I managed to "fix" it by grabbing the built-in tags from the DB, editing the color, and then re-adding/"updating" the modified tags to the DB. Why this works, and why I couldn't do it in one step, I don't know...

# Tags
for tag in json_lib.tags:
    self.add_tag(
        Tag(
            id=tag.id,
            name=tag.name,
            shorthand=tag.shorthand,
            color=TagColor.get_color_from_str(tag.color),
        )
    )
    # Apply user edits to built-in JSON tags.
    if tag.id in range(RESERVED_TAG_START, RESERVED_TAG_END + 1):
        updated_tag = self.get_tag(tag.id)
        updated_tag.color = TagColor.get_color_from_str(tag.color)
        self.update_tag(updated_tag)  # NOTE: This just calls add_tag?

I have the strong suspicion that it is ORM shenanigans. Probably something like: The ORM notices that the Tag object that is being added is already being tracked by it and so instead of being INSERTed it gets UPDATEd instead

[CyanVoxel]

Thank you SO much for the review, @Computerdores and @Tishj!! I know this was a massive one to look over so I really appreciate it!

[Tishj]

Thanks for your continued work on TagStudio, it's great to see the strides that the project is making! 💪