Defining extensible and enabling more use cases

[wizardwatch]

Let me start by admitting I need to become more familiar with the code base. My cursory search may have misinformed me about future intentions or current features. It is my understanding that there are currently no implementations of the goal of extensibility. Selfishly, then, I suggest an implementation that would cater to my particular use case.

Assuming the definition and goal of extensibility is to enable users to write Turing complete rules for organizing files, I suggest integrating a Lisp, Python, or other general-purpose language for this goal. For example, the use case that led me to make this discussion is automatically sorting files downloaded by a browser, such that file types like PDFs can be automatically printed and categorized based on the download source. Doing this would mean integrating into protocols like XDG or defining rules that watch file changes as a background service to trigger user input. The behavior desired by the user in these cases is likely very complex, so scripting is the only way to satisfy every use case.

Thus, the complete proposition is this: The macro system will best exist as a series of expressions connected by rules defined by Tag Studio. Rules would either be represented in the GUI, or in the code as tagstudio.{something}

Imagine this chain of events. In Tag Studio one defines an event such as "on new file" which triggers (pseudo code)

tagstudio.message("Categorize the new file");
tagstudio.categorize(tagstudio.getFile)

or another example
"on new file"
"if filetype pdf"

printDocument(tagstuidio.getFile)

This could allow the user to define their desired behavior with unlimited options, made even more useful if these scripts could implement new rules to expose in the GUI.

Potential downsides:
Now there is an API to break.
You are locking into a specific language, which may not be the best choice if tag studio is eventually ported.

If those downsides are too significant, I suggest allowing users to use the same GUI to make macros but instead execute files with arguments and accept input from the CLI program. This would not involve an API and would not lock in any decisions, but it is obviously less powerful for creating a complete system for interacting with files.

[Computerdores]

but it is obviously less powerful for creating a complete system for interacting with files.

also potentially less secure, because if the other approach uses proper sandboxing the worst that could happen is that the library data is messed up.

[wizardwatch]

That is very true. I would like to add that if integrating sandboxed code into the program is the approach taken, an escape hatch that allows arbitrary code to run would be very useful. I don't think printing, emailing, transcoding, etc files will ever be core features, but I would appreciate the ability to add them myself. Perhaps a checkbox per macro/script that says "allow unsafe operations."

[CyanVoxel]

Hello! I’ve caught up on the discussions here as well as in the Discord, and first wanted to give an outline of what’s been planned before this post and how I feel about the proposals in this discussion and where that can lead things.

Current Planned Features

Related to “Extensibility”:

Macros

When? Maybe v9.5, but could be v9.6+

• Actions that are triggered or ran automatically based on various conditions. These would include but not be limited to:
  • Triggers: (manual, added to library, upon opening the library, etc.)
  • Conditions: Is file type “image”, is file extension .png, does filename contain “vacation”, is file tag count <4 and file added date greater than 2024-09-06-12:31? Etc...
• These macro scripts would be mainly intended to be built within the TagStudio program’s GUI, however they could also be stored in a human readable/editable format on disk.
• This feature has been partially in the program since v8.x, but has not had a full and proper implementation yet.

API/Interface for 3rd Party Programs

When? When it’s ready, this is likely a long term change. v10?

• An interface for external programs to operate upon a TagStudio library. This is mainly focused around CRUD operations.
• This goes hand in hand with separating out a core TagStudio library for use internally and in 3rd-party applications/plugins/etc.

Plugin System

When? After the API. We’re talking post-v10 for this probably.

• The ability to add on external plugins to TagStudio.
• I don’t have any idea how this might operate. This may use the API listed above, or a sperate plugin API.
  • The TagStudio Core is meant to contain a subset of the TagStudio program’s features. This line has yet to be firmly drawn, as features and changes are always shifting around the program.

Now the Rest

It seems we generally want similar things, especially when it comes to conditional operations, however my scope is admittedly much more limited than giving users free range over an entire scripting language. I believe that creating a curated set of conditions and operations will be able to cover what most users want to achieve or that I’d consider in the scope of the program, without the need for explicitly supporting a Turing-complete scripting language or requiring any sort of sandboxed environment. I would consider having arbitrary file system access, internet access, and code execution being outside the scope of the program.

There is also the important distinction to make between the proposed “TagStudio Core” library/system/protocol and the “TagStudio Application” - both of which are currently intermingled but planned to be combed apart over time. Macros are tricky because they would obviously need access to an internal API, and that’s something we plan on building, however I would also consider many Macro operations being out of the scope of the “core” program. This line is not set in stone at the moment, but as the core library continues to be worked on then it will need to be solidified. Over time features could be backported to be considered “core”, but let’s not worry about that now.

So, where do I see things going right now? I think that getting a good base of Macro triggers and conditions implemented, along with a GUI to create and edit them, as well an agreed upon human-readable save format/syntax would be a reasonable next step. I feel gradually working on a comprehensive number of (in-scope) conditions rather than opening up the doors to a full scripting language would be a better approach for this program specifically. “Plugins” could be the answer to more advanced operations that I probably wouldn’t want to allow for or support myself, but I feel those would be best as the last layer on this stack of features.