Path forward for Book Browser

[imnotjames]

Proposal

Regarding the Book Browser view, we should switch back to filtering & sorting on the frontend using the old Book API endpoint for now.

See #935 for an example of what this change looks like.

Overview

The Book Browser as it stands in 3.0.2 is not fully operational and has a significant number of features missing. Given the book browser is a pretty big part of what Grimmory does I believe this should be considered something we should correct in a timely manner - eg, within the week.

While I am certain that pagination & and a good backend API is the long term solution, I am less certain we can accomplish all of the features that the original book browser supported on the AppBookService without significant tradeoffs on stability, maintenance, & usability -- at least, not in a reasonable time frame.

So, what can we do about it? I see a few paths forward.

Option A: Bring back client side filtering and sorting. It was working before, it can work again. I did bring this back in a branch. , but with the number of changes that have happened since them I'm worried I broke some of the other performance improvements we've made.

Option B: If we want to retain using the backend for everything, the most straightforward approach would be to fetch all records and sort + filter in Java. This means removing all of the database filtering and sorting, and re-implementing them in application-land. We have a PR that started this for some sorts. This has most of the drawbacks of doing it on the client side, but does cut down on network calls to the browser. At the same time, though, this means we cannot rely on the client computer for processing, memory, etc -- and we also have to do this fresh for every page. This means overall more database calls than client side. Other sorts that we have - such as "random" - would still be difficult to implement as part of this. (I have ideas on how, but none of them are great.)

Option C: Implement all of the sorts (+ multi-sort), support physical books, fix filters, add fields to AppBookService using database queries. This is going to be more difficult and have more unknowns. I am sure we could get the AppBookService to work, but it would require significant effort and focus + could lead to more regressions. Some sorts we would probably drop without deprecation notices -- leading to user frustration -- so there's a limited amount of wiggle room in some cases to retain the existing API definition. I feel like this is where we'd feel quite a bit of time pressure & could end up making concessions that we aren't happy with because of it.

Option D: Create a new set of APIs which are more appropriate for our use cases & can support all of the features our users expect. This will take the longest amount of time, but could set us up for a very strong and long lasting API.

It's possible to do many of these options simultaneously, as well.

Of these, Option A and then Option D at a later date make the most sense to me.

Reasoning

1. I don't want to cut features that are already existing in a patch release or minor release. It feels frustrating for users.
2. I would prefer if we could get a patch release soon which has the book browser working again.
3. I would prefer the book browser work before we start reworking the rest of the UI visually.
4. I would prefer if we had more time to understand the problem space better that we're trying to solve instead of trying to shoehorn things in.
5. In 3.0.2 we are still fetching all of the books from the /v1/books endpoint on the book browser page. There's limited performance improvement because we are fetching both paginated + unpaginated.
6. Long term, we want a new set of clear API endpoints with strong contracts.

Results

Vote occurred on 2026-05-03 with Three +1, One -1. The proposal will be moved forward.

[imnotjames]

This was a question brought up multiple times:

Will this negatively impact performance?

[imnotjames]

We already are fetching all of the data I am suggesting we fetch. Between that & the tanstack virtual grid, this seems to be relatively similar in performance. Profiling was done as part of this and the memory usage and CPU % was within margin of error. (eg, the browser took up 1.05GB memory vs 1.1GB memory)

This leaves in place the performance improvements for the frontend from the tanstack virtual and other aspects which we've improved.

[imnotjames]

This was asked in a sync we had.

Is this a permanent change?

[imnotjames]

This is a UI only change which brings no change to state, no migrations, no API changes, and no new web requests.

This is a change that could be removed at any time, and is a two-way-door decision which is easily reversible.

Other options include additions to the API, code migrations, and changes in state which are harder to undo.

[imnotjames]

This is a paraphrasing of a concern that was brought up:

Does this replace the work we've done on the App API?

[imnotjames]

What does this mean for the work on the app API: nothing, the app API will not be removed and improvements can continue to be made.

This is primarily a change that is temp until we can find a better long term approach - whether that is the app API, a new API, or soemthing else entirely. We have been improving the app API, but there's a lot of gaps & code we're less confident of.

[imnotjames]

At this point we're going to do a vote. Please respond to this thread with your vote - for, against, or abstain. Voting quorum is met with @obviouslyallie , @balazs-szucs, @zachyale, @imnotjames

For posterity, the voting:

So we're going to be moving forward. There might be a better way to do this but it's what I got.

[imnotjames]

For

[zachyale]

For

[balazs-szucs]

Aganist

[obviouslyallie]

For