Deleted books are never removed from Kobo devices, and reappear as duplicates when re-added

[imouro]

What went wrong?

Up front: I'm a user, not a developer. I hit this on my own library, and the code analysis further down was done with an AI tool at my direction — details in the disclosure at the end. The symptoms and the numbers are my own observations.

When I delete a book from my library, it stays on my Kobo forever. Sync reports success, but the book never disappears from the device. My reader had drifted 38 books ahead of the library before I noticed.

It also produces duplicates. If I delete a book and later add the same file again — which happens while reorganising folders — the reader shows it twice: the new copy, plus the old one that can never be withdrawn.

This looks structural rather than environmental: Grimmory can only announce a removal for a book that still exists in the database, but a removal is only ever detected after the book is gone.

How can we reproduce it?

1-Put a book on the Kobo shelf and sync a Kobo device so it downloads.
2- Delete the book from the library (a soft delete is enough).
3- Sync the Kobo again, as many times as you like.

What happened: the book is still on the device. No removal is ever sent. What I expected: the device drops the book.

To also reproduce the duplicate: add the same file back after step 3, then sync. The device now shows the book twice.

What Build of Grimmory are you on?

Stable

Your setup

.- Grimmory version: v3.2.4 (grimmory/grimmory:latest, revision 6fd21da)
.- How I'm running it: Docker Compose, MariaDB 11.4.5
.- Operating system: Ubuntu

Screenshots or error messages (optional)

Where it breaks

syncLibrary() diffs the previous and current snapshots to find removals, then calls entitlementService.generateChangedEntitlements(removedIds, token, true) (KoboLibrarySyncService.java:118).

That method sources its data from bookQueryService.findAllWithMetadataByIds(bookIds) (KoboEntitlementService.java:75), whose query is:

SELECT b FROM BookEntity b WHERE b.id IN :bookIds AND (b.deleted IS NULL OR b.deleted = false)
By the time a removal is detected the book is already deleted, so the query returns nothing, the stream is empty, and no ChangedEntitlement with removed=true is ever emitted. Soft delete does not help: deleted = true is excluded by the same predicate, and DeletedBooksCleanupTask purges the rows after 7 days.

The duplicates come from the same place: the entitlement ID is the Grimmory book ID (String.valueOf(book.getId())), so a re-imported file gets a new ID and lands as a new title, while the old entitlement can never be withdrawn.

What was checked on my install

All 38 stuck books corresponded to book rows that no longer existed — no exceptions.
Cross-referencing snapshot file hashes, 5 of them shared a hash with a live book under a different ID. Those were exactly the 5 duplicates I could see on the reader.
To confirm this is the only blocker, a repair was run against my database (AI-assisted, under my supervision, with a full dump taken first): minimal book + book_file (EPUB) rows were recreated for the 38 stuck IDs, kept off the Kobo shelf, and the IDs seeded into the existing snapshots. Nothing else was changed, and the temporary rows were deleted afterwards.
What I saw myself: my reader had been showing 1,609 titles against 1,564 in Grimmory. After that change, I synced both Kobos and each of them dropped exactly 38 books, landing on 1,571. The 5 duplicates are gone. I did not touch the devices by hand at any point.
Possibly the same root cause as #1210

Discussion #1210 ("Kobo Sync - Book not found for entitlement ID") reports orphaned references left behind by deleted books, with the cause not pinned down. It may be the same thing seen from a different angle.

Possible direction, not a patch

A removal entitlement only needs the book ID, so it could be built from the snapshot rows instead of a live BookEntity lookup; or the lookup for removed=true could deliberately include soft-deleted rows.

Minor and separate, while I was in there: at KoboLibrarySyncService.java:170 the deleted-book progress cursor is cleared with syncToken.getOngoingSyncPointId(), which is null on a single-page sync, so cursor rows leak until the snapshot is pruned. Harmless as far as I can tell, but it looks like it was meant to be currSnapshot.getId().

AI disclosure, being precise about who did what:

I'm not a developer. I noticed the symptom, and everything in the "Where it breaks" section was produced by Claude (Claude Code) working on my server at my direction: decompiling the jar from my running container and cross-checking it against main.
The database queries and the temporary repair were also run by that tool under my supervision, on my install, with a dump taken beforehand.
Mine: the symptom, the device counts before and after, the decision to report it, and reading this through before posting. I can explain the cause in my own words and I'm happy to run anything you want checked — but if you ask me to read Java, I'll be honest and say I'd need help with it.
If that mix isn't what you want in Issue Triage, tell me and I'll cut it back to the plain user-facing report.

Before submitting

• I've searched existing discussions and this hasn't been reported yet
• I've reviewed this report and have completed it as fully as I can

[imnotjames]

This is not a defect, this is a design decision. As such I am counting this as a feature request.