DeDRM failed to convert 2 books to EPUB

[MegMM]

Question / bug report

First, the plugin has been doing a terrific job on several hundred books tonight, so thank you for that!
Only two failed to convert. I own both books, but they aren't on Amazon's active site anymore.
[UPDATE] Sorry! Forgot to specify this is when converting to EPUB, so I'm not 100% sure it's DeDRM or not.

Steps I tried:

1. I was able to successfully download the file using K4PC 1.17.
2. I can read the book on my Kindle 4th gen ereader.
3. I removed the DeDRM plug-in and tried uploading directly from my Kindle 4th gen. But it didn't affect the message.
   That both failed to convert with the same error (XXXX is just my substitution for the file position):

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xed in position XXXX: invalid continuation byte

Thank you!

Which version of Calibre are you running?

7.26

Which version of the DeDRM plugin are you running?

10.0.9

If applicable, which version of the Kindle software are you running?

No response

Log output

DeDRM v10.0.9: Trying to decrypt Prophesy (The King & Alpha Seri - A.E. Via.azw3
MobiDeDrm v1.1.
Copyright © 2008-2022 The Dark Reverser, Apprentice Harper et al.
Decrypting Kindle Format 8 ebook: Prophesy (The King & Alpha Series Book 1)
Got DSN key from database default_key
Got DSN key from database kindle_key_1720940058_1
Found 12 keys to try after 0.8 seconds
Crypto Type is: 0
This book is not encrypted.
Decryption succeeded after 0.8 seconds
DeDRM v10.0.9: Finished after 0.8 seconds
InputFormatPlugin: MOBI Input running
on C:\Users\megha\AppData\Local\Temp\calibre_e5vayy2a\p8gwl8aj.azw3
Found KF8 MOBI of type 'standalone'
Failed to run pipe worker with command: from calibre.srv.render_book import viewer_main; viewer_main()
Traceback (most recent call last):
  File "runpy.py", line 198, in _run_module_as_main
  File "runpy.py", line 88, in _run_code
  File "site.py", line 83, in <module>
  File "site.py", line 78, in main
  File "site.py", line 50, in run_entry_point
  File "calibre\utils\ipc\worker.py", line 196, in main
  File "<string>", line 1, in <module>
  File "calibre\srv\render_book.py", line 845, in viewer_main
  File "calibre\srv\render_book.py", line 836, in render_for_viewer
  File "calibre\srv\render_book.py", line 814, in render
  File "calibre\srv\render_book.py", line 656, in process_exploded_book
  File "concurrent\futures\_base.py", line 456, in result
  File "concurrent\futures\_base.py", line 401, in __get_result
  File "concurrent\futures\thread.py", line 58, in run
  File "calibre\srv\render_book.py", line 551, in process_book_files
  File "calibre\ebooks\oeb\polish\container.py", line 641, in parsed
  File "calibre\ebooks\oeb\polish\container.py", line 610, in parse
  File "calibre\ebooks\oeb\polish\container.py", line 210, in parse_xhtml
  File "calibre\ebooks\oeb\polish\parsing.py", line 69, in parse
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xed in position 6543: invalid continuation byte

and

calibre, version 7.26.0
ERROR: Loading book failed: Failed to open the book at C:\Calibre Library\amazon\JM Wolf\Jace's Trial (Trials in Abingdon Bo (3922)\Jace's Trial (Trials in Abingdo - JM Wolf.azw3. Click "Show details" for more info.

Failed to convert book: C:\Calibre Library\amazon\JM Wolf\Jace's Trial (Trials in Abingdon Bo (3922)\Jace's Trial (Trials in Abingdo - JM Wolf.azw3 with error:
Failed to initialize plugin: 'C:\\Users\\megha\\AppData\\Roaming\\calibre\\plugins\\GetFileName.zip'
DeDRM v10.0.9: Trying to decrypt Jace's Trial (Trials in Abingdo - JM Wolf.azw3
MobiDeDrm v1.1.
Copyright © 2008-2022 The Dark Reverser, Apprentice Harper et al.
Decrypting Kindle Format 8 ebook: Jace's Trial (Trials in Abingdon Book 1)
Got DSN key from database default_key
Got DSN key from database kindle_key_1720940058_1
Found 12 keys to try after 0.9 seconds
Crypto Type is: 0
This book is not encrypted.
Decryption succeeded after 0.9 seconds
DeDRM v10.0.9: Finished after 0.9 seconds
InputFormatPlugin: MOBI Input running
on C:\Users\megha\AppData\Local\Temp\calibre_vlozscqz\h9bonjak.azw3
Found KF8 MOBI of type 'standalone'
KF8 has no metadata Table of Contents
Failed to run pipe worker with command: from calibre.srv.render_book import viewer_main; viewer_main()
Traceback (most recent call last):
  File "runpy.py", line 198, in _run_module_as_main
  File "runpy.py", line 88, in _run_code
  File "site.py", line 83, in <module>
  File "site.py", line 78, in main
  File "site.py", line 50, in run_entry_point
  File "calibre\utils\ipc\worker.py", line 196, in main
  File "<string>", line 1, in <module>
  File "calibre\srv\render_book.py", line 845, in viewer_main
  File "calibre\srv\render_book.py", line 836, in render_for_viewer
  File "calibre\srv\render_book.py", line 814, in render
  File "calibre\srv\render_book.py", line 656, in process_exploded_book
  File "concurrent\futures\_base.py", line 456, in result
  File "concurrent\futures\_base.py", line 401, in __get_result
  File "concurrent\futures\thread.py", line 58, in run
  File "calibre\srv\render_book.py", line 551, in process_book_files
  File "calibre\ebooks\oeb\polish\container.py", line 641, in parsed
  File "calibre\ebooks\oeb\polish\container.py", line 610, in parse
  File "calibre\ebooks\oeb\polish\container.py", line 210, in parse_xhtml
  File "calibre\ebooks\oeb\polish\parsing.py", line 69, in parse
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xed in position 242894: invalid continuation byte

[ElleKayEm]

Both logs say the book is unencrypted so doesn't seem to be DRM. I'd try fresh downloads of the books.

[MegMM]

Ok, thanks! I'll check with Mobileread.

As I said, I had already retried downloading both using K4PC and Kindle 4th Gen. And I tried loading directly from the Kindle 4th Gen to Calibre and got the error. Both books are readable in the app and the ereader.

[MegMM]

Sorry I closed it too soon. So after I removed the DeDRM plug-in, restarted Calibre, removed and re-downloaded the books. They do have DRM from both the Kindle 4th Gen and K4PC 1.17. So the DeDRM plug-in is required for these books.

After adding the DeDRM plugin back, restarting, removing, re-downloading the books and trying again, errors are slightly different. The errors below are for one book, using the file from K4PC and the 4th gen eReader. I think they're the same and I think the second book had the same errors too. I ran them through the Converter again, AZW3 to AZW3, and saved the full Debugging to separate folders. The text gets generated and saved, but the book isn't put together correctly.

So some people may have correctly DeDRM'd books, but the books aren't being reassembled correctly. I'm not actually sure what's going on.

Errors from K4PC -> Calibre -> convert to EPUB

calibre, version 7.26.0 (win32, embedded-python: True)
Conversion error: Failed: Convert book 1 of 1 (Prophesy (The King & Alpha Series Book 1))

Convert book 1 of 1 (Prophesy (The King & Alpha Series Book 1))
DeDRM v10.0.9: Trying to decrypt _xaswk58.azw3
MobiDeDrm v1.1.
Copyright © 2008-2022 The Dark Reverser, Apprentice Harper et al.
Decrypting Kindle Format 8 ebook: Prophesy (The King & Alpha Series Book 1)
Got DSN key from database default_key
Got DSN key from database kindle_key_1720940058_1
Found 12 keys to try after 0.7 seconds
Crypto Type is: 0
This book is not encrypted.
Decryption succeeded after 0.7 seconds
DeDRM v10.0.9: Finished after 0.7 seconds
Conversion options changed from defaults:
  cover: 'C:\\Users\\megha\\AppData\\Local\\Temp\\calibre_e9lhupdi\\4s2sfg4u.jpeg'
  verbose: 2
  output_profile: 'kindle_voyage'
  read_metadata_from_opf: 'C:\\Users\\megha\\AppData\\Local\\Temp\\calibre_e9lhupdi\\jm3naefc.opf'
Resolved conversion options
calibre version: 7.26.0
{'add_alt_text_to_img': False,
 'asciiize': False,
 'author_sort': None,
 'authors': None,
 'base_font_size': 0.0,
 'book_producer': None,
 'change_justification': 'original',
 'chapter': "//*[((name()='h1' or name()='h2') and re:test(., "
            "'\\s*((chapter|book|section|part)\\s+)|((prolog|prologue|epilogue)(\\s+|$))', "
            "'i')) or @class = 'chapter']",
 'chapter_mark': 'pagebreak',
 'comments': None,
 'cover': 'C:\\Users\\megha\\AppData\\Local\\Temp\\calibre_e9lhupdi\\4s2sfg4u.jpeg',
 'debug_pipeline': None,
 'dehyphenate': True,
 'delete_blank_paragraphs': True,
 'disable_font_rescaling': False,
 'dont_split_on_page_breaks': False,
 'duplicate_links_in_toc': False,
 'embed_all_fonts': False,
 'embed_font_family': None,
 'enable_heuristics': False,
 'epub_flatten': False,
 'epub_inline_toc': False,
 'epub_max_image_size': 'none',
 'epub_toc_at_end': False,
 'epub_version': '2',
 'expand_css': False,
 'extra_css': None,
 'extract_to': None,
 'filter_css': '',
 'fix_indents': True,
 'flow_size': 260,
 'font_size_mapping': None,
 'format_scene_breaks': True,
 'html_unwrap_factor': 0.4,
 'input_encoding': None,
 'input_profile': <calibre.customize.profiles.InputProfile object at 0x00000124FBA170D0>,
 'insert_blank_line': False,
 'insert_blank_line_size': 0.5,
 'insert_metadata': False,
 'isbn': None,
 'italicize_common_cases': True,
 'keep_ligatures': False,
 'language': None,
 'level1_toc': None,
 'level2_toc': None,
 'level3_toc': None,
 'line_height': 0.0,
 'linearize_tables': False,
 'margin_bottom': 5.0,
 'margin_left': 5.0,
 'margin_right': 5.0,
 'margin_top': 5.0,
 'markup_chapter_headings': True,
 'max_toc_links': 50,
 'minimum_line_height': 120.0,
 'no_chapters_in_toc': False,
 'no_default_epub_cover': False,
 'no_inline_navbars': False,
 'no_svg_cover': False,
 'output_profile': <calibre.customize.profiles.KindleVoyageOutput object at 0x00000124FBA62990>,
 'page_breaks_before': "//*[name()='h1' or name()='h2']",
 'prefer_metadata_cover': False,
 'preserve_cover_aspect_ratio': False,
 'pretty_print': True,
 'pubdate': None,
 'publisher': None,
 'rating': None,
 'read_metadata_from_opf': 'C:\\Users\\megha\\AppData\\Local\\Temp\\calibre_e9lhupdi\\jm3naefc.opf',
 'remove_fake_margins': True,
 'remove_first_image': False,
 'remove_paragraph_spacing': False,
 'remove_paragraph_spacing_indent_size': 1.5,
 'renumber_headings': True,
 'replace_scene_breaks': '',
 'search_replace': '[]',
 'series': None,
 'series_index': None,
 'smarten_punctuation': False,
 'sr1_replace': None,
 'sr1_search': None,
 'sr2_replace': None,
 'sr2_search': None,
 'sr3_replace': None,
 'sr3_search': None,
 'start_reading_at': None,
 'subset_embedded_fonts': False,
 'tags': None,
 'timestamp': None,
 'title': None,
 'title_sort': None,
 'toc_filter': None,
 'toc_threshold': 6,
 'toc_title': None,
 'transform_css_rules': '[]',
 'transform_html_rules': '[]',
 'unsmarten_punctuation': False,
 'unwrap_lines': True,
 'use_auto_toc': False,
 'verbose': 2}
DeDRM v10.0.9: Trying to decrypt 4pnfrem8.azw3
MobiDeDrm v1.1.
Copyright © 2008-2022 The Dark Reverser, Apprentice Harper et al.
Decrypting Kindle Format 8 ebook: Prophesy (The King & Alpha Series Book 1)
Got DSN key from database default_key
Got DSN key from database kindle_key_1720940058_1
Found 12 keys to try after 0.0 seconds
Crypto Type is: 0
This book is not encrypted.
Decryption succeeded after 0.0 seconds
DeDRM v10.0.9: Finished after 0.0 seconds
InputFormatPlugin: MOBI Input running
on C:\Users\megha\AppData\Local\Temp\calibre_e9lhupdi\n_84gv5m.azw3
Found KF8 MOBI of type 'standalone'
Extracting text...
Parsing all content...
Parsing styles/0001.css ...
Parsing text/part0000.html ...
Failed to parse content in text/part0000.html
Traceback (most recent call last):
  File "calibre\ebooks\oeb\reader.py", line 197, in _manifest_prune_invalid
  File "calibre\ebooks\oeb\base.py", line 1056, in data
  File "calibre\ebooks\oeb\base.py", line 966, in _parse_xhtml
  File "calibre\ebooks\oeb\parse_utils.py", line 203, in parse_html
  File "calibre\ebooks\conversion\preprocess.py", line 542, in __call__
  File "calibre\__init__.py", line 483, in entity_to_unicode
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xed in position 0: invalid continuation byte

Referenced file 'text/part0000.html' not in manifest
Parsing text/part0000.html ...
Failed to read from manifest entry with id: added, ignoring
Traceback (most recent call last):
  File "calibre\ebooks\oeb\reader.py", line 219, in _manifest_add_missing
  File "calibre\ebooks\oeb\base.py", line 1056, in data
  File "calibre\ebooks\oeb\base.py", line 966, in _parse_xhtml
  File "calibre\ebooks\oeb\parse_utils.py", line 203, in parse_html
  File "calibre\ebooks\conversion\preprocess.py", line 542, in __call__
  File "calibre\__init__.py", line 483, in entity_to_unicode
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xed in position 0: invalid continuation byte

Spine item 'id62' not found
Traceback (most recent call last):
  File "runpy.py", line 198, in _run_module_as_main
  File "runpy.py", line 88, in _run_code
  File "site.py", line 83, in <module>
  File "site.py", line 78, in main
  File "site.py", line 50, in run_entry_point
  File "calibre\utils\ipc\worker.py", line 215, in main
  File "calibre\gui2\convert\gui_conversion.py", line 38, in gui_convert_override
  File "calibre\gui2\convert\gui_conversion.py", line 25, in gui_convert
  File "calibre\ebooks\conversion\plumber.py", line 1138, in run
  File "calibre\ebooks\conversion\plumber.py", line 1348, in create_oebbook
  File "calibre\ebooks\oeb\reader.py", line 97, in __call__
  File "calibre\ebooks\oeb\reader.py", line 724, in _all_from_opf
  File "calibre\ebooks\oeb\reader.py", line 369, in _spine_from_opf
calibre.ebooks.oeb.base.OEBError: Spine is empty

and

Errors from Kindle 4th Gen -> Calibre -> convert to EPUB

Convert book 1 of 1 (Prophesy (The King & Alpha Series Book 1))
DeDRM v10.0.9: Trying to decrypt _zamoqko.azw3
MobiDeDrm v1.1.
Copyright © 2008-2022 The Dark Reverser, Apprentice Harper et al.
Decrypting Kindle Format 8 ebook: Prophesy (The King & Alpha Series Book 1)
Got DSN key from database default_key
Got DSN key from database kindle_key_1720940058_1
Found 12 keys to try after 0.7 seconds
Crypto Type is: 0
This book is not encrypted.
Decryption succeeded after 0.7 seconds
DeDRM v10.0.9: Finished after 0.7 seconds
Conversion options changed from defaults:
  output_profile: 'kindle_voyage'
  read_metadata_from_opf: 'C:\\Users\\megha\\AppData\\Local\\Temp\\calibre_e9lhupdi\\di5ck8mu.opf'
  verbose: 2
  cover: 'C:\\Users\\megha\\AppData\\Local\\Temp\\calibre_e9lhupdi\\h6c3q6b4.jpeg'
Resolved conversion options
calibre version: 7.26.0
{'add_alt_text_to_img': False,
 'asciiize': False,
 'author_sort': None,
 'authors': None,
 'base_font_size': 0.0,
 'book_producer': None,
 'change_justification': 'original',
 'chapter': "//*[((name()='h1' or name()='h2') and re:test(., "
            "'\\s*((chapter|book|section|part)\\s+)|((prolog|prologue|epilogue)(\\s+|$))', "
            "'i')) or @class = 'chapter']",
 'chapter_mark': 'pagebreak',
 'comments': None,
 'cover': 'C:\\Users\\megha\\AppData\\Local\\Temp\\calibre_e9lhupdi\\h6c3q6b4.jpeg',
 'debug_pipeline': None,
 'dehyphenate': True,
 'delete_blank_paragraphs': True,
 'disable_font_rescaling': False,
 'dont_split_on_page_breaks': False,
 'duplicate_links_in_toc': False,
 'embed_all_fonts': False,
 'embed_font_family': None,
 'enable_heuristics': False,
 'epub_flatten': False,
 'epub_inline_toc': False,
 'epub_max_image_size': 'none',
 'epub_toc_at_end': False,
 'epub_version': '2',
 'expand_css': False,
 'extra_css': None,
 'extract_to': None,
 'filter_css': '',
 'fix_indents': True,
 'flow_size': 260,
 'font_size_mapping': None,
 'format_scene_breaks': True,
 'html_unwrap_factor': 0.4,
 'input_encoding': None,
 'input_profile': <calibre.customize.profiles.InputProfile object at 0x000001B212CF70D0>,
 'insert_blank_line': False,
 'insert_blank_line_size': 0.5,
 'insert_metadata': False,
 'isbn': None,
 'italicize_common_cases': True,
 'keep_ligatures': False,
 'language': None,
 'level1_toc': None,
 'level2_toc': None,
 'level3_toc': None,
 'line_height': 0.0,
 'linearize_tables': False,
 'margin_bottom': 5.0,
 'margin_left': 5.0,
 'margin_right': 5.0,
 'margin_top': 5.0,
 'markup_chapter_headings': True,
 'max_toc_links': 50,
 'minimum_line_height': 120.0,
 'no_chapters_in_toc': False,
 'no_default_epub_cover': False,
 'no_inline_navbars': False,
 'no_svg_cover': False,
 'output_profile': <calibre.customize.profiles.KindleVoyageOutput object at 0x000001B212D42910>,
 'page_breaks_before': "//*[name()='h1' or name()='h2']",
 'prefer_metadata_cover': False,
 'preserve_cover_aspect_ratio': False,
 'pretty_print': True,
 'pubdate': None,
 'publisher': None,
 'rating': None,
 'read_metadata_from_opf': 'C:\\Users\\megha\\AppData\\Local\\Temp\\calibre_e9lhupdi\\di5ck8mu.opf',
 'remove_fake_margins': True,
 'remove_first_image': False,
 'remove_paragraph_spacing': False,
 'remove_paragraph_spacing_indent_size': 1.5,
 'renumber_headings': True,
 'replace_scene_breaks': '',
 'search_replace': '[]',
 'series': None,
 'series_index': None,
 'smarten_punctuation': False,
 'sr1_replace': None,
 'sr1_search': None,
 'sr2_replace': None,
 'sr2_search': None,
 'sr3_replace': None,
 'sr3_search': None,
 'start_reading_at': None,
 'subset_embedded_fonts': False,
 'tags': None,
 'timestamp': None,
 'title': None,
 'title_sort': None,
 'toc_filter': None,
 'toc_threshold': 6,
 'toc_title': None,
 'transform_css_rules': '[]',
 'transform_html_rules': '[]',
 'unsmarten_punctuation': False,
 'unwrap_lines': True,
 'use_auto_toc': False,
 'verbose': 2}
DeDRM v10.0.9: Trying to decrypt kxsq3l6t.azw3
MobiDeDrm v1.1.
Copyright © 2008-2022 The Dark Reverser, Apprentice Harper et al.
Decrypting Kindle Format 8 ebook: Prophesy (The King & Alpha Series Book 1)
Got DSN key from database default_key
Got DSN key from database kindle_key_1720940058_1
Found 12 keys to try after 0.0 seconds
Crypto Type is: 0
This book is not encrypted.
Decryption succeeded after 0.0 seconds
DeDRM v10.0.9: Finished after 0.0 seconds
InputFormatPlugin: MOBI Input running
on C:\Users\megha\AppData\Local\Temp\calibre_e9lhupdi\ta9tppgr.azw3
Found KF8 MOBI of type 'standalone'
Extracting text...
Parsing all content...
Parsing text/part0000.html ...
Failed to parse content in text/part0000.html
Traceback (most recent call last):
  File "calibre\ebooks\oeb\reader.py", line 197, in _manifest_prune_invalid
  File "calibre\ebooks\oeb\base.py", line 1056, in data
  File "calibre\ebooks\oeb\base.py", line 966, in _parse_xhtml
  File "calibre\ebooks\oeb\parse_utils.py", line 203, in parse_html
  File "calibre\ebooks\conversion\preprocess.py", line 542, in __call__
  File "calibre\__init__.py", line 483, in entity_to_unicode
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xed in position 0: invalid continuation byte

Parsing styles/0001.css ...
Referenced file 'text/part0000.html' not in manifest
Parsing text/part0000.html ...
Failed to read from manifest entry with id: added, ignoring
Traceback (most recent call last):
  File "calibre\ebooks\oeb\reader.py", line 219, in _manifest_add_missing
  File "calibre\ebooks\oeb\base.py", line 1056, in data
  File "calibre\ebooks\oeb\base.py", line 966, in _parse_xhtml
  File "calibre\ebooks\oeb\parse_utils.py", line 203, in parse_html
  File "calibre\ebooks\conversion\preprocess.py", line 542, in __call__
  File "calibre\__init__.py", line 483, in entity_to_unicode
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xed in position 0: invalid continuation byte

Spine item 'id62' not found
Traceback (most recent call last):
  File "runpy.py", line 198, in _run_module_as_main
  File "runpy.py", line 88, in _run_code
  File "site.py", line 83, in <module>
  File "site.py", line 78, in main
  File "site.py", line 50, in run_entry_point
  File "calibre\utils\ipc\worker.py", line 215, in main
  File "calibre\gui2\convert\gui_conversion.py", line 38, in gui_convert_override
  File "calibre\gui2\convert\gui_conversion.py", line 25, in gui_convert
  File "calibre\ebooks\conversion\plumber.py", line 1138, in run
  File "calibre\ebooks\conversion\plumber.py", line 1348, in create_oebbook
  File "calibre\ebooks\oeb\reader.py", line 97, in __call__
  File "calibre\ebooks\oeb\reader.py", line 724, in _all_from_opf
  File "calibre\ebooks\oeb\reader.py", line 369, in _spine_from_opf
calibre.ebooks.oeb.base.OEBError: Spine is empty

[ElleKayEm]

The DRM gets removed during import. Let's see a debug log from that: https://github.com/noDRM/DeDRM_tools/blob/master/FAQs.md#i-cannot-solve-my-problem-with-the-dedrm-plugin-and-now-i-need-to-post-a-log-how-do-i-do-that

For the copy from your Kindle, do you have that Kindle's serial number entered into DeDRM's customization?