sword-to-org-insert-outline; many lines have "David's Psalm of Praise." appended

[dmfunk]

Invoking Gen-Rev as option for sword-to-org-insert-outline results in muddled output?

Sword 1.9.0, Diatheke 4.8

Sample:

**** Revelation of John 18:10

Standing afar off for the fear of her torment, saying, Alas, alas, that great city Babylon, that mighty city! for in one hour is thy judgment come.David's Psalm of praise.

**** Revelation of John 18:11

And the merchants of the earth shall weep and mourn over her; for no man buyeth their merchandise any more:David's Psalm of praise.

[alphapapa]

Unfortunately, the Sword format, libraries, and utilities are not stable, so issues like this come up from time to time. It probably also depends on the module being used. Which one is that from? (I guess a KJV one, but specifics likely matter.)

[dmfunk]

Yes, KJV.

Hadn't realized the digital armour of G-d was so full of chinks—but if we are imperfect, it stands to reason that our tools might be, too.

Noticed similar issues with dtk having trouble parsing things predictably.

Have opted to use bible-mode for now, as it seems to produce the most legible outputs of sword-bible-kjv in emacs, thus far.

Thanks for the prompt reply, and happy belated Easter.

[alphapapa]

I meant to ask which KJV module you're using. i.e. what is the filename, where did it come from, is there a version number, or a contents hash to identify it, etc. There are likely many different Sword modules of the KJV, and some may exhibit this issue while others don't.

Unfortunately, there are too many variables at play for it to matter much, anyway. What's really unfortunate is that the Sword project has chosen to make the source code of the encoding/decoding libraries the definition of the file formats, and changes them at will, which makes it impractical to provide alternative implementations of them, and which leads to issues like this developing over time. It further doesn't help that the versions of tools like libsword and Diatheke in the wild tend to be many, as different ones are provided in various Linux distros and such, so two users may have different versions, one with the problem, another without.

You may be interested in this other repo of mine: https://github.com/alphapapa/sword-converter In it you'll find tools that can be used to convert a Sword module to a SQLite database. Note that it hasn't been updated in years, so who knows whether it would work with the module you have, or the version of libsword you have, etc. But if you were to use it to convert a Sword module to a SQLite database, you'd have a stable file that could be used with other tools reliably. (Note as well that it was only designed to convert basic text, not annotations and various metadata that may be included in some modules.) The notes include some testing of various SQLite full-text indexing options (IMO, showing that SQLite is a suitable replacement for the Sword module format).

I hope that someday the Sword project will "see the light" and move toward standard file formats that would be more interoperable; unfortunately, interoperability and stability don't seem to be goals of theirs, so we are left with this ever-shifting landscape.

Ultimately, you might even consider a solution like downloading a version of the Bible in HTML from a Web site that provides it; even HTML could be converted to other formats and searched with standard tools more reliably in the long term.

Happy Easter to you as well.

[dmfunk]

Originally KJV 2.3 via macports, now updated to KJV 3.1; still experiencing the same behaviour.

Interestingly, invoking sword-to-org-insert-outline on a single book, i.e. James, generates output as expected.

The issue starts when Psalms get involved, either as a standalone query, or in a range like Psalms-James.

I think it's because Psalms 3+ [in this KJV, perhaps others] have an extra line of context, before each line of verse.

$ diatheke -f plain -b KJV -k Ps 3

A Psalm of David, when he fled from Absalom his son.
Psalms 3:1: LORD, how are they increased that trouble me! many are they that rise up against me.
A Psalm of David, when he fled from Absalom his son.
Psalms 3:2: Many there be which say of my soul, There is no help for him in God. Selah.
A Psalm of David, when he fled from Absalom his son.
Psalms 3:3: But thou, O LORD, art a shield for me; my glory, and the lifter up of mine head.
A Psalm of David, when he fled from Absalom his son.
Psalms 3:4: I cried unto the LORD with my voice, and he heard me out of his holy hill. Selah.
A Psalm of David, when he fled from Absalom his son.
Psalms 3:5: I laid me down and slept; I awaked; for the LORD sustained me.
A Psalm of David, when he fled from Absalom his son.
Psalms 3:6: I will not be afraid of ten thousands of people, that have set themselves against me round about.
A Psalm of David, when he fled from Absalom his son.
Psalms 3:7: Arise, O LORD; save me, O my God: for thou hast smitten all mine enemies upon the cheek bone; thou hast broken the teeth of the ungodly.
A Psalm of David, when he fled from Absalom his son.
Psalms 3:8: Salvation belongeth unto the LORD: thy blessing is upon thy people. Selah.
(KJV)

Back in emacs, things seem to break on the last passage of the second book of Psalms, i.e. Psalms 2:12.

Where we can see the extra context line of Psalms 3:1 creep backwards into 2:12, popping out of the proverbial stack.

And changing as the extra line of context changes between chapters.

**** Psalms 2:12

Kiss the Son, lest he be angry, and ye perish from the way, when his wrath is kindled but a little. Blessed are all they that put their trust in him.A Psalm of David, when he fled from Absalom his son.

"A Psalm of David, when he fled from Absalom his son." is appended on subsequent lines until:

**** Psalms 3:8

Salvation belongeth unto the LORD: thy blessing is upon thy people. Selah.To the chief Musician on Neginoth, A Psalm of David.

"To the chief Musician on Neginoth, A Psalm of David." is appended on subsequent lines until:

**** Psalms 4:8

I will both lay me down in peace, and sleep: for thou, LORD, only makest me dwell in safety.To the chief Musician upon Nehiloth, A Psalm of David.

"To the chief Musician upon Neginoth, A Psalm of David." is appended on subsequent lines until:

**** Psalms 5:12

For thou, LORD, wilt bless the righteous; with favour wilt thou compass him as with a shield.To the chief Musician on Neginoth upon Sheminith, A Psalm of David.

and so forth.

I began debugging sword-to-org.el, but lack the time and experience to comprehend it presently.

24hrs ago, I hadn't even heard of sword, diatheke, etc…! So there has been a learning curve.

[An enjoyable one, it must be said.]

Thanks for your suggestions, hope this debug info is useful.

[alphapapa]

I think it's because Psalms 3+ [in this KJV, perhaps others] have an extra line of context, before each line of verse.

$ diatheke -f plain -b KJV -k Ps 3

A Psalm of David, when he fled from Absalom his son.
Psalms 3:1: LORD, how are they increased that trouble me! many are they that rise up against me.
A Psalm of David, when he fled from Absalom his son.
Psalms 3:2: Many there be which say of my soul, There is no help for him in God. Selah.
A Psalm of David, when he fled from Absalom his son.
Psalms 3:3: But thou, O LORD, art a shield for me; my glory, and the lifter up of mine head.
A Psalm of David, when he fled from Absalom his son.
Psalms 3:4: I cried unto the LORD with my voice, and he heard me out of his holy hill. Selah.
A Psalm of David, when he fled from Absalom his son.
Psalms 3:5: I laid me down and slept; I awaked; for the LORD sustained me.
A Psalm of David, when he fled from Absalom his son.
Psalms 3:6: I will not be afraid of ten thousands of people, that have set themselves against me round about.
A Psalm of David, when he fled from Absalom his son.
Psalms 3:7: Arise, O LORD; save me, O my God: for thou hast smitten all mine enemies upon the cheek bone; thou hast broken the teeth of the ungodly.
A Psalm of David, when he fled from Absalom his son.
Psalms 3:8: Salvation belongeth unto the LORD: thy blessing is upon thy people. Selah.
(KJV)

Hm, that looks like a bug in either Diatheke or that KJV module file to me; that prologue shouldn't be output before each verse.

There might also be an issue in that this Elisp library doesn't account for such prologues, but that such prologues keep getting output until the end of the module looks like a bug that's outside of this project.

I would guess that maybe there's a variable in Diatheke (or one of its supporting libraries) that holds the prologue, and that it isn't getting cleared after being output, so it keeps getting output once it is set.

So my best suggestion would be to isolate the problem from this Elisp library and then report the bug to the Sword project, without any reference to this Elisp project. (After ensuring that you have the latest versions of the software and modules, of course; it's always possible that this bug has already been solved.)

Then if it proves necessary to work around that bug in this library, we could consider how to do that.