`LargeString` leading to failing insert

[tsteckenborn]

If you're using a LargeString change-tracking can lead to an insert issue, due to it being to large for changelog_changes (which is likely defined as a string):

inserted value too large for column: TrexColumnUpdate failed on table '08992D9A763244238BFA5B3FCFD8C6C2:SAP_CHANGELOG_CHANGES' with error: AttributeEngine: indexed value not within specified range;checkCesu8StringColumn(): max length (5000) violated on pos 1, value='...',column '$column$', rc=6959

[nkaputnik]

@tsteckenborn
what would you suggest as default behavior for large strings from your side?
We are using String(5000) for change-tracking, since large strings would cause too much of a performance impact (they are essentially blobs on DB level), so anything over 5k chanracters would need to be truncated somehow.
Now - this could leave the old value and new value to essentially be identical after truncating, so this could look meaningless to users.
We could

• truncate after 5k characters
• insert only the last 5k
• write a small differ util to only show the changes made (could prove difficult if entier text or multiple parts are changed)
• write the total number of characters that are different
• only write a standard text that text is too large to be stored, but that for sure changes were made to it

Other than that, I need to verify that inserting large strings into the DB doesn't cause an error by itself.

Best,
Nick

[tsteckenborn]

The documentation doesn't state any supported (or unsupported) data types as far as I can see. Maybe a first step could be to maintain a list of supported (or unsupported) types.
If possible some sort of design time / build time failure would be great. In this case it only fails during runtime, when such a call is being made.

From "solution" perspective my personal preference would be 3 (differ), followed by 1 (truncation with eventually a truncation explaining that there were X more characters not saved) (if we can't utilize LargeStrings in general)

[trifork-gdo]

What about doing a min effort approach (e.g. option 1) and then provide a handler/hook where the developer can jump in and write their own code in doing special handling? That way I can write my own "special" diff function based on my specific requirements.

[lfcatrifork]

Thanks for raising this! I’d suggest a practical approach to avoid DB errors due to oversized strings: before inserting into the database, we could limit the string length to 4,997 characters and append "..." at the end (ensuring the total stays within the 5,000-character limit.)

if (c.valueChangedTo && c.valueChangedTo.length > 5000) { c.valueChangedTo = c.valueChangedTo.substring(0, 4997) + '...'; }

This way, we preserve the indication that truncation occurred, while staying safely within the defined String(5000) constraint.

I'm happy to implement this and submit a pull request if the team agrees.

[schiwekM]

Hi @tsteckenborn,

with v2 of the plugin this is fixed and values having more than 5k characters behave according to the practical approach from @lfcatrifork. v2 is currently in beta on npm and we intend to fully release it till end of Month.

BR,
Marten