Can't decompress data which is compressed by rust code.

[WindSoilder]

Here is simplest reproducing example:

import zstd
zstd.decompress(b'\x28\xb5\x2f\xfd\x00\x58\x11\x00\x00\x7b\x7d')

It raises an error: Error: Input data invalid or missing content size in frame header.

More context

I'm trying to rewrite a client application in rust, it sends compressed data to server, then server decompresses it. Unfortunally the server failed to decompress data.

Here is how I do it in client side:

use std::io::Cursor;
use zstd;

fn main() {
    let body = zstd::encode_all(Cursor::new("{}".as_bytes()), 3).unwrap();
    for x in body.iter() {
        print!("\\x{x:x?}");
    }
}

And I copied the body and decompressed it in python, and it failed.

---

If I tried to compress data({} in my example) in python, and decompressed in rust, it successes. So I think it's the issue in python side.

In rust, I'm using zstd-rs for compressing/decompressing

[sergey-dryabzhinsky]

try to not skip first 4 bytes пн, 12 авг. 2024 г. в 10:50, Wind ***@***.***>:

…

Here is simplest reproducing example: import zstdzstd.decompress(b'\x28\xb5\x2f\xfd\x00\x58\x11\x00\x00\x7b\x7d') It raises an error: Error: Input data invalid or missing content size in frame header. More context I'm trying to rewrite a client application in rust, it sends compressed data to server, then server decompresses it. Unfortunally the server failed to decompress data. Here is how I do it in client side: use std::io::Cursor;use zstd; fn main() { let body = zstd::encode_all(Cursor::new("{}".as_bytes()), 3).unwrap(); for x in body.iter() { print!("\\x{x:x?}"); }} And I copied the body and decompressed it in python, and it failed. ------------------------------ If I tried to compress data({} in my example) in python, and decompressed in rust, it successes. So I think it's the issue in python side. In rust, I'm using zstd-rs <https://github.com/gyscos/zstd-rs> for compressing/decompressing — Reply to this email directly, view it on GitHub <#100>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFPGHIFTCGZUBSV5PKACZLZRBSKZAVCNFSM6AAAAABMLT2TJOVHI2DSMVQWIX3LMV43ASLTON2WKOZSGQ3DAMRXHEYTGMY> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[gyscos]

It looks like python-zstd is unable to decompress zstd data when the content-size is missing from the frame header.

https://github.com/sergey-dryabzhinsky/python-zstd/blob/master/src/python-zstd.c#L157C1-L157C96

The C library (or the rust binding) do not require this, or include it by default, which is why data compressed by the rust binding cannot currently be decompressed by python-zstd.

[sergey-dryabzhinsky]

What original data is?

[sergey-dryabzhinsky]

Can you decompress data using cli version of zstd?

[WindSoilder]

Hi, the original data is {}. And I can decompress data using cli version of zstd.

[sergey-dryabzhinsky]

It's clearly no valid zstd block, there is no header, but maybe it's zstd stream chunk.