Is it possible load a gzip-ed csv file?

[llimllib]

The docs say that files can be pulled from zip or tar.gz files:

File attachments can be also be pulled from archives. The following archive extensions are supported:
.zip - for the ZIP archive format
.tar - for tarballs
.tar.gz and .tgz - for compressed tarballs

Is it possible to load a gzipped file that's not in a tar repository?

I tried this:

const stats = await FileAttachment("test.csv.gz").csv();

but it, oddly, succeeded with an extremely corrupt result; it should presumably throw an error rather than succeeding in a way that is likely to surprise users

[Fil]

Not possible atm. For this task you can use fflate directly on the client.

The goal of "pulling from archives" is to allow a data loader to output multiple files, which can then be extracted read one by one.

Note that if we added a built-in "gz" data loader extension, you would call const stats = await FileAttachment("test.csv").csv(); without the .gz—but the text.csv.gz file would be the source of that file. However it would still be built as test.csv, so I'm not sure what the benefit of the additional complexity would be?

[llimllib]

Sure, the reason I wanted it was the same reason I ended up trying to figure out parquet: I wanted a smaller file on disk, as I was trying to replicate your HTTP response time example and we have a lot of requests. Allowing a gzipped file would let me trade space on disk for speed at load time.

Thanks for the answer!

[Fil]

In all my tests, parquet beats csv.gz (by … a lot). See https://mastodon.social/@severo/111957633001467414 for a recent story.

[llimllib]

right, I saw that toot actually, but then my parquet file failed to load as per #873