Add zstd compression support

[Zaczero]

It seems you currently use https://github.com/fullonic/brotli-asgi [^1], which supports gzip and brotli compression. My recommendation is to switch to https://github.com/Zaczero/starlette-compress. It is backwards compatible, compatible with Python implementations other than CPython, generally better maintained, and also supports the new zstd compression.

[vincentsarago]

there is also https://github.com/developmentseed/starlette-cramjam 🤷

[Zaczero]

@vincentsarago I have taken a look at this library and in my opinion it's a worse alternative. Here's why (ordered from most to least impactful):

1. Bad default configuration
   It uses 11 for brotli by default which is a very bad choice for real-time applications (such as web apps). Such a high level is especially slow on brotli, which will hurt overall performance unless changed explicitly. It should only be reserved for operations where latency is not important. [ref]

2. Requires more configuration to get it set up
   By default it will compress all content-types, which is an odd decision for this kind of library. This will increase maintenance efforts and increase the chance of introducing a performance bug. It's common practice for compression middlewares to provide a default whitelist for compressible content-types, and starlette-compress does that. [ref]

3. Worse Brotli performance
   As per their own benchmarks, they show that their Brotli implementation is slower than the brotli package. [ref]

4. Suboptimal startup performance
   I quickly noticed that this package makes no use of the TYPE_CHECKING flag, which enables type-only import of starlette.types. This potentially increases the startup time. [ref]

5. Incomplete typing support
   It claims to support typing but the package does not contain a py.typed marker file that some tools use to detect whether a package is typed. [ref]

[vincentsarago]

🙏 thanks for the feedback on my lib. I'll try to fix some of the issues 🙏

Because we're doing free-review, here is mine on starlette-compress:

1. it depends on brotli (which hasn't been updated since 2023)
2. alternatively depends on brotlicffi (which hasn't been updated since 2023)
3. weak header parsing https://github.com/Zaczero/starlette-compress/blob/main/starlette_compress/__init__.py#L77-L85

To be fair, I've shared starlette-cramjam, because it doesn't depends on brotlipy.

I didn't want to have starlette-cramjam here because I didn't wanted to impose my own library (we use it on out own stac-fastapi deployment) but if you feel starlette-compress and zstd compression is a must have to stac-fastapi user please feel free to open a PR 🙏

[Zaczero]

Hello, here's my justification:

1. As far as I know, it's the most popular Python brotli solution. The same brotli dependency is used by brotli-asgi currently. It simply works and is well-recognized.

2. Sadly, brotli is only compatible with the CPython implementation, and for others, brotlicffi is installed. This is also a common technique in other Python brotli-based packages that aim to be widely compatible. If you run on CPython, it will not be installed.

3. That's a good point. It's actually a deliberate performance choice because, currently, to my knowledge and experience, the zstd->brotli->gzip fallback order seems to be the only widely used approach. I have never seen other client preference orderings (other than synthetic tests). Because of that, I decided to simply ignore q-values and use a hard-coded preference order.