Error for requests with gzip content encoding and json content type

[oleg-babintsev]

Request headers:

headers = {
    'content-type': 'application/json',
    'charset': 'utf-8',
    'content-encoding': 'gzip',
}

Output in console:

SyntaxError: Unexpected token ▼ in JSON at position 0
    at JSON.parse (<anonymous>)
    at /app/index.js:48:22
    at Layer.handle [as handle_request] (/app/node_modules/express/lib/router/layer.js:95:5)
    at next (/app/node_modules/express/lib/router/route.js:137:13)
    at next (/app/node_modules/express/lib/router/route.js:131:14)
    at next (/app/node_modules/express/lib/router/route.js:131:14)
    at next (/app/node_modules/express/lib/router/route.js:131:14)
    at next (/app/node_modules/express/lib/router/route.js:131:14)
    at next (/app/node_modules/express/lib/router/route.js:131:14)
    at next (/app/node_modules/express/lib/router/route.js:131:14)

[mendhak]

How can I recreate the issue? I tried this:

curl -X POST -H "Content-Type: application/json" -d '{"a":"b"}' -H "Charset: utf-8" -H "Content-Encoding: gzip" http://localhost:8080/test/abc

But everything seems fine.

[oleg-babintsev]

I think you need send content in gzip format as binary data, like this:

echo '{ "mydummy" : "json" }' | gzip > body.gz
curl -v -i http://localhost/mymodule -H'Content-Encoding: gzip' --data-binary @body.gz

But I can't check it right now.

[mendhak]

I was able to recreate the error/message using

echo '{ "mydummy" : "json" }' | gzip > body.gz
curl -v -i http://localhost:8080/mymodule -H 'Content-Encoding: gzip' -H 'Content-Type: application/json' --data-binary @body.gz

I then tried decompressing the body using body-parser, which is supposed to be able to work with gzip, but that didn't work.

// parse application/x-www-form-urlencoded
app.use(bodyParser.urlencoded({ extended: false }))
 
// parse application/json
app.use(bodyParser.json())

I'll try it a bit more later. I'm looking for a solution that doesn't involve me writing custom code, to avoid complexity.

[thieman]

This is a bug in the server, not something that needs to be fixed in the request. Amazon Data Firehose is an example of a well-known application that hits this bug when GZIP encoding is enabled.

[mendhak]

This is now available in mendhak/http-https-echo:36 thanks to @matt-mercer for the PR.