Error 413 (Request Entity Too Large) When Using "Select Recent Configurations" in Airflow UI

[Glitch33r]

Description:

I am encountering an issue in the Airflow UI when attempting to run a DAG using the "Select Recent Configurations" option. Instead of running successfully, I receive a 413 Request Entity Too Large error.

JSON Size in Issue: 10,000+ lines

Steps to Reproduce:

• Open the form to manually provide JSON data for the DAG run (our JSON can be quite large, up to 10,000 lines).
• Submit the JSON configuration. This step works perfectly, and the DAG runs without issues.
• Attempt to use "Select Recent Configurations" to reuse a previously submitted JSON configuration.
• Encounter the 413 Request Entity Too Large error.

Environment:

Airflow Version: 2.10.3

Additional Information:

The error suggests that the payload is too large for the server to handle. However, submitting the same data via the form works perfectly.

Questions:

• Is this a known limitation or bug in the Airflow UI?
• Are there any configuration options (e.g., for the web server) that can address this issue?
• Could this be resolved by modifying request limits in the server configuration?

Any help or insights would be greatly appreciated!

[potiuk]

https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#allowed-payload-size

[Glitch33r]

Thank you! I missed this conf variable

[HuanjieGuo]

Hi @Glitch33r , does it work for you after you tune the payload size? For our case, it doesn't work even after we tuned it.

[Glitch33r]

@HuanjieGuo Hi, yes. It works for my docker instance and on production as well. Maybe try as env variable (I have done in this way), also try to increase the value and check if this particular config is used

[HuanjieGuo]

Thanks @Glitch33r , let me change it to env variable. Currently we use airflow config.

[HuanjieGuo]

sys env doesn't help in 2.10.5 either.

[HuanjieGuo]

Same here, use Airflow 2.10.5

run a DAG using the "Select Recent Configurations" option. Instead of running successfully, I receive a 413 Request Entity Too Large error.

Already set the allowed-payload-size and it doesn't work .

The payload only takes around 111 kb.

[potiuk]

Actually the 111KB is likely the HTML page with "Request Entity too large' message - not the actual payload. If you received 413 error it means that you DID NOT get what you wanted because it was too large.

Also it could be your proxy or whatever you have between you and airflow that might cause it.

[HuanjieGuo]

We deployed our cluster in k8s and the request has passed the service.

"Actually the 111KB is likely the HTML page with "Request Entity too large' message" if it is from HTML, that should due to the webserver code right?

It's weird that tune the allowed_payload_size doesn't work for it. Any other parameter we can try for the html size you think?