Make Gradio UI button.click run in parallel

[imbajin]

refer

incubator-hugegraph-ai/hugegraph-llm/src/hugegraph_llm/demo/rag_demo/rag_block.py

Line 176 in 2ae610c

btn.click( # pylint: disable=no-member

Gradio should be the default fully asynchronous implementation, generating a task_id like value with each (component)click, and then executing multiple requests in parallel.

However, in the current conversation, it has been tested that accessing multiple windows simultaneously can cause blocking and queuing. It is necessary to confirm the cause of the problem and enable parallelization (control parallelism)

[Kryst4lDem0ni4s]

@imbajin Please assign this task to me.
Cause of the Problem:
Blocking and Queuing Behavior: Gradio, by default, has a concurrency limit of 1. This causes blocking when multiple users or windows are accessing the application simultaneously.
No Asynchronous Execution: The current implementation does not specify concurrency_limit, leading to sequential execution.
Queue Configuration: If queue() is not configured properly, tasks are processed one at a time.

[imbajin]

@imbajin Please assign this task to me. Cause of the Problem: Blocking and Queuing Behavior: Gradio, by default, has a concurrency limit of 1. This causes blocking when multiple users or windows are accessing the application simultaneously. No Asynchronous Execution: The current implementation does not specify concurrency_limit, leading to sequential execution. Queue Configuration: If queue() is not configured properly, tasks are processed one at a time.

Thanks & assigned, but why Gradio set it(concurrency) to 1 by default? (Really confused)

[Kryst4lDem0ni4s]

I found it in the documentation, concurrency_limit: int | None | Literal['default']
default = "default"
If set, this is the maximum number of this event that can be running simultaneously. Can be set to None to mean no concurrency_limit (any number of this event can be running simultaneously). Set to "default" to use the default concurrency limit (defined by the default_concurrency_limit parameter in Blocks.queue(), which itself is 1 by default).

Links : https://www.gradio.app/docs/gradio/blocks
https://www.gradio.app/docs/gradio/button

So the default is 1, otherwise when it is explicitly set to None, it would be unlimited concurrent requests. I am exploring two methods here.
The first method is to set this parameter in the button gradio.Button() and the other method is to utilize Blocks with gradio.Blocks().

[imbajin]

I found it in the documentation, concurrency_limit: int | None | Literal['default'] default = "default" If set, this is the maximum number of this event that can be running simultaneously. Can be set to None to mean no concurrency_limit (any number of this event can be running simultaneously). Set to "default" to use the default concurrency limit (defined by the default_concurrency_limit parameter in Blocks.queue(), which itself is 1 by default).

Links : gradio.app/docs/gradio/blocks gradio.app/docs/gradio/button

So the default is 1, otherwise when it is explicitly set to None, it would be unlimited concurrent requests. I am exploring two methods here. The first method is to set this parameter in the button gradio.Button() and the other method is to utilize Blocks with gradio.Blocks().

@Kryst4lDem0ni4s I think both fine for us, we can first use the global settings in Blocks, modify the Button if we need to set it independently in future :)