Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(llm): enable tasks concurrency configs in Gradio #188

Open
wants to merge 11 commits into
base: main
Choose a base branch
from
Open

fix(llm): enable tasks concurrency configs in Gradio #188

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
e4b430c
[Feature] #185: Improvement of Prompt Configuration for rag_demo Grap…
Kryst4lDem0ni4s Mar 2, 2025
50678a7
feat(rag_demo): add task ID generation to enabled concurrency with Gr…
Kryst4lDem0ni4s Mar 2, 2025
50a8966
[Feature] #185: Improvement of Prompt Configuration for rag_demo Grap…
Kryst4lDem0ni4s Mar 2, 2025
885514d
Merge branch 'gradioconcurrency' of https://github.com/Kryst4lDem0ni4…
Kryst4lDem0ni4s Mar 2, 2025
d65881a
Merge branch 'main' into gradioconcurrency
imbajin Mar 3, 2025
418df29
move decorator
imbajin Mar 3, 2025
2e11b1f
fix #176: refactor(demo) switched gr.Blocks() concurrency to gr.Butto…
Kryst4lDem0ni4s Mar 3, 2025
c1aa28f
fix: improve exception handling in HGraphConfig for API version retri…
Kryst4lDem0ni4s Mar 3, 2025
ddd3f46
refactor: enhance task ID logging and improve exception handling in H…
Kryst4lDem0ni4s Mar 3, 2025
c684d45
Merge branch 'main' into gradioconcurrency
imbajin Mar 6, 2025
75abdf9
Merge branch 'main' into gradioconcurrency
imbajin Mar 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 15 additions & 4 deletions hugegraph-llm/src/hugegraph_llm/demo/rag_demo/rag_block.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,11 @@

from hugegraph_llm.config import resource_path, prompt, huge_settings, llm_settings
from hugegraph_llm.operators.graph_rag_task import RAGPipeline
from hugegraph_llm.utils.decorators import with_task_id
from hugegraph_llm.utils.log import log


@with_task_id
def rag_answer(
text: str,
raw_answer: bool,
Expand Down Expand Up @@ -126,17 +128,22 @@ def create_rag_block():
gr.Markdown("""## 1. HugeGraph RAG Query""")
with gr.Row():
with gr.Column(scale=2):
# with gr.Blocks().queue(max_size=20, default_concurrency_limit=5):
inp = gr.Textbox(value=prompt.default_question, label="Question", show_copy_button=True, lines=3)

# TODO: Only support inline formula now. Should support block formula
gr.Markdown("Basic LLM Answer", elem_classes="output-box-label")
raw_out = gr.Markdown(elem_classes="output-box", show_copy_button=True, latex_delimiters=[{"left":"$", "right":"$", "display":False}])
raw_out = gr.Markdown(elem_classes="output-box", show_copy_button=True,
latex_delimiters=[{"left": "$", "right": "$", "display": False}])
gr.Markdown("Vector-only Answer", elem_classes="output-box-label")
vector_only_out = gr.Markdown(elem_classes="output-box", show_copy_button=True, latex_delimiters=[{"left":"$", "right":"$", "display":False}])
vector_only_out = gr.Markdown(elem_classes="output-box", show_copy_button=True,
latex_delimiters=[{"left": "$", "right": "$", "display": False}])
gr.Markdown("Graph-only Answer", elem_classes="output-box-label")
graph_only_out = gr.Markdown(elem_classes="output-box", show_copy_button=True, latex_delimiters=[{"left":"$", "right":"$", "display":False}])
graph_only_out = gr.Markdown(elem_classes="output-box", show_copy_button=True,
latex_delimiters=[{"left": "$", "right": "$", "display": False}])
gr.Markdown("Graph-Vector Answer", elem_classes="output-box-label")
graph_vector_out = gr.Markdown(elem_classes="output-box", show_copy_button=True, latex_delimiters=[{"left":"$", "right":"$", "display":False}])
graph_vector_out = gr.Markdown(elem_classes="output-box", show_copy_button=True,
latex_delimiters=[{"left": "$", "right": "$", "display": False}])

answer_prompt_input = gr.Textbox(
value=prompt.answer_prompt, label="Query Prompt", show_copy_button=True, lines=7
Expand All @@ -147,6 +154,7 @@ def create_rag_block():
show_copy_button=True,
lines=7,
)

with gr.Column(scale=1):
with gr.Row():
raw_radio = gr.Radio(choices=[True, False], value=False, label="Basic LLM Answer")
Expand Down Expand Up @@ -200,6 +208,9 @@ def toggle_slider(enable):
example_num,
],
outputs=[raw_out, vector_only_out, graph_only_out, graph_vector_out],
queue=True, # Enable queueing for this event
concurrency_limit=5, # Maximum of 5 concurrent executions
concurrency_id="rag_answer_task"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so as here

)

gr.Markdown(
Expand Down
11 changes: 11 additions & 0 deletions hugegraph-llm/src/hugegraph_llm/utils/decorators.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,3 +90,14 @@ def wrapper(*args: Any, **kwargs: Any) -> Any:
log.debug("%s QPS: %f/s", args[0].__class__.__name__, qps)
return result
return wrapper

# A decorator to wrap functions with a task id generation and logging
def with_task_id(func):
def wrapper(*args, **kwargs):
import uuid
task_id = str(uuid.uuid4())
log.info(f"New task created with id: {task_id}")
# Optionally, you could also pass the task_id to the function if needed:
# kwargs['task_id'] = task_id
return func(*args, **kwargs)
return wrapper
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this decorator still useful now or in the future? Is the main purpose of logging newly created tasks?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes this one is useful, it's for generating concurrency ids, while the other is a more static general id useful for logging references (the one in the button) and can be modified or removed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes this one is useful, it's for generating concurrency ids, while the other is a more static general id useful for logging references (the one in the button) and can be modified or removed.

We can keep it, but I still don't understand the purpose of recording this taskID separately in the click function? It seems that there is no significant difference even if I don't use this decorator and record it?

BTW, our logs are already too numerous and need to be streamlined as much as possible 😄

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Understood, I'll try to streamline the logs next.
And you're right, there isnt any significant usage of the click function's taskID, it is merely for identifying the ongoing process so that it is clearer to the dev, which process' ID was being generated, but we can reduce this too if it is unneccesary. As for the decorator, the task_id = str(uuid.uuid4()) is being used to identify any concurrent processes being run, so that we can differentiate ongoing processes per task_id for further debugging as per #176 . I hope this is useful for further development and scaling

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Understood, I'll try to streamline the logs next.
And you're right, there isnt any significant usage of the click function's taskID, it is merely for identifying the ongoing process so that it is clearer to the dev, which process' ID was being generated, but we can reduce this too if it is unneccesary. As for the decorator, the task_id = str(uuid.uuid4()) is being used to identify any concurrent processes being run, so that we can differentiate ongoing processes per task_id for further debugging as per #176 . I hope this is useful for further development and scaling

I haven't found out how it works in my local tests. Can you give me an example of how to use it?

13 changes: 8 additions & 5 deletions hugegraph-python-client/src/pyhugegraph/utils/huge_config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
# under the License.

import re
import sys
import traceback
from dataclasses import dataclass, field
from typing import List, Optional
Expand Down Expand Up @@ -63,8 +64,10 @@ def __post_init__(self):
)

except Exception as e: # pylint: disable=broad-exception-caught
traceback.print_exception(e)
self.gs_supported = False
log.warning(
"Failed to retrieve API version information from the server, reverting to default v1."
)
try:
traceback.print_exception(e)
self.gs_supported = False
except Exception:
exc_type, exc_value, tb = sys.exc_info()
traceback.print_exception(exc_type, exc_value, tb)
log.warning("Failed to retrieve API version information from the server, reverting to default v1.")
Loading