[Discussion] The selection of Agentic/Taskflow frame

[imbajin]

Feature Description

Some Context Here:

Our goal at GraphRAG is to focus on providing better I/O capabilities related to LLM/RAG system. We need agentic but will not attempt to create a large and comprehensive (agent/rag) framework, so we tend to choose the highest performance/flexible workflow + agentic framework for integration(Any suggestions and feedback are welcome)

Below is a simplified summary table assisted by LLM. Currently, the focus is on the first 4 (CrewAI / Agno / LLamaIndex / Pydanic-AI), with prioritization to be determined. They generally have their own built-in workflow-like designs and have few dependencies (relatively lightweight).

Table:

Framework	Core Features	Documentation/Community Popularity	Workflow	Advantages	Disadvantages	GitHub ⭐️	Lightweight	Notes
CrewAI	Well-known multi-agent collaboration framework, supports sequential/hierarchical structures, dynamic workflow design, and dual-mode implementation of workflows	High	Crew + Flow design (the latter is event-driven)	Few dependencies, provides both Crew (role collaboration) + Flows (process control) usage modes, many examples and documentation	1. Requires understanding of Crew/Flow architecture and YAML configuration 2. Some tools may depend on Rust/C++ underlying libraries 3. Performance is unclear	26.7k+	Medium	Suitable for complex scenarios such as project management and strategic analysis, can be directly combined/compatible with LangChain/LLamaindex, disable info collection when using
Agno	High-performance multimodal framework, supports text/image/audio/video, model-agnostic, extremely low memory footprint. Claims performance is X1000 times that of LangGraph	High	workflow no graph/no chain design, supports caching/persistence. Simply put, it does not provide syntactic sugar, the process is manually controlled, which is not very user-friendly	Extremely fast startup speed, native multimodal support, knowledge base integration	Low community maturity, relies on third-party model APIs	19k+	Medium-High	Claims memory usage is only 1/50 of LangGraph, suitable for high-concurrency scenarios
Llamaindex		High	Event-driven workflow	Classic and established		39k+	Medium-Low?
Pydanic-AI	Produced by the Pydantic team, quality is relatively guaranteed, design should also be relatively good	Low	Classic Graph implementation	New generation	Officially stated to be in Beta, not perfect enough, API changes at any time, use with caution in production	6.5k+	Medium-High?
Lagent	Domestic lightweight multi-Agent framework, communication based on AgentMessage, defaults to providing synchronous/asynchronous interfaces	Low-Medium	Chained through the call method	Lightweight + good domestic model support	Relatively basic functionality, limited multi-agent support, no official documentation link?	2k+	Medium-High	Developed by Shanghai AI Laboratory, suitable for rapid experimentation
ControlFlow	Agent implementation based on the Prefect scheduling framework, integrated with the LangChain ecosystem	Medium	Uses Prefect + Graph design	Task granularity is controllable, supports hybrid orchestration of traditional workflows and Agents		1.2k+	Low	Based on Prefect
Swarm	OpenAI company open source, stateless lightweight design + Handoff (handoff mechanism), dynamic context management, automatic function generation, highest friendliness with OpenAI features	Medium-High	Unknown	Based on OpenAI technology, high theoretical performance potential, engineering design and OpenAI integration should be very good	Experimental version, less documentation and applications, low community participation, does not come with compatibility for other LLMs by default	18.7k+	Medium-High	Mainly to see the design highlights
…

---

Heavyweight Agent Frameworks (omitted)

This section is not core, mainly listing common/well-known frameworks, which can also be studied for good ideas.

• LangChain's LangGraph (9k+ ⭐️), its workflow design is relatively classic and the description is relatively complete (supports both Graph + Task/Function calling methods)
  • Note that in the latest version, LangGraph can be used independently and is no longer strongly coupled with LangChain (other differences can be roughly referenced in LLM QA)
  • So it can also be used as a reference, especially noting its performance (slow + high runtime overhead according to many user feedback)
• Microsoft's AutoGen (40k+ ⭐️)
• MetaGPT (46k+ ⭐️)
• Dify/FastGPT/RAGFlow and other RAG frameworks also have Agent functions
• AutoGPT (classic frame)
• ...

[chiruu12]

Pydantic-AI could have been an excellent option for us. However, since it's still in beta, adopting it would require updating our codebase, which would increase our workload. Therefore, I suggest going with Agno. That said, Agno's manually controlled workflow has its own pros and cons, so we will need to make some compromises with that approach as well.

[chiruu12]

@imbajin sir I went through all the repositories for the agentic frameworks you have mentioned above sorry took quite a while in doing that. I believe we should be looking at crewai or agno only.
I believe there are significant opportunities to reduce their weight by removing unnecessary components for our use and we could potentially create a more streamlined solution by extracting just the essential components we need.
Would you agree with taking a more minimalist approach by identifying and integrating only the critical components from these frameworks? Also i am working on a small demo of how an ai agent can be made for the same can i get a few api call examples for it? so that I can use those examples in my demo?

[imbajin]

integrating only the critical components from these frameworks?

@chiruu12 It would be best if we could do this, not be bound by unnecessary design of the framework, it will also be easier to switch and iterate over time (in fact the Agentic GraphRAG I understand is still similar to a single-Agent framework, and the logic should be kept as clear and concise as possible)

[chiruu12]

@imbajin yes sir and as most of these frameworks are open source and them being already lightweight it is easy to go through the whole code though it will take some time, we can use their code at the places where we need I do have a understanding of Agno framework once we decide which framework we will be using I will go through the whole code for the same for the time being I will try to understand the code for hugegraph-ai I went through the docs and will be trying it out.
I hope I will be useful in making this project a level above from what you expect (btw I replied lightning quick because I was checking the docs!).
My understanding is a little different for the Agentic GraphRag sir. I believe for increasing the accuracy we will have to take some extra steps in the agent (like a manager or validator/ fallback kind of agent) so that it is able to make decisions more accurately,
for that we will have to settle for the optimal trade-off for the accuracy and time.

[Aryankb]

Hey @imbajin
I was working with agentic graph RAG recently. The project aimed at Document QnA from a story book. It required Agentic graph-RAG approach as in a story, different characters can have different relationships with each other. Also different incidents are linked to each other. So, here agentic graph-RAG worked well if someone asks deep questions from story. So i made graph where centre node was book --> first level nodes were chunks (each chapter of that book) --> second level nodes were atomic_facts from that chunk--> final level nodes were characters. Same characters from different atomic_facts were connected (so as to know that character was present in which parts of the story). Also different characters were connected to each other having some relationships.

So for finding the answer of user's question, I used LANGGRAPH. It was really lightweight , Fast and simple to implement it.

There were agents -

• character finder (to find all characters, user's question contains)
• atomic fact finder ( worked at finding the relevant atomic facts for each character, according to user query. Using similarity search)
• information validator (checked if information is enough)
• atomic fact extractor (if information is not enough, then extract the nearby atomic fact, or nearby events in that story)
• final composer (compose final answer using extracted information and user query.

It was a fixed agentic system, worked at graphs of one kind only

I have a question, regarding what is the requirement ? Do we need to create a Interface, where user can create his own agentic system for his Graph , just using drag-drop or prompts (without code)
If yes, then using LANGGRAPH or completely doing it manually would be the best approach, because there will be transparency , and user would know what is happening. Using CrewAI would not be a good idea , as there we don't know what is happening under the hood.

Here is the project :-
https://github.com/Aryankb/DOC_QNA

[chiruu12]

So for finding the answer of user's question, I used LANGGRAPH. It was really lightweight , Fast and simple to implement it.

gpt uh!!

[chiruu12]

I believe for our use case we will have to train to model to also get us the text-to-cypher? As we wont know what kind of input the user is going to give us so for that we will have to fine tune a model for the same as the agent might need to write the cypher query on it's own?

Hey @imbajin I was working with agentic graph RAG recently. The project aimed at Document QnA from a story book. It required Agentic graph-RAG approach as in a story, different characters can have different relationships with each other. Also different incidents are linked to each other. So, here agentic graph-RAG worked well if someone asks deep questions from story. So i made graph where centre node was book --> first level nodes were chunks (each chapter of that book) --> second level nodes were atomic_facts from that chunk--> final level nodes were characters. Same characters from different atomic_facts were connected (so as to know that character was present in which parts of the story). Also different characters were connected to each other having some relationships.

So for finding the answer of user's question, I used LANGGRAPH. It was really lightweight , Fast and simple to implement it.

There were agents -

• character finder (to find all characters, user's question contains)
• atomic fact finder ( worked at finding the relevant atomic facts for each character, according to user query. Using similarity search)
• information validator (checked if information is enough)
• atomic fact extractor (if information is not enough, then extract the nearby atomic fact, or nearby events in that story)
• final composer (compose final answer using extracted information and user query.

It was a fixed agentic system, worked at graphs of one kind only

I have a question, regarding what is the requirement ? Do we need to create a Interface, where user can create his own agentic system for his Graph , just using drag-drop or prompts (without code) If yes, then using LANGGRAPH or completely doing it manually would be the best approach, because there will be transparency , and user would know what is happening. Using CrewAI would not be a good idea , as there we don't know what is happening under the hood.

Here is the project :- https://github.com/Aryankb/DOC_QNA

[Aryankb]

@chiruu12 yes, that's a good idea. If we need to create a system, where user can create different agents, each agent will have it's own job , means it's own cypher-query to extract information from the graph. Some agents might do other work like information validator.
Also , we can give user two options, either write own cypher-query, or generate query using LLM. In the second case, we might need fine tuning .

[chiruu12]

@Aryankb I think most of the people won't be proficient enough to write their own queries I have worked quite a bit with graph rag in my intern and at first even I had a bit trouble in writing those.
So, I would suggest that we can get a description for the knowledge that the user will be providing us if they don't, we will by default use a LLM to get what the knowledge or text is about and then make an agent write the query for us and use that query ?
@imbajin sir what is your opinion on this?

[Aryankb]

Also @imbajin sir just wanted to know , what is the actual requirement.

• Is it direct prompt -->information
  (where user just write a prompt, subtasks will be divided and agents gets created based on the knowledge graph structure)

• or user will have capability to create and orchestrate his own agents ( to extract specific information from graph)

And also where we are currently ?