Skip to content

Improve code coverage of mcp-agent #215

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 93 commits into from
May 28, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
93 commits
Select commit Hold shift + click to select a range
c1dec1c
fix: Rename and update content conversion functions for OpenAI integr…
StreetLamb May 13, 2025
53564e3
Refactor LLM provider tests and remove outdated event progress test
StreetLamb May 13, 2025
9b0e076
add support for Azure Identity instead of keys (#190)
demoray May 13, 2025
8f05f91
Add streamable HTTP support to mcp-agent (#193)
saqadri May 13, 2025
b97fd35
fix: Conditionally include toolConfig in BedrockAugmentedLLM argument…
StreetLamb May 14, 2025
da1554e
fix get prompt no return (#194)
zmetry May 14, 2025
8779dfb
fix: Remove unused imports from test_augmented_llm_openai.py
StreetLamb May 14, 2025
3f5d453
test: Add comprehensive tests for AzureAugmentedLLM functionality
StreetLamb May 14, 2025
4edc57f
feat: Add pytest-cov for test coverage reporting
StreetLamb May 14, 2025
75c85e5
fix: Update installation instructions in README
StreetLamb May 14, 2025
741cfbd
fix: Set model to an empty string in MCPAzureTypeConverter
StreetLamb May 14, 2025
dadcf02
fix: Update coverage command examples in README for clarity
StreetLamb May 14, 2025
47a7d8e
fix: Correct bedrock content and mcp message param handling in Bedroc…
StreetLamb May 15, 2025
9211913
test: Add comprehensive tests for BedrockAugmentedLLM functionality
StreetLamb May 15, 2025
2ceb8dc
fix: correct role handling and set model to empty string in GoogleMCP…
StreetLamb May 15, 2025
d38b77c
test: Add comprehensive tests for GoogleAugmentedLLM functionality
StreetLamb May 15, 2025
1b39119
refactor: Update import statement for OpenAI in OllamaAugmentedLLM
StreetLamb May 15, 2025
fcbc090
test: Add unit tests for OllamaAugmentedLLM functionality
StreetLamb May 15, 2025
32745f4
fix: enable content conversion handling for pydantic models and dict …
StreetLamb May 15, 2025
25ea5ee
test: Enhance tests for AnthropicAugmentedLLM
StreetLamb May 15, 2025
a2d1dcf
Merge branch 'main' of https://github.com/lastmile-ai/mcp-agent into …
StreetLamb May 15, 2025
4589214
test: Add unit tests for Agent class and mock fixtures
StreetLamb May 15, 2025
6319a8b
test: Add comprehensive unit tests for MCPApp class functionality
StreetLamb May 15, 2025
272fa69
test: Add unit tests for OpenAIEmbeddingIntentClassifier and CohereEm…
StreetLamb May 16, 2025
ae4fa5d
test: Add unit tests for OpenAILLMIntentClassifier and AnthropicLLMIn…
StreetLamb May 16, 2025
f3a7e0c
test: Add tests for orchestrator workflow module
StreetLamb May 16, 2025
ef01720
fix: Validate plan_type in Orchestrator constructor to ensure it is e…
StreetLamb May 16, 2025
21465b6
Merge branch 'main' of https://github.com/lastmile-ai/mcp-agent into …
StreetLamb May 16, 2025
2ca05d0
test: Add unit tests for FanOut and ParallelLLM classes
StreetLamb May 17, 2025
8d4386d
test: Add tests for embedding routers and llm routers
StreetLamb May 17, 2025
749cad4
fix: Await embedding computation in EmbeddingRouter and fix EmeddingR…
StreetLamb May 17, 2025
e570ee7
test: Add unit tests for swarm workflows
StreetLamb May 18, 2025
6c3a9cb
fix: Change AgentFunctionResult to AgentFunctionResultResource in Swa…
StreetLamb May 18, 2025
f3ded33
test: Add tests for workflow signals
StreetLamb May 19, 2025
77ab6b4
fix: Generate unique signal name in AsyncioSignalHandler
StreetLamb May 19, 2025
e13dec4
test: Add tests for MCPAggregator
StreetLamb May 19, 2025
86c7e41
fix: Reset initialized flag in MCPAggregator when connection persiste…
StreetLamb May 19, 2025
34fa51c
fix: Add trio dependency for 'dev'
StreetLamb May 19, 2025
1b9ddc7
test: Add tests for MCPConnectionManager
StreetLamb May 19, 2025
e7aa41f
fix: Improve error handling and tool call execution in AzureAugmentedLLM
StreetLamb May 21, 2025
6f5e24f
test: Add additional tests for AzureAugmentedLLM
StreetLamb May 21, 2025
921a1bd
Merge branch 'main' of https://github.com/lastmile-ai/mcp-agent into …
StreetLamb May 21, 2025
404c3a3
fix: Update import paths for Context and ServerRegistry
StreetLamb May 22, 2025
e5a96ce
fix: Update validators in NestedLocation and ComplexModel to use clas…
StreetLamb May 22, 2025
d2b218c
fix: Update agent tests due to changes in agents.py
StreetLamb May 22, 2025
c927399
fix: Initialize event queue in AsyncEventBus constructor
StreetLamb May 22, 2025
63d3827
fix: Update MCPApp tests
StreetLamb May 22, 2025
ac8fbb7
fix: Improve PydanticTypeSerializer serializing and deserializing logic
StreetLamb May 22, 2025
da76d19
fix: Update PydanticTypeSerializer tests
StreetLamb May 22, 2025
8fefd4e
fix: Remove unused AsyncMock import and update LLM instantiation in t…
StreetLamb May 24, 2025
7f33dc1
fix: Fix broken AzureAugmentedLLM tests
StreetLamb May 24, 2025
fd6dffe
fix: Conditionally pass instruction to Agent only if not None
StreetLamb May 24, 2025
3b28ad2
fix: fix broken AnthropicAugmentedLLM tests
StreetLamb May 24, 2025
f312076
fix: fix broken openai and google tests
StreetLamb May 24, 2025
2d995f6
fix: fix broken BedrockAugmentedLLM tests
StreetLamb May 24, 2025
5a85857
fix: update broken OllamaAugmentedLLM tests
StreetLamb May 24, 2025
0fb1eaf
fix: initialize synthesizer in Orchestrator and correct typo in gener…
StreetLamb May 24, 2025
56a973d
fix: update broken orchestrator tests
StreetLamb May 24, 2025
c5a01be
fix: update broken orchestrator and fan out tests
StreetLamb May 25, 2025
ec5f367
fix: update mocks to fix router tests
StreetLamb May 25, 2025
31ce0e9
fix: remove unused imports and clean up test files
StreetLamb May 25, 2025
ec59981
fix: update broken swarm tests
StreetLamb May 25, 2025
66e1915
fix: set default values for server_names and functions in SwarmAgent
StreetLamb May 25, 2025
cf75ef5
test: add unit tests for TemporalSignalHandler, TemporalExecutor, and…
StreetLamb May 25, 2025
bb74646
test: add unit tests for WorkflowState, WorkflowResult and Workflow c…
StreetLamb May 25, 2025
044a5fd
test: add unit tests for EvaluatorOptimizerLLM and related components
StreetLamb May 25, 2025
f75a5c6
Merge branch 'main' of https://github.com/lastmile-ai/mcp-agent into …
StreetLamb May 25, 2025
e72b80f
test: update mock_context to include tracer attributes in various tests
StreetLamb May 25, 2025
0acd277
fix: Ensure private attributes are initialized in PydanticTypeSerializer
StreetLamb May 25, 2025
5881f5a
fix: Update response usage access to dictionary format in AzureAugmen…
StreetLamb May 25, 2025
2ad4a63
fix: Correct typo in synthesizer method call in Orchestrator class
StreetLamb May 25, 2025
65de0b2
fix: Append ToolMessage results to responses in AzureAugmentedLLM
StreetLamb May 25, 2025
4284d6d
refactor: Remove unused imports from test fixtures in conftest.py and…
StreetLamb May 25, 2025
bab2afa
fix: Move pytest-cov dependency to the correct section in pyproject.toml
StreetLamb May 25, 2025
672c895
refactor: Improve embedding generation in MockEmbeddingModel for bett…
StreetLamb May 25, 2025
7f6fa4b
fix: Update default parameters for server_names and functions in Swar…
StreetLamb May 25, 2025
20c083e
fix: Update import paths in test_mcp_connection_manager.py to remove …
StreetLamb May 25, 2025
cff4a3e
test: Add test_resume_workflow_signal_error test case
StreetLamb May 25, 2025
91d96d6
fix: Add explicit assertions for workflow state transitions in test_r…
StreetLamb May 25, 2025
2f2d3ec
refactor: Simplify ParallelLLM fixture setup and improve assertions i…
StreetLamb May 25, 2025
fc23a56
fix: Improve randomness in mock embedding generation and enhance asse…
StreetLamb May 25, 2025
8aac5c4
fix: Make embed_side_effect an async coroutine to honour the protocol
StreetLamb May 25, 2025
931c1e3
fix: Await asynchronous mock methods in MockAugmentedLLM class
StreetLamb May 25, 2025
be8e799
fix: Add agent_name to CallToolRequest in test_swarm for improved con…
StreetLamb May 25, 2025
c43e2b9
fix: Refactor test_call_tool methods to use real SwarmAgent instance …
StreetLamb May 25, 2025
0c4a532
fix: Initialize context in TestSwarmAgent
StreetLamb May 25, 2025
5afb263
fix: Update content formatting in OpenAIAugmentedLLM to use list comp…
StreetLamb May 25, 2025
8282269
fix: Remove unused mock_aggregator fixture from LLM test classes
StreetLamb May 25, 2025
c12c995
fix: Handle non-JSON-serialisable Literal values in PydanticTypeSeria…
StreetLamb May 25, 2025
4d3ab76
fix: Preserve compatibility with Pydantic v1 for handling of required…
StreetLamb May 25, 2025
5a56dff
fix: Remove mock_context fixture and update tests to use Context dire…
StreetLamb May 25, 2025
847062f
fix: Refactor initialization tests for OllamaAugmentedLLM to improve …
StreetLamb May 25, 2025
34f29c6
fix: Refactor DummyClient to DummyRegistry for non-persistent session…
StreetLamb May 25, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,8 @@ dev = [
"pytest>=7.4.0",
"pytest-asyncio>=0.21.1",
"boto3-stubs[bedrock-runtime]>=1.37.23",
"trio>=0.30.0",
"pytest-cov>=6.1.1",
"httpx>=0.28.1",
]

Expand Down
12 changes: 8 additions & 4 deletions src/mcp_agent/executor/workflow_signal.py
Original file line number Diff line number Diff line change
Expand Up @@ -225,11 +225,11 @@ async def wait_for_signal(
self, signal, timeout_seconds: int | None = None
) -> SignalValueT:
event = asyncio.Event()
unique_name = str(uuid.uuid4())
unique_signal_name = f"{signal.name}_{uuid.uuid4()}"

registration = SignalRegistration(
signal_name=signal.name,
unique_name=unique_name,
unique_name=unique_signal_name,
workflow_id=signal.workflow_id,
run_id=signal.run_id,
)
Expand Down Expand Up @@ -257,20 +257,24 @@ async def wait_for_signal(
self._pending_signals[signal.name] = [
ps
for ps in self._pending_signals[signal.name]
if ps.registration.unique_name != unique_name
if ps.registration.unique_name != unique_signal_name
]
if not self._pending_signals[signal.name]:
del self._pending_signals[signal.name]

def on_signal(self, signal_name):
def decorator(func):
unique_signal_name = f"{signal_name}_{uuid.uuid4()}"

async def wrapped(value: SignalValueT):
if asyncio.iscoroutinefunction(func):
await func(value)
else:
func(value)

self._handlers.setdefault(signal_name, []).append(wrapped)
self._handlers.setdefault(signal_name, []).append(
[unique_signal_name, wrapped]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean _handlers is a tuple?

)
return wrapped

return decorator
Expand Down
1 change: 1 addition & 0 deletions src/mcp_agent/logging/transport.py
Original file line number Diff line number Diff line change
Expand Up @@ -273,6 +273,7 @@ def __init__(self, transport: EventTransport | None = None):
self.listeners: Dict[str, EventListener] = {}
self._task: asyncio.Task | None = None
self._running = False
self.init_queue()

def init_queue(self):
if self._running:
Expand Down
6 changes: 4 additions & 2 deletions src/mcp_agent/mcp/mcp_aggregator.py
Original file line number Diff line number Diff line change
Expand Up @@ -187,6 +187,7 @@ async def close(self):
not self.connection_persistence
or not self._persistent_connection_manager
):
self.initialized = False
return

try:
Expand Down Expand Up @@ -979,8 +980,9 @@ def getter(item: NamespacedPrompt):
else:
raise ValueError(f"Unsupported capability: {capability}")

# Search across all servers
for srv_name, items in capability_map.items():
# Search servers in the order of self.server_names
for srv_name in self.server_names:
items = capability_map.get(srv_name, [])
for item in items:
if getter(item) == name:
return srv_name, name
Expand Down
108 changes: 79 additions & 29 deletions src/mcp_agent/utils/pydantic_type_serializer.py
Original file line number Diff line number Diff line change
Expand Up @@ -181,6 +181,14 @@ def serialize_type(typ: Any) -> Dict[str, Any]:
origin = get_origin(typ)
if origin is not None:
args = get_args(typ)
# Special handling for Literal: store raw values, not types
if origin is Literal:
return {
"kind": "generic",
"origin": "Literal",
"literal_values": [make_serializable(a) for a in args],
"repr": str(typ),
}
serialized_args = [
PydanticTypeSerializer.serialize_type(arg) for arg in args
]
Expand Down Expand Up @@ -292,7 +300,7 @@ def _serialize_validators(model_class: Type[BaseModel]) -> List[Dict[str, Any]]:
name,
validator,
) in model_class.__pydantic_decorators__.field_validators.items():
field_names = [str(f) for f in validator.fields]
field_names = [str(f) for f in validator.info.fields]
validators.append(
{
"type": "field_validator",
Expand Down Expand Up @@ -382,9 +390,11 @@ def _serialize_fields(model_class: Type[BaseModel]) -> Dict[str, Dict[str, Any]]
"description": make_serializable(
getattr(field_info, "description", None)
),
"required": make_serializable(
getattr(field_info, "required", True)
),
"required": getattr(
field_info,
"is_required",
lambda: getattr(field_info, "required", True),
)(),
}

# Add constraints if defined
Expand All @@ -411,10 +421,10 @@ def _serialize_fields(model_class: Type[BaseModel]) -> Dict[str, Dict[str, Any]]
else:
default = make_serializable(default)

# Use type_ if available (Pydantic v2), else fallback to Any
attr_type = getattr(private_attr, "type_", Any)
private_attrs[name] = {
"type": PydanticTypeSerializer.serialize_type(
private_attr.annotation
),
"type": PydanticTypeSerializer.serialize_type(attr_type),
"default": default,
}

Expand All @@ -436,7 +446,18 @@ def _serialize_config(model_class: Type[BaseModel]) -> Dict[str, Any]:
else:
return config_dict

# Extract serializable config values
# If config_source is a dict or ConfigDict (Pydantic v2), just copy its items
if isinstance(config_source, dict):
for key, value in config_source.items():
if not str(key).startswith("_"):
try:
json.dumps({key: value})
config_dict[key] = value
except (TypeError, OverflowError):
config_dict[key] = str(value)
return config_dict

# Otherwise, use inspect.getmembers (for class-based config)
for key, value in inspect.getmembers(config_source):
if (
not key.startswith("_")
Expand Down Expand Up @@ -516,6 +537,12 @@ def deserialize_type(serialized: Dict[str, Any]) -> Any:
elif kind == "generic":
# Handle generics like List[int], Dict[str, Model], etc.
origin_name = serialized["origin"]

# Special handling for Literal: use literal_values if present
if origin_name == "Literal" and "literal_values" in serialized:
literal_values = serialized["literal_values"]
return Literal.__getitem__(tuple(literal_values))

args = [
PydanticTypeSerializer.deserialize_type(arg)
for arg in serialized["args"]
Expand Down Expand Up @@ -621,15 +648,13 @@ def reconstruct_model(serialized: Dict[str, Any]) -> Type[BaseModel]:
# Get the field type
field_type = PydanticTypeSerializer.deserialize_type(field_info["type"])

# Handle default values
# Determine if the field is required
is_required = field_info.get("required", True)
default = field_info.get("default", ...)
if default is None and not field_info.get("required", True):
default = None

# Handle default factories
default_factory = field_info.get("default_factory")

# This logic ensures that fields with a default or default_factory are not required
if default_factory:
# This is a simplification - ideally we'd recreate the actual factory
if default_factory == "list":
default_factory = list
elif default_factory == "dict":
Expand Down Expand Up @@ -658,25 +683,61 @@ def reconstruct_model(serialized: Dict[str, Any]) -> Type[BaseModel]:

# Add the field definition
if constraints or default_factory:
# If there is a default_factory, always use default=... and set default_factory
field_definitions[field_name] = (
field_type,
Field(
default=default if default_factory is None else ...,
default=... if default_factory is not None else default,
default_factory=default_factory,
**constraints,
),
)
else:
field_definitions[field_name] = (field_type, default)
if is_required:
field_definitions[field_name] = (field_type, Field(default=...))
else:
field_definitions[field_name] = (
field_type,
Field(
default=default,
),
)

Comment on lines 687 to 705
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Avoid passing default_factory=None & reconcile required logic

Field(..., default_factory=None) is semantically meaningless and may
raise in future Pydantic releases.
Also, when is_required=False but default is still ..., the field
remains required.

-                    Field(
-                        default=... if default_factory is not None else default,
-                        default_factory=default_factory,
+                    kwargs = {**constraints}
+                    if default_factory is not None:
+                        kwargs["default_factory"] = default_factory
+                        kwargs["default"] = ...
+                    elif default is not ...:
+                        kwargs["default"] = default
+                    else:
+                        kwargs["default"] = ...
+                    Field(**kwargs),

Committable suggestion skipped: line range outside the PR's diff.

🤖 Prompt for AI Agents
In src/mcp_agent/utils/pydantic_type_serializer.py around lines 683 to 701,
avoid passing default_factory=None to Field as it is meaningless and may cause
errors in future Pydantic versions. Also, ensure that when is_required is False,
the default is not set to ... which makes the field required. Fix this by only
passing default_factory if it is not None and by setting default to a proper
non-required value when is_required is False.

# Create model config
model_config = ConfigDict(**config_dict) if config_dict else None

# Create the basic model
# Collect private attributes to pass to create_model
private_attr_kwargs = {}
if "__private_attrs__" in fields:
for name, attr_info in fields["__private_attrs__"].items():
default = attr_info.get("default")
if default == "None":
default = None
private_attr_kwargs[name] = PrivateAttr(default=default)

# Create the basic model, including private attributes in the class namespace
reconstructed_model = create_model(
name, __config__=model_config, **field_definitions
name, __config__=model_config, **field_definitions, **private_attr_kwargs
)

# Patch __init__ to ensure private attributes are initialized on instance
private_attrs = getattr(reconstructed_model, "__private_attributes__", {})
if private_attrs:
orig_init = reconstructed_model.__init__

def _init_with_private_attrs(self, *args, **kwargs):
orig_init(self, *args, **kwargs)
for attr_name, private_attr in private_attrs.items():
# Only set if not already set
if not hasattr(self, attr_name):
default = private_attr.default
# If default is ... (Ellipsis), treat as None
if default is ...:
default = None
setattr(self, attr_name, default)

reconstructed_model.__init__ = _init_with_private_attrs
Comment on lines +723 to +739
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Monkey-patching __init__ may break dataclass-like signatures

Overwriting __init__:

  1. Removes the rich signature Pydantic generates (hurts tooling &
    validation).
  2. Breaks multiple inheritance if models are later subclassed.
  3. Runs for every reconstructed model, even when Pydantic would have
    already initialised private attrs (v1 & v2).

Prefer letting Pydantic handle private attributes:

-        private_attrs = getattr(reconstructed_model, "__private_attributes__", {})
-        if private_attrs:
-            orig_init = reconstructed_model.__init__
-            def _init_with_private_attrs(self, *args, **kwargs):
-                orig_init(self, *args, **kwargs)
-                ...
-            reconstructed_model.__init__ = _init_with_private_attrs
+        # Pydantic takes care of initialising PrivateAttr defaults; no patch needed.

If a specific bug required this hook, add a comment with a minimal
reproduction and scope the patch to that scenario.

Committable suggestion skipped: line range outside the PR's diff.

🤖 Prompt for AI Agents
In src/mcp_agent/utils/pydantic_type_serializer.py around lines 719 to 735, the
current approach monkey-patches the __init__ method of reconstructed_model to
initialize private attributes, which removes Pydantic's rich __init__ signature,
breaks multiple inheritance, and runs unnecessarily for all models. To fix this,
remove the __init__ monkey-patching entirely and rely on Pydantic's built-in
handling of private attributes. If a specific bug requires this patch, add a
detailed comment explaining the minimal reproduction and limit the patch only to
that case.


# Add validators (this gets complex and may require exec/eval)
if validators:
for validator in validators:
Expand Down Expand Up @@ -719,17 +780,6 @@ def reconstruct_model(serialized: Dict[str, Any]) -> Type[BaseModel]:
except Exception as e:
logger.error(f"Error recreating validator: {e}")

# Add private attributes (simplified)
if "__private_attrs__" in fields:
for name, attr_info in fields["__private_attrs__"].items():
attr_type = PydanticTypeSerializer.deserialize_type(attr_info["type"])
default = attr_info.get("default")
if default == "None":
default = None
reconstructed_model.__private_attributes__[name] = PrivateAttr(
default=default, annotation=attr_type
)

return reconstructed_model

@classmethod
Expand Down
7 changes: 6 additions & 1 deletion src/mcp_agent/workflows/llm/augmented_llm.py
Original file line number Diff line number Diff line change
Expand Up @@ -253,7 +253,12 @@ def __init__(

self.agent = Agent(
name=self.name,
instruction=self.instruction,
# Only pass instruction if it's not None
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are there cases where instruction is None? Should we fail early in that case?

**(
{"instruction": self.instruction}
if self.instruction is not None
else {}
),
server_names=server_names or [],
llm=self,
)
Expand Down
28 changes: 14 additions & 14 deletions src/mcp_agent/workflows/llm/augmented_llm_anthropic.py
Original file line number Diff line number Diff line change
Expand Up @@ -953,27 +953,27 @@ def anthropic_content_to_mcp_content(
mcp_content.append(TextContent(type="text", text=content))
else:
for block in content:
if block.type == "text":
mcp_content.append(TextContent(type="text", text=block.text))
elif block.type == "image":
# Handle pydantic models (ContentBlock) and dict blocks
if isinstance(block, BaseModel):
block_type = block.type
block_text = block.text
else:
block_type = block["type"]
block_text = block["text"]

if block_type == "text":
mcp_content.append(TextContent(type="text", text=block_text))
elif block_type == "image":
raise NotImplementedError("Image content conversion not implemented")
elif block.type == "tool_use":
# Best effort to convert a tool use to text (since there's no ToolUseContent)
mcp_content.append(
TextContent(
type="text",
text=to_string(block),
)
)
elif block.type == "tool_result":
# Best effort to convert a tool result to text (since there's no ToolResultContent)
elif block_type == "tool_use" or block_type == "tool_result":
# Best effort to convert a tool use and tool result to text (since there's no ToolUseContent or ToolResultContent)
mcp_content.append(
TextContent(
type="text",
text=to_string(block),
)
)
elif block.type == "document":
elif block_type == "document":
raise NotImplementedError("Document content conversion not implemented")
else:
# Last effort to convert the content to a string
Expand Down
Loading
Loading