Add databricks-agents framework for building discoverable agents

[stuagano]

Add databricks-agents framework for building discoverable agents

Summary

Adds databricks-agents, a lightweight Python framework for building discoverable AI agents on Databricks Apps. This framework makes it trivial to turn any Databricks App into a standards-compliant agent with auto-generated A2A protocol endpoints.

What Problem Does This Solve?

Before: Building agents on Databricks required:

• Manual A2A protocol implementation
• No standard way to make apps discoverable
• Complex agent-to-agent communication setup
• Agents tied to Model Serving endpoints

After: With databricks-agents:

• 5 lines of code to create an agent
• Auto-generated discovery endpoints
• Built-in workspace and UC discovery
• Agents can be full applications (not just serving endpoints)

Key Innovation: Agent = App

This framework treats Databricks Apps as first-class agents, enabling:

• Full application logic with custom UI
• Stateful operations and workflows
• Integration with Databricks data, UC, and AI services
• Standards-based discovery and communication

Example Usage

from databricks_agents import AgentApp

# Create agent (5 lines!)
app = AgentApp(
    name="customer_research",
    description="Research customer information",
    capabilities=["search", "analysis"],
)

@app.tool(description="Search companies")
async def search_companies(industry: str) -> dict:
    return {"results": [...]}

# Deploy to Databricks Apps → Auto-registered in UC!

This automatically provides:

• ✅ /.well-known/agent.json - A2A protocol agent card
• ✅ /.well-known/openid-configuration - OIDC delegation
• ✅ /health - Health check endpoint
• ✅ /api/mcp - MCP server for tools
• ✅ /api/tools/<tool_name> - Tool endpoints
• ✅ Unity Catalog registration

Framework Components

Core (src/databricks_agents/core/)

• agent_app.py - AgentApp class (FastAPI wrapper with agent capabilities)

Discovery (src/databricks_agents/discovery/)

• agent_discovery.py - Workspace scanning for agent-enabled apps
• a2a_client.py - A2A protocol client for agent communication

Registry (src/databricks_agents/registry/)

• uc_registry.py - Register agents in Unity Catalog as AGENT objects

MCP (src/databricks_agents/mcp/)

• mcp_server.py - Model Context Protocol server
• uc_functions.py - Discover and expose UC Functions as tools

CI/CD & Documentation

Workflows (.github/workflows/)

• test.yml - Automated testing (Python 3.10-3.12, linting, coverage)
• publish.yml - PyPI publishing on releases
• docs.yml - GitHub Pages documentation deployment

Documentation (docs/)

• MkDocs Material theme
• Quick Start guide
• API reference structure
• Complete examples

Examples

• customer_research_agent.py - Basic agent with custom tools
• discover_agents.py - Workspace agent discovery
• communicate_with_agent.py - A2A protocol communication
• full_featured_agent.py - Complete example with all features

Testing

cd databricks-agents
pip install -e ".[dev]"
pytest tests/ -v

All tests pass with >80% coverage.

Project Structure

databricks-agents/
├── src/databricks_agents/        # Core framework (~2000 LOC)
│   ├── core/                      # AgentApp, tool registration
│   ├── discovery/                 # Agent discovery, A2A client
│   ├── mcp/                       # MCP server, UC Functions
│   ├── registry/                  # Unity Catalog integration
│   └── orchestration/             # (Future: multi-agent patterns)
├── examples/                      # 4 complete examples
├── tests/                         # Test suite
├── docs/                          # MkDocs documentation
├── .github/workflows/             # 3 CI/CD workflows
├── README.md                      # Comprehensive documentation
├── CONTRIBUTING.md                # Contribution guidelines
├── LICENSE                        # Apache 2.0
└── pyproject.toml                 # Package configuration

Why Sandbox?

This is the perfect sandbox project because it:

✅ Early-stage but valuable - Works today, provides immediate value
✅ Innovative approach - New pattern for agent building on Databricks
✅ Community-driven - Ideal for gathering feedback and contributions
✅ Low friction - 5 lines to create an agent
✅ Building block - Foundation for multi-agent systems

Graduation Path

Sandbox (0.1.x - 0.5.x)

• Community validation and feedback
• Real-world usage patterns
• Feature stabilization

Full Repo (1.0+) - When proven adoption:

• 100+ GitHub stars
• 1000+ PyPI downloads/month
• 10+ contributors
• 5+ community examples

Platform Integration - Long term:

• Influence native Databricks agent features
• UC AGENT type (when available)
• Built-in orchestration primitives

Dependencies

All standard, well-maintained packages:

• fastapi>=0.115.0
• uvicorn[standard]>=0.30.0
• pydantic>=2.0.0
• httpx>=0.27.0
• databricks-sdk>=0.30.0

Checklist

• Code is complete and tested
• Documentation is comprehensive
• Examples work end-to-end
• CI/CD workflows configured
• Apache 2.0 license
• CONTRIBUTING.md included
• manifest.yaml created
• All files follow sandbox conventions

Related Work

Builds on Databricks platform primitives:

• Databricks Apps
• Unity Catalog
• Databricks SDK
• Workspace OIDC

Implements open standards:

• A2A Protocol
• MCP (Model Context Protocol)

Questions?

Happy to address any feedback or concerns! This framework solves a real gap in the Databricks agent ecosystem and provides immediate value to users building multi-agent systems.

[github-actions]

All commits in PR should be signed ('git commit -S ...'). See https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits