About the project

This started as a small testbed to quickly prototype and validate features while building the Marfa AI assistant platform. Over time it has become a practical companion app that runs in the background during development, letting me query it whenever needed.

I’ve expanded it with capabilities such as wake-word activation so it stays idle unless triggered—and built-in web search for pulling recent information. I’ll continue extending it in parallel with the main platform as new features are developed.

Current features:

• General conversations
• Web search for latest news, weather or any real-time info
• Can search/manage your gmail and google calendar (Composio configuration needed)

Dev setup

Clone the repository and install dependencies to a virtual environment:

cd voice_ai_assistant
uv sync

Create .env.local file based on .env.example file and add required keys

Livekit can local hosted or cloud. Get the Livekit keys

• LIVEKIT_URL
• LIVEKIT_API_KEY
• LIVEKIT_API_SECRET

Get other required keys for TTS, STT and LLM (including Perplexity). OR You can use livekit inference so you won't need separate api keys, just livekit cloud keys will be enough. TTS, STT and LLM can be changed easily.. more here

For Gmail and google calendar I am setting up the integration via Composio. So you need to setup your account and authenticate gmail and calendar there. You can ignore this if you don't want these integration.. Just remove get_composio_tools() from Assistant class. If you want to try that, once all setup is done in composio side get the API Key and add that on COMPOSIO_API_KEY

For easy access, I have also hardcoded user id fom composio COMPOSIO_USER_ID. Authenticate your user in composio dashboard, get user_id and add that in .env.local and you are good to go.

You can extend the capabilities of assistant by creating new tool or using MCP integration

Run the Assistant

Before your first run, you must download certain models such as Silero VAD and the LiveKit turn detector:

uv run python src/agent.py download-files

Next, run this command to speak to your agent directly in your terminal:

uv run python src/agent.py console