Agora Deepgram Voice Agent

A standalone voice-agent project with a Next.js web client and Python FastAPI backend. Agora carries RTC audio and RTM events; the agent uses a fixed Deepgram Nova-3 English STT, managed OpenAI gpt-4o-mini, and Deepgram Aura-2 Thalia TTS pipeline.

Prerequisites

• Python 3.10+
• Bun
• An Agora project with App ID, App Certificate, RTC, RTM, and Conversational AI enabled
• A Deepgram API key

Configure

Install dependencies:

bun install
cd server
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt -r requirements-dev.txt
cd ..

Create server/.env.local yourself from the variables documented in server/.env.example:

AGORA_APP_ID=your_agora_app_id
AGORA_APP_CERTIFICATE=your_agora_app_certificate
DEEPGRAM_API_KEY=your_deepgram_api_key
AGENT_GREETING=Hi there! I'm Ada, your virtual assistant from Agora. How can I help?
PORT=8001

The backend also accepts the older Agora CLI aliases APP_ID and APP_CERTIFICATE. AGENT_BACKEND_URL is a web-server variable; local root scripts set it to http://localhost:8001.

Run

bun run dev

• Web: http://localhost:3000
• Backend: http://localhost:8001
• Backend API docs: http://localhost:8001/docs

API Contract

The browser calls Next.js /api/* URLs, which are rewritten to FastAPI through AGENT_BACKEND_URL.

GET  /api/get_config
POST /api/startAgent  { channelName, rtcUid, userUid, parameters? }
POST /api/stopAgent   { agentId }

/startAgent has no provider selector in either request or response. The speech pipeline is fixed by server/src/agent.py.

Verify

No live Agora session is required for these checks:

bun run verify:backend
cd server && pytest tests -v
cd ../web && bun test
cd .. && bun run verify:web:api
bun run verify:web:proxy

Checks that need installed local backend dependencies or a production web build:

bun run verify:local:fastapi
bun run verify:web:build
bun run verify:local

Deploy

Deploy web/ to a Next.js-compatible host and set:

AGENT_BACKEND_URL=https://your-fastapi-service.example.com

Deploy the root Dockerfile or server/ separately with the five backend variables above. The container exposes port 8001.

Project Map

• web/: Next.js UI, RTC/RTM lifecycle, transcripts, connection state, and fixed pipeline presentation
• server/: FastAPI token and agent lifecycle service
• server/src/agent.py: fixed Deepgram and managed OpenAI agent configuration
• ARCHITECTURE.md: request and session flow
• AGENTS.md: contributor instructions

Troubleshooting

• Backend initialization fails: verify Agora credentials and DEEPGRAM_API_KEY are real, non-placeholder values.
• Browser API calls return 404: set AGENT_BACKEND_URL for the Next.js process.
• Agent does not join or transcripts are missing: verify RTC, RTM, and Conversational AI are enabled for the Agora project.
• Deepgram synthesis fails: verify the backend can reach wss://api.deepgram.com and the API key is valid.

License

Released under the MIT License.

deepgram-agent