MCP servers
Docs on building MCP servers and defining the external platforms at Gaby AI's disposal.
This section details Gaby's MCP architecture. It is Gaby's personal action space, and allows connections to external platforms and a web browser.
Requirements & Usage
| Python Library | Usage / Role |
|---|---|
livekit | Handles text-to-speech (TTS) and speech-to-text (STT) for user interactions. |
mcp-agent-servers | Equips Gaby with function tools via API routes to external platforms (or, in more modern terms, AI agents). |
huggingface-cli | Gives access to open-source machine learning models on Hugging Face for inference. |
Main Challenge
The main objective is to control the agent's responses over time, so the websocket wrappers and handler functions must adapt to the livekit framework.
Database Selection
For the sake of prototyping Gaby as an application for the Google BigQuery competition, Google Storage was used.
MCP Servers & Usage in Gaby
| MCP Agent (Source Link) | Usage |
|---|---|
| Data Visualization -https://github.com/antvis/mcp-server-chart?tab=readme-ov-file. | For visualizing data insights upon users' request. |
| Jira Integration - https://github.com/tom28881/mcp-jira-server | Monitor Team members' workflow for teams running on Jira. |
| Notion Integration | Used as Agent's Workspace to document and report data findings to user. |
Backend API
A backend API built with FastAPI and static HTML/CSS, streaming the agent's actions and decision-making. This endpoint unifies Gaby's services, database authorisation, management and ML workflows.
2025 Cloud Services Comparison & Usage
| Source / Server | Usage Relative to AI Agent Stack | Usage Relative to All Other |
|---|---|---|
| Ollama |
|
|
| Google Stack BigQuery and Vertex AI | This serves as Gaby’s database and memory Gatekeeper. Its main responsibilities include assisting with service workflows, storing analytics from his observations, and enabling fast knowledge retrieval, such as entity relationships. |
|
| Google Vertex AI |
|
|
| Lightning Studio |
|
|
| Hugging Face Space |
|
|
| Google: Firestore Database | Used this for my Tarot Reading Agent so this is my go-to at the moment but BigQuery AI is a more scalable solution (i.e. will not have to worry that much about database setups / runs) | |
| Google API keys to workspace e.g. Google Sheets | These keys are used for users to view their cleaned dataset in tabular form. This is also handy in connecting IP subnets to users' database e.g. MongoDB. |
Service checklist
Gaby AI's data services, as a law-abiding citizen.
Gaby as a standalone app summary Checklist
The following table defines the initial architecture checklist for the API server / Model tuning behind Gaby's available services as the final product.
Gaby Application Services
| Action | Description | Multi-agent network? Which platform? | Gaby Feedback loop? i.e. whether this collected data is used in Gaby’s reward based learning system | Gaby ML models required? How many? | Data (e.g. Insight) Cache required? i.e. Whether the fortnightly insights are cached in a database. | LLM Prompt Stack Required? How many? | Gaby Temporal Memory required? (Temporal - in the last 2-4 weeks) | Gaby Long Term Memory required? |
|---|---|---|---|---|---|---|---|---|
| Data Quality Summary / Sprint Dashboard | Multi-agent Connection + Retrieve Data fields metrics and score (based on custom benchmark) | Yes, dependent database | No | No | Yes | Yes, 3 | Yes | Yes |
| Software Team Summary / Sprint Dashboard e.g. Software Developer team, Data team | e.g. Team Members Work Progression per sprint | Yes, Jira, Slack. | No | No | No | Yes, 3 | Yes | Yes |
| Finance Reporting & Accounting | Company’s Ingoing and outgoing | Maybe | No | No | Yes | Yes | Yes | Yes |
| Engagement & Behavior | -- | No | Yes | Yes, 2 → human behaviour mimicking, human inference | Yes | Yes, minimum 5 and 1 for backup [Base] | No | Yes |
| Entire Company Tech Stack Projects / Database | Company’s general monitoring | Yes, Jira, Slack, Github, Notion, Confluence | Yes | Yes, 2 → Recommendation system built on Temporal memory for faster data retrieval specific to the company, Entity graph of company’s users | Yes | No | Yes | Yes |
| End of Week Documentation for employee / teams | Recommendation system that is used to store | Yes, Typeform, Notion, Confluence etc. | No | Yes, 1 → Recommendation System | Yes | Yes, ~5 | Yes | Yes |
API integration checklist
This section outlines potential integration partners and platform targets for Gaby AI, focusing on API compatibility, ecosystem synergy, and deployment scalability.
The following table tracks my targeted platforms for partnerships and integration into Gaby's services.
| Platform type | External platform | Usage | Partnered? | Integrated? |
|---|---|---|---|---|
| Database | Google Workspace e.g. Google Sheets / Docs | Data reporting and insights | ✅ | ❌ |
| Cloud | Google BigQuery AI/ML | AI/ML schedulers; internal & external database management | ✅ | ❌ |
| Cloud | Azure / Microsoft: Power BI (legacy system and skewed market because it is Power BI) | Data reporting and insights | ❌ | ❌ |
| Database | MongoDB | Database connection endpoint | ❌ | ❌ |
| Database | Supabase | Database connection endpoint | ❌ | ❌ |
| Project management | Notion | Marketing/exposure; team documentation | ✅ | ✅ |
| Project management | Atlassian e.g. Jira | Marketing/exposure; team documentation | ✅ | ❌ |
| Database | HuggingFace | Internal database handling & workflows; access data to demo Gaby's services | ❌ | ❌ |
| Project management | TypeForm | Documentation; marketing/exposure | ✅ | ❌ |
| Community networks | Github MCP Network Hub & Copilot | Marketing/exposure | ❌ | ❌ |
| Community networks | Kaggle | Marketing/exposure; access data to demo Gaby's services | ✅ | ✅ |
| Cloud | AWS Bedrock & Sagemaker | AI/ML schedulers; internal & external database management; local serving MCP networks (with security!); Jupyter Notebook servers as AI Agent's playground | ❌ | ✅ |