Overview
Developed during my Software Engineering Internship at HealthFirstPriority (HFP), MeshMind is a distributed clinical assistant and analytical platform. It features over 20 REST API endpoints, distributed inference using Caddy load balancing and Tailscale VPN mesh networking across multiple AI worker nodes, RAG semantic search, JWT authentication, and real-time streaming summaries.
Problem & Approach
Clinicians face high cognitive load when parsing lengthy medical records. They need accurate, low-latency AI summaries without risking data privacy or experiencing slow API response bottlenecks.
Engineered a distributed microservice architecture pairing a FastAPI backend with Go middleware, a PostgreSQL database, and a React frontend. Configured Tailscale VPN mesh networking and Caddy reverse proxy to distribute inference across worker nodes.
System Architecture
- Frontend Interface
- React, Next.js, and TypeScript with real-time stream consumer hooks.
- Backend API & Middleware
- FastAPI (Python) and Go middleware for fast request routing and authentication.
- Database
- PostgreSQL storing patient records, embeddings, and conversation histories.
- Networking & Mesh
- Caddy load balancer and Tailscale zero-trust private mesh connecting distributed inference workers.
Features & Deliverables
- 01Distributed AI Inference NetworkImplemented
Multi-node inference pipeline balanced by Caddy across a secure Tailscale VPN mesh.
- 02Retrieval-Augmented Generation (RAG)Implemented
Semantic search and document chunk retrieval over clinical records for evidence-grounded AI responses.
- 03Real-Time Streaming ResponsesImplemented
Server-Sent Events (SSE) streaming delivering conversational output token-by-token directly to the UI.
- 0420+ Validated REST EndpointsImplemented
Comprehensive API suite covering authentication, patient context, document ingestion, and analytics.
Technical Challenges
ProblemHigh API response latency during heavy clinical document parsing and inference.
Resolution:Profiled bottlenecks across database queries and Python execution; optimized indexing and refactored distributed endpoints, achieving approximately 30% reduction in API response time.
ProblemEnsuring secure, zero-trust communication between geographically distributed inference worker nodes.
Resolution:Integrated Tailscale VPN mesh to provide end-to-end encrypted node-to-node routing without exposing internal ports to the public internet.
Contributions & Takeaways
My Contributions
- Built and tested Python backend code across 15+ core features as part of a 4-developer engineering team.
- Designed and implemented distributed inference routing with Caddy and Tailscale.
- Developed RAG semantic retrieval, JWT authentication, and SSE streaming endpoints.
- Collaborated on responsive clinical interface improvements using React.
Key Learnings
- Distributed AI systems require robust network telemetry and zero-trust mesh infrastructure to ensure reliability.
- Server-Sent Events provide significantly better perceived UX for LLM interactions compared to blocking HTTP requests.
Interested in discussing this project or working together?
Get in touch