Ruthvik Garlapati

Real-time LLM inference benchmarking: streams 4 providers (Groq, Cerebras, Cloudflare, Fireworks) in parallel via SSE with TTFT, throughput, P95 latency, and cost metrics.

Self-hosted GPU job scheduling on k3s NVIDIA A10 VM — FastAPI + Next.js + Redis/RQ via Helm, DCGM exporter for GPU metrics, SSE live log streaming.

End-to-end LLM inference stack on EKS g4dn spot with vLLM (Llama-3.1-8B), DCGM → Prometheus → Grafana — all infra as Terraform IaC.

MLOps pipeline: MLflow model versioning, FastAPI/Docker inference server, ArgoCD GitOps canary releases via nginx, Prometheus/Grafana observability.

Autonomous K8s validation agent: runs health/performance/security/smoke tests, analyzes metrics/logs/traces, streams real-time risk assessments with ArgoCD rollback.

Polyglot Bazel monorepo (C++/Python): custom GEMM/ReLU/softmax kernels, proto-defined API, HTTP inference server, typed client SDK, p99 latency benchmark.

vLLM + Triton on k3s NVIDIA VM, KEDA event-driven autoscaling triggered by Prometheus queue-depth, benchmarked 100 concurrent users with Locust.

Benchmarked Mistral-7B on Modal A10G vs EKS/T4: quantified 10x cold-start gap, 3x TCO difference, ~15 req/min break-even for enterprise GPU platform selection.

Full-stack LLM fine-tuning platform with LoRA training on Modal GPU, HuggingFace Hub auto-push, and real-time log streaming via Supabase.

Multi-agent LLM pipeline on GCP (LangGraph + Gemini) with triage/specialist/escalation routing, FAISS/Vertex AI RAG, streaming UI, Cloud Run via Terraform.

Agentic system with Gemini 2.5: 5-tool multi-turn loop (15-turn budget), lazy client init, graceful fallback, structured LLM output — automated PR security review end-to-end.

Self-reflecting GenAI pipeline on Google ADK: parallel specialist agents (RAG/web/cost) feeding a synthesis agent that challenges its own output; Cloud Run auto-deploy.

Production observability for multi-agent LLMs: per-call token counts, latency histograms, USD cost attribution across providers, anomaly alerting at 3× rolling baseline.

Full-stack LLM eval platform (FastAPI + Next.js + PostgreSQL): auto-generates adversarial test suites for Claude deployments, streams diagnostics via SSE, clusters failures with LLM-as-judge.

Production MCP server connecting Claude to ServiceNow — 9 tools for incident triage, KB search, CMDB dependency mapping; 87% P1/P2 priority accuracy on 35-case eval suite.

Multi-tenant observability platform: Kafka/Redpanda topic isolation, Spark Structured Streaming SLO aggregation, ArgoCD auto-rollback on SLO breach, per-tenant Grafana.

Secure MCP gateway with local DuoGuard-0.5B classifiers for inbound prompt injection screening and real-time outbound PII masking (GDPR/HIPAA compliant).

Fault-tolerant AV perception: YOLOv8n primary + LLM fallback (NVIDIA NIM) validated by NeMo Guardrails; SRE watchdog redirects vehicle control on SLO breach.

Self-healing K8s canary orchestrator in Go on Temporal: saga-based auto-rollback, human approval gate, crash-resilient execution, Kyverno policies, audit trail.

Linux incident debugger CLI in Go — simulates and diagnoses production faults via stress-ng, strace, perf, tcpdump, and eBPF.

Capacity-planning and autoscaling simulator on Kubernetes: Erlang-C (M/M/c) queueing model with regression traffic forecasting predicts replicas 30s ahead and preemptively raises HPA floors — full capacity within ~10s of a 6× spike while reactive CPU-based HPA sat 4× under-provisioned.

HA MySQL cluster (primary + 2 replicas, GTID semi-sync) with automated failover via ProxySQL and a custom Python controller — chaos-validated (process kill, network partition, disk exhaustion) with sub-9s MTTR and zero read-path downtime.

Containerized HPC cluster (SLURM, LDAP/SSSD, NFS/autofs, Prometheus/Grafana) with LLM-assisted triage CLI — detects stuck jobs and drained nodes, proposes remediations via NVIDIA NIM, executes only with human approval.