Manu Ajith
Portrait of Manu Ajith

Manu Ajith

Distributed systems, production AI infrastructure, 2x startup CTO

Software gets slow and brittle in quiet ways. I build the parts that don't.

I've spent fifteen years building production software from the product layer down to the infrastructure it runs on. Most of that has been distributed systems in Elixir, alongside Rust, Go, and Python. Lately my focus is self-hosted LLM inference on multi-GPU clusters, evaluation pipelines, and data systems that have to stay upright under real operational load.

I've built two startups from the first commit and hired the engineering teams behind them, staying hands-on in the code both times. I live in Toronto after working out of more than 40 countries, and I write up the production bugs that surprised me on til.codes.

Six things I've learned the hard way

The worst AI failure is not a crash. It is a model quietly getting worse.

Crashes page you right away. A prompt tweak or aggressive quantization can drop structured output accuracy from 99% to 82% while every health check stays green. The only real defense is scoring candidate models against live production traces, with latency and GPU cost measured in the same loop.

til.codes: When quantizing a 70B model broke everything

Measure the bottleneck before touching the architecture.

Four-way tensor parallelism looks faster on paper until PCIe all-reduce makes time to first token 2.8x slower than single-GPU replicas. A single WHERE clause sitting outside a Postgres window function can hide a 76x speedup. Intuition proposes; production traces decide.

til.codes: Why tensor parallelism made our inference slower

Boring, reproducible infrastructure beats clever tooling.

Most teams lose more hours fighting flaky CI, drifting environments, and fragile deploys than solving actual product problems. Making deployments predictable with Terraform, Kubernetes, and Nix is rarely flashy, and almost always the highest-leverage work on the team.

til.codes: When the Nix eval cache serves a ghost derivation

Pair AI with deterministic rules, not a pile of agents.

Use models where inputs are genuinely messy, like emails, support threads, and operational messages. Hand the structured output straight to deterministic pipelines with explicit backpressure and proper observability.

til.codes: Debugging a silent Langfuse integration in production

Flaky tests and slow feedback loops are distributed systems problems.

When an end-to-end test fails one time in ten, it is almost never a random browser glitch. It is an unmodeled race between async state, database commits, and client reconnection. Developer tooling deserves the same rigor as production.

til.codes: Flaky Playwright tests are a distributed systems problem

Stay close to the code, even when you run the team.

I have been a startup CTO and founding engineer twice. The architectural calls that age best come from someone who still reads query plans, profiles memory leaks, and feels the friction of shipping to production.

til.codes: How CUDA graphs leaked memory in production

What I've shipped, without the logos

Self-hosted AI inference and evaluation

Running vLLM on multi-GPU clusters, sizing KV caches and tracking CUDA graph leaks, training SFT pipelines, and building LLM-as-Judge eval harnesses wired into Langfuse.

Distributed Elixir and high-volume pipelines

Nearly a decade of production BEAM systems: clustered Elixir surviving election-day peaks, Broadway and Redpanda backpressure pipelines handling millions of daily events at 99.9% uptime, and Phoenix LiveView internal tools built for operations teams.

Large-scale data and platform migrations

Zero-downtime dual-write and shadow-read cutovers moving 25+ billion rows from PostgreSQL to ClickHouse, breaking monoliths into Elixir, Rust, and Go services, and cutting infrastructure spend by 40%.

Open source and modern agent workflows

Author of the open-source Langfuse and Svix Elixir clients, contributor to jjui, and builder of Jujutsu workspace tooling for running parallel coding agents.

Work together

I work with early-stage founders as an advisor, fixed-scope systems specialist, or fractional CTO. I am also always glad to hear from teams tackling hard infrastructure or AI serving problems.

Send a note to [email protected], or read how consulting works.