VantaLLM
private repo flagshipA from-scratch framework for training decoder-only LLMs. I designed the architecture and made the technical calls, with AI tooling in the loop the way modern teams work. Every piece of it, I can sit down and explain: why it is there, what broke, how we fixed it. A cybersecurity-native LLM stack: my own cybersec tokenizer, adversarial eval hooks, persona-conditioned serving. I built it knowing someone like me would try to break it.
- Ran the 567M config for 30,000 steps on a single 20GB A100. The same code scales to a 125B bring-up profile by editing a JSON file. No forks, no branches.
- When ZeRO-3 param-gathers hung the run, we replaced a 128-expert Python loop with three batched tensors. Debugging that is where the real understanding came from.
- KV-cache decode is bit-identical to no-cache decode, and the tokenizer round-trips SQLi payloads and CVE strings exactly. I test the boring things because the boring things break.
- The tokenizer is trained on security corpora, so vocab covers CVE strings, payloads, and exploit syntax that general tokenizers shred into garbage.
- Adversarial eval hooks run against every checkpoint: injection resistance, jailbreak probes, extraction attempts.
- The multi-node launcher fails fast on NAT and duplicate-IP setups instead of hanging at 0% GPU. Paranoia as a design principle.
→ read the full case study · private · walkthrough on request
┌────────────────────────────────────────────┐ │ VantaLLM · decoder-only · 567,068,416 params│ ├────────────────────────────────────────────┤ │ tok(151,003) → emb → 12× [ RMSNorm │ │ → GQA + RoPE (shared lazy cache) │ │ → dense FFN ⇄ MoE 16e top-2 │ │ (batched grouped-GEMM, cap=1.25) ] │ │ → RMSNorm → lm_head │ ├────────────────────────────────────────────┤ │ train: DDP / DeepSpeed ZeRO-2·3 · bf16 │ │ serve: FastAPI · SSE · persona routing │ └────────────────────────────────────────────┘
