Robert Zhang
I work in the CTO office at , where most of my time goes into making large models run fast on hardware that wasn't built around them.
Before that I was at and , and in . I'm currently researching at and a fellow at .
I write here about kernels, interconnects, and the systems in between, and keep a few photos from outside the lab.
Recent
Everything- Nov 15Tensor Parallelism Without NVLink: Where the All-Reduce Time Actually GoesResearch
- Nov 15NCCL's Defaults Are Tuned for NVLink. Off It, They Pick Wrong.Research
- Nov 15PCIe vs Ethernet Scale-Up for Inference: When Each One WinsResearch
- Nov 15Reading a Multi-GPU vLLM Run Without Fooling YourselfResearch
- FebPipeline ParallelismResearch
- MLA ProfilingProject
- JanFast Multidimensional Matrix Multiplication on CPU from ScratchResearch