Senior/Staff Software Engineer - LLM Inference Systems & Open-Source

Summary

Design and develop LLM inference systems for Qualcomm AI accelerators, working on open-source projects like llm-d and other serving frameworks. Build distributed inference capabilities including prefill/decode disaggregation, KV-cache management, model parallelism, and multi-node scaling. Optimize end-to-end performance across inference engines, runtimes, and hardware, integrating with Kubernetes and cloud-native infrastructure. Collaborate with hardware, compiler, and product teams to deliver production-quality software in Python, C++, Go, or Rust, with expertise in machine learning systems, distributed systems, or AI accelerator software.