Member of Technical Staff, ML Inference Engineering

About the Role

Sanas is bringing real-time speech and language models on-premise — deployed at scale directly inside sovereign data centers, not served from behind a hosted cloud endpoint. It's one of the most demanding environments in the industry: strict latency budgets, massive concurrency, and infrastructure that needs to be private and reliable.

We're looking for a deeply hands-on, experienced engineer to help lead that build. This is someone who shapes core infrastructure and architecture decisions rather than just executing against a specification, and who naturally raises the level of the engineers working alongside them.

What You'll Do

Inference Optimizations

● Implement custom kernels and low-level optimizations to push the absolute limits of GPU compute.

● Apply graph optimization, operator fusion, and hardware-specific code generation at the ML compiler level.

● Profile, analyze, and resolve deep system bottlenecks to radically improve latency, throughput, and memory efficiency.

● Drive model-level execution improvements, including mixed precision and advanced quantization strategies.

Serving Optimizations

● Write and optimize custom inference server backends to handle complex business logic, scripting, and state management.

● Own and scale our overarching model serving infrastructure across multi-GPU, multi-node deployments to meet strict on-premise latency budgets.

● Build robust, fault-tolerant runtime services, complete with comprehensive performance benchmarking and monitoring.

Requirements

● 8+ years of experience writing high-quality, high-performance code, including at least 5 years focused on machine learning systems.

● Experience with one of C/C++/Rust and Python.

● Deep familiarity with modern NVIDIA GPU architectures (e.g., Ada Lovelace, Blackwell), CUDA, and low-level system profiling.

● Hands-on experience with ML compilers and optimization frameworks (e.g., Apache TVM, TorchInductor/Dynamo, TensorRT, Triton).

● Experience building or extending model serving infrastructure, specifically writing custom C++ backends for Triton Inference Server (or similar serving engines).

● Fluency in the AI serving stack, from kernels and quantization up to schedulers, state management, and autoscaling.

● A record of shipping research or systems that other people build on, whether in a lab or in industry.

Nice-to-have:

● Experience serving low-precision (FP4/FP8) models, multiple LoRA adapters within one model instance (Multi-LoRA), or models distributed across several GPU nodes.

● A research-leaning or systems background in Speech (STT, TTS, S2S) or LLM inference, with work you can point to.

● Experience operating large-scale, on-premise AI training or inference clusters, including bare-metal provisioning and Kubernetes management.

● Familiarity with high-performance networking (InfiniBand or RoCE), distributed storage systems, and hardware health monitoring.

● Experience maintaining or contributing to open-source ML or systems projects.