Search jobs

About the company

Our client is a software company.

The role

Raydar is recruiting for this role on behalf of our client. Own the path that takes trained machine learning models into reliable production serving, with a focus on speed, efficiency and dependability at high request volume. The work combines performance tuning of models and GPU-based code with the design of distributed serving systems that run across many machines.

What you'll do

• Optimize serving latency and throughput for machine learning models.

• Take models from research through containerization and into stable production deployment.

• Profile and tune code to make efficient use of GPU hardware.

• Design and run distributed serving systems across multiple GPUs and nodes.

• Apply model acceleration methods such as quantization, batching and caching.

Requirements

What we're looking for

• Senior-level experience, comparable to a senior software or inference engineer at a large technology company.

• Track record of developing server-side or learning-based software and getting it running live for real users.

• Hands-on experience with machine learning inference optimization frameworks.

• Proficiency in systems programming and GPU optimization in languages such as C++, CUDA, Rust or optimized Python.

• Familiarity with methods that speed up and shrink models, such as weight compression, knowledge transfer between models or token prediction.

• Experience with distributed systems, container orchestration and handling high concurrency.

• Based in an eligible country for this remote role.

Bonus points

• Public technical work, such as open-source contributions to inference engines or detailed technical write-ups.