Search jobs

This role is for one of our clients

Role Type: Contract

Engagement Duration: 3 months

Location: Remote – North America preferred

Preferred Geographies: United States (approved regions) and Canada

Start Date: Immediate

Experience Required: 5+ years

Compensation: $100–$150 per hour

Availability: Minimum 4 hours per day and 20 hours per week, with at least 4 hours of overlap with Pacific Standard Time (PST)

We are looking for a highly skilled Senior Software Engineer with expertise in Python, TypeScript, JavaScript, or Go to support advanced AI development and evaluation initiatives. The ideal candidate combines strong software engineering fundamentals with exceptional attention to quality, hands-on experience in preference data collection or rubric creation, and the ability to translate complex technical work into clear, actionable insights.

This role is suited to engineers who can operate effectively in ambiguous, fast-paced environments, exercise sound technical judgment, and prioritize practical execution. Candidates should be able to distinguish high-quality examples from mediocre ones, articulate their reasoning, and help turn qualitative assessments into scalable evaluation processes.

Prior experience working with large, complex codebases, such as Visual Studio Code (VS Code), is an advantage.

Requirements

Key Responsibilities

• Rubric Development: Design, refine, and iterate on rubrics, evaluation frameworks, and quality criteria for preference data.
• Data Review and Quality Assurance: Review, assess, and label data while identifying subtle inconsistencies, quality issues, and opportunities for improvement.
• Engineering Infrastructure: Build, maintain, and optimize pipelines and infrastructure supporting data generation, collection, processing, and evaluation workflows.
• Technical Documentation: Synthesize findings from data analysis and evaluation activities into concise reports, updates, recommendations, and actionable summaries.
• Cross-Functional Collaboration: Work closely with researchers and engineers to translate qualitative judgments into structured, repeatable, and scalable processes.
• Codebase Navigation: Understand, contribute to, and maintain complex software repositories while ensuring code quality, reliability, and performance.
• Continuous Improvement: Identify workflow inefficiencies and implement practical improvements to data quality, evaluation consistency, and engineering processes.

Required Skills and Qualifications