Overview
The role is for a Staff Software Engineer focusing on evaluating interactions with AI coding agents like OpenAI Codex and Claude Code. This is a contract position, starting ASAP and lasting through early May, with a pay rate between $100 and $200 per hour.
Key Responsibilities
- Evaluate AI-generated coding interactions end-to-end.
- Judge whether outputs are useful, correct at a high level, and aligned with strong engineering thought.
- Assess quality of explanations and reasoning beyond just the code.
- Distinguish different levels of response quality.
- Provide clear, opinionated feedback on what worked and what didn’t.
- Help define expectations for interactions with tools like Cursor.
Requirements
- Staff/Principal-level engineer status or equivalent experience.
- Strong background in TypeScript, JavaScript, or Python.
- Hands-on experience with OpenAI Codex, Claude Code, and Cursor.
- Deep familiarity with modern AI-assisted development workflows.
- Able to evaluate code without full execution or deep review of every line.
- Comfortable providing direct, opinionated feedback.
- High standards for what constitutes good engineering.
Benefits
- Pay rate of $100–$200/hour.
- Flexible hours, approximately 10–20 hours per week.
- Potential for contract extension beyond early May.
Location
Remote
How to Apply
Interested candidates should complete the take-home evaluation exercise and participate in a behavioral interview.