Overview
The role is for a Senior Software Engineer focused on evaluating the quality of interactions with AI coding agents like OpenAI Codex and Claude Code. This position is a contract role that requires a strong engineering background, predominantly assessing how these models behave and their effectiveness in real-world coding scenarios.
Key Responsibilities
- Evaluate AI-generated coding interactions end-to-end.
- Judge whether outputs are useful and correct at a high level.
- Assess the quality of explanations and reasoning.
- Distinguish between different levels of response quality.
- Provide clear, opinionated feedback on interactions.
- Help define standards for quality interactions with tools like Cursor.
Requirements
- Staff / Principal-level engineer or equivalent experience.
- Strong background in Python, TypeScript, or JavaScript.
- Hands-on experience with OpenAI Codex, Claude Code, and Cursor.
- Deep familiarity with modern AI-assisted development workflows.
- Able to evaluate code without needing to fully execute or deeply review every line.
- Comfortable giving direct, opinionated feedback.
- A high bar for what constitutes good engineering.
Benefits
- Rate: $100–$200/hour.
- Hours: ~10–20 hours/week.
- Contract duration through early May, with possible extension.
- Start: ASAP.
Location
Remote
How to Apply
For further information, refer to the provided Loom video.
Deadline
No specific deadline stated.