Overview
The role involves working as a Senior Software Engineer (contract) to evaluate the quality of interactions with AI coding agents like OpenAI Codex and Claude Code. This position emphasizes assessing engineering judgment rather than writing production code.
Key Responsibilities
- Evaluate AI-generated coding interactions end-to-end.
- Judge whether outputs are useful, correct, and aligned with strong engineering thought.
- Assess the quality of explanations and reasoning behind the code.
- Differentiate between various levels of response quality.
- Provide clear feedback on what worked well and what felt misleading.
- Help define quality standards for interactions with tools like Cursor.
Requirements
- Staff or Principal-level engineer experience or equivalent.
- Strong background in TypeScript/JavaScript or Python.
- Hands-on experience with OpenAI Codex, Claude Code, and Cursor.
- Familiarity with AI-assisted development workflows.
- Able to evaluate code and provide opinionated feedback.
- High standards for good engineering practices.
Benefits
- Rate: $100–$200/hour.
- Hours: Approximately 10–20 hours per week.
- Contract duration: Through early May with a possibility of extension.
- Start date: ASAP.
Location
Remote
How to Apply
Details on the application process include a take-home evaluation exercise followed by a behavioral interview.