Sophont has announced an opportunity for a Founding Lead Researcher – Medical AI Evaluations, offering an experienced artificial intelligence researcher the opportunity to lead the evaluation strategy behind a range of medical AI systems.

The fully remote position is designed for an exceptional researcher with expertise in machine learning evaluation, statistics, experimental design, biomedical AI, and scientific measurement. The successful candidate will play a central role in developing evaluation standards that could influence how medical foundation models and clinical AI systems are tested, compared, and assessed for readiness.

Sophont is a public-benefit corporation focused on developing open, universal medical AI capable of working across areas including pathology, neuroimaging, clinical text, and other medical domains.

About the Sophont Medical AI Research Role

The Founding Lead Researcher will own the evaluation layer across Sophont’s medical AI research portfolio. This includes language models, pathology foundation models, neuroimaging models, multimodal systems, agentic AI, retrieval systems, and clinical-facing products.

The role sits at the intersection of several technical disciplines, including:

Machine learning benchmark design.
Statistical analysis and experimental design.
Evaluation infrastructure.
Biomedical data quality.
Scientific interpretation.
Clinical AI assessment.
Model validation.

The successful candidate will be responsible for developing evaluation systems and standards that allow researchers and partners to make reliable comparisons between medical AI models.

Key Responsibilities

The researcher will lead the development of reusable evaluation methodologies across multiple categories of medical AI systems.

Core responsibilities include:

Developing evaluation methodologies for medical language, pathology, neuroimaging, multimodal, retrieval, agentic, and clinical AI systems.
Building benchmark infrastructure supporting public and internal leaderboards.
Creating reproducible evaluation jobs and standardised result packages.
Designing high-quality datasets in collaboration with researchers, clinicians, and MedARC contributors.
Developing annotation rubrics and quality-control processes.
Conducting leakage checks and measuring inter-rater reliability.
Establishing standards for source-grounded medical AI systems.
Evaluating retrieval quality, evidence recall, evidence precision, uncertainty calibration, and medical fact verification.
Designing clinically meaningful evaluation protocols.
Conducting statistically defensible model comparisons.
Performing confidence-interval analysis, subgroup analysis, calibration, robustness checks, and ablations.
Reporting negative results alongside successful findings.
Translating evaluation findings into research papers, technical reports, blog posts, partner materials, and internal release decisions.
Leading MedARC contributors through structured and reproducible research workflows.

Required Qualifications and Skills

Sophont is seeking candidates with substantial experience developing rigorous machine learning evaluations, benchmarks, model validation systems, or scientific measurement infrastructure.

Applicants should demonstrate strong knowledge of statistics, experimental design, data quality analysis, and scientific interpretation.

Key requirements include:

Strong Python and software engineering skills.
Experience building data pipelines.
Experience with reproducible experiments.
Experience developing model evaluation infrastructure.
Ability to design annotation protocols.
Ability to establish and reason about ground truth in noisy biomedical datasets.
Experience with LLMs, agents, computer vision models, biomedical foundation models, or similarly complex AI systems.
Ability to balance evaluation cost, latency, model quality, and statistical power.
Strong scientific and technical writing skills.
Ability to operate effectively in an early-stage organisation while establishing new processes.

Preferred Experience

Experience working with medical, biological, clinical, pathology, or neuroimaging datasets is preferred.

Additional experience with retrieval-augmented generation, hybrid search, BM25, dense retrieval, ColBERT-style retrieval, LLM-as-judge systems, or human evaluation pipelines would be advantageous.

Sophont also values candidates with experience building public benchmarks, leaderboards, crowdsourcing platforms, annotation systems, GPU infrastructure, distributed evaluation systems, experiment tracking, or model-serving infrastructure.

A strong publication record or open-source contribution history in machine learning, biomedical AI, or evaluation methodology is also desirable. Experience leading open-source research communities or distributed contributor groups would provide an additional advantage.

Fully Remote Medical AI Research Position

The Founding Lead Researcher role is fully remote, giving the successful candidate significant flexibility over working hours. Employees are expected to attend core collaboration meetings while having considerable autonomy over their schedules.

The position offers an opportunity to work within an open research environment where employees are encouraged to publish their work, contribute to open-source projects, and participate in workshops, hackathons, and academic conferences.

Salary and Benefits

Sophont is offering a salary range of $100,000 to $300,000 USD per year, depending on experience and qualifications.

The compensation package includes:

Meaningful equity.
OpenAI and Anthropic subscriptions.
401(k) with a 4% company match.
Medical insurance.
Dental insurance.
Vision insurance.
Basic life insurance.

The combination of salary, equity, research autonomy, and opportunities for publication makes the position potentially attractive to senior researchers seeking to work at the intersection of artificial intelligence and healthcare.

How to Apply

Interested candidates should submit their CV by email to hiring@sophontai.com. The required subject line is:

[Your Full Name] CV – Medical AI Evaluations

Applicants should highlight relevant experience in machine learning evaluation, biomedical AI, statistics, scientific research, software engineering, benchmarks, or related areas.

The opportunity is particularly suited to researchers who want to take ownership of medical AI evaluation standards and contribute to open, transparent research in an early-stage environment.

Sophont states that it is an equal opportunity employer and welcomes applicants from all backgrounds.

VISIT OFFICIAL WEBSITE TO APPLY

For more opportunities such as these please follow us on Facebook, Instagram, Twitter, LinkedIn and WPChannel

Disclaimer: Global South Opportunities (GSO) shares jobs, internships, fellowships, grants, scholarships, and other opportunities for informational purposes only. GSO is not affiliated with Magamba Network or the Fired Up initiative and does not participate in the recruitment or selection process. Applicants should verify the vacancy details, eligibility requirements, application process, and deadline through the official organisation before applying.

Sophont Opens Remote Founding Lead Researcher Role in Medical AI Evaluations – Salary Range $100,000-$300,000 USD @$300,000 USD