Careers
Multilingual RLHF Quality Reviewer
Reinforcement learning from human feedback is only as good as the judgement behind it, and at scale that judgement has to be audited. You will be the audit. You review annotator work, adjudicate disagreements, and identify whether a failure originates in an individual, the training, or the guideline itself.
What you will do
- Review preference judgements, ratings and written responses for correctness and bias
- Adjudicate disagreements with a documented rationale a client can audit
- Sample and audit batches, reporting quality metrics against thresholds
- Distinguish systematic error from individual error, because the remedies differ
- Support red team and safety evaluation programmes (optional, flagged clearly)
What you bring
- English at C1 with rigorous written reasoning, plus C1 Turkish, Arabic, German, Russian or Farsi
- At least two years in annotation, linguistic quality, editorial, translation review or similar
- Ability to reason about ambiguous cases and write down the reasoning
- Sound judgement on safety, bias and factual accuracy and total reliability
Nice to have
- Prior RLHF or model evaluation experience
- A postgraduate qualification in linguistics, philosophy, law or a science
- Subject depth in medicine, law, finance or engineering
What we offer
- Rates that rise with review tier and language scarcity
- Fully remote and flexible within weekly minimums
- Paid guideline training per programme and steady volume
- Work that directly shapes model behaviour and a clear route into programme leadership
Apply
The first step takes under two minutes. We respond to every application.
Let us build your Türkiye team
Tell us your function, your scale and your language needs. We will come back to you within six hours.