| Data acquisition |
Speech, images, tool-use traces, domain documents |
Generalists → specialists |
Screening, consent, geo/language filters |
Per item / project |
| Annotation & enrichment |
Classification, extraction, transcription — with human verification of prelabels |
Calibrated generalists + reviewers |
Gold tasks, overlap, consensus |
Per row / page / HIT |
| Alignment & post-training |
SFT demos, pairwise ranking, preference data, reasoning traces |
Specialists, vetted experts |
Domain quals, rubrics, adjudication |
Per judgment / SOW |
| Evaluation & safety |
Benchmarks, factuality, hallucination, red team, model comparison |
Experts, auditors, red teamers |
Multi-stage QA, references, escalation |
Pilot / retainer |
| Runtime oversight |
Approvals, exception queues, agent escalation, tool-use intervention |
Operators + domain experts |
Audit logs, SLA routing, stop controls |
Per queue / platform + usage |