1. 1 Projects
  2. 2 Details
  3. 3 Source: private client work

Management | Scale AI | Jul 2025 to Jul 2026

LLM coding-data programme

At Scale AI I led a distributed team of 35 on an LLM coding-data programme: task allocation, rubrics and quality bars, mentoring, and delivery. Together we shipped 11,000+ hours at a 3.9/5 average quality rating, and my own work scored 4.3/5 across 51 reviews.

  • Leadership
  • Rubric design
  • Code review
  • LLM evaluation
  • Python
  • LoRA and PEFT
  • RLHF and SFT
  • RAG
35
people in the squad I led
11,000+
hours shipped with my squad
3.9/5
average quality rating
4.3/5
my own score across 51 reviews

Squad lead

Running a squad of 35

The squad was distributed, and my job covered the whole loop: what we would build, what good looked like, how every piece of work was checked, and getting it delivered. Rubrics were designed so that scores from different reviewers are comparable. These are the four stages I owned.

  1. Scope

    Scoped the task sets the squad worked from.

  2. Standards

    Wrote the evaluation rubrics and the technical assessments.

  3. Review

    Built the review workflow that every piece of work went through before delivery.

  4. Delivery

    Ran delivery for the distributed squad. Together we shipped 11,000+ hours of task work at a 3.9/5 average quality rating.

Lead reviewer

Multi-model coding and reasoning evaluation

Alongside the squad, I was lead reviewer on more than half of the projects I was assigned to, multi-model coding and reasoning evaluation among them. My own work was reviewed as well, and 51 internal quality reviews averaged 4.3/5.

I was promoted into the platform's senior reviewer tier and held its top contributor designation.

51
internal quality reviews of my work
4.3/5
average across those reviews
Senior
reviewer tier, promoted into it
Top
contributor designation, held

Engineering

Validation and fine-tuning pipelines

Validation pipelines

Python pipelines for data quality assurance and for monitoring model output.

RLHF and SFT feedback loops

Feedback loops for reinforcement learning from human feedback and for supervised fine-tuning.

Fine-tuning with LoRA and PEFT

Parameter-efficient fine-tuning of models with LoRA and the PEFT library.

RAG with Model Context Protocol

Retrieval-augmented generation pipelines that use Model Context Protocol to give evaluations the right context.

Step 3 of 3

Source code

This was client work, so the task sets, rubrics and pipelines are not public. Ask me how the squad and the review workflow ran, or read the code of my open-source projects.