Calibration is the process of unifying grading standards between QA raters — so that in the same conversation, two raters give the same result. Without calibration, each grader understands the criteria differently: employees are graded 3 points by one person, 5 points by another person — the points are no longer meaningful, the training is in the wrong direction. Zenify recorded a 21% increase in conversion rate (approved) for businesses that do regular calibrations.
TL;DR
- Calibration = unification of grading standards between graders.
- Same conversation, different scorers give different results = not calibrated.
- Organize periodically: select samples, the whole team scores, compares, and agrees.
- Calibration helps QA scores reliably for correct training.
- More than one grader → calibration required.
Why is calibration needed?
Problem: same conversation, different scorers.
Cause:
Understand different criteria ("how polite" is enough).
Different strictness (some people grade easily, others grade hard).
Different priorities (people value results, people value process).
Consequences:
Employees don't know if they are right or wrong.
QA scores are not reliable → training in the wrong direction.
Employees are dissatisfied because "doing the same thing but scoring differently".
How Calibration works
Regular calibration cycle (monthly/quarterly):
1. Choose a conversation template
3-5 representative conversations: good, average, borderline.
Choose "difficult" conversations — where people easily differ.
2. The whole group scores independently
Everyone scores the same sample according to the criteria table.
No discussion in advance — to see the real difference.
3. Compare and discuss
Compare points, find discrepancies.
Discussion: "Why do you give me 3 and I give you 5?" — give reasons.
4. Unify and update
Finalize the understanding of common criteria.
Update scoring instructions if necessary.
Record the decision for later application.
Signs that need calibration
Two people marking the same conversation will give a score difference of 2 points or more.
Employees complain of "being judged harshly".
QA score depends on "who scores" rather than the actual quality.
The group just had a new untrained rater.
Do effective calibration
1. Do it periodically, not just once
Standardization is not done once - changes according to reality.
Monthly/quarterly is a reasonable rhythm.
2. Someone leads
One person (lead QA/manager) presides and keeps the meeting focused.
Make sure it doesn't turn into a "point argument".
3. Documentation
Clear scoring instructions, specific examples for each level.
New employees participating in grading must study before starting work.
4. Measure the level of consensus
After several calibrations, the difference between examiners decreases = effective calibration.