Label bias arises when the outcome labels used as ground truth for training reflect the biases of the humans or processes that generated them. In a recruitment context, the label “hired/not hired” encodes the decisions of human recruiters who may have been influenced by conscious or unconscious bias. Training a model on biased labels teaches the model to replicate that bias.
Inter-rater reliability analysis is the primary detection method. Multiple independent labellers rate the same instances, and agreement is measured using Cohen’s kappa (for two raters) or Krippendorff’s alpha (for multiple raters). Low inter-rater agreement indicates subjective or inconsistent labelling, which increases the risk that labels encode individual biases rather than objective ground truth. The AISDP documents the inter-rater reliability statistics, the labeller qualifications and training, the annotation guidelines provided, and the method used to resolve disagreements.
Where inter-rater reliability is low, or where analysis reveals systematic patterns in labelling differences across protected characteristic subgroups, relabelling by diverse panels provides a corrective dataset. The relabelled subset is compared against the original labels to quantify the label bias. The AISDP records the relabelling methodology, the panel composition, the divergence between original and relabelled outcomes, and the decision on how to handle the divergence (replace original labels, use relabelled data as a validation benchmark, or apply bias-aware label smoothing).
The annotation process should also be assessed for conditions that support quality. Annotators should be compensated fairly and working under conditions that do not incentivise speed over accuracy. Annotation quality directly affects label accuracy, which directly affects model fairness and performance.
Key outputs
- Inter-rater reliability statistics (Cohen’s kappa, Krippendorff’s alpha)
- Label bias assessment (subgroup-level analysis)
- Relabelling methodology and results (where applicable)