Choroba

Prague.pm

Saturday, 21 November 2026 16:00

20 minutes

Any

English 

When several people score or evaluate many items (imagine e.g. radiologists interpreting X-ray images), we might need to measure how consistent they are and to what degree they agree with each other. Besides simple percentage, there are a bit more advanced methods, namely (Cohen's, Fleiss', weighted) Kappa and Krippendorff's coefficient Alpha.

I needed some of them for my work, but I couldn't find any modules implementing them on CPAN. So I implemented them and uploaded the code to CPAN. When the data I was processing grew considerably larger, I discovered my implementation was too slow. I experimented with AI to optimise the code, but in the end I had to do the heavy lifting myself.

I'll explain the methods and describe the journey. 

statistics