Testing for Inter-rater reliability

I haven't used a stats package in a long time. I remember back in grad school how interesting my Biostats and BioInformatics courses were, especially Janet Hughes' classes. But in my professional life, I hardly ventured beyond the first five tasks in the function button on the standard Excel toolbar. My QI (Quality Improvement) team has been pre-testing some clinical assessment questionnaires for Health Centers. The questionnaires consist of a checklist and a rating, for two assessors to fill out while observing a specific consultation eg an Antenatal Care visit. The checklist scores are weighted with qualitative ratings. For example, of the items which the midwife conducted during the Physical Examination, the assessors rate the … [Read more...]