• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar

Kampuchea Crossings

Bump to baby on the beaten expat track

  • Home
  • Public Health Portfolio
  • Work Posts
  • Contact

reliability

Testing for Inter-rater reliability

22 November 2011 by Nathalie Abejero Leave a Comment

I haven’t used a stats package in a long time. I remember back in grad school how interesting my Biostats and BioInformatics courses were, especially Janet Hughes‘ classes. But in my professional life, I hardly ventured beyond the first five tasks in the function button on the standard Excel toolbar.

via http://www.effective-time-management-strategies.com

My QI (Quality Improvement) team has been pre-testing some clinical assessment questionnaires for Health Centers. The questionnaires consist of a checklist and a rating, for two assessors to fill out while observing a specific consultation eg an Antenatal Care visit. The checklist scores are weighted with qualitative ratings. For example, of the items which the midwife conducted during the Physical Examination, the assessors rate the quality in which these items were carried out.

There’s a lot of room for bias in any data collection method, but at least I want to know that my instrument is reliable. So I want to test the inter-rater agreement (eg did they see the same things? did they rate the items similarly?). You’d think this is a simple enough calculation to carry out in excel, in a neat and tidy point-and-click experience. But no. Apparently this seems to be an obscure corner of statistics that you need macros for to carry out in SAS and SPSS.

I ask for the billionth time in my short life: what did people do before internet? I’m a bit less stumped since I’ve found these handy sources below, which provide a free or low-cost stats package for content analysis. Whew!

  • kappa2 – free downloadable automated excel workbook (although I’ve had some problems using this one)
  • ReCal – free to use, online interface to enter your data
  • AgreeStat – $45 downloadable automated Excel workbook

Filed Under: Work Tagged With: content analysis, inter-rater, inter-rater reliability, QI, quality improvement, reliability, reliability testing, statistics, testing

Primary Sidebar

Those little feet pitter-pattering about rule our lives lately. But on the occasional free moment I get to tap out scatterbrained bursts of consciousness about raising toddlers in Cambodia, traveling with them and working abroad. These posts are my personal updates to friends and family. But since you’re here, have a look around. Thanks for stopping by…

Read More…

Tweets by nabejero

Blog Post Categories

  • Interests
  • Life
  • Travels
  • Work

Latest posts

  • Kids project: Micro-loans to women entrepreneurs
  • I ran the 50th NYC Marathon!
  • Bok l’hong with Margaritas or, memories from the Mekong
  • Getting the kids to like ampalaya (bitter gourd)
  • Gender differences in athletic training
  • Commuting and office work in the time of Covid
  • Until Covid-19 messaging improves, who do you turn to?
  • Filipino snack

Tags

aid baby Bangkok bush Cambodia christmas coconut covid-19 cuisine delivery development expat expatriate Filipino food food foreign aid holiday hurricane inauguration katrina Khmer Khmer cuisine Khmer food Khmer New Year kids levy louisiana mango Manila medical tourism mekong new orleans nola nyc obama parenthood parenting Philippines Phnom Penh Poipet running Thailand travel US xmas
  • Facebook
  • Google+
  • LinkedIn
  • Pinterest
  • RSS
  • Twitter

Copyright © 2023 · Genesis Sample on Genesis Framework · WordPress · Log in