After finishing training on Monday, our research team spent Tuesday-Thursday field testing the responsive care tool and spent Friday-Saturday piloting it. These two steps are super important in any research project, as they allow us to work out all of the issues and quirks the tool has before we initiate the formal data collection process. Since our tool is observing caregiver-chid interactions during play, we provide each caregiver and child with a toy to facilitate that interaction. The toy has to be age-appropriate, stimulating enough to elicit a reaction from the child, and something they haven't seen before so that it doesn't bore them. During field testing we went through a trial and error process of figuring out which toys to use for each age group. We also spent a lot of time observing and live coding the caregiver-child interactions, while video recording them, and then going frame by frame to discuss any discrepancies in our coding.
By day three, all of our coders scores were within 10% of each other, and we moved to the pilot phase. We are piloting this tool on 20 caregiver-child dyads, with four of the interactions coded by both a RA and Saima, to check for inter-rater reliability (IRR). IRR refers to the degree of agreement between our coders. We need this to be high because our RA's will be visiting households independently, and we need their coding to be consistent in order to get reliable results. Based on our first day of piloting the IRR between each RA and the gold standard was between 0.88-0.90, which is considered to be an excellent IRR.
Saima and I have been ecstatic that the training, field testing, and pilot study have been successful so far. Next week we will begin on the training and field testing for the other tools we will use in our study. I am so thankful for this highly motivated team who has worked through this extreme heat! We have been in the field in 90-120 degree weather all week, but no complaints here as the scenery has been amazing and our study participants have been so gracious to welcome us into their homes.
