What Would I Do Differently?

In a recent post (see What Am I Working On?) Abagail asked me the following questions in reference to my dissertation:

1. You talked a lot about member checking but with the students, it didn't seem to have a high response rate. What would you do differently, if anything, to increase member checking?
2. What device did you use to record your interviews by phone?
   
3. If you were to do it again, what would you have done differently? (i.e. do you wish that you had used the various surveys you adapted or designed on a larger student sample? Would you have used more focus groups than individual interviews? and Why? etc)

I figured that this would be a good venue to address those three questions.

1. One of the difficulties was that there was a significant time lag between when I did the interviews and when I was able to transcribe them. This meant that approximately half of my participants had graduated by the time I got around to having them member check the transcriptions (which I believe accounted for many of the students who didn't do the member check). Second, in most instances the students said that they read through the interview transcripts, they just didn't make any changes. That could be because they didn't really read it, it could be because they weren't as invested in the research as I was so whatever they said at the time was okay (which is what I think happened), or it could be because they read through them and felt that the transcripts still represented their beliefs (which could have also bee some of what was going on).
2. I purchased a telephone recording control from Radio Shack and attached my mini-cassette tape recorder to one plug and my telephone to the other plug (see Radio Shack website). Note that this device doesn't work with certain mini-cassette recorders.
3. The biggest thing that I would have changed about my dissertation study was that I would have conducted a pilot study or even just a simple pilot test of my interview protocol. This would have prevented the need to re-develop the protocol for the initial interview and the creation of additional protocols for the follow-up interviews. The other thing I would have like to have done was spend more time in the school actually observing the students. I was there for a full month, but I would have like to have spent more time (particularly given the month that I selected). As for your other questions, it was never my intention that the surveys would be used for the purposes of statistical analysis so I wouldn't hope for a larger sample with those surveys. As I have used one of the surveys (the student perceptions one) in another study, I already had a data set in which I have utilized that survey for. As for the other three, the extended version of the ESPRI is no longer relevant as Roblyer et al have fairly strong reliability on a much shorter instrument (and I will be working with them in the new year on a project using that shorter ESPRI). The learning styles inventory and the transactional distance survey would make for interesting stand alone studies with a larger sample and may be something that I look at in future years, but not in the next twelve months. As for the focus groups over individual interviews, I would still have used individual interviews. I think that the one-on-one interviews were useful in that it got each student's voice included - assuming all twelve had stayed in the study (or at least that portion of the data collection). When I did the initial focus group, there was one dominant voice out of the three participants, one less participatory, and then one who simply agreed with the other two. In this situation there was one voice that was relatively silent, which didn't happen with the individual interviews.

So, what about others who have completed theses, dissertations, or other large scale research projects with virtual schools... What would you have done differently? Did you use interview or focus groups? Why did you choose one over the other?

Tags: virtual school, cyber school, high school, education