Table 5 Illustrative quotations for response options
From: Cognitive interviews guide design of a new CAM patient expectations questionnaire
Response option | Quote | |
|---|---|---|
5.1 Word Sets | ||
5.1.1 | Issue: Participant difficulty recalling words | Participant attempting to recall response choices: “ Disagree strongly, disagree slightly, agree, agree moderately agree slightly, don’t have a clue. Don’t know what you just said. I would need to have these written down or write them down myself to look at otherwise I can’t remember and I spend more time trying to think memorize the answer choices but forget the question.” SEA-416 |
5.1.2 | Issues: Meanings of words inconsistent among participants; leads to non-linear scale | “When you started giving options, I sort of, in my mind, pictured the one going up to ten, so I think your first option was a little worse, if that was the scale, that would be the one. And as it went up, “back feels better than before”, from the treatment, I figured that as the ten and you sort of mentioned, the option after that, it sort of felt like it went from one to ten and then sort of went backwards again, it was a natural order in my mind, so, what was the option after “back pain”?” SEA-412 |
5.2.1 | Item 1: “My pain will improve a lot” | “I just felt confused, like do I say I agree with the statement? Do I have to disagree with the statement? I just felt like ugh. I feel like depending on how a question was worded, especially if it was negative I feel like I could easily have said the opposite of what I meant. Just by being confused.” AZ-411 |
Respondents confused about how to convey their expectations on this scale; not in agreement about appropriate response | ||
Scenario (1): If you only thought your pain would improve a little, what response would you choose? | ||
- I would probably say disagree moderately… I’m disagreeing moderately that my pain will improve a lot, because I’m still trying to keep an open mind but I am sort of feeling like I’ve tried so many other things that I’m not really sure if it’s going to help that much. SEA-410 | ||
- Agree slightly. I would endorse it slightly with “A little bit” toward a smaller degree. Then moderate would be sort of, I think the next step up, but not “agree with strongly” which would be a lot of improvement. I think “more slightly” is the lowest degree of positive, if that makes sense. SEA-412 | ||
5.2.2 | Concerns with interpretation of Item 2: “I will be better able to cope with my pain” | “If I’m not coping anymore do I say strongly disagree or I strongly agree? The question doesn’t make sense anymore with an agree, disagree kind of response.” AZ-411 |
5.2.3 | Item 4: “My energy will improve” Responses to scenarios lacked consistency among respondents | Scenario (1): If you thought your energy would increase just a little, what would your response be? |
- I slightly agree? When I was trying to say, or moderately agree is what I would respond to try to say it increases a little bit. AZ-411 | ||
- I would probably say agree moderately SEA-410 | ||
- Neither agree nor disagree SEA-416 | ||
- Agree slightly. SEA-412 | ||
Scenario (2): If you thought your energy level was fine now, how would you answer that? | ||
- Huh, my energy level is fine, I’d probably say I don’t agree to that statement? AZ-411 | ||
- I think I’d still slightly agree. SEA-416 | ||
- If I felt my energy level was fine, I would say “neither agree nor disagree.” SEA-412 | ||
5.3 Numerical Response Options | ||
5.3.1 | 0-10 Scale | “There was a movie out once where they called a girl a “10” because she was very good looking. So zero to ten is kind of, 10 has always been “the best” and zero is “no good”. And that’s the same scale they use for pain, zero to ten.” AZ-813 |
“I think the one to ten scale is kind of the easiest thing for people to relate to, I don’t think people relate very well to using percentages or negative numbers, I think they would just find that confusing. One to 100 seems a little bit arbitrarily more detailed than you need it to be. I mean is somebody gonna choose a 37.4 out of 100?” SEA-430 | ||
5.3.2 | 0-100 Scale | “I like zero to ten better… It was just like, oh, there’s so many more numbers to choose from. Like 44 or 88… the question is too general for that sort of precision but there was space for that sort of precision so I was like, oh…” AZ-427 |
5.4 Anchors | ||
5.4.1 | Concrete upper anchor, support | What does “extreme improvement” mean to you? It means total relief. Total relief would be the same thing. Total relief would be better, but extreme improvement is still okay. it still gives me a good idea of what you’re asking. AZ-816 |
I think it was easy, but I’m actually thinking a better word would be total improvement rather than extreme improvement. ‘Cause zero’s no improvement, so ten would be total improvement. It would be a better choice.’ SEA-803 | ||
5.4.2 | Lower Anchor: No Change/Worse | If there is no change or getting worse it’s going to be zero. If there is like complete relief of pain that’s going to be ten. So it makes sense. AZ-529 |
5.4.3 | Upper Anchor: Complete Relief | I think complete relief speaks more clearly. Yeah. Because it’s referring to the pain. But then again completely resolved does put it at, it’s the main problem in your life too. Which it does create depression and a huge change in everything in your life. So I can also relate to that. AZ-831 |
I would say completely resolved makes more sense because, well, given my own thoughts about it, because completely resolved to me, means it’s not a problem anymore, don’t worry about it, complete relief could be, it’s good for now, but it might come back later. And I think that resolved is a more solid result. SEA-803 | ||
I kind of like relief better, ‘cause it really describes more, I think, what people are after with back pain.’ SEA-831 | ||
5.4.4 | Reliance on midpoint anchor | I think in some ways when you do that slightly moderately part because the variation is sometimes so little and you have to twist your head around to see how your answer fits with those words-it’s hard to pick. I always find myself doing these things to one side of the middle. SEA-416 |
5.4.5 | Discrepancy in midpoint anchor word meaning | I thought there was going to- how much improvement- to me the scale, when it was 1 to 10 before I had put two or three as somewhat, but then you changed that to 5 so I chose 5. SO YOU CHOSE 5 BECAUSE I SAID 5 IS SOMEWHAT SUCCESSFUL? Yeah, on a zero to ten I thought of 5 as like 50% improvement, so to me somewhat successful is a bit less than that. Somewhat successful to me is like 20 to 30%. 50% improvement is more noticeable. AZ-427 |