Visual analogue scales vs. slider scales in Web surveys: Pros and cons
Paper presented at the
Midterm Conference "Quantitative Methods in European Sociology" of the
European Sociological Association (ESA),
Research Network RN21
November 9 & 10, 2012 in Maribor (Slovenia)
The present paper reports findings from an experimental comparison of two graphic rating scales for Web-based research, namely visual analogue scales (VAS) and slider scales. On the first look, VASs and slider scales are quite similar. Both scales are usually made from a simple horizontal line where only the ends have (e.g., verbal) anchors. In contrast to slider scales VASs are not implemented as scales with a limited number of response options but mostly as continuous rating scales. In computerized data collection every pixel in length serves as a possible rating (for a working example see http://vasgenerator.net). There is evidence that respondents are capable of using the multitude of response options in a meaningful way and that data collected with VASs are on the level of an interval scale (see Reips & Funke, 2008).
In comparison to categorical rating scales, slider scales suffer from high rates of break-off, especially for respondents with low formal education (see Funke, Reips, & Thomas, 2011). Furthermore, there are indicators that respondents have difficulties with understanding how to use the scale, especially regarding the category the slider’s handle is positioned in the beginning. Research on semantic differentials made from categorical rating scales versus VASs could show beneficial effects of VASs (see Funke & Reips, 2012). No direct comparison between VAS and slider scales has been published so far.
The experiment was included in a Web survey (N = 1807). In a fully crossed 3 X 3 factorial between-subjects design the rating scales provided to answer three closed-ended items were either VASs, slider scales, or categorical scales made from standard HTML radio buttons. The rating scales had three, five, or seven response options. Furthermore, continuous VASs with 200 response options were used.
Break-Off. Slider scales made more respondents break-off than the other scales, especially those with a low formal education.
Item Nonresponse. VAS and radio button scales did not differ regarding the share of item nonresponse. However, an inconsiderably large share of respondents did not change the initial position of the slider.
Distribution of Ratings. Overall, slider scales lead to fewer ratings of middle intensity than VAS and radio buttons, which did not differ from one another.
Response Time. Analyzing client-sided paradata, VAS and slider scales lead to higher response times than radio button scales.
Overall, VASs and radio button scales did not differ significantly. The number of response options had no effect.
Considering present as well as previous findings, there is no point in using slider scales. Slider scales may substantially increase break-off and harm data quality. Analysis of the distribution of values suggests, that even respondents with a high formal education and a high computer literacy have problems using this type of rating scale. Furthermore, slider scales are likely to foster response styles like acquiescence.
Whenever categorical ratings are sufficiently precise, it is recommended to use standard, low-tech rating scales made from radio buttons in Web surveys. However, VASs can be an interesting alternative whenever measurement of small differences is relevant.