When comparing social science phenomena through a time perspective, absolute and relative difference (RD) are the two typical presentation formats used to communicate interpretations to the audience, while time distance (TD) is the least frequently used of such formats. This article argues that the chosen presentation format is extremely important because the various formats suggest different substantive interpretations. To elaborate upon this issue, researchers from the Statistical Office of the Republic of Slovenia, National institute of Public Health, and general academia were invited to participate in an experiment with alternative presentation formats that describe changes in certain social science phenomena over time. The results revealed a prevailing tendency of respondents to rely on interpretations related to absolute differences, which was additionally reinforced with graphical presentation formats. Therefore, whenever RD or TD is more proper for substantive interpretations, the corresponding presentation format must be designed with special attention.
This paper elaborates upon differences in socially desirable responding as beingthe result of mode effects between web, telephone, and face-to-face survey modes.Social desirability is one of the main threats to comparability of data between dif-ferent modes. The paper conceptualises socially desirable responding as a specifictype of mode effect, which is not only a result of inherent characteristics of a surveymode, but is also mediated and moderated by complex interdependencies of spe-cific survey implementations, contextual factors, and characteristics and behavioursof respondents. While web surveys are generally less prone to socially desirable re-sponding, it is essential to be wary of circumstances that may reduce the perceivedprivacy of the survey situation and lead to biased reporting. The presented empiricalstudy analyses the answers to a large number of items used in a pilot implementationof the Generations and Gender Survey across the three modes to gain insights intothe incidence of socially desirable responding and its role in the observed differencesin estimates. The comparison of means, distributions, and proportions of extreme re-sponses to scale questions is performed across 89 survey items. The results are inline with the previous findings on lower susceptibility of web surveys to social de-sirability bias. More importantly, the findings suggest that the problem of sociallydesirable responding is likely to be a major contributor to the differences in meanestimates, response distributions, and the level of extreme responding between the studied modes.
The article addresses the issues related to capturing and processing response time paradata – data on the process of answering the survey questionnaire that researchers have been intensively using in the past two decades to evaluate and improve survey instruments as well as to understand the survey response process. This article focuses on approaches for identifying and processing response time and response time outliers. It presents a systematic overview of scientific papers on response time and response time outliers in web surveys. The results show that despite the great potential of response time paradata and the considerable body of existing research a large gap remains in understanding the true potential of response time paradata. This is mostly due to lacking deeper insights into the links between response time paradata and response quality of respondents.
Meta-analytic techniques have become the standard methods for aggregating the results from thematically related studies in the behavioral, health, and economic sciences. The primary objective of the systematic review was to identify metaanalyses, which were previously conducted in survey methodology and classify them according to the thematic areas that they addressed. The findings are based on a systematic search of two bibliographic harvesters (together covering 265 bibliographic databases), which yielded 54 eligible manuscripts reporting 60 meta-analytic studies and 91 effect sizes. The thematic areas were structured under the seven dimensions of the total survey error (TSE). The results showed that past meta-analyses partly cover two of the seven TSE dimensions, namely measurement and nonresponse error, where paradata have an important role, while research questions pertaining to survey methodology under the remaining TSE dimensions have not yet been investigated by the meta-analyses. In this context, the systematic review discussed gaps in current research and potential opportunities for future meta-analyses in survey methodology. The analysis also shows a relatively large untapped potential of paradata.
Do web surveys still yield lower response rates compared with other survey modes? To answer this question, we replicated and extended a meta-analysis done in 2008 which found that, based on 45 experimental comparisons, web surveys had an 11 percentage points lower response rate compared with other survey modes. Paradata also have an important role in this analysis, especially when monitoring the number and invitation mode of contacts. Fundamental changes in internet accessibility and use since the publication of the original meta-analysis would suggest that people's propensity to participate in web surveys has changed considerably in the meantime. However, in our replication and extension study, which comprised 114 experimental comparisons between web and other survey modes, we found almost no change: web surveys still yielded lower response rates than other modes (a difference of 12 percentage points in response rates). Furthermore, we found that prenotifications, the sample recruitment strategy, the survey's solicitation mode, the type of target population, the number of contact attempts, and the country in which the survey was conducted moderated the magnitude of the response rate differences. These findings have substantial implications for web survey methodology and operations.