Projects / Programmes
The integration of Internet technology into probability-based sample surveys
| Code |
Science |
Field |
Subfield |
| 7.00.00 |
Interdisciplinary research |
|
|
| Code |
Science |
Field |
| S274 |
Social sciences |
Research methodology in science |
quantitative research, statistics, survey methodology, web survey, probability sampling, Internet technologies, Web 2.0, operations research, optimization of costs and errors
Researchers (8)
Organisations (1)
Abstract
Contemporary survey research nowadays encounters ever more problems related to declining response rates and growing costs, which is leading to a serious deterioration of data quality. Within this context, web surveys hold the potential for cost reductions but they are also increasingly introducing a non-probability sampling approach (self selection of respondents, internet panels). The latter is, however, very problematic for academic and official surveys which strictly require probability (scientific) samples. In addition, web surveys based on probability samples suffer from low response rates, limited sampling frames and solicitation problems. The integration of web surveys into probability samples is thus one of the biggest challenges of contemporary social science research methodology. This is confirmed by the amount of these issues which appear at the leading conferences in this area and by the large national research projects, which have recently addressed these problems (i.e. in the USA, the Netherlands, the UK, Germany etc). The anticipated methodological research within the European Social Survey (ESS) also addresses web surveys as a mode for future ESS surveys.
The project will analyse the potential of the optimal integration of the web into the probability sample surveys in Slovenia. At the same time, it will contribute to the understanding of these problems in the global scientific community.
The project will establish a model for the optimal introduction of the web in the mixed-mode context. First, a meta-analysis of factors influencing participation (based on the WebSM portal) on mixed-mode options will be performed. Next, pilot experiments will be conducted (surveys of the RIS project and of the Statistical Office) to analyse background variables (socio-demographics, social networks, attitudes) and to estimate the probabilities to participate in various types (combinations) of survey modes. This will create the basis for an on-line decision tool for an individually tailored strategy for each respondent (i.e. selection of the next contact mode, the future timing of contacts, type of incentives, type of questionnaire etc), which will be optimised with respect to costs and errors (Means Squared Error). The model will be implemented in an innovative pilot on-line tool (Web 2.0) that will integrate support for all three key modes of a computer-assisted survey (Web, CATI, CAPI) and will be made publicly available for use and further development at the address http://oneclicksurvey.websm.org.
The project combines social science research, survey methodology, statistics, operational research and informatics. It upgrades past work of the research group and the EU Framework programme (Web Survey Methodology, http://WebSM.org) co-ordinated by the proposing principal investigator of this project. It is also linked to corresponding activities of the European Social Survey (ESS), in which the Faculty of Social Sciences collaborates as a partner, while this project’s proposing principal investigator serves as an expert member of the ESS Methods group.
Significance for science
The results of the project have contributed new important findings in the field of web survey methodology and survey methodology in general. Namely, the project addressed the problem of inclusion of the web mode into mixed-mode survey data collection, with the purpose of optimizing the whole survey process, which is one of the central challenges of the modern survey methodology. The importance of such approaches is underlined by the fact that even the largest academic survey studies, like European Social Survey, intensively investigate mixed-mode (including web) surveys as alternatives to expensive face-to-face data collection.
The empirical study showed that less cost demanding mixed-mode surveys can perform comparably to face-to-face surveys or even surpass them in terms of response rates. In this regards, the most important contribution of the project is development of the methodology for evaluation of optimality of various survey designs by simultaneous inclusion of data quality and costs criteria. Both, the empirical study and evaluation of the existing literature confirmed that only such comprehensive approach can ensure sufficiently high quality of data at preserved cost effectiveness. The findings reject still prevailing use of response rates as the key indicator of survey data quality and emphasize the importance of considering all survey error sources.
With the establishment of the optimization model and its implementation we offered the survey methodology an important tool for further development of survey research, which will enable fulfilling the survey quality in its broader sense, not only regarding data quality, but also timeliness of data collection, appropriate costs and other criteria. Academic and research institutions can achieve a higher-quality and more cost effective introduction of new technologies into their research processes and thus optimize the whole survey process.
The application 1CS, which has been developed during the project, presents a very powerful solution for implementation of web surveys and other survey modes. Advanced features enable scientific and other users to prepare and conduct the most complex web surveys that adhere to modern standards in the field. Due to its open-source nature, the application is freely available for use and development to all interested users, which ensures a high level of flexibility for various research problems and allows effective enhancements of functionalities with new advancements in survey methodology. By simultaneous application of other project results, users are offered a comprehensive solution for more optimal conduction of the survey research processes, which is becoming increasingly important also in scientific research.
The enhancement of the contents of the WebSM portal on the basis of project research activities, we offered the scientific community and other users even richer information source on web survey methodology. We included a large number of new existing studies and all published findings that were obtained within the project. The portal thus preserves and further strengthens its position of the leading website in this field and is an important resource for research work and obtainment of new findings for a number of researchers. At the same time, it represents a central source for a widespread international dissemination of the project results.
Finally, it should be stressed that the project results have also built an important foundation for further research work in the field. Since the project addressed one of the most topical problems of survey research, it has established a basis on which it will be possible to apply for new projects, funded by the EU and other international and national research organizations.
Significance for the country
The project results have and will further contribute to the development of survey research in Slovenia, enhancements of findings by the Slovenian science and the consolidation of its role in the international community.
In the narrower sense, the project contributed a new knowledge about the possibilities of integration of the Internet technology into representative surveys in Slovenia. Investigations of effectiveness of any new approaches to survey research must always inevitably consider specific socio-cultural environment in which they being are performed. The project represents a pioneering contribution to a systematic research on mixed-mode surveys with the web mode in Slovenia.
The empirical research within the project, which was entirely performed in Slovenia, and the development of the optimization model, are especially beneficial for Slovenian research organizations, since they can directly apply the project findings into their research activities. The utilization of the developed solutions enable them a comprehensive evaluation of different approaches to survey data collection, offering a higher level of optimization of research processes in terms of data quality and costs. This can also substantially increase the competitiveness of research services in Slovenia. Especially important in this regards is the cooperation with the Statistical Office of the Republic of Slovenia. Namely, the field of official research requires a high level of data quality (accuracy of estimates), making data collection with the traditional survey modes highly expensive. The Statistical Office therefore intensively investigates different alternative possibilities for lowering research costs at simultaneous preservation of an adequate data quality level. The application of the proposed models for estimation of costs and optimality of different survey modes has here a large potential for increased cost-effectiveness of official research.
Of a special importance for the Slovenian research area is also the web application 1CS, which is (also) available in Slovenian language. This has enormously encouraged its use in the Slovenian academic research and among students. Furthermore, it is being used by some large research institutions, including the Statistical Office and the National Health Institute. A wide availability and friendly user interface also contribute to further popularization of web survey research with of a higher quality among a wide spectrum of users, from pupils and students to scientific researchers.
By focusing on the implementation of modern information-communication technologies into the scientific research work, the project research activities have contributed to a broader accomplishment of the priorities of the Ministry of Higher Education, Science and Technology. The activities thus address one of the areas that according to the Resolution on National Research and Development Programme for the period 2006-2010 demonstrate the highest potential for advancement of economic competitiveness and productivity as well as the technological and developmental advancement of the economy and the whole society.
Finally, the project activities and its results represent a continuation of a successful scientific work of the research centre on the Faculty of Social Sciences and advancement and strengthening of its competitiveness in the scientific community. The centre thus preserves the position of one of the world’s leading research groups in the field of web survey methodology. The corresponding dissemination of the project’s research results substantially contribute to the international reputation and impact of the Slovenian research activities and allow strengthening of the position of Slovenian science in the international scientific community.
Most important scientific results
Annual report
2008,
2009,
final report,
complete report on dLib.si
Most important socioeconomically and culturally relevant results
Annual report
2008,
2009,
final report,
complete report on dLib.si
