Mehr lesen
A new and updated definitive resource for survey questionnaire testing and evaluation
Building on the success of the first Questionnaire Development, Evaluation, and Testing (QDET) conference in 2002, this book brings together leading papers from the Second International Conference on Questionnaire Design, Development, Evaluation, and Testing (QDET2) held in 2016. The volume assesses the current state of the art and science of QDET; examines the importance of methodological attention to the questionnaire in the present world of information collection; and ponders how the QDET field can anticipate new trends and directions as information needs and data collection methods continue to evolve.
Featuring contributions from international experts in survey methodology, Advances in Questionnaire Design, Development, Evaluation and Testing includes latest insights on question characteristics, usability testing, web probing, and other pretesting approaches, as well as:
* Recent developments in the design and evaluation of digital and self-administered surveys
* Strategies for comparing and combining questionnaire evaluation methods
* Approaches for cross-cultural and cross-national questionnaire development
* New data sources and methodological innovations during the last 15 years
* Case studies and practical applications
Advances in Questionnaire Design, Development, Evaluation and Testing serves as a forum to prepare researchers to meet the next generation of challenges, making it an excellent resource for researchers and practitioners in government, academia, and the private sector.
Inhaltsverzeichnis
List of Contributors xvii
Preface xxiii
Part I Assessing the Current Methodology for Questionnaire Design, Development, Testing, and Evaluation 1
1 Questionnaire Design, Development, Evaluation, and Testing: Where are We, and Where are We Headed? 3
Gordon B. Willis
1.1 Current State of the Art and Science of QDET 3
1.2 Relevance of QDET in the Evolving World of Surveys 11
1.3 Looking Ahead: Further Developments in QDET 16
1.4 Conclusion 19
References 20
2 Asking the Right Questions in the Right Way: Six Needed Changes in Questionnaire Evaluation and Testing Methods 25
Don A. Dillman
2.1 Personal Experiences with Cognitive Interviews and Focus Groups 25
2.2 My 2002 Experience at QDET 29
2.3 Six Changes in Survey Research that Require New Perspectives on Questionnaire Evaluation and Testing 33
2.4 Conclusion 42
References 43
3 A Framework for Making Decisions about Question Evaluation Methods 47
Roger Tourangeau, Aaron Maitland, Darby Steiger, and Ting Yan
3.1 Introduction 47
3.2 Expert Reviews 48
3.3 Laboratory Methods 51
3.4 Field Methods 55
3.5 Statistical Modeling for Data Quality 59
3.6 Comparing Different Methods 63
3.7 Recommendations 67
References 69
4 A Comparison of Five Question Evaluation Methods in Predicting the Validity of Respondent Answers to Factual Items 75
Aaron Maitland and Stanley Presser
4.1 Introduction 75
4.2 Methods 76
4.3 Results 79
4.4 Discussion 84
References 85
5 Combining Multiple Question Evaluation Methods: What Does tt Mean When the Data Appear to Conflict? 91
Jo d'Ardenne and Debbie Collins
5.1 Introduction 91
5.2 Questionnaire Development Stages 92
5.3 Selection of Case Studies 93
5.4 Case Study 1: Conflicting Findings Between Focus Groups and Cognitive Interviews 95
5.5 Case Study 2: Conflicting Findings Between Eye-Tracking, Respondent Debriefing Questions, and Interviewer Feedback 97
5.6 Case Study 3: Complementary Findings Between Cognitive Interviews and Interviewer Feedback 100
5.7 Case Study 4: Combining Qualitative and Quantitative Data to Assess Changes to a Travel Diary 104
5.8 Framework of QT Methods 110
5.9 Summary and Discussion 110
References 114
Part II Question Characteristics, Response Burden, and Data Quality 117
6 The Role of Question Characteristics in Designing and Evaluating Survey Questions 119
Jennifer Dykema, Nora Cate Schaeffer, Dana Garbarski, and Michael Hout
6.1 Introduction 119
6.2 Overview of Some of the Approaches Used to Conceptualize, Measure, and Code Question Characteristics 120
6.3 Taxonomy of Question Characteristics 127
6.4 Case Studies 132
6.5 Discussion 141
Acknowledgments 147
References 148
7 Exploring the Associations Between Question Characteristics, Respondent Characteristics, Interviewer Performance Measures, and Survey Data Quality 153
James M. Dahlhamer, Aaron Maitland, Heather Ridolfo, Antuane Allen, and Dynesha Brooks
7.1 Introduction 153
7.2 Methods 157
7.3 Results 174
7.4 Discussion 182
Disclaimer 191
References 191
8 Response Burden: What is it and What Predicts It? 193
Ting Yan, Scott Fricker, and Shirley Tsai
8.1 Introduction 193
8.2 Methods 197
8.3 Results 202
8.4 Conclusions and Discussion 206
Acknowledgments 210
<
Über den Autor / die Autorin
PAUL C. BEATTY is Chief of the Center for Behavioral Science Methods at the U.S. Census Bureau.
DEBBIE COLLINS is a Senior Research Director at the National Centre for Social Research, UK.
LYN KAYE is a consultant in Survey Research Methods, and previously Statistics New Zealand's Senior Researcher.
JOSE???LUIS PADILLA is Professor of Methodology of Behavioral Sciences at University of Granada, Spain.
GORDON B. WILLIS is Cognitive Psychologist at the National Cancer Institute, National Institutes of Health, USA.
AMANDA WILMOT is a Senior Study Director at Westat, USA.
Zusammenfassung
A new and updated definitive resource for survey questionnaire testing and evaluation
Building on the success of the first Questionnaire Development, Evaluation, and Testing (QDET) conference in 2002, this book brings together leading papers from the Second International Conference on Questionnaire Design, Development, Evaluation, and Testing (QDET2) held in 2016. The volume assesses the current state of the art and science of QDET; examines the importance of methodological attention to the questionnaire in the present world of information collection; and ponders how the QDET field can anticipate new trends and directions as information needs and data collection methods continue to evolve.
Featuring contributions from international experts in survey methodology, Advances in Questionnaire Design, Development, Evaluation and Testing includes latest insights on question characteristics, usability testing, web probing, and other pretesting approaches, as well as:
* Recent developments in the design and evaluation of digital and self-administered surveys
* Strategies for comparing and combining questionnaire evaluation methods
* Approaches for cross-cultural and cross-national questionnaire development
* New data sources and methodological innovations during the last 15 years
* Case studies and practical applications
Advances in Questionnaire Design, Development, Evaluation and Testing serves as a forum to prepare researchers to meet the next generation of challenges, making it an excellent resource for researchers and practitioners in government, academia, and the private sector.
