Evaluation Essentials - Methods for Conducting Sound Evaluation Research

An indispensable primer on program evaluation, with step-by-step instructions and real world examples The field of evaluation, which has its roots in the evaluation of Federal programs, has grown into an important aspect of the nonprofit funding cycle. Many funders require that initiatives they fund be subject to some form of evaluation, and the demand for trained evaluators is greater than ever before. Evaluation Essentials offers an introduction to the theory and practice of evaluation, with examples from sectors including public policy, public health, non-profit management, social work, arts management, education, international assistance, and labor. Beth Osborne DaPonte, a leading authority in program evaluation, describes a step-by-step approach to forming evaluation questions, describing programs using Theory of Change and program logic models, understanding causation as it relates to evaluation, using quasi-experimental design, and creating meaningful outcome measures. This Second Edition covers the latest best practices and advances in the field, helping readers develop the essential skills they'll need to work as an evaluator or to wisely hire an evaluator. With Evaluation Essentials, nonprofit leaders, funders, government agencies, NGOs and students gain a thorough and thoughtful approach to demonstrating real impact.

List of contents

Preface ix
Acknowledgments x
One: Introduction 1
Learning Objectives 1
Introduction 2
Size and Importance of Nonprofit Sector 3
The Evaluation Framework 5
Structure of Book 9
Summary 9
Key Terms 10
Discussion Questions 10
Two: Describing the Initiative 11
Learning Objectives 11
Introduction 12
Reasons to Rigorously Describe the Program 13
The Logistics of Describing the Program 14
Common Mistakes Made Describing Initiative and Programs 16
The Program Is Alive and So Is Its Description 18
Theory of Change 19
The Program Logic Model 25
Analyzing the Program Logic Model 34
Challenges of Multisite Programs 35
Program Implementation Model 36
Examples of Program Descriptions 37
Summary 67
Key Terms 68
Discussion Questions 68
Three: Laying the Evaluation Ground Work 69
Learning Objectives 69
Evaluation Approaches 70
Framing Evaluation Questions 72
Insincere Reasons for Evaluation 75
High Stakes Evaluation 75
Who Will do the Evaluation? 76
External Evaluators 77
Internal Evaluators 79
Independence of the Evaluator 79
External Versus Internal Evaluators 80
Confidentiality and Ownership of Evaluations 81
Endorsement and Action on Recommendations 82
Evaluation Policy 83
The Evaluation Report 84
Summary 86
Key Terms 86
Discussion Questions 86
Four: Causation 87
Learning Objectives 87
Introduction 88
Necessary and Sufficient 88
Setting Cutoff Points and Causal Interpretation 91
Intervening Variables 95
Types of Causal Effects 97
Lagged Effects 98
Permanency of Effects 98
Functional Form of Impact 99
Spectacular Causes and Effects 100
Summary 101
Key Terms 102
Discussion Questions 102
Five: the Prisms of Validity 103
Learning Objectives 103
Introduction 104
Statistical Conclusion Validity 105
Types of Erroneous Conclusions: Type 1 and Type 2 Errors 106
Threats to Statistical Conclusion Validity 107
Internal Validity 111
Threats to Internal Validity 111
Summary 118
Construct Validity 119
Threats to Construct Validity 120
Summary 122
External Validity 122
Considerations for Determining the External Validity of Studies 122
Summary 123
Conclusions 124
Key Terms 124
Discussion Questions 124
Six: Attributing Outcomes to the Program: Quasi-Experimental Design 126
Learning Objectives 126
Introduction 127
Quasi-Experimental Notation 127
Frequently used Designs that do not Show Causation 128
One-Group Posttest-Only 128
Posttest-Only with Nonequivalent Groups 129
Participants' Pretest-Posttest 130
Designs that Generally Permit Causal Inferences 131
Untreated Control Group Design with Pretest and Posttest 131
Delayed Treatment Control Group 137
Different Samples from the Same Populations Design 140
Nonequivalent Observations Drawn from One Group 141
Equivalent Groups Using Switched Measures 142
Cohort Designs 144
Time Series Designs 146
Summary 148
Key Terms 149
Discussion Questions 149
Seven: Collecting Data 150
Learning Objectives 150
Introduction 151
Informal and Loosely Structured Interviews 152
Focus Groups 154
Surveys 160
Survey Instrument Design 161
Target Populations and Sampling 169
Informal Sampling Approaches 170
Formal Sampling Approaches 171
Best Practices in Administering Surveys to a Purposive Sample 172
Follow-up for Nonresponse 173
Secondary Data Sources 173
Summary 174
Key Terms 174
Discussion Questions 175
Eight: Conclusions 176
Learning Objectives 176
Using Evaluation Tools to Write Grant Proposals 177
Hiring Evaluation Consultants 177
Writing Recommendations 180
The Tone of Evaluation Reports 180
Finalization Process of Evaluation Reports 181
Conclusions 182
Discussion Questions 182
Glossary 183
Index 188

About the author

Beth Osborne DaPonte, Ph.D. has taught program evaluation at the graduate level at Yale University's School of Management, Carnegie Mellon University, and the University of Pittsburgh. Additionally, Beth is Principal and Owner of Social Science Consultants, LLC, a firm that provides consulting and advisory services to the non-profit/NGO, religious, governmental, and international sectors. Clients have included the United Nations (secretariat and UN-System funds, programs, and agencies); foundations (e.g., Forbes Funds, Heinz Endowments, Hillman Family Foundations); educational institutions (e.g., University of Pittsburgh); and US-based nonprofits (e.g., Greater Pittsburgh Community Food Bank, Allies for Children).

Product details

• Authors: Beth Osborne Daponte, Beth Osborne DaPonte, Beth Osborne (Yale University) Daponte, Daponte Beth Osborne
• Publisher: Wiley, John and Sons Ltd
• Languages: English
• Product format: Paperback / Softback
• Released: 23.04.2025
• EAN: 9781394234783
• ISBN: 978-1-394-23478-3
• No. of pages: 208
• Subjects: Children's and young people's books > Non-fiction books / Non-fiction picture books > General, reference works; Humanities, art, music > Education > Education system; EDUCATION / Evaluation & Assessment, Education: examinations and assessment, Examinations & assessment