Approaches
click here to
return to methods page
Canadian Evaluation Society Project in Support of Advocacy and
Professional Development http://consultation.evaluationcanada.ca/results.htm
presents
a
quick
list of potential benefits that can be derived from evaluation,
outputs that result from evaluation activities, and knowledge
elements required to carry out evaluation activities.
Joint Committee on Standards for Educational Evaluation has Program
Evaluation Standards. A summary is here but people are encouraged to
read the whole book.
http://www.jcsee.org/program-evaluation-standards/program-evaluation-standards-statements
"Joint Committee is a coalition of major professional associations
concerned with the quality of evaluation. ...
The Joint Committee is accredited by the American National Standards
Institute (ANSI). Standards approved by ANSI become American
National Standards."
American Evaluation Assocation has Guiding Principals http://www.eval.org/Publications/GuidingPrinciples.asp
such as Systematic Inquiry, Competence, Integrity/Honesty, Respect
for People, and Responsibilities for General and Public Welfare
Action research
Action research International http://www.scu.edu.au/schools/gcm/ar/arhome.html
see
the
FAQ
page about action research http://www.scu.edu.au/schools/gcm/ar/arp/arfaq.html
Also
see
Action
Research International, an on line journal http://www.scu.edu.au/schools/gcm/ar/ari/arihome.html
Understanding Action Research http://cadres.pepperdine.edu/ccar/define.html
an overview
Action Research made simple http://www.youtube.com/watch?v=Qg83f72_6Gw
Checklists
Evaluation Checklists http://www.wmich.edu/evalctr/checklists/
This site provides refereed checklists for designing, budgeting,
contracting, staffing, managing, and assessing evaluations of
programs, personnel, students, and other evaluands; collecting,
analyzing, and reporting evaluation information; and determining
merit, worth, and significance
Michael Scriven has some checklists http://michaelscriven.info/papersandpublications.html
about meta evaluation and key evaluation
Collaborative, Participatory, and
Empowerment Evaluation
Collaborative, Participatory, and Empowerment
Evaluation http://homepage.mac.com/profdavidf/empowermentevaluation.htm
This is the home of the AEA tig for this topic.
Empowerment Evaluation and the Internet: A Synergistic
Relationship http://cie.asu.edu/volume1/number4/index.html
an article about empowerment evaluation, by David M.
Fetterman
Guide to Project Evaluation: A Participatory Approach
http://www.phac-aspc.gc.ca/ph-sp/resources-ressources/guide/index-eng.php
This includes a brief description of what is participatory
evaluation.
Impact Evaluation
Network of Networks on Impact Evaluation (NONIE) http://www.worldbank.org/ieg/nonie/index.html
has
some
stuff,
more coming soon. Some of their documents are on their members
page.
World Bank's impact
evaluation another page on impact evaluation, has this
definition: "An impact evaluation assesses the changes in the
well-being of individuals that can be attributed to a particular
project, program or policy. This website aims at disseminating
information and providing resources for people and organizations
working to assess and improve the effectiveness of interventions
aimed at reducing poverty."
And lists other resources.
The World Bank has this http://www.worldbank.org/ieg/influential_evaluations/
Influential
Evaluations:
Evaluations
that Improved Performance and Impacts of Development Programs
Impact Evaluation: Methodological and Operational Issues
http://www.adb.org/Documents/Handbooks/Impact-Analysis/default.asp
"This
quick
reference
provides an overview of methods available for evaluating impacts of
development programs, and addresses some common operational concerns
about their practical application." From the Asian Development
Bank.
Final Report of the Evaluation Gap Working Group: "When Will We Ever
Learn? Improving Lives through Impact Evaluation" http://www.cgdev.org/section/initiatives/_active/evalgap
"The
Evaluation
Gap
Working Group was convened to understand the reasons for the lack of
good impact evaluation and the possible ways to make significant
progress toward solving the problem. After a year of research and
deliberation and further consultation among a broad set of
interested parties, the group reached a recommendation for what the
international community must do to close the "evaluation gap": take
individual action to reinforce existing efforts, and make a major
advance through a collective endeavor."
and in response:
International Workshop on Impact Evaluation for Development, 15
November 2006 - Hosted by the World Bank and the DAC Evaluation
Network http://www.oecd.org/document/56/0,3343,en_21571361_34047972_37626936_1_1_1_1,00.html
presentations available here.
Impact Evaluation – Its Purpose and Use http://www.oag.govt.nz/central-govt/1st-report-2000
This
report
includes
a discussion of impact evaluation, along with how its used in the
govt.
Evaluation designs to assess program impact http://www.fhi.org/en/Youth/YouthNet/Publications/Focus/ToolsGuides/index.htm
Chapter 5 has some info This chapter "Offers guidance on an
considerations around the need for impact evaluation, Reviews study
designs you can use to carry out an impact evaluation, Outlines the
technical requirements and resources needed for each type of
evaluation." From Family Health International.
Systems
Systems thinking and program evaluation http://facilitatedsystems.com/weblog/2005/06/systems-thinking-and-program.html
a
portal
to
more info.
Integrated/overall approach
Easy Outcomes http://www.easyoutcomes.org/
"Easy
Outcomes
is
a visual way of working which lets you do strategic planning,
monitoring, evaluation, reporting and contracting for any project,
program, organization, collaboration or sector."
Other
IFADs new approach to evaluation http://www.ifad.org/evaluation/oe/approach/1_0.htm
This
includes
1.
A strong service and partnership orientation, 2.Evaluation
work is working together, 3.Moving evaluation to a higher
plane and 4.Evaluation work needs to be evaluated.
Also, evaluation work is inseparable from learning, and its
resolve to learn with its partners and develop together with them
the recommendations and lessons learned to help improve the
performance of IFAD’s operations.
Realistic Evaluation: An Overview, Nick Tilley. http://www.evidence-basedmanagement.com/research_practice/
This paper was presented at the 2000 Founding Conference of
the Danish Evaluation Society. Tilley spells "out what I think
evaluation can usefully contribute to social policy
development." Tilley writes: "Whereas the question which was
asked in traditional experimentation was, 'Does this work?' or
'What works?', the question asked by us in realistic
evaluation is 'What works for whom in what
circumstances?'"
How research and evaluation evidence contributes to policy
making http://www.nationalschool.gov.uk/policyhub/evaluating_policy/how_res_eval_evid.asp
shows the many factors, including evaluation, that
contribute to policy making. Also shows that values (ie politics)
contribute to evaluation.
Real world evaluation http://www.realworldevaluation.org/
"It is not uncommon for evaluators to be asked to perform an
evaluation of a project that is well underway or almost finished,
yet when they ask questions about comparable baseline data and
control groups their clients respond with blank stares. Too
typically time and resources are few, yet clients expect “rigorous
impact evaluation.” And there are also political pressures to deal
with. The RealWorld Evaluation approach was developed specifically
to address the need for practical strategies on how to ensure the
highest level of methodological rigor consistent with the
circumstances under which the evaluation has to be
conducted." The book is by Michael Bamberger, Jim Rugh
and Linda Mabry.
Logic Models
Just to start, a logic model is
a formal method to organize what you know about your program
(goals, activities, methods, expected outcomes, etc) so you can
clearly specify what you want to evaluate.
State Program Evaluation Guides: Developing and Using a Logic
Model http://www.cdc.gov/DHDSP/programs/nhdsp_program/evaluation_guides/logic_model.htm
another overview
Logic models http://www.evaluationwiki.org/index.php/Logic_Model
Overview
of
what
they are.
Logic model http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html
oveview
from
UW
Cooperative Extension
Logic model development guide http://www.wkkf.org/knowledge-center/publications-and-resources.aspx
Good outcomes model guidelines http://www.easyoutcomes.org/guidelines/outcomesguidelines.html
and here http://www.outcomesmodels.org/guidelines.html
similar to logic model, but different name
Logic model presentation http://www.usablellc.net/html/links_we_like.html#logic_models
slide
show
and
talk about what are logic models.
Politics of evaluation
Welcome to Logic Models: The Basics! http://apps.nccd.cdc.gov/DASHOET/Logic_model/
"This tutorial will provide you with information and
resources to assist you as you plan and develop a logic model to
describe your program and help guide program evaluation."
Decision-Making in Government: The Role of Program
Evaluation http://www.tbs-sct.gc.ca/eval/tools_outils/Aucoin/Aucoin_e.asp
"Decision-making
in
government
is a process in which evidence, both from systematic research and
practical experience, mixes with a complex interaction of ideas,
interests, ideologies, institutions and individuals. These several
factors are the determinants of decisions at the political and
administrative levels. At different times and under different
regimes, the decision-making process will be structured and managed
in ways that seek to give more or less weight to evidence. No one
process is necessarily or always more ‘rational’ than the others. It
all depends on what questions need to be asked by decision-makers in
the circumstances and context of the times in order to make the best
possible decisions for their agendas and/or public expectations of
good governance.
As a consequence, the importance attached to the
use of evidence in decision-making invariable waxes and wanes over
time."
Improving agricultural extension. A reference manual http://www.fao.org/docrep/W5830E/w5830e00.htm
see
chapter
11
http://www.fao.org/docrep/W5830E/w5830e0d.htm
Evaluating
extension
programmes,
some discussion about politics of evaluation.
From Addressing Barriers to Learning, Vol. 3 (1), Winter 1998.
Accountability: Is it Becoming a Mantra? http://smhp.psych.ucla.edu/account.htm
School
Mental
Health
Project-UCLA, Center for Mental Health in Schools.
"evaluation is not simply a technical process.
Evaluation involves decisions about what, how, and when to measure,
and these decisions are based in great part on values and beliefs.
As a result, limitedknowledge, bias, vested interests, and ethical
issues are constantly influencing evaluation processes and the
decisions made with respect to accountability."
From The Johns Hopkins University School for Advanced International
Studies http://www.sais-jhu.edu/cmtoolkit/issues-in-practice/evaluation/effectiveness.htm
"evaluation is inevitably a political act – evaluation is
always carried out for someone and for some purpose. At times this
may undermine the integrity of the evaluation." Some further
discussion.
Evaluation Journal of Australasia http://www.aes.asn.au/publications/
has
several
articles
about
politics in evaluation. Examples are 'A balancing act':
resolving multiple stakeholder interests in program evaluation -
Anne Markiewicz, (March/April 2005, vol 4, no 1,2) and The
politics of program logic – Brian English, Lisette Kaleveld (August
2003, vol 3, no 1).
Chapter 11 - Evaluating extension programmes http://www.fao.org/docrep/W5830E/w5830e0d.htm
has a section "Recognizing the politics of evaluation: Know the
stakeholders or you will be sorry"
Organization of evaluation
The Importance of Evaluation Associations and Networks http://www.agenciadeavaliacao.org.br/artigos/&id=5
By Jean Serge Quesnel, listed at the La Agencia Brasileña de
Evaluación website. Describes how evaluation associations and
networks are organized and tied together.
click here to
return to methods page
last updated September 2011
last verified September 2011