RESEARCH ABOUT RESEARCH
METHODS
click here to
return to methods page
Journals
African Evaluation Journal http://www.aejonline.org/index.php/aej/index
"The African Evaluation Journal will publish high quality
peer-reviewed articles of merit on any subject related to
evaluation, and provide targeted information of professional
interest to members of AfrEA and its national associations and
evaluators across the globe."
Electronic Journal of Business Research Methods http://www.ejbrm.com/
"provides
perspectives
on
topics
relevant
to
research in the field of business and management."
Harvard Family Research Project's Evaluation Exchange http://www.hfrp.org/evaluation/the-evaluation-exchange
on
line
periodical.
"addresses
current
issues
facing program evaluators of all levels, with articles written by
the most prominent evaluators in the field. "
The International Journal of Qualitative Methods http://ejournals.library.ualberta.ca/index.php/IJQM/index
"The journal is multilingual and multidisciplinary, with a
focus on qualitative research methods."
Journal of Methods and Measurement in the Social Sciences (JMM)
https://journals.uair.arizona.edu/index.php/jmmss/index
"is an online scholarly publication focusing on methodology
and research design, measurement, and data analysis"
The Journal of Modern Applied Statistical Methods http://digitalcommons.wayne.edu/jmasm/
"an independent, peer-reviewed, open access journal designed to
provide an outlet for the scholarly works of applied nonparametric
or parametric statisticians, data analysts, researchers, classical
or modern psychometricians, quantitative or qualitative evaluators,
and methodologists."
Journal of
Official Statistics http://www.degruyter.com/view/j/jos
many articles about
research methods. Articles from 2012 and before are here http://www.jos.nu/Contents/jos_online.asp
.
Journal of MultiDisciplinary Evaluation http://journals.sfu.ca/jmde/index.php/jmde_1/index
articles describing evalaution projects, and evaluation
issues, methods, etc.
Journal of Research Practice http://jrp.icaap.org/index.php/jrp
"The
Journal
aims
to
highlight
the
dynamics of research practice, as it unfolds in the life of a
researcher, in the growth and decline of a field, and in relation
to a changing social and institutional environment."
Marketing Bulletin http://marketing-bulletin.massey.ac.nz/
On
like
journal
about
marketing,
but
many articles are about survey methods or other research methods.
All are available for free. Some articles are, "Theatre
Audience on Stage: Three Experiments Analysing the Effects of Survey
Design Features on Survey Response in Audience Research", and "An
Experiment with Skip Instructions."
As of 2/1/16, last issue was in 2014
Methods, data, analyses http://www.gesis.org/en/services/publications/mda/
"publishes research on all questions important to quantitative
methods, with a special emphasis on survey methodology."
Pakistan Journal of Statistics http://www.pakjs.com/
To get to the journals, click on PJS Library. One issue, Vol. 27, October 2011,
No. 4, has a bunch of articles about surveys and sampling. Previous
issues have various articles of interest.
Practical Assessment, Research and Evaluation http://pareonline.net/
some articles about general evaluation and research methodology. For
example, "Too reliable to be true? Response bias as a potential
source of inflation in paper-and-pencil questionnaire
reliability" Peer, Eyal & Gamliel, Eyal. Volume 16, Number
9, June 2011.
Qualitative Social Research
http://www.qualitative-research.net/index.php/fqs/index
"A
peer-reviewed,
free
and
interdisciplinary
online
journal for qualitative research." a journal of research about
qualitative methods. Some of the forums include: "Subjectivity
and Reflexivity in Qualitative Research II", and "Text, Archive,
Re-Analysis", and others.
Qualitative Sociology Review http://www.qualitativesociologyreview.org/ENG/index_eng.php
Research
using
qualitative
methods,
and
some
about qualitative methods. For example: "Oh yes, I remember it
well!" Reflections on Using the Life-Grid in Qualitative Interviews
with Couples .
The qualitative report http://tqr.nova.edu/
edited on line journal about qualitative research. A
number of articles about qualitative methods.
Social Research Update http://sru.soc.surrey.ac.uk/
lots
of
articles
about
many
different
techniques. published quarterly by the Department of Sociology,
University of Surrey, England. "Each issue, we will cover one topic
in sufficient depth to indicate the main directions of recent
developments and provide a bibliography for further reading. If
there is room, we shall also list courses, addresses and other
useful information."
Last volume was in 2014
Statistics surveys http://imstat.org/ss/
"publishes
survey
articles
in
theoretical,
computational,
and applied statistics. The style of articles may range from reviews
of recent research to graduate textbook exposition." From
several statistical associations.
Also see Probability statistics http://www.i-journals.org/ps/
sponsored by mathematical associations.
Surveillance and Society http://library.queensu.ca/ojs/index.php/surveillance-and-society/index
has a couple of interesting issues. Issue 2/3, 'Doing
Surveillance Studies': Methodology Issue, and issue 4, 'People
Watching People': Personal Issue.
Survey Methods: Insights from the Field. http://surveyinsights.org/
"The journal will be orientated towards the daily work of
surveying". The journal is edited jointly by the Swiss Foundation
for Research in the Social Sciences and GESIS, the Leibniz Institute
for the Social Sciences.
Survey Methodology http://www5.statcan.gc.ca/olc-cel/olc.action?ObjId=12-001-X&ObjType=2&lang=en&limit=0
From StatCan. "To draw on the expertise of internationally
recognized professionals, look no further than Survey Methodology —
a publication containing the very latest information on the
development and application of survey techniques. Published twice
yearly, Survey Methodology focuses on the development and evaluation
of techniques used in data collection and interpretation."
Also see their series on research. http://www.statcan.gc.ca/start-debut-eng.html
click
on
information
for
researchers, expand the All Subjects choice, and one choice is for
statistical methods (including survey, and other research related
subjects)
Also see their Statistics Canada International
Symposium Series - Proceedings http://www.statcan.gc.ca/bsolc/olc-cel/olc-cel?catno=11-522-X&CHROPG=1&lang=eng
has various papers, such as "Center Sampling: A strategy for
surveying difficult to reach populations", and "Data collection,
sometimes all it takes is a little persuasion ... in the form of
information". The last symposium proceeding is from 2014.
Survey Practice http://surveypractice.org/index.php/SurveyPractice
A new on line journal from American Association of Public
Opinion Researchers. This journal is "designed to provide current
information on issues in survey research and public opinion. The
articles in Survey Practice emphasize useful and
practical information designed to enhance survey quality by
providing a forum to share advances in practical survey methods,
current information on conditions affecting survey research, and
interesting features about surveys and people who work in survey
research."
Survey Research Methods http://www.surveymethods.org/
actually forwards to https://ojs.ub.uni-konstanz.de/srm/
on line peer reviewed journal. This journal "is
the official journal of The European Survey Research
Association" http://www.europeansurveyresearch.org/home/
Survey Statistician http://isi.cbs.nl/iass/allUK.htm
this is a newsletter from the International Association of Survey
Statisticians. But some of the items are about survey methods.
Project Euclid http://projecteuclid.org/
has
a
whole bunch of free on line statistical journals. One of the
journals is Statistics Surveys, listed above.
Directory of Open Access Journals http://www.doaj.org/
links to a bunch of statistics journals
For developing countries, check out the Health InterNetwork Access
to Research Initiative http://www.who.int/hinari/en/
with
free
or
low
cost
access
to journals, for non profit institutions in developing countries.
Includes these journals: American Journal of Evaluation, Evaluation
and Program Planning, International Journal of Social Research
Methodology, and others which may have related information.
List of journals checked and updated 2/1/16
Individual
papers,
proceedings, conferences, etc.
From the US govt
Choi BCK, Pak AWP. A catalog of biases in questionnaires. Prev
Chronic Dis [serial online] 2005 Jan [date cited]. Available from:
URL:
http://www.cdc.gov/pcd/issues/2005/jan/04_0050.htm
Good overview of errors in questionnaires.
Papers from the Census Bureau www.census.gov/srd/www/byyear.html
Includes :Working Papers in Survey
Methodology (SM), which includes research about
questionnaire design, identifying the causes of nonsampling errors
and evaluations of alternative designs. Another section is The Research Report Series (RR) which
covers research in statistical methodology and estimation. Examples
of papers include: Improving survey quality through pretesting. by T
Demaio, J Rothgeb and J Hess; Questionnaire Design Effects on
Interview Outcomes by Jeffrey C. Moore and Laureen H. Moyer; and
Better Formatting for Lower Response Burden by Andrew Zukerberg and
Meredith Lee.
Census papers from Survey of Program
Dynamics http://www.census.gov/spd/pubs.html
Two
papers
in
particular
are
"SPD98-1:
Teens Talk: Are Adolescents Willing and Able to Answer Survey
Questions?" and "SPD99-2 Comparing Reponse Rates for SPD, PSID and
NLSY"
CDC conference proceedings http://www.cdc.gov/nchs/products/misc_pubs.htm
has several procedings from Health Survey Research Methods,
most recently from 2010.
FCSM http://fcsm.sites.usa.gov/
The fcsm is the Federal Committee on Statistical
Methodology. They have papers available from conferences, and
links to research sections of federal agencies.
Other Conferences
Australasian Evaluation Society http://www.aes.asn.au/
has
some
papers
from
their
conferences
available http://www.aes.asn.au/professional-learning/international-conference/previous-conferences.html
Many are research oriented.
Providing Incentives to Survey Respondents: Final Report
http://www.copafs.org/ report in text, from copafs,
Council of Professional Associations on Federal Statistics at
their home page, from 1993.
Proceedings of the ISI World Statistics Congresses
http://www.isi-web.org/publications/proceedings
some of the proceedings are available. Some are quite large
and will take a long time to download.
Proceedings of the Survey Research Methods Section http://www.amstat.org/sections/SRMS/proceedings/
Many
of
the
past
proceedings
are
available on line. Has many research papers on various topics.
SRMS also has links to past conferences http://www.amstat.org/sections/SRMS/conferences.html
some
of
which
have
papers
available.
Others, organizations,
individuals.
Herman Aguinis has several research methods papers http://mypage.iu.edu/~haguinis/pubs.html
including Methodological wishes for the next decade and how to
make wishes come true. with Jeffrey R. Edwards. in press in
Journal of Management Studies, andAn ounce of prevention is worth
a pound of cure: Improving research quality before data collection.
with Robert J. Vandenberg. Annual Review of Organizational
Psychology and Organizational Behavior, 2014.
Frederick Conrad http://sitemaker.umich.edu/fred.conrad/home
has papers on Conversational Interviewing, web surveys,
cognitive interviewing. Main focus seems to be on not using standardized
interviewing formats, as they can lead to misunderstanding, for
example.
Evaluation South Asia http://www.unicef.org/rosa/monitoring_evaluation.html
"Evaluation
South
Asia
has
been
recently
published. This publication ... looks at some important challenges
to the direction in which evaluation in the developing world is
heading, and provides some answers to those challenges. It
aims to ... promote an understanding among public institutions and
civil societies that evaluation feeds into knowledge
management. It further seeks to show evaluation as a strong
tool for improving effectiveness, efficiency and sustainability of
programmes, and to demonstrate innovative evaluation methods."
Gary King's papers http://gking.harvard.edu/publications
Papers
include
"Avoiding
Randomization
Failure
in Program Evaluation" Population Health Management 14 (2011):
S11-S22 , and many more.
Institute for Social and Economic Research. Some
publications http://www.iser.essex.ac.uk/publications
also are about methods, like Extended field efforts to reduce the
risk of non-response bias: do they pay off? ISER Working Paper
Series: 2011-24. A search for Peter Lynn will find many of them.
Jon Krosnick http://comm.stanford.edu/faculty/krosnick/
papers
include
Response
Rates
in
Surveys by the News Media and Government Contractor Survey
Research Firms, The Survey Response Process in Telephone and
Face-to-Face Surveys: Differences in Respondent Satisficing and
Social Desirability Response Bias, and many many more.
RTI has research papers http://www.rti.org/
see one of their conference pages http://www.rti.org/page.cfm?objectid=848A548F-5661-4F34-9380E267533C3AED
for several survey papers
Statistical Significance Testing Fall 1998 Issue of Research in
the schools http://www.personal.psu.edu/users/d/m/dmr/sigtest/sigtest.htm
the
link
to
the
journal
is
in pdf format, linking to other pdf chapters. For the
journal of research in the schools, but many chapters are general,
and can apply to any researcher.
Washington Statistical Society http://www.washstat.org/
See their section: Event Material. Some presentations
from previous seminars are there.
Propensity Score Estimation with Boosted Regression for Evaluating
Causal Effects in Observational Studies http://www.rand.org/pubs/reprints/RP1164/
about
how
to
deal
with
causal
effect modeling with naturalistic rather than experimental data.
Websm http://www.websm.org/
has bunches of
papers on all sorts of topics.
Joint Program in Survey Methodology http://www.jpsm.umd.edu/jpsm/
has
some
on
line
presentations
(click
on on line media). I can't quite get them to work but they look
very interesting.
Research about qualitative methods
Association for Qualitative research http://www.aqr.org.uk/index.shtml
has
some
in
brief
comments,
http://www.aqr.org.uk/inbrief/index.shtml
not exactly research
Research Methods Division, Academy of Management, In-Depth
Discussions and Helpful Tutorials http://division.aomonline.org/rm/
Annual publications, includes articles about various
methods, such as Ethnostatistics, Research Methods and
Organizational Behavior, Not just a numbers thing:
Tactics for improving reliability and validity in Qualitative
Research. and Some Thoughts on Publishing Qualitative
Research. These are available up through 2002.
Specifically about web survey methods
Conducting Research Surveys via E-mail and the Web http://www.rand.org/pubs/monograph_reports/MR1480/
"The
authors
discuss
the
advantages
and
disadvantages of using e-mail and the Web to conduct research
surveys, and also offer practical suggestions for designingand
implementing Internet surveys most effectively"
Journal of Computer Mediated Communication http://jcmc.indiana.edu/
has
some
articles
of
interest,
for
example from April 2005, Researching Internet-Based
Populations: Advantages and Disadvantages of Online Survey Research,
Online Questionnaire Authoring Software Packages, and Web Survey
Services at http://jcmc.indiana.edu/vol10/issue3/wright.html
and
a
few
other
articles
on
the site.
Don Dillman's papers page
http://www.sesrc.wsu.edu/dillman/
includes, for example, Millar, Morgan M., Allison C.
O'Neill and Don A. Dillman.. 2009, Are Mode Preferences Real?
(about survey modes), and Benjamin L. Messer and Don A. Dillman..
2010, Using Address Based Sampling to Survey the General Public by
Mail vs. 'Web plus Mail'
ANZMAC Australian and New Zealand Marketing Academy http://www.anzmac.org/
They
have
conference
proceedings,
to
four
conferences, up to 2002. Click on "proceedings archives"
then you can the conferences.Check the papers section. See
for example Medlin, Christopher, Roy, Subroto, and Ham Chai,
Theong ( 1999): "World Wide Web Versus Mail Surveys: A Comparison
And Report". There is no description of the papers, so
it's kind of hit and miss.
Virtual Surveys site, http://www.virtualsurveys.com/
See
their
site
map
and
then
papers page, which include Pop-up Surveys. What works, what
doesn't work and what will work in the future.
and a few others.
Mertler, Craig A. (2003). Patterns of
response and nonresponse from teachers to traditional and web
surveys. http://pareonline.net/getvn.asp?v=8&n=22
found different response rates but similar responses.
Designing an Intranet User Survey http://www.intranetjournal.com/articles/200412/ij_12_13_04a.html
brief intro. Paul Chin 12/13/2004
Public Opinion Quarterly. http://poq.oxfordjournals.org/content/68/1.toc
Papers
from
Spring
2004
are
free access. One paper, for example is "A Comparison of Web and
Mail Survey Response Rates" http://poq.oxfordjournals.org/cgi/content/full/68/1/94
Michael
D.
Kaplowitz,
Timothy
D.
Hadlock
and Ralph Levine. Public Opinion Quarterly 2004
68(1):94-101; doi:10.1093/poq/nfh006
also articles from 2007, vol 71, number 5 http://poq.oxfordjournals.org/content/71/5.toc
are free. This issue is about cell phones.
also see this Survey Methods for Public Health
Researchers http://www.oxfordjournals.org/our_journals/poq/collectionspage.html
selected readings
and this 2010 special issue http://www.oxfordjournals.org/our_journals/poq/special.html
about total survey error.
Comparing the Results of Probability and Non-Probability Sample
Surveys http://comm.stanford.edu/faculty/krosnick/
by
Jon
Krosnick,
about
comparing
internet
surveys with other methods.
Cell
phones
Public Opinion Quarterly, articles from 2007, vol 71, number
5 http://poq.oxfordjournals.org/content/vol71/issue5/index.dtl
are
free.
This
issue
is
about
cell phones.
DC-AAPOR Workshop on Cell Phone Numbers and Telephone Surveying
in the U.S. http://www.dc-aapor.org/cpworkshop.php
sept 4, 2008
SITES ABOUT MISSING DATA, non
response, multiple imputation, etc
Nonresponse in Social Science Surveys: A Research Agenda
http://www.nap.edu/catalog.php?record_id=18293
Feb 15, 2013. Authors: Panel on a Research Agenda for the Future of
Social Science Data Collection; Committee on National Statistics;
Division on Behavioral and Social Sciences and Education; National
Research Council. This review addresses the core issues
regarding survey nonresponse. It considers why response rates are
declining and what that means for the accuracy of survey results.
Journal of Official Statistics, http://www.jos.nu/Contents/issue.asp?vol=27&no=2
June
2011, special issue on non response.
Public Opinion Quarterly. http://poq.oxfordjournals.org/content/74/5.toc
Special
2010
issue
on total survey error. How to address missing, non response,
measurement error, etc.
Missingdata.org.uk http://missingdata.lshtm.ac.uk/
The site says "This site exists: to support researchers from the
social and medical sciences with the analysis of incomplete
datasets, and as a focus of statistical research." In
particular, see "Introduction to missing data"
Richard Williams http://www3.nd.edu/~rwilliam/xsoc63993/index.html
has class notes, from 2015, on missing data.
Marina Soley-Bori. http://www.bu.edu/sph/files/2014/05/Marina-tech-report.pdf
Dealing with missing data: Key assumptions and methods for
applied analysis. Technical Report No. 4 May 6, 2013. "This tech
report presents the basic concepts and methods used to deal with
missing data. After explaining the missing data mechanisms and the
patterns of missingness, the main conventional methodologies are
reviewed, including Listwise deletion, Imputation methods, Multiple
Imputation, Maximum Likelihood and Bayesian methods."
Prevention and Treatment of Item Nonresponse http://www.jos.nu/Articles/abstract.asp?article=192153
from 2003, Journal of Official Statistics. Edith D. de
Leeuw, Joop Hox, and Mark Huisman. "we present a typology of missing
data patterns and their origins. Based on this typology and the
potential sources of item nonresponse, we outline how missing data
can be prevented as much as possible. Finally, we discuss how
knowledge of the data collection process can improve the statistical
treatment of the remaining missing data."
Univ of Texas Statistical FAQs, missing data
http://www.utexas.edu/its-archive/rc/answers/general/gen25.html
overview
of
missing
data,
how
to
handle, some software links.
The multiple imputation FAQ page http://sites.stat.psu.edu/~jls/mifaq.html
questions and answers about multiple imputation. Also
see his home page http://sites.stat.psu.edu/~jls/
for some presentations on multiple imputation. He isn't at
Penn State now, don't know where he is.
Andridge RR, Little RJ. A Review of Hot Deck Imputation for Survey
Non-response. Int Stat Rev. 2010 Apr;78(1):40-64. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3130338/
Survey sampling topics http://www.restore.ac.uk/PEAS/theory.php
has a section about Non response adjustments by weighting and
another section about imputation. These are introductory stuff, easy
to read, good.
Machelle D. Wilson and Kerstin Lueck http://www.hindawi.com/journals/jam/2014/368791/
Working with Missing Data: Imputation of Nonresponse Items in
Categorical Survey Data with a Non-Monotone Missing Pattern. Journal
of Applied Mathematics. Volume 2014. This article also includes a
couple of sections that are overviews of missing data.
Technical Methods Report: What to Do When Data Are Missing in Group
Randomized Controlled Trials. http://ies.ed.gov/ncee/pubs/20090049/index.asp
October 2009. National Center for Education Evaluation and
Regional Assistance
Research
about
survey
scales,
e.g.,
use
of
neutral, don't know, and so on.
The "Don't Know", "Undecided", and "Neutral" Response Options
from statpac http://www.statpac.com/surveys/undecided-category.htm
Krosnick, J.A., et al. 2002. 'The Impact of "No Opinion'
Response Options on Data Quality-Non Attitude Reduction or an
Invitation to Satisfice?" Public Opinion
Quarterly 66: 371-403.
http://are.berkeley.edu/~hanemann/
Jolene D. Smyth, Don A. Dillman, Leah Melani Christian, and
Michael J. Stern. 2005.
Comparing Check-All and Forced-Choice Question Formats in Web
Surveys: The Role of Satisficing, Depth of Processing, and
Acquiescence in Explaining Differences. Social and Economic
Science Research Center Technical Report 05-029.
Washington State University: Pullman. 30pp. http://www.sesrc.wsu.edu/dillman/papers.html
And if you got to here, check out
this very interesting SPOOF
Journal of Obnoxious Statistics http://www.xs4all.nl/~edithl/jobs.htm
The
Journal
of
Obnoxious
Statistics
(JOBS)
is created specially for Lars Lyberg by his friends and colleagues
to commemorate the 20th anniversary of the Journal of Official
Statistics.
last verified 2/28/15
Last updated 5/30/2016
click here to
return to methods page