Journals
From the US govt
Choi BCK, Pak AWP. A catalog of biases in questionnaires. Prev
Chronic Dis [serial online] 2005 Jan [date cited]. Available from: URL:
http://www.cdc.gov/pcd/issues/2005/jan/04_0050.htm
Good overview of errors in questionnaires.
Census papers from Survey of Program Dynamicshttp://www.sipp.census.gov/spd/pubsmain.htm
Two papers in particular are "SPD98-1: Teens Talk: Are Adolescents
Willing
and Able to Answer Survey Questions?" and "SPD99-2 Comparing Reponse
Rates
for SPD, PSID and NLSY"
CDC conference proceedings http://www.cdc.gov/nchs/products/pubs/pubd/other/miscpub/miscpub.htm
lists two publications specifically. One is the Health
Survey Research Methods Conference Proceedings. April 1996,
described
briefly at http://www.cdc.gov/nchs/products/pubs/pubd/other/miscpub/6hsrm.htm
and downloadable at the miscpub.htm
site listed above. A more recent conference proceedings is also listed
at the miscpub.htm
, from 2004, Health Survey
Research Methods: Eighth Conference. The other is A
New Agenda for Interdisciplinary
Survey
Research Methods: Proceedings of the CASM II Seminar, which reviews
current needs and proposes future directions for
interdisciplinary
survey methods research. Described and downloadable from http://www.cdc.gov/nchs/products/pubs/pubd/other/miscpub/casm.htm
Be warned, both are pdf files and large, so over a modem will take a
long
time to download.
FCSM http://www.fcsm.gov/
The fcsm is the Federal Committee on Statistical Methodology, and has
conference
papers available from some conferences. 2001 papers are available
at http://www.fcsm.gov/events/papers2001.html
listing papers such as Experiences with Incentives in a Public
Attitudes
Survey Using RDD and Are Late/Difficult Cases in Demographic Survey
Interviews
Worth the Effort?: A Review of Several Federal Surveys. The FCSM
site also has a report: Measuring and Reporting Sources of
Error
in Surveys as pdf. 2003 conference papers are available
here http://www.fcsm.gov/events/papers2003.html
The FCSM site also links to other federal agencies with research
papers,
for example the BLS site:
BLS research papers http://www.bls.gov/search/ore.asp
then select statistical survey papers and then search.
Some
papers include "Response Analysis Surveys for Assessing Response Errors
in Establishment Surveys", "How the Memorability of Events Affects
Frequency
Judgments," and many others. Most are from conferences, eg, from
SRMS, AAPOR, etc.
BLS also has this http://www.bls.gov/ore/home.htm
Office of Survey Methods Research (OSMR) which has a number of research
papers about survey methods.
Studies of Welfare Populations: Data Collection and Research
Issues http://aspe.hhs.gov/hsp/welf-res-data-issues02/
has sections on response rates, non response, missing data,
measurement error.
Conferences
Providing Incentives to Survey Respondents: Final Report http://members.aol.com/copafs/incentives.htm
report in text, from copafs, Council of Professional Associations on
Federal
Statistics at http://members.aol.com/copafs/
from 1993.
Proceedings from Bulletin of the International
Statistical Institute http://www.stat.fi/isi99/proceedings.html
Click on all for a list of all papers.
Some of the papers are about surveys, such as "A Rolling Sample Survey
for Yearly and Decennial Uses", and "Alternative Designs for the U.S.
Consumer
Expenditure Survey", and "The Application and Development of Survey
Sampling
in China", and lots of others.
Also see the Baltic-Nordic Conference on Survey
Sampling, 2002 http://www.matstat.umu.se/personal/Gunnar/banocoss/
which has some papers available.
The International Conference on Improving Surveys, 2002 http://www.icis.dk/
(requires java) some papers available on various topics.
See
the papers link.
International Conference on Questionnaire Development,
Evaluation,
and Testing Methods http://www.jpsm.umd.edu/qdet/qdet-set.html
some of the invited papers are here for downloading, in pdf format.
MOLS2006. Methodology of Longitudinal Surveys http://www.iser.essex.ac.uk/ulsc/mols2006/
has abstracts now but will soon have papers.
Statistics Canada International Symposium Series -
Proceedings http://www.statcan.ca/bsolc/english/bsolc?catno=11-522-X&CHROPG=1
has various papers, such as "Center Sampling: A strategy for surveying
difficult to reach populations", and "Data collection, sometimes all it
takes is a little persuasion ... in the form of information".
Others, organizations,
individuals.
Center for Evaluation Research & Methodology http://www.vanderbilt.edu/cerm/articles.htm
has various papers on different issues of evaluation and
research methods, e.g. about moderators in meta analysis, a copy
of Evaluation methods
for social intervention. Annual Review of Psychology,
2000, 51, 345-375, and other articles.
Frederick Conrad http://sitemaker.umich.edu/fred.conrad/home
has papers on Conversational Interviewing, web surveys, cognitive
interviewing. Main focus seems to be on not using standardized interviewing
formats, as they can lead to misunderstanding, for example.
Evaluation South Asia http://www.unicef.org/rosa/monitoring_evaluation.html
"Evaluation South Asia has been recently published. This publication
... looks at some important challenges to the direction in which
evaluation in the developing world is heading, and provides some
answers to those challenges. It aims to ... promote an
understanding among public institutions and civil societies that
evaluation feeds into knowledge management. It further seeks to
show evaluation as a strong tool for improving effectiveness,
efficiency and sustainability of programmes, and to demonstrate
innovative evaluation methods."
Gary King's papers
http://gking.harvard.edu/preprints.shtml
Papers include "Enhancing the Validity and Cross-cultural
Comparability of Survey
Research" and "Misunderstandings among Experimentalists and
Observationalists about Causal Inference"
Institute for Social and Economic Research http://www.iser.essex.ac.uk/pubs/workpaps/
working papers include The Impact of a Mixed Mode Data Collection
Design on Non-Response Bias on a Business Survey (2005-16), A
Review of Methodological Research Pertinent to Longitudinal Survey
Design and Data Collection (2005-29), many others. A search for Peter
Lynn will find many of them.
Jon Krosnick http://communication.stanford.edu/faculty/krosnick.html
papers include Response Rates in Surveys by the News Media and
Government Contractor Survey Research Firms, The Survey Response
Process in Telephone and Face-to-Face Surveys: Differences in
Respondent Satisficing and Social Desirability Response Bias, and many
many more.
Statistical Significance Testing Fall 1998
Issue
of Research in the schools http://www.personal.psu.edu/users/d/m/dmr/sigtest/sigtest.htm
the link to the journal is in pdf format, linking to other pdf
chapters.
For the journal of research in the schools, but many chapters are
general,
and can apply to any researcher.
Werner W. Wittmann 's international web page http://www.psychologie.uni-mannheim.de/psycho2/psycho2.en.php3?language=en
click on publications, then papers. Two papers include "The
reliability of change scores: Many misinterpretations of Lord and
Cronbach by many others. Revisiting some basics for longitudinal
research.", and "
How to Fool Yourself with Experiments in Testing Theories in Program
Evaluation and Psychological Research."
Washington Statistical Society Methods Seminars, on respondent
incentives http://www.scs.gmu.edu/~wss/methods/index.html
a number of papers on various incentive topics.
National Centre for Social Research, working papers from Department
of Work and Pensions http://www.dwp.gov.uk/asd/asd5/wp-index.asp
includes several research papers, such as these from 2002: Encouraging
and Maintaining Participation in the Families and Children Survey:
Understanding
why people take part. This research sought to explore the reasons
why
people (continue to) participate in FACS or to withdraw their
co-operation
from the survey. Longitudinal qualitative research approaches in
evaluation
studies. This paper focuses on the specific role that
longitudinal
qualitative research can play in addressing the range of research
questions
that arise in developing and assessing policy. And this paper from
2001:
Research
Methods for Policy Evaluation.
Propensity Score Estimation with Boosted Regression for Evaluating
Causal Effects in Observational Studies http://www.rand.org/pubs/reprints/RP1164/
about how to deal with causal effect modeling with naturalistic
rather than experimental data.
Websm http://www.websm.org/
has bunches of papers
on all sorts of topics.
Research about qualitative methods
Research Methods Division, Academy of Management, In-Depth
Discussions
and Helpful Tutorials http://division.aomonline.org/rm/
Annual publications, includes articles about various methods, such as Ethnostatistics,
Research Methods and Organizational Behavior, Not just a
numbers
thing: Tactics for improving reliability and validity in Qualitative
Research.
and Some Thoughts on Publishing Qualitative Research.
These are available up through 2002.
Specifically about web survey methods
Conducting Research Surveys via E-mail and the Web http://www.rand.org/pubs/monograph_reports/MR1480/
"The authors discuss the advantages and disadvantages of using e-mail
and the Web to conduct research surveys, and also offer practical
suggestions for designingand implementing Internet surveys most
effectively"
Don Dillman's papers page http://survey.sesrc.wsu.edu/dillman/papers.htm
as of June 15, included Melevin, Paul T., Don A. Dillman, Rodney
Baxter,
and C. Ellen Lamiman. 1998. Personal Delivery of Mail Questionnaires
for
Household Surveys: A Test of Four Retrieval Methods. Revised version
forthcoming
in Journal of Applied Sociology. And others.
ANZMAC Australian and New Zealand Marketing Academy http://www.anzmac.org/
They have conference proceedings, to four conferences, up to
2002.
Click on "proceedings archives" then you can the conferences.Check the
papers section. See for example Medlin, Christopher, Roy,
Subroto,
and Ham Chai, Theong ( 1999): "World Wide Web Versus Mail Surveys: A
Comparison
And Report". There is no description of the papers, so it's
kind of hit and miss.
Virtual Surveys site, http://www.virtualsurveys.com/
See their site map and then papers page, which include Pop-up
Surveys.
What works, what doesn't work and what will work in the
future.
and a few others.
Mertler, Craig A. (2003). Patterns of response and nonresponse from teachers to traditional and web surveys. http://pareonline.net/getvn.asp?v=8&n=22 found different response rates but similar responses.
Designing an Intranet User Survey http://www.intranetjournal.com/articles/200412/ij_12_13_04a.html
brief intro. Paul Chin 12/13/2004
Public Opinion Quarterly. http://poq.oxfordjournals.org/archive/
Papers from 2004 are free access. One paper, for example is "A
Comparison of Web and Mail Survey Response Rates" http://poq.oxfordjournals.org/cgi/content/full/68/1/94
Michael D. Kaplowitz, Timothy D. Hadlock and Ralph Levine. Public
Opinion Quarterly 2004 68(1):94-101; doi:10.1093/poq/nfh006
Are representative internet surveys possible? http://www.statcan.ca/bsolc/english/bsolc?catno=11-522-X20010016291
2001 study by Tom Smith.
Comparing the Results of Probability and Non-Probability Sample
Surveys http://communication.stanford.edu/faculty/krosnick.html
by Jon Krosnick, about comparing internet surveys with other methods.
SITES ABOUT MISSING DATA, non response,
multiple imputation, etc
Prevention and Treatment of Item Nonresponse http://www.jos.nu/Articles/abstract.asp?article=192153
Edith D. de Leeuw, Joop Hox, and Mark Huisman. "we present a typology
of missing data patterns and their origins. Based on this typology and
the potential sources of item nonresponse, we outline how missing data
can be prevented as much as possible. Finally, we discuss how knowledge
of the data collection process can improve the statistical treatment of
the remaining missing data."
International Conference on Survey Nonresponse, 1999 http://www.jpsm.umd.edu/icsn/papers/
These papers include "Telephone Reminder Calls and Mail Survey Response
Rates", "Socio-Economic Bias in Surveys: Some Attempts at
Interpretation" and many more.
Univ of Texas Statistical FAQs, missing data
http://www.utexas.edu/its-archive/rc/answers/general/gen25.html
overview of missing data, how to handle, some software links.
The multiple imputation FAQ page http://www.stat.psu.edu/~jls/mifaq.html
questions and answers about multiple imputation. Also see his
home
page http://www.stat.psu.edu/~jls/
for some presentations on multiple imputation.
Methodology for collecting data about survey non-respondents, an
ISER working paper, listed at http://www.irc.essex.ac.uk/pubs/workpaps/index.php
Click on working paper box,
then its 2002, paper 2002-05.
This paper reports the development of a method to
collect
limited survey data from non-respondents to personal interview
surveys.
This site also has some other interesting working papers on various
methods
issues. For example, "Respondent Behaviour in Panel Studies - A Case
Study
for Income-Nonresponse by Means of the British Household Panel Study"
is
a study of motives for non response about income. This is a March 2003
working paper.
Interagency Household Survey Nonresponse Group (IHSNG) http://www.fcsm.gov/committees/ihsng/ihsng.htm
bunch of papers about non response, for example, "Measuring Survey
Nonresponse
by Race and Ethnicity".
Studies of Welfare Populations: Data Collection and Research
Issues http://aspe.hhs.gov/hsp/welf-res-data-issues02/index.htm
several chapters about response rates, non response, missing.
Statistical Clearinghouse http://www.nss.gov.au/nss/home.NSF/pages/About+SCH?OpenDocument
click on "seminars". One is
"Drawing Conclusions from Surveys with a Low Response Rate" but there
are
other seminars of interest. Also see research papers here
http://www.nss.gov.au/nss/home.NSF/ResearchPapers?OpenView
with papers such as Low Response Rates and Their Effects on
Survey Results.
SOME REPORTS from the National Opinion Research Center, University of Chicago listed at: http://webapp.icpsr.umich.edu/GSS/index.html go to reports, then methodological reports.
Trends in Non-Response Rates, by Tom W. Smith
Little Things Matter: A Sampler of How Differences in Questionnaire Format Can Affect Survey Responses, by Tom W. Smith
Thoughts on the Nature of Context Effects
By Tom W. Smith, article about order effect.
Research
about survey scales, e.g., use of neutral, don't know, and so on.
THE SCIENCE OF ASKING QUESTIONS, Nora Cate Schaeffer and
Stanley Presser
http://arjournals.annualreviews.org/doi/abs/10.1146/annurev.soc.29.110702.110112
Annual Review of Sociology, Vol. 29: 65-88 (Volume publication date
August 2003)
(doi:10.1146/annurev.soc.29.110702.110112)
people in non university settings (like me) can only see abstract, but
a personal use copy is available here
http://www.ssc.wisc.edu/cde/faculty/schaeffe.htm
provided by the author. For personal use only.
The "Don't Know", "Undecided", and "Neutral" Response Options
from statpac http://www.statpac.com/surveys/undecided-category.htm
Krosnick, J.A., et al. 2002. 'The Impact of "No Opinion'
Response Options on Data Quality-Non Attitude Reduction or an
Invitation to Satisfice?" Public Opinion
Quarterly 66: 371-403.
http://are.berkeley.edu/~hanemann/
Jolene D. Smyth, Don A. Dillman, Leah Melani Christian, and Michael J.
Stern. 2005.
Comparing Check-All and Forced-Choice Question Formats in Web Surveys:
The Role of Satisficing, Depth of Processing, and Acquiescence in
Explaining Differences. Social and Economic Science Research Center
Technical Report 05-029.
Washington State University: Pullman. 30pp.
http://survey.sesrc.wsu.edu/dillman/papers.htm
And if you got to here, check out this very interesting SPOOF
Journal of Obnoxious Statistics http://www.xs4all.nl/~edithl/jobs.htm
The Journal of Obnoxious Statistics (JOBS) is created specially for
Lars Lyberg by his friends and colleagues to commemorate the 20th
anniversary of the Journal of Official Statistics.
last verified October 2007
Last updated October 2007
click here to
return
to methods page
I'm trying to figure out how to organize this page better. Check back
some time.