Research Methods

Click one of the links below or scroll down the page to see:

Lecture/Discussion Topics

• Operational Definitions
• Scale Construction: Examples Related To The Environment
• Find the Flaw: Does End Time Belief Cause Climate Change Apathy?

Class Activities

• Introduction to Empirical Articles
• Research Questions
• Original Research Project on Humans and the Environment
• Demonstrating Correlation
• Thinking Critically (with Penn & Teller) About “Environmental Hysteria”
• Evaluating the Ecological Footprint as a Measurement Tool

Multimedia Resources

• Film: Extraneous and Confounding Variables
• Film: The Consensus Research Project

Suggested Readings for Students

• Click Here To Jump To List

References Cited in this Section

• Click Here To Jump To List

Lecture/Discussion Topics

Operational Definitions

Ask students to generate operational definitions for each of the following constructs (or generate others for your class). You could model one or two first, and then split students into small groups for the remainder. Have them discuss their ideas, and choose the “best” definitions. What makes some operational definitions better than others?

Lecture/Discussion Topics

Scale Construction: Examples Related To The Environment

Several measures related to environmental attitudes and behaviors can be used to illustrate scale construction issues, including how to write items, response formats, item analysis, validity, reliability, internal consistency, generalizability concerns (e.g., age, cross-cultural variation), etc. Examples can be found in the following articles:

Mayer, F. S., & Frantz, C. M. (2004). The connectedness to nature scale: A measure of individuals’ feeling in community with nature. Journal of Environmental Psychology, 24, 503-515.

This article describes five studies assessing the validity and reliability of the connectedness to nature scale (CNS), an individual-difference measure of affective connection to the natural world. The scale shows decent internal consistency, unidimensionality, test-retest reliability, convergent validity, and is a good predictor of relevant behavior (i.e., ecological behavior, lifestyle patterns, and students’ curriculum choices). This scale is an example of a operational definition of an ecopsychological concept. The article includes items and response scale. Sample items: “I often feel a sense of oneness with the natural world around me,” “My personal welfare is independent of the welfare of the natural world.”

Clayton, S. (2003). Environmental identity: A conceptual and an operational definition. In S. Clayton & S. Opotow (Eds.), Identity and the natural environment (pp. 45-65). Cambridge, MA: MIT Press.

In this chapter, social psychologist Susan Clayton presents the theoretical and empirical justification for her 24-item Environmental Identity Scale (EID) that was designed to measure “the extent to which the natural environment plays an important part in a person’s self-definition” (p. 52). Inspired by others’ work on social identity, Clayton included items to address the salience of nature (to what extent does the individual interact with nature), idealogy associated with the identity (support for environmental education and a sustainable lifestyle), and associated positive emotions (enjoyment obtained in nature). Clayton describes how she tested convergent, discriminant, and predictive validity. The scale items are presented in an appendix.

Dunlap, R., Van Liere, K., Mertig, A., & Jones, R. E., (2000). Measuring endorsement of the New Ecological Paradigm: A revised NEP scale. Journal of Social Issues,56, 425-442.

The original 12-item New Environmental Paradigm scale can be found in Dunlap & Van Liere (1978). In this 2000 article, the original authors and colleagues present a revision in which ecological worldview is assessed more broadly. Specifically, the authors have expanded the facets of this worldview from the original three, “balance of nature,” “limits to growth,” and “anti-anthropocentrism,” to also include the idea of human exemption from the constraints of nature and the belief that there is an impending global environmental crisis. The authors have also modified the scale so that pro-NEP and anti-NEP items are evenly balanced and gender-fair language is used (i. e., ” humans” instead of ” mankind”). The article offers a summary of research on the validity and dimensionality of the original NEP scale. In 2008, to commemorate the 30th anniversary of the NEP, the Journal of Environmental Education reprinted the original 1978 article along with a new essay by Dunlap (2008) in which he discussed the development, revisions, criticisms and current uses of the NEP Scale. This would be good to use with students to help demonstrate how psychological instruments evolve as a result of their use.

Kuhn, R. G., & Jackson, G. L. (1989). Stability of factor structures in the measurement of public environmental attitudes. Journal of Environmental Education, 20(3) , 27-32.

These authors administered a 21-item scale that combined modified items from Dunlap & Van Liere’s (1978) New Environmental Paradigm (NEP) and Dominant Social Paradigm (DSP) scales. After administering the items to hundreds of Canadian participants, the researchers factor analyzed the data. Their results yielded four factors that they suggest represent four primary areas of concern: the consequences of growth and technology, quality of life, the relationship between humans and nature, and limits to the biosphere.

Lindeman, M., & Vaananen, M. (2000). Measurement of ethical food choice motives.Appetite,34, 55-59.

In this article the authors describe how they developed three sets of items based on previous research on vegetarianism and “ethical food choice motives,” administered the items to samples of adults, and factor analyzed the results. They reported that their confirmatory factor analysis supported their theoretical distinctions between the three motives for making ethical food choices: ecological welfare, political values, and religion.

Kaiser, F. G., & Wilson M. (2000). Assessing people’s general ecological behavior: A cross-cultural measure. Journal of Applied Social Psychology, 30, 952-978.

The authors presented a revision of Kaiser’s (1998) General Ecological Behavior (GEB) scale for cross-cultural application. In addition to the cross-cultural issue, the authors discuss issues related to response format and testing of reliability, internal consistency, and discriminant validity.

Musser, L. M., & Malkus, A. J. (1994). The Children’s Attitudes Toward the Environment scale Journal of Environmental Education, 25(2), 22-26.

This scale, which was designed for grade-school children, includes three types of statements (“I think…”, “I do…”, and “I like…”) addressing a variety of environmental issues (recycling, conservation, animal rights/protection, nature appreciation, pollution). The article includes scale items and scoring instructions. The first author later created a version for use with preschool children that can be found in Musser & Diamond (1999).

Schultz, P. W. (2001). Assessing the structure of environmental concern: Concern for the self, other people, and the biosphere. Journal of Environmental Psychology, 21, 327-339.

This single-item measures the degree to which nature is included within an individual’s representation of self. Respondents indicate which of seven pairs of circles labled “me” and “nature,” and varying from non-overlapping to nearly completely overlapping, best represents their sense of self connection to nature.

Smith-Sebasto, N. J., & D’Costa, A. (1995). Designing a Likert-style scale to predict environmentally responsible behavior in undergraduate students: A multistep process. Journal of Environmental Education, 27, 14-20.

This article provides an overview of some steps involved in scale construction. The authors organize the article by the following subheadings:

• Step 1- Define environmental action domain and develop test items
• Step 2- Verify categories or dimensionality of test
• Step 3- Select response strategy or test
• Step 4- Select and revise items based on item analysis
• Step 5- Conduct validation studies
• Step 6- Conduct discrimination and classification studies

Lecture/Discussion Topics

Find the Flaw: Does End Time Belief Cause Climate Change Apathy?

A study by political scientists Barker and Bearce (2013) concluded that Christians who believe in coming “end-times” are less likely to support global warming legislation than other U.S. citizens. However, in a critique of their study, Veldman (2013) pointed out a significant flaw in Barker and Bearce’s attitude measure, which asked participants to indicate the degree to which they agreed with the statement, “Global warming is a problem that requires immediate government action, in order to prevent environmental devastation and catastrophic loss of life for future generations.”  Instructors could ask students to identify the flaw(s) with that measure; i.e., that there are actually four parts to the statement, each of which could prompt differing degrees of agreement.

Veldman conducted a series of focus groups with Christians, and observed a) a generally negative view of government; b) resistance to apocalyptic framing by scientists regarding climate change; and, importantly, c) a sense of “ethical responsibilities vis-à-vis the environment (see also exercise on Faith-Based Responses to Climate Change on the Individual Differences page). 

CLASS ACTIVITIES

Introduction to Empirical Articles

Have students read a recent article from the empirical literature, and report on the methodology that was utilized: identify constructs, variables and how they were operationalized; the sample/population of interest; the research design: experimental (true, with random assignment to conditions? or quasi-experimental? or correlational?); type of analyses; were the results statistically significant? How should the results be interpreted? Would a Type I or Type II error be more problematic in this area of research? Any potential confounding variables or other study limitations?

Instructors could assign one of the studies listed below (compiled and abstracted by a group at the University of British Columbia Canada; submitted by Kevin Hamilton), a study cited in the course text, or ask students to identify one based on their personal interests.

Bolderdijk, J. W., Steg, L., Geller, E. S., Lehman, P. K., & Postmes, T. (2013). Comparing the effectiveness of monetary versus moral motives in environmental campaigning. Nature Climate Change, 3(4), 413-416.

The authors investigated relative effectiveness of economic versus moral appeals on subjective affect and behaviour.  Economic (egocentric) versus moral (biospheric) appeals to motivate tire-checking behaviour were manipulated experimentally and in a field study.  Measures included predicted subjective affect about complying with the behavioural appeals (studies 1, 2), and whether or not subjects took advantage of a “free tire check” coupon in a field study (study 3).  The authors found that participants were more likely respond to, and anticipate feeling better acting according to biospheric appeals, while economic appeals made participants less likely to take advantage of the free tire check than a control condition.

Osbaldiston, R., & Schott, J. P. (2011). Environmental sustainability and behavioral science: Meta-analysis of proenvironmental behavior experiments. Environment and Behavior, 44, 257-299.

What types of experimental treatments have most effectively promoted pro-environmental behaviour?  The authors performed a meta-analysis on 87 published reports containing 253 experiments. It was found that treatments incorporating elements of cognitive dissonance, goal setting, social modelling, and prompts yielded the largest effect sizes.  

Yoeli, E., Hoffman, M., Rand, D. G., & Nowak, M. A. (2013). Powering up with indirect reciprocity in a large-scale field experiment. Proceedings of the National Academy of Sciences, 110, 10424-10429.

Does observability increase public cooperation behavior?  The authors manipulated observability of participation in a public electricity-reduction program.  The key dependent measure was actual rate of participation in the program. The authors conclude that observability increases participation in the public good program, especially in apartment buildings, where observability is high.  

CLASS ACTIVITIES

Research Questions

In groups of 2-3, ask students to generate a hypothesis about sustainability-relevant behaviors (or, alternatively, assign each group a research question), and ask them to discuss how they could test the hypothesis/answer the question using an experimental, quasi-experimental, or correlational design.  They could then discuss their ideas with the rest of the class. Students could then consider some of the ideas described by their peers, and weigh the pros and cons of using a different methodology than what was proposed. The intention is to demonstrate the relative advantages and disadvantages of field studies, experimentation, quasi-experimentation, correlational studies, interviews, etc. for different kinds of research questions.

CLASS ACTIVITIES

Original Research Project on Humans and the Environment

Many psychology research methods classes involve students conducting original research. Instructors may want to consider requiring the content of original research to be environmentally-related. For example, Cay Anderson-Hanley gives her students the following assignment: You will have the opportunity to work collaboratively with others to investigate some aspect of a current environmental issue on campus (i. e., the use and future of the North Woods). Early in the semester, students review prior research on the North Woods and commence a study to investigate some psychological aspects of the environmental issue (perhaps perceptions of benefits and costs, restorative experiences, etc.). During the semester student groups meet periodically and collect data from a particular ” user” group (e. g., bikers, dog-walkers, academic users, decision-makers). Mid-semester, students report on their group’ s progress. Near the end of the semester, each group enters their data into an EXCEL spreadsheet and meets with the Instructor to review the results. At the end of the semester, groups create PowerPoint presentations of their data, analyses, conclusions, and recommendations, with each group member contributing to the presentation (remind them to practice ahead of time, make eye contact, use an engaging voice, etc…). Individuals will also turn in brief papers summarizing their groups’ findings and describing the experience of participating in research on environmental psychology. Most students find the experience very interesting and intellectually stimulating, especially when the topic is local and immediate. (Contributed by Cay Anderson-Hanley.)

Class Activities

Demonstrating Correlation

When discussing correlational studies, instructors could administer the following two scales in class (adapted from Kaiser, Wolfing & Fuhrer, 1999). It is important to reverse score some of the behavioral items; after they complete the measure, ask students to identify which ones (#3, 4, 6, 7, 8, 11, 12, 13). Then calculate the correlation coefficient between the two. Note that in sustainability focused classes, the correlation may be quite small, due to restriction of range. The instructor could then replicate the study in a different class; ask students to hypothesize which class(es) might yield different results. This could lead into a discussion of sampling (random vs. convenience vs. biased). Finally, discuss the inability to infer causality (i.e., directionality and third variable problems). Have students generate hypothesized relationships between the variables, including potential third variables.

Class Activities

Thinking Critically (with Penn & Teller) About “Environmental Hysteria”

Penn Jillette and Teller are masters of illusion who are following in the footsteps of their fellow magicians Harry Houdini and The Amazing (James) Randi. On their Showtime television series, “Bullshit!” they strongly (with abundant expletives) advocate critical thinking as they debunk various hucksters who prey on people’s gullability to make an easy buck. During the first season (2003) one episode focused on “Environmental Hysteria.” This 29-min. segment is a good illustration of what it means to “think critically” and adopt the skeptical perspective of a scientist, and also of the ways that even self-proclaimed bullshit debunkers can exhibit bias in their “critical thinking.” (Part 2 of the film is available here.)

Instructors can address source credibility as Penn & Teller explore the global warming debate by comparing the stances taken by environmental journalist Ross Gelbspan and Jerry Taylor, Director of National Resource Studies at the Cato Institute. (Ask students whether the sleepy student in Gelbspan’s lecture is evidence that his arguments as not compelling, as Penn & Teller suggest.) Instructors can encourage students to think about quality of evidence and research design after showing students the scene in which Penn & Teller send a confederate into the midst of a crowd of environmental demonstrators in Washington, D.C. to collect signatures on a petition to ban “dihydrogen monoxide” (water). The confederate explains that dihydrogen monoxoide is pervasive in our environment, that it is in our lakes and rivers, it is used in pesticide mixtures, etc. Not surprisingly, she manages to gather some signatures from credulous individuals. Penn and Teller use this behavior as evidence that environmental activists are a bunch of unquestioning “joiners.” Instructors can encourage their students to think of alternative explanations for the finding (e.g., signers may have been relying on heuristics when deciding whether to sign an anti-chemical petition at an environmental rally). Ask students to consider what might happen if the same phenomenon was tested among a different crowd (e.g., circulating a petition at a pesticide convention to license the unrestricted use of dihydrogen monoxide). Click here to view a clip from the episode. The DVD of the show’s first season (including this episode) is available from the Penn & Teller store.

Class Activities

Evaluating the Ecological Footprint as a Measurement Tool

Mathis Wackernagel and William Rees (1996) coined the term “ecological footprint” to describe the impact of an individual human or group of humans on the earth based on their consumption of resources including water, energy, food, space, and various materials. The measurement of ecological footprint is used to estimate the amount of resources and space that would be needed to sustainably support a given lifestyle on a global scale (i.e., how many planets we would need for every individual to live a lifestyle with a particular ecological footprint). Environmental educators and advocates use the ecological footprint as a heuristic tool for raising awareness and inspiring lifestyle change among individuals. Instructors can introduce students to the theoretical concept of the ecological footprint and then have them evaluate the online tools that have been designed to measure it (e.g., at the Global Footprint Network or Center for Sustainable Economy). Ask students to consider the following questions:

• Is the online quiz a good operational definition of the ecological footprint concept?
• Are the items easy/difficult to answer? Do the response options present any problems?
• What factors might affect the test-retest reliability of this tool?
• Is this measure likely to be equally valid for all samples of people? (consider age, geography, socioeconomic status, etc.)

MULTIMEDIA RESOURCES

Film: Extraneous and Confounding Variables

There’s a short animation that illustrates extraneous and confounding variables, random assignment, and error variance.

Film: The Consensus Research Project

The Consensus research project in Ireland “uses social science and collaborative research methods to explore innovative policy, technology and educational initiatives for sustainable household consumption practices relating to food, water, energy, and mobility.“  They also have a  trailer about transitions to sustainable consumption (2013, 1:53).

REFERENCES CITED IN THIS SECTION

Aizenstat, S. (1995). Jungian psychology and the world unconscious. In T. Roszak, M. E. Gomes, & A.D. Kanner (Eds.) Ecopsychology: Restoring the earth, healing the mind (pp. 92-100). San Francisco: Sierra Club Books.

Angell, J. R. (1909). Darwin’s influence on psychology. Psychological Review, 16, 152-169.

Baldwin, E. (1993). The case for animal research in psychology. Journal of Social Issues, 49, 121-131.

Barker, D. C., & Bearce, D. H. (2013). End-times theology, the shadow of the future, and public resistance to addressing global climate change. Political Research Quarterly, 66, 267-279. doi: 10.1177/1065912912442243

Bell, P.A., (2012). Handouts for Exercises in Environmental Psychology, Colorado State University.

Bowd, A., & Shapiro, K. J. (1993). The case against laboratory animal research in psychology. Journal of Social Issues, 49, 133-142.

Bjork, D. W. (1997). William James: The center of his vision. Washington, DC: American Psychological Association.

Boston, T. (1996). Ecopsychology: An earth-psyche bond. Trumpeter. Retrieved August 16, 2006 from http://trumpeter.athabascau.ca/index.php/trumpet/rt/printerfriendly/269/402.

Cahalan, W. (1995). The earth is our real body: Cultivating ecological groundedness in Gestal therapy, Gestalt Journal, 18, 99-100.

Davis, J. (1998). The transpersonal dimensions of ecopsychology: Nature, nonduality, and spiritual practice. Humanistic Psychologist, 26, 51-67.

Dewsbury, D. (1990). Early interactions between animal psychologists and animal activists and the founding of the APA Committee on Precautions in Animal Experimentation. American Psychologist, 45, 315-327.

Fisher, A. (2002). Radical ecopsychology: Psychology in the service of life. New York: State University of New York Press.

James, W. (1899/1912). On some of life’s ideals: On a certain blindness in human beings: What makes a life significant (pp. 3-46). New York: Henry Holt & Co.

James, W. (1890). The Principles of Psychology. New York: Holt.

Kaiser, F. G., Wolfing, S., & Fuhrer, U. (1999). Environmental attitude and ecological behavior. Journal of Environmental Psychology, 19, 1-19.

Knight, S., Vrij, A., Bard, K., & Brandon, D. (2009). Science versus human welfare? Understanding attitudes toward animal use. Journal of Social Issues, 65(3), 463-483. doi: 10.1111/j.1540-4560.2009.01609.x

Kuhn, J. (2001). Toward an ecological humanistic psychology. Journal of Humanistic Psychology, 41, 9-24.

Penn, D., & Mysterud, I. (Eds.) (2005). Evolutionary perspectives on environmental problems (Evolutionary foundations of human behavior). Somerset, NJ: Aldine Transaction.

Pinker, S. (2002). The blank slate: The modern denial of human nature. New York: Viking.

Reser, J. P. (1995). Whither environmental psychology? The transpersonal ecopsychology crossroads. Journal of Environmental Psychology, 15, 235-257.

Richards, J. M., Jr. (2000). A history of Division 34: The division of population and environmental psychology. In D. A. Dewsbury (Ed.) Unification through division, histories of the divisions of the American Psychological Association, (Vol. 5, pp. 113-136). Washington, DC: American Psychological Association.

Sabini, M. (Ed.) (2002). The Earth has a soul: The nature writings of C.G. Jung. Berkeley, CA: North Atlantic Books.

Saucier, D. A., & Cain, M. E. (2006). The foundations of attitudes about animal research. Ethics & Behavior, 16, 117-133. http://dx.doi.org/10.1207/s15327019eb1602_3

Saunders, C. D. (2003). The emerging field of conservation psychology. Human Ecology Review, 10, 137-149.

Schroll, M. A. (2003). Remembering ecopsychology’s origins: A chronicle of meetings, conversations, and significant publications. Gatherings: Journal of the International Community for Ecopsychology [online]. Retrieved August 16, 2006 from http://www.ecopsychology.org/journal/ezine/ep_origins.html

Scull, J. (1999). Ecopsychology: Where does it fit in psychology? Online verion of paper presented at the annual psychology conference, Malaspina University College, March 26, 1999. Retrieved August 16, 2006 from http://members.shaw.ca/jscull/ecointro.htm.

Skinner, B. F. (1948). Walden Two. New York: Macmillan.

Skinner, B. F. (1987). Upon further reflection. Englewood Cliffs, NJ: Prentice-Hall.

Swanson, J. (1995). The call for Gestalt’s contribution to ecopsychology: Figuring in the environmental field. Gestalt Journal, 18, 47-85.

Thoreau, H. D. (1854). Walden; or, Life in the woods. Boston: Ticknor and Fields.

Van Vugt, M., Griskevicius, V., & Schultz, P. W. (2014). Naturally green: Harnessing stone age psychological biases to foster environmental behavior. Social Issues and Policy Review, 8(1), 1-32. doi: 10.1111/sipr.12000

Veldman, R. G. (2013, August 20). Does end time belief really cause climate change apathy? Religion Dispatches. Retrieved 10/8/15 from http://religiondispatches.org/does-end-time-belief-really-cause-climate-change-apathy/

Wilson M., Daly M., & Gordon S. (1998) The evolved psychological apparatus of human decision-making is one source of environmental problems. In T Caro (Ed.), Behavioral ecology and conservation biology (pp.501-523). New York: Oxford University Press.

Yunt, J. D. (2001). Jung’s contribution to an ecological psychology. Journal of Humanistic Psychology, 41, 96-121.