2002-03 Hunt Grant Research Report
Students’ Implicit and Explicit Attitudes Toward Psychological Research and Practice
Jessica L. Snowden and Len B. Lecci
University of North Carolina at Wilmington
The present study examined the relation between explicit and implicit attitude measures within the context of student interests in psychological research and practice. Using a sample of college students completing an undergraduate research methods class, implicit and explicit attitude measures were compared and used to predict grades and career interests. Implicit attitude measures were not significant predictors for either criterion, yet explicit attitudes significantly predicted for both grades and career interests. The predictive validity of explicit attitudes for grades was not greater than that of grade point average. The possible utility of explicit attitudes for grade prediction in classes is discussed.
Psychology is a scientific discipline that places great emphasis on the necessity and benefits of empirical research (Lecci & Snowden, 2002). While research as a career is generally carried out only by those continuing to do graduate work, the importance of research to psychology is so substantial that most universities have historically required students to take undergraduate coursework in research methods or experimental psychology as a graduation requirement (Luchins, 1949). Given that empirical research is often a daunting prospect, it is important to consider what attitudes students hold toward research and experimental coursework.
Attitudes have increasingly been studied using implicit measures (Greenwald & Banaji, 1995), given their potential utility at uncovering attitudes individuals may be unable or unwilling to report. Within the last decade, some researchers have found evidence of a weak relation between explicit and implicit attitude measures (Dasgupta, McGhee, Greenwald, & Banaji, 2000; Greenwald & Banaji, 1995; Greenwald, McGhee, & Schwartz, 1998; Ottaway, Hayden, & Oakes, 2001), suggesting that the two types of attitudes may be relatively independent. Alternatively, other research has yielded evidence of a moderate relation between implicit and explicit measures (Cunningham, Preacher, & Banaji, 2001; Greenwald et al., 1998; Monteith, Voils, & Ashburn-Nardo, 2001; Rudman, Greenwald, Mellott, & Schwartz, 1999). Thus, the present study examined the relation between explicit and implicit attitude measures within the context of student attitudes toward psychological research and practice to attempt to clarify the conflicting findings on such a relation, as well as to examine how such attitudes may relate to career interests and course grades in a research methods course.
The present study examined four specific issues.
First, students' perceptions of research were examined. Students’ attitudes toward research were expected to be more negative than their attitudes toward practice due to the higher proportion of students who end up working in a practice-related field as opposed to a research-related field.
The second issue of interest was whether students’ attitudes toward research, on both explicit and implicit measures, are consistent. Consistent with the bulk of previous research, students’ implicit and explicit attitudes toward research were expected to be only weakly related.
Third, the predictive validity of students’ attitudes toward research on grades in a research methods course was examined. Students’ explicit and implicit attitudes were expected to predict grades, such that more positive attitudes toward research on both measures should predict better grades in the course.
Finally, the change in students’ explicit and implicit attitudes toward research were examined over the course of the semester in an attempt to discern what effect research methods coursework may have on attitudes towards research. Students’ attitudes were expected to be more positive, on both implicit and explicit measures, at the end of a research methods course.
Method
Participants
Participants were undergraduate students at the University of North Carolina at Wilmington, a small regional university in the southeastern United States. Two test sessions were conducted. At the first test session, participants were 31 psychology majors drawn from three undergraduate psychology classes in research methods, as well as a control group of 54 psychology students who had completed at least 30 credit hours, but who had not yet taken a research methods course. Only 4 control students were not psychology majors and, of these, 2 were psychology minors. Participants in the control group were largely recruited from statistics classes, as this is a prerequisite for the research methods course.
The sample at the first test session was 76.5% female and 23.5% male and the mean age was 21.3 (SD = 3.9), ranging from 19 to 52. Four research methods students and 8 control students did not return for the second session (an overall attrition rate of 14.11%). The second test session sample was 69.9% female and 30.1% male with a mean age of 21.6 (SD = 4.1). Participation was voluntary and students were compensated with $5 for each of the two half-hour test sessions.
Procedure
Within the first seven weeks of the semester, participants completed the first test session. At each session, they completed a self-report (explicit) attitude questionnaire and the Implicit Association Test (IAT; Greenwald et al., 1998), which is a computerized word pairing task that has been used to examine implicit attitudes in a variety of areas. The IAT employed in this study involved the pairing of words relating to either psychological research or psychological practice with pleasant and unpleasant words. Each target category (i.e., research and practice) contained five words that had been validated in a pilot study in order to match for familiarity and the extent to which the words represented the categories of research and practice.
Each attribute category contained eight words previously used in IAT research (Greenwald et al., 1998). The research category words were laboratory, grants, animals, sampling, and design. The practice category words were couch, payments, appointment, reassurance, and empathy. Reaction times for the different discrimination tasks, compatible (research/unpleasant; practice/unpleasant) and incompatible (research/pleasant; practice/unpleasant), were recorded. The order of the IAT discrimination task completion was not randomized within participants, but was randomized between participants (i.e., half of the participants completed the compatible discrimination task first at both sessions and half of the participants completed the incompatible discrimination task first at both sessions).
After completing the IAT, participants completed the paper-and-pencil explicit attitude questionnaire. The explicit questionnaire contained items assessing: (a) the extent to which a student is currently considering a career in research, as well as the extent to which a student would be willing to consider a research career; (b) a student’s research experience, both perceived and actual experience; and (c) student perceptions of empirical research (negative and positive perceptions were assessed independently). A parallel set of questions regarding attitudes toward and experience with psychological practice was also included. For all items except items assessing actual experience with research and practice, responses were Likert scaled from 0 to 5. The items assessing negative perceptions of psychological practice and research were reverse coded such that higher scores indicate fewer negative opinions toward these areas.
The second test session was completed during the last four weeks of the semester. The second test session procedure was identical to that of the first test session, other than the fact that participants also completed a grade release form authorizing the researchers to obtain the students’ grades from the university.
Results
The first two trials of each testing block were deleted in accordance with the Implicit Association Test (IAT) procedure (Greenwald et al., 1998). Nine participants’ data were excluded from the IAT analyses due to error rates in excess of the 75% criterion commonly used with the IAT (Cunningham et al., 2001; Greenwald et al., 1998; Ottaway et al., 2001). Less than 1% of all trials were deleted due to extreme responses (reaction time latencies shorter than 300 ms or longer than 3,000 ms). Again, this is in accordance with traditional IAT procedures (Cunningham et al., 2001; Greenwald et al., 1998; Ottaway et al., 2001).
Due to the exploratory nature of the analyses, all statistical tests were two-tailed tests using an alpha value of .05. Given the number of comparisons made, alpha inflation is a valid concern. As such, an ordered Bonferroni correction was performed and all statistical analyses with p-values less than .004 are significant after the ordered Bonferroni correction.
Creating scales to assess attitudes
Cronbach’s alpha coefficients were calculated to determine the feasibility of combining certain items from the explicit questionnaire into scales. Alpha coefficients were calculated for the research items and for the practice items for the first and second test sessions. The modest alpha coefficients for all five Likert-scaled research (a1 = .70; a2 = .67) and practice items (a1 = .67; a2 = .69) combined indicated that analyses should not be conducted on an aggregate score for either research or practice. However, the career-related questions for both research (a1 = .89; a2 = .87) and practice (a1 = .90; a2 = .89) did yield high alpha coefficients and were therefore combined. In addition, there was no significant change on these items across the semester, so a research career interest and a practice career interest scale were created from the arithmetic mean of the responses to the appropriate two items for these interest areas for the first and second test sessions. For the items assessing positive and negative perceptions of research and practice, alpha coefficients were generally low (at or below .70). Thus, attitudes toward research and practice were assessed at an item, rather than scale, level.
Responses for explicit questionnaire items at the first and second test sessions were compared to examine whether student responses changed significantly over the course of the semester. Change scores were calculated by subtracting first test session item responses from the second test session item responses to examine whether changes occurred over the course of the semester. Please refer to Table 1 for the results of change analyses.
Items assessing students’ negative perception of psychological practice, t(71) = 2.24, p = .028, perceived experience with research, t(71) = –2.66, p = .010, and actual experience with research, t(72) = -2.48, p = .016, changed significantly from the first test session to the second test session, indicating that students’ perceptions of practice became slightly more negative over the course of the semester and that they reported more research experience, both actual and perceived, at the end of the semester. There were no significant differences between research methods students and the control psychology students for changes across the semester. Thus, for all other analyses using the explicit questionnaire, the arithmetic mean of a given participant’s item responses for the first and second test sessions was calculated and used as the explicit measure.
Explicit ratings of research and practice
As a whole, psychology students held significantly more positive views of psychological practice (M = 4.35, SD = .66) than they did toward research (M = 4.11, SD = .77), t(84) = –3.02, p = .003, although they had relatively high opinions of both psychological areas. Psychology students also held significantly more negative opinions of research (M = 4.04, SD = .85) as compared to practice (M = 4.37, SD = .79), t(83) = –2.96, p = .004. Finally, psychology students were significantly more likely to report career interests in psychological practice (M = 3.70, SD = 1.17) than in research (M = 2.96, SD = 1.14), t(84) = –4.21, p < .001. Other than the research methods students’ stronger preference (M = .56, SD = .85) for practice over research, compared to control psychology students (M = .06, SD = .63), t(83) = -3.08, p = .003, there were no differences between the student groups for these trends. Please refer to Table 2 to view significant differences between explicit attitudes of psychology students towards research and practice.
Implicit ratings of research and practice
IAT analyses were conducted on log transformed data in accordance with prior research (Greenwald et al., 1998), although analyses on the original data yielded the same results. Cursory examination of the data indicated that no significant differences existed between the IAT scores of research methods students and control psychology students at either the first, F(1,74) = .072, p = .790, or second, F(1,62) = .241, p = .626, test sessions. In addition, there was no difference between the two student groups for the change in IAT scores from the first to second test session, t(41.39) = .17, p = .866. Thus, the two groups were aggregated for the purposes of the implicit attitudes analyses.
For results of IAT analyses, please refer to Table 3. At the first test session, students held implicitly more positive attitudes toward psychological practice than toward psychological research as demonstrated by a significant IAT effect (mean difference score of 111.90, SD = 157.81), t(75) = 6.23, p < .001, such that reaction times were significantly quicker for the compatible discrimination task (i.e., pairing practice related words with pleasant words) than for the incompatible discrimination task (i.e., pairing research related words with pleasant words) at the first test session. This same finding did not occur at the second test session (mean difference of 17.91, SD = 167.26), t(63) = 1.01, p = .315. The disappearance of the IAT effect at the second test session appears to stem from a significant decrease in the length of the incompatible discrimination task mean response latency from the first (M = 885.81, SD = 188.434) to the second (M = 780.91, SD = 164.30) test session, t(60) = 6.40, p < .001. There was no significant difference between the compatible discrimination task mean response latency for the first (M = 773.91, SD = 147.17) and second (M = 763.00, SD = 167.26) test session, t(60) = .79, p = .434.
A comparison of research methods students and control students
Differences between research methods students and control psychology students were examined for all explicit questionnaire items or scales and are presented in Table 4. Research methods students (M = 2.50, SD = 1.21) were significantly less likely to state that they were interested in a career in psychological research compared to control psychology students (M = 3.22, SD = 1.03), t(83) = 2.92, p = .004. In addition, research methods students reported significantly lower positive feelings (M = 3.71, SD = .95) toward research compared to control students (M = 4.33, SD = .54), t(41.47) = 3.37, p = .002, and significantly more negative feelings (M = 3.71, SD = .89) toward psychological research than did control students (M = 4.24, SD = .76), t(83) = 2.90, p = .005. Notably, there were no significant differences between the research experiences that research methods students reported or perceived themselves to have as compared to that reported by the control students at either the first or second test session.
With regard to attitudes and perceptions of psychological practice, no significant differences emerged between the interest in a career in practice reported by research methods students (M = 3.64, SD = 1.13) and the interest reported by control psychology students (M = 3.73, SD = 1.20), t(83) = .36, p = .723. There was no significant difference between the positive attitudes of research methods students (M = 4.40, SD = .64) toward practice as compared to control students (M = 4.27, SD = .71), nor between the negative attitudes of research methods students and control psychology students at either the first, t(82) = 1.28, p = .205, or second test session, t(71) = 1.96, p = .054. However, research methods students had more actual experience (M = 1.40, SD = 1.08) with psychological practice, t(46.46) = -2.53, p = .015, and reported having slightly more experience (M = 2.21, SD = 1.65) with practice, t(50.50) = -1.86, p = .073, than did control psychology students (M = .85, SD = .74; M = 1.58, SD = 1.27).
Relating implicit and explicit measures
A significant IAT effect was only found at the first test session. Thus, only the first session IAT data was examined in relation to the explicit questionnaire items and as a predictor of career interests and research methods course grades. The IAT was not found to significantly relate to any of the explicit responses reflecting student attitudes toward research and practice. Thus, this may support the widely held belief that explicit and implicit attitudes largely function independently of one another (Greenwald & Banaji, 1995). The relations between the implicit and explicit measures are presented in Table 5.
Predicting outcomes
Course grades in research methods were strongly related to both psychology grade point averages, r(25) = .73, p < .001, and cumulative undergraduate grade point averages, r(25) = .66, p < .001. The IAT had no significant predictive validity for research methods course grades. However, explicit questionnaire items assessing attitudes towards practice and, in particular, attitudes towards research did display significant predictive validity for research methods course grades. These findings are presented in Table 6. Average positive attitudes towards research, r(25) = .58, p = .002, and negative attitudes towards research, r(25) = .50, p = .010, significantly predicted research methods course grades. Unexpectedly, average positive perceptions of psychological practice also significantly predicted research methods course grades, r(25) = .40, p = .041. However, regression analyses indicated that explicit attitudes toward research did not provide a significant increase in predictive validity over that provided by cumulative grade point average or psychology grade point average when predicting course grades.
Finally, several items from the explicit questionnaire were significantly related to career interests and these correlations are presented in Table 7. The IAT displayed no predictive validity for career interest in either research, r(75) = –.03, p = .811, or in practice, r(75) = .11, p = .337. However, the explicit positive and negative perceptions that a student held toward both psychological research and practice displayed predictive validity for career interests in those respective areas. Specifically, the more positive a student’s attitude toward practice, the greater the likelihood that student reported being interested in a career in psychological practice, r(84) = .45, p < .001. Similarly, the less negative the attitudes toward psychological practice, the more likely a student was to report being interested in a career in practice at both the first test session, r(84) = .22, p = .049, and the second test session, r(84) = .37, p = .001. The more positive a student’s reported perceptions of research, the more likely they were to report interest in a career in psychological research, r(84) = .41, p < .001. Negative perceptions of psychological research had no significant relation to career interests in either research, r(84) = .16, p = .139, or practice, r(84) = .21, p = .054. Student perceptions of practice (positive or negative) had no significant relation to the consideration of a career in research.
Discussion
All psychology students appeared to have more positive attitudes toward psychological practice than toward research on both an implicit and explicit level. However, while there appeared to be no change in explicit attitudes toward research, all students developed more positive implicit attitudes toward research over the course of the semester, as indicated by the lack of an IAT effect at the second test session. While it is possible that the disappearance of the IAT effect is due to practice effects, it is unlikely due to the length of time between test sessions (approximately two months). In addition, the pattern of the compatible and incompatible means at the first and second test sessions demonstrates that the important change is a significant decrease in the average incompatible discrimination task mean response latency at the second test session, rather than decreases in means for both discrimination tasks at the second session (as one might expect were practice effects an issue).
Given that there was no difference between control psychology students and research methods students on implicit attitudes toward psychology at either the first or second test session, the positive change in attitudes toward research cannot necessarily be attributed to the effects of experimental coursework in particular, but perhaps rather a maturation effect common to all psychology students. The overwhelming majority of control psychology students were drawn out of statistics classes, so it is still possible that they were exposed to some of the issues central to research that will be further discussed when they later enroll in a research methods course. Thus, the disappearance of the IAT effect at the second test session may still, in fact, suggest that negative attitudes toward research can be ameliorated through exposure to research-based courses (i.e., statistics and research methods). Control psychology subjects were selected from statistics classes, as it was expected that they would be the closest match for the students enrolled in the research methods course in both credit hours and prior coursework. In retrospect, perhaps these students were not the best controls and their inclusion may have contributed to the difficulty in clearly interpreting the findings.
The finding of any IAT effect between the attitudes toward psychological research and practice itself is noteworthy given that both of the target categories used in the current IAT consisted of only five target stimuli. This bolsters the original findings and assertion by Greenwald et al. (1998) that the IAT may be sensitive to differences even when using only a few target stimuli, as well as effective at assessing attitudes for categories with few associated stimuli.
One of the most problematic issues in the present study is how to interpret the differences between the generally more negative attitudes toward research held explicitly by research methods students as compared to control psychology students who have not yet taken a research methods course. These findings may be due either to some influence of the research methods course on these students or to some uncontrolled characteristic in this group. The students in the research methods course had been in the course for between two to seven weeks prior to the first test session, so they definitely had some exposure to research based material prior to their first responses. It is possible that research methods students, due to their presence in a research-based course in which they would later be evaluated and receive a grade, may be primed to be apprehensive about research more so than other students and thus their ratings of research tended to be more negative.
Equally conceivable is the possibility that some characteristic of this sample of research methods students, independent of the coursework itself, was responsible for their more negative opinions toward research. For example, the findings that experimental psychology students reported more actual experience with psychological practice may reflect such a bias. Regardless, the findings are interesting given that, ideally, students coming out of a research based course would have more positive evaluations of research than students who had not yet been exposed to such material.
The finding that attitudes toward research and practice influence how likely a student is to consider a career in those areas is not surprising and seems rather intuitive, yet is still relevant. However, perhaps the most noteworthy finding, particularly for psychology students, is that of the predictive validity of explicit attitudes toward research for grades in a research methods course. While grade point average, both cumulative or within psychology, was a stronger predictor of grade point average, attitudes are likely to be more easily assessed and obtained as predictors of grades. Regardless, students who held more positive attitudes, or at least less negative attitudes, toward research were more likely to obtain a high grade in the research methods course. Clearly, the fact that the more receptive a student is at the outset of the course, the more likely that student is to achieve a desirable grade is an important finding.
The present study explored several issues regarding the explicit and implicit attitudes that psychology students hold towards psychological research and practice, including the positive and negative associations held towards each area, attitude change across the semester, and the predictive validity of these attitudes with regard to career interests and course grades. Due to the exploratory nature of the study, there is merit to several interpretations of the findings. Future research may clarify which interpretation is most accurate.
In addition, it is probable that assessments of explicit and implicit attitudes may be more likely to be related to, or predictive of, future outcomes when the attitudes under examination are those for which people have stronger, more durable histories (e.g., race beliefs). This study represents a launching point for a program of research comparing implicit and explicit attitudes and examining the relation between the two, as well as how such attitudes may predict future behavior.
References
Cunningham, W. A., Preacher, K. J., & Banaji, M. R. (2001). Implicit attitude measures: Consistency, stability, and convergent validity. Psychological Science, 12, 163–170.
Dasgupta, N., McGhee, D., Greenwald, A.G., & Banaji, M. R. (2000). Automatic preference for white Americans: Eliminating the familiarity explanation. Journal of Experimental Social Psychology, 36, 316–328.
Greenwald, A., & Banaji, M. (1995). Implicit social cognition: Attitudes, self-esteem, and stereotypes. Psychological Review, 102, 4–27.
Greenwald, A., McGhee, D., & Schwartz, J. (1998). Measuring individual differences in implicit cognition: The Implicit Association Test. Journal of Personality and Social Psychology, 74, 1464–1480.
Lecci, L., & Snowden, J. (2002). Education and training (not principles) govern the actions of psychologists. Journal of Forensic Psychology Practice, 2, 81–88.
Luchins, A. (1949). Teaching experimental psychology to clinical students. American Psychologist, 4, 48–51.
Monteith, M. J., Voils, C. I., & Ashburn-Nardo, L. (2001). Taking a look underground: Detecting, interpreting, and reacting to implicit racial biases. Social Cognition, 19, 395–417.
Ottaway, S. A., Hayden, D. C., & Oakes, M. A. (2001). Implicit attitudes and racism: Effects of word familiarity and frequency on the Implicit Association Test. Social Cognition, 19, 97–144.
Rudman, L. A, Greenwald, A. G., Mellott, D. S., & Schwartz, J. L. K. (1999). Measuring the automatic components of prejudice: The flexibility and generality of the Implicit Association Test. Social Cognition, 17, 437–465.
Author Note
Jessica L. Snowden is currently a M.A. candidate in General Psychology at the University of North Carolina at Wilmington. She received her B.A. in Psychology and Criminal Justice from the same institution. Her research has primarily focused on juror bias.
Special thanks to Naomi Debbaut, Laurie Jachym, and Julie Gardner for their assistance with data collection for this study. This project was supported by a Psi Chi Thelma Hunt Grant. Portions of this paper were presented at the 15th Annual Convention of the American Psychological Society in Atlanta, GA. Please address all correspondence to Len Lecci, Department of Psychology, University of North Carolina at Wilmington, 601 South College Rd., Wilmington, NC, 28403-5612. E-mail: leccil@uncw.edu.
____________________________________________
Winter 2004 issue of Eye on Psi Chi (Vol. 8, No. 2, pp. 50-55), published by Psi Chi, The National Honor Society in Psychology (Chattanooga, TN). Copyright, 2004, Psi Chi, The National Honor Society in Psychology. All rights reserved.