If you are a frequent reader of Eye on Psi Chi, you already know the importance of the undergraduate research experience for students planning to attend graduate school. The importance of this experience has been chronicled not only in the literature (Keith- Spiegel, 1991; Keith-Spiegel & Wiederman, 2000; Landrum & Clark, 2005; Landrum, Jeglum, & Cashin, 1994), but also in the pages of Eye on Psi Chi (Kaiser, Kaiser, Richardson, & Fox, 2007; Slattery & Park, 2002; Sleigh & Ritzer, 2007). The tasks of the undergraduate research assistant have also been defined in numerous articles. Some of these articles include rankings of the importance of tasks (Bauer & Bennett, 2003; Kaiser, et al., 2007; Kardash, 2000; Landrum & Nelsen, 2002), and other articles provide general information on the tasks to be performed by undergraduate research assistants (Sleigh & Ritzer, 2007). It is clear from the data that different faculty may have different expectations for undergraduates involved in research. What are the specific expectations that faculty members have for their undergraduate researchers? Do faculty members communicate these expectations, and do these expectations form the basis of evaluative criteria for the undergraduate research experience?
|Evaluating the Undergraduate Research Assistantship Experience|
|R. Eric Landrum, PhD, Boise State University (ID) |
Although instructors often provide detailed instructions in a course syllabus, instructors rarely provide detailed information about how undergraduate research experiences are to be evaluated. For instance, Slattery and Park (2002) reported that only 21.7% of faculty reported always giving detailed descriptions of the expectations of students’ research work. Luckily, faculty members are paying greater attention to this issue. Recently, Roig (2007) published a sample student-faculty research agreement that not only outlines a weekly schedule of tasks to-be-completed, but also provides evaluative criteria that are largely based on the requirement that student researchers produce a manuscript in the publication format of the American Psychological Association (APA). If a manuscript is the intended product of the undergraduate research assistantship, then Roig’s evaluation system would work well. But in this article, my suggestion is that faculty members must individually determine the desired outcomes for their own undergraduate research assistants, and then communicate those desired outcomes to students (much like faculty members would distribute the paper grading rubric to students before the paper is due so that the students will know what is important).
How might a faculty member start this process? Like any good psychological question, start with a review of the literature. By looking at those items that have been identified as undergraduate research assistant tasks, a faculty member can begin to form his/her own rubric. For example, Sleigh and Ritzer (2007) presented a comprehensive listing of typical research tasks comprising 14 major categories and 132 individual tasks and skills. That many items would be overwhelming for evaluation purposes! Thus, faculty members need to think about the most important tasks and skills for their students. This has been studied from a number of perspectives. For example, Kaiser et al. (2007) asked graduate admissions directors to rate the importance of 39 undergraduate research experiences, and Landrum and Nelsen (2002) asked undergraduate psychology educators to rate 40 potential benefits, skills, or abilities gained from the undergraduate research assistantship. Bauer and Bennett (2003) surveyed alumni about their perceptions of the undergraduate research experience, and Kardash (2000) asked both the undergraduate research assistants and their mentors to simultaneously rate different aspects of the undergraduate research experience, both at the beginning and the end of the research experience.
So where is the universal evaluation of undergraduate research experiences? There isn’t one. The moral to this story is that each faculty member must determine the important aspects of the undergraduate research experience, and then develop an evaluative scale to meet those needs. There is not a one-size- fits-all evaluation, just as there is no universal teaching effectiveness evaluation. And I would take this one step further— a faculty member’s goals for one undergraduate research assistant might actually be different from the goals for another assistant, depending on the research and on the student. Now is the moment for self-disclosure. I am particularly interested in this topic, because I have not done a good job in evaluating my research assistants. I have worked with over 200 undergraduate students in my 19-year career, and I’ve never rigorously evaluated anyone based on premeditated evaluative criteria. But that is about to change. Based on my own review of the articles I have cited here, and in reflecting upon what I believe is important to my research assistants, I have developed a Research Assistant Evaluation Form (Table 1) that I will begin to use during the Spring 2008 semester. I have divided the goals into two major areas: (a) specific skills and abilities, and (b) interpersonal goals. I share that form with you here, in hopes that it might stimulate other faculty members to think about what is important for their undergraduate research assistants, and also for faculty to consider sharing their evaluation form openly with students, as I will.
As I developed this evaluative rubric, a couple of important ideas came to mind. First, I think my preference will be to use this as a pre-test/post-test type of instrument. It may be that growth in particular areas is more important than the eventual post-experience evaluation (e.g., excellent, good). Second, I need to realize that the undergraduate research experience, even as brilliantly as I design it, may not achieve these goals. In other words, a student’s ability to achieve a score of "excellent” in increasing self-confidence can only occur if I provide opportunities to achieve this goal. Furthermore, if a student begins an undergraduate research assistantship with a high level of self-confidence, then the ceiling effect may prevent any significant improvement, regardless of how well designed the research experience may be. What I will take from this endeavor is the value of communicating with students, up front, what is expected of them and how they will be evaluated. This should alleviate many potential sources of confusion about progress towards research goals, and ultimately grade determinations by the faculty member. Ultimately, time will tell if this strategy works!
|TABLE 1 | Research Assistance Evaluation Form|
|SPECIFIC SKILLS AND ABILITIES||Excellent||Good||Fair||Poor|
|Data Analysis Abilities|
|Familiarity with SPSS||4||3||2||1|
|Improved math skills||4||3||2||1|
|Generate clear research ideas||4||3||2||1|
|Choose appropriate measures||4||3||2||1|
|Develop surveys, questionnaires||4||3||2||1|
|Ask relevant research questions||4||3||2||1|
|Troubleshoot research project issues||4||3||2||1|
|Conference submission: oral, poster||4||3||2||1|
|Preparation of tables, graphs||4||3||2||1|
|Mastery of APA format||4||3||2||1|
|Conduct literature searchers||4||3||2||1|
|Ability to lead other students||4||3||2||1|
|Apply ethical principles||4||3||2||1|
|Time management ||4||3||2||1|
|Cope with deadlines||4||3||2||1|
|Building Mentoring Relationship|
|Gets to know faculty member||4||3||2||1|
|Forms relationship for strong letter of recommendation||4||3||2||1|
|Improve communication skills ||4||3||2||1|
|Aid in graduate school decision-making||4||3||2||1|
Bauer, K. W., & Bennett, J. S. (2003). Alumni perceptions used to assess undergraduate research experience. The Journal of Higher Education, 74, 210-230.
Kaiser, J. C., Kaiser, A. J., Richardson, N. J., & Fox, E. J. (2007, Winter). Perceptions of graduate admissions directors: Undergraduate student research experiences: "Are all research experiences rated equally?” Eye on Psi Chi, 11 (2), 22-24.
Kardash, C. M. (2000). Evaluation of an undergraduate research experience: Perceptions of undergraduate interns and their faculty mentors. Journal of Educational Psychology, 92, 191-201.
Keith-Spiegel, P. (1991). The complete guide to graduate school admission: Psychology and related fields. Hillsdale, NJ: Erlbaum.
Keith-Spiegel, P., & Wiederman, M. W. (2000). The complete guide to graduate school admission: Psychology, counseling, and related professions (2nd ed.). Mahwah, NJ: Erlbaum.
Landrum, R. E., & Clark, J. (2005). Graduate admissions criteria in psychology: An update. Psychological Reports, 97, 481-484.
Landrum, R. E., Jeglum, E. B., & Cashin, J. R. (1994). The decision-making process of graduate admissions committees in psychology. Journal of Social Behavior and Personality, 9, 239-248.
Landrum, R. E., & Nelsen, L. R. (2002). The undergraduate research assistantship: An analysis of the benefits. Teaching of Psychology, 29, 15-19.
Roig, M. (2007). A student-faculty research agreement. Retrieved from the Society for the Teaching of Psychology OTRP online, November 14, 2007, from http://www.teachpsych.org/otrp/resources/ mr07research.pdf
Slattery, J. M., & Park, C. L. (2002, Spring). Predictors of successful supervision of undergraduate researchers by faculty. Eye on Psi Chi, 6 (3), 29-33.
Sleigh, M. J., & Ritzer, D. R. (2007, Spring). Undergraduate research experience: Preparation for the job market. Eye on Psi Chi, 11 (3), 27-30.Leadership
|R. Eric Landrum, PhD, is currently a professor of psychology at Boise State University. He received his PhD in cognitive psychology (with an emphasis in quantitative methodology) from Southern Illinois University– Carbondale in 1989. His research interests center on the study of educational issues, identifying those conditions that best facilitate student success. He has over 200 professional presentations at conferences and published 17 books or book chapters, and has published over 60 professional articles in scholarly, peer-reviewed journals. He is the author of the newly published Undergraduate Writing in Psychology: Learning to Tell the Scientific Story (2008, APA Books), and the lead author (with Steve Davis) of The Psychology Major: Career Options and Strategies for Success (3rd ed., 2007, Prentice Hall). He has worked with over 200 undergraduate research assistants, and in 16 years at Boise State, he has taught over 10,000 students.|
Copyright 2008 (Volume 12, Issue 3) by Psi Chi, the
International Honor Society in Psychology
Eye on Psi Chi is a magazine designed to keep members
and alumni up-to-date with all the latest information about Psi Chi’s programs,
awards, and chapter activities. It features informative articles about careers,
graduate school admission, chapter ideas, personal development, the various
fields of psychology, and important issues related to our discipline.
Eye on Psi Chi is published quarterly: