Margaret W. Matlin
State University of New York at Geneseo
James W. Kalat
North Carolina State University
In the United States, about half of doctoral-level psychology programs and a third of master's-level programs require applicants to submit scores from the Graduate Records Examination (GRE) Psychology Test. Still other programs recommend the GRE Psychology Test, but they do not require it (Norcross, Hanych, & Terranova, 1996).
Undergraduate students typically find that psychology professors do not have much information about the GRE Psychology Test. Most of the information we'll discuss in this article is available in free but seldom-read publications from Educational Testing Service1 (P.O. Box 6000, Princeton, NJ 08541-0001). The rest comes from our combined experience in serving on the committee that guides the development of the GRE Psychology Test. (James Kalat served as the committee chair from 1990 to 1998, and Margaret Matlin became the committee chair in 1998.)
The GRE Psychology Test is a paper-based test in a multiple-choice format, with five choices per item. The test includes about 215 items; students are allotted 2 hours and 50 minutes to complete the test. The GRE Psychology Test is administered on three dates each year, at test locations throughout the world. In the present article, we will discuss how the GRE Psychology Test is scored and how the test is developed, as well as information on the test's usefulness.
The Scoring System for the GRE Psychology Test
The GRE Psychology Test is designed to be challenging. You may be accustomed to earning
scores in the neighborhood of 90% on the examinations in your psychology courses. As a result, you may worry about the large number of items that you cannot answer. Here's some reassuring information: You can earn a score of 500 on
the GRE Psychology Test by answering 43% of the questions correctly, answering 25% of the questions incorrectly, and leaving 32% of the questions unanswered (Graduate Records Examination Board, 1999b, p. 7). Remember, too, that students
who take the GRE Psychology Test constitute a selected sample of all psychology undergraduates. Keep in mind, then, that a score of 90% correct on this exam would be extremely unusual!
The mean score on the GRE Psychology Test is 554, with a standard deviation of 99, based on the performance of all examinees who took the test between October 1, 1995, and September 30, 1998 (Graduate Records Examinations Programs,
1999a, p. 15). The reliability of the test is .95, as estimated in an analysis of a sample of 4,130 scores (Graduate Record Examinations Board, 1999a, p. 22). The 99th percentile on this test starts at 770; a score of 340 is in
the first percentile.
Your score on the GRE Psychology Test will be based on the number of questions you answered correctly, minus one fourth of the questions you answered incorrectly (Graduate Records Examination Board, 1999b, p. 4). The question often
arises: Should you guess when you do not know the answer? This question has a complex answer. Many students believe that this formula will hurt them if they guess, rather than omitting an answer. Across all examinees who take the
test, the mean score of the whole population will not be affected by random guessing. That is, the number of points gained by guessing will be balanced by the correction factor for incorrect answers. However, by chance, some
individual students will lose points by guessing (and other students will gain points by guessing).
What should you do if you can eliminate one or more of the five possible answers, or if one answer seems like a slightly better guess than the others. Here, your chances of answering correctly are improved. In these cases, you should
certainly fill in an answer, because this type of "strategic guessing" may indeed improve your score.
Who Prepares the GRE Psychology Test, and How
Most psychology faculty members do not know how the GRE Psychology Test is prepared. In fact, neither of us knew about the procedures
when we were initially asked to serve on the committee. The GRE Program of the Educational Testing Service (ETS) appoints a committee of six psychology faculty members from U.S. colleges and universities. The committee members
are selected to be diverse in terms of their research specialties, the geographic location of their universities, and so forth. Furthermore, they must have a broad background in psychology as a whole, in addition to expertise in
just a single area.
Each committee member writes about 15 potential test questions per year. The ETS test developers also invite other faculty members to write additional questions. The committee members individually review the hundreds of questions that
are submitted each year. Then they meet for a three-day session once a year to discuss each question. During this session, they review whether the question addresses an important issue, whether it is clearly stated, and whether
the question has only one correct answer.
On the GRE Psychology Test, 40% of the questions deal with material in cognition, sensation and perception, learning, and biological psychology; another 43% deal with social psychology, personality, development, and abnormal behavior.
The remaining 17% cover material such as research methods, history of psychology, and applied psychology. On the actual test, however, the three types of questions are distributed throughout, rather than appearing in three separate
sections. Your overall score for the GRE Psychology Test is based on questions from all three of these areas.
After each administration of the test—and before scores are reported at ETS—statisticians at ETS
calculate extensive statistics on each question. The information includes the percent of students who answered each question correctly, the point biserial (that is, the correlation between performance on that question and performance
on the test as a whole), and information on how well the students who selected each choice for a given item performed on the rest of the test. For example, the results might indicate that the students who chose answer "B"
—the correct answer—had the highest overall scores on the test, whereas those who answered "D" had the worst. These statistics call attention to items with
possible problems, and almost any test has a small number of these items. Mostly, these are cases in which the point biserial is less than .30, and a large percentage of the best students had marked a choice other than the
intended correct answer.
The staff then consults with committee members to decide whether the item might have a second acceptable answer. Questions that are judged to be problematic are not used in computing scores. The staff at ETS continually reviews the
performance of test questions. If performance patterns on a specific question change, they will stop considering that question in the scores. Furthermore, they will not use that question in future tests. A change in performance
patterns may occur for many reasons. However, the most common problem occurs when new research either disproves an old theory or supports a new explanation of behavior. Thus, the GRE Psychology Test is updated to reflect changes
in our knowledge about psychological phenomena.
How Useful is the GRE Psychology Test?
Psychology departments consider many kinds of information when they evaluate applicants for admission to graduate school. They generally
emphasize grade point average (GPA), GRE General Test scores, GRE Psychology Test scores, and letters of recommendation. They may also examine information such as the student's research experience, the prestige of the student's
undergraduate institution, and the match between student and faculty interests (Keith-Spiegel, Tabachnick, & Spiegel, 1994).
The GRE Board (1999a) decided to investigate the correlation between several quantitative measures and a student's first year grades in graduate school. The Board collected quantitative data from 110 graduate departments of psychology
for the years 1986 through 1990; the study was based on 1,151 students who had pursued graduate work. The data included the student's undergraduate GPA, GRE General Test scores (verbal, quantitative, and analytical), and scores
on the GRE Psychology Test. Of these measures, which one would you guess correlates best with first-year grades in graduate school?
As Table 1 shows, both the GRE Psychology Test and the undergraduate GPA are correlated .37 with first-year graduate grades. This equivalence is especially interesting, because the Psychology Test is based on students' performance
on just one examination, whereas the undergraduate GPA is established over four years of college courses. Table 1 also shows that scores on the GRE Psychology Test are more highly correlated with first-year graduate grades than
are any of the GRE General Test measures, alone or in combination.
Some researchers have argued that first-year grades may not be an appropriate measure of success in graduate school (Sternberg & Williams, 1997). However, we have no evidence that other measures of success—such as faculty evaluations of graduate students—are reliable or valid (Thayer & Kalat, 1998). Also, first-year grades are important because they can determine
a graduate student's future. At many universities, graduate students who fail to reach some minimum criterion in their first-year grades, such as a B average, are dismissed from the program or denied financial aid.
Kuncel, Hezlett, and Ones (1998) examined data on graduate students in all academic fields, not just psychology. In their meta-analysis of 201 studies of 27,039 graduate students, the GRE Subject Test was correlated .46 with first-year
graduate GPA, .48 with scores on comprehensive examinations in graduate school, .53 with faculty ratings, and .21 with completion of a graduate degree. With the exception of degree completion, the GRE Subject Tests correlated more
strongly with these measures than either the GRE General Test or the undergraduate GPA.
Future Directions of the GRE Psychology Test
Several years ago the GRE Board considered offering the Subject Tests in a computer-based format, consistent with the current
GRE General Test. However, constructing the computer-based GRE General Test required adding an enormous number of new items to the test pool. The cost of constructing a similar number of new items for the Psychology Test pool would
be prohibitively expensive. In other words, you can count on the paper-and-pencil format for the foreseeable future!
The GRE Psychology Committee has been more concerned about another issue, the nature of the test questions. Many people—both students taking the GRE Psychology Test
and the committee members themselves—have complained that the test may overemphasize rote recall of facts, some of them not especially interesting or important.
Items of that kind are the easiest to write, and they usually produce decent item statistics. However, they do not reflect what most psychologists consider the best side of psychology.
In recent years, the GRE Psychology Committee has tried to reduce the number of items that require simple recognition of terms or psychologists' names. We have replaced these items with more conceptual questions. However, students
often obtain commercially available preparation books for the Subject Test. Some of these books contain numerous "name that psychologist" items as well as outdated "factual" questions. Students may be misled by their scores on
the practice tests in these books. The books will also encourage students to use inappropriate review strategies in preparing for the GRE Psychology Test. These strategies will be especially counterproductive as the GRE Psychology
Committee increases the number of conceptual questions.
The official guide—GRE: Practicing to Take the Psychology Test—will
give you better information and more realistic practice items. You can order this resource from the GRE website at www.gre.org or by calling 1-800-537-3160 or (609) 771-7243.
Since the mid-1990s, new editions of the test have included some "analysis-of-evidence sets" among the methods items. For these questions, you will read a paragraph describing a hypothetical research study and its results. Then you'll
answer two to five questions about proper conclusions to be drawn, potential weaknesses of the study as described, and ways to improve the research design or to follow up with additional research. So far, students' performance
on these item sets has correlated well with the rest of the test. Future forms of the GRE Psychology Test will continue to use such items. These analysis-of-evidence items attempt to assess critical thinking, consistent with the
increasing emphasis on critical thinking in psychology (e.g., Nunmedal & Halpern, 1995).
In summary, the GRE Psychology Test continues to evolve. With fewer factual items and a larger number of conceptual and critical-thinking items, the test will have greater content validity. As a result, the test will assess more completely
those skills that psychologists value most highly in their graduate students.
Graduate Record Examinations Board. (1999a). Guide to the use of scores: Graduate Records Examinations, 1999-2000. Princeton,
NJ: Educational Testing Service.
Graduate Record Examinations Board. (1999b). Psychology Test descriptive booklet, 1999-2001. Princeton, NJ: Educational Testing Service.
TKalat, J. W., & Matlin, M. W. (2000). The GRE Psychology Test: A useful but poorly understood test. Teaching of Psychology, 27, 23-26.
Keith-Spiegel, P., Tabachnick, B. G., & Spiegel, G. B. (1994). When demand exceeds supply: Second-order criteria used by graduate school selection committees. Teaching of Psychology, 21, 79-81.
Kuncel, N. R., Hezlett, S. A., & Ones, D. S. (1998, April). The predictive validity of the Graduate Record Examination: A meta-analysis. In J. P. Campbell & D. S. Ones (Chairs), Selection Into I-O programs: Focus on GRE validity. Symposium conducted at the 13th annual conference of the Society for Industrial and Organizational Psychology, Dallas, TX.
Norcross, J. C., Hanych, J. M., & Terranova, R. D. (1996). Graduate study in psychology: 1992-1993. American Psychologist, 51, 631-643.
Nummedal, S. G., & Halpern, D. F. (1995). Introduction: Making the case for "psychologists teach critical thinking." Teaching of Psychology, 22, 4-5.
Sternberg, R. J., & Williams, W. M. (1997). Does the Graduate Record Examination predict meaningful success in the graduate training of psychologists? A case study. American Psychologist, 52, 630-641.
Thayer, P. W., & Kalat, J. W. (1998). Questionable criteria. American Psychologist, 53, 566.
Margaret W. Matlin received her bachelor's degree from Stanford University and her PhD from the University of Michigan. She currently holds the title of Distinguished Teaching Professor of Psychology at SUNY Geneseo, where she has taught for 29 years.
Margaret Matlin is the author of four current textbooks, Psychology (third edition), Psychology of Women (fourth edition), Cognition (fifth edition in press), and Sensation and Perception (with coauthor Hugh J. Foley, fourth edition). Dr. Matlin has also won several teaching awards, including the American Psychological Association Teaching of Psychology Award for Four-Year Institutions (1985), and the American Psychological Foundation's Distinguished Teaching in Psychology Award (1995).
Among Margaret Matlin's interests are travel, foreign films, and 19th-century British novels. In 1990, she and her husband, Dr. Arnold Matlin, founded a preschool educational and nutritional program in El Sauce, Nicaragua; about 300 children have subsequently attended the program.
James W. Kalat, professor of psychology at North Carolina State University, received his PhD from the University of Pennsylvania in 1971. After a few years of doing research on taste-aversion learning in rats, he shifted his focus to textbook writing. He has written Biological Psychology, currently in its seventh edition, and Introduction to Psychology, with the sixth edition scheduled for summer 2001. Both reflect his experience in teaching both courses every semester for many years. He has served on the GRE Psychology Committee and the Convention Program Committee of the American Psychological Society. His nonacademic interests include his children, grandchildren, and birdwatching.
Authors' note. We thank Walter Emmerich, Robin Durso, and Judson Sheridan for their comments on an earlier version of this article. We also thank Heather L. Gitlin, Susan A. Obetz, and Dawn Robinson for their comments on this current article.
Send correspondence to Margaret W. Matlin, Department of Psychology, SUNY Geneseo, Geneseo, NY 14454; e-mail: firstname.lastname@example.org, or to James W. Kalat, Department of Psychology, Box 7801, North Carolina State University, Raleigh, NC 27695-7801; e-mail: email@example.com.
For more details on the scoring system and the test preparation, consult our article in the journal Teaching of Psychology (Kalat & Matlin, 2000).
Copyright 2001 (Vol. 5, Iss. 2) Psi Chi, the International Honor Society in Psychology