In the United States, about half of doctoral-level psychology programs and a third of master's-level programs require applicants to submit scores from the Graduate Records Examination (GRE) Psychology Test. Still other programs recommend the GRE Psychology Test, but they do not require it (Norcross, Hanych, & Terranova, 1996).
Undergraduate students typically find that psychology professors do not have much information about the GRE Psychology Test. Most of the information we'll discuss in this article is available in free but seldom-read publications from Educational Testing Service1 (P.O. Box 6000, Princeton, NJ 08541-0001). The rest comes from our combined experience in serving on the committee that guides the development of the GRE Psychology Test. (James Kalat served as the committee chair from 1990 to 1998, and Margaret Matlin became the committee chair in 1998.)
The GRE Psychology Test is a paper-based test in a multiple-choice format, with five choices per item. The test includes about 215 items; students are allotted 2 hours and 50 minutes to complete the test. The GRE Psychology Test is administered on three dates each year, at test locations throughout the world. In the present article, we will discuss how the GRE Psychology Test is scored and how the test is developed, as well as information on the test's usefulness.
The Scoring System for the GRE Psychology Test
The GRE Psychology Test is designed to be challenging. You may be accustomed to earning scores in the neighborhood of 90% on the examinations in your psychology courses. As a result, you may worry about the large number of items that you cannot answer. Here's some reassuring information: You can earn a score of 500 on the GRE Psychology Test by answering 43% of the questions correctly, answering 25% of the questions incorrectly, and leaving 32% of the questions unanswered (Graduate Records Examination Board, 1999b, p. 7). Remember, too, that students who take the GRE Psychology Test constitute a selected sample of all psychology undergraduates. Keep in mind, then, that a score of 90% correct on this exam would be extremely unusual!
The mean score on the GRE Psychology Test is 554, with a standard deviation of 99, based on the performance of all examinees who took the test between October 1, 1995, and September 30, 1998 (Graduate Records Examinations Programs, 1999a, p. 15). The reliability of the test is .95, as estimated in an analysis of a sample of 4,130 scores (Graduate Record Examinations Board, 1999a, p. 22). The 99th percentile on this test starts at 770; a score of 340 is in the first percentile.
Your score on the GRE Psychology Test will be based on the number of questions you answered correctly, minus one fourth of the questions you answered incorrectly (Graduate Records Examination Board, 1999b, p. 4). The question often arises: Should you guess when you do not know the answer? This question has a complex answer. Many students believe that this formula will hurt them if they guess, rather than omitting an answer. Across all examinees who take the test, the mean score of the whole population will not be affected by random guessing. That is, the number of points gained by guessing will be balanced by the correction factor for incorrect answers. However, by chance, some individual students will lose points by guessing (and other students will gain points by guessing).
What should you do if you can eliminate one or more of the five possible answers, or if one answer seems like a slightly better guess than the others. Here, your chances of answering correctly are improved. In these cases, you should certainly fill in an answer, because this type of "strategic guessing" may indeed improve your score.
Who Prepares the GRE Psychology Test, and How
Most psychology faculty members do not know how the GRE Psychology Test is prepared. In fact, neither of us knew about the procedures when we were initially asked to serve on the committee. The GRE Program of the Educational Testing Service (ETS) appoints a committee of six psychology faculty members from U.S. colleges and universities. The committee members are selected to be diverse in terms of their research specialties, the geographic location of their universities, and so forth. Furthermore, they must have a broad background in psychology as a whole, in addition to expertise in just a single area.
Each committee member writes about 15 potential test questions per year. The ETS test developers also invite other faculty members to write additional questions. The committee members individually review the hundreds of questions that are submitted each year. Then they meet for a three-day session once a year to discuss each question. During this session, they review whether the question addresses an important issue, whether it is clearly stated, and whether the question has only one correct answer.
On the GRE Psychology Test, 40% of the questions deal with material in cognition, sensation and perception, learning, and biological psychology; another 43% deal with social psychology, personality, development, and abnormal behavior. The remaining 17% cover material such as research methods, history of psychology, and applied psychology. On the actual test, however, the three types of questions are distributed throughout, rather than appearing in three separate sections. Your overall score for the GRE Psychology Test is based on questions from all three of these areas.
After each administration of the test—and before scores are reported at ETS—statisticians at ETS calculate extensive statistics on each question. The information includes the percent of students who answered each question correctly, the point biserial (that is, the correlation between performance on that question and performance on the test as a whole), and information on how well the students who selected each choice for a given item performed on the rest of the test. For example, the results might indicate that the students who chose answer "B"—the correct answer—had the highest overall scores on the test, whereas those who answered "D" had the worst. These statistics call attention to items with possible problems, and almost any test has a small number of these items. Mostly, these are cases in which the point biserial is less than .30, and a large percentage of the best students had marked a choice other than the intended correct answer.
The staff then consults with committee members to decide whether the item might have a second acceptable answer. Questions that are judged to be problematic are not used in computing scores. The staff at ETS continually reviews the performance of test questions. If performance patterns on a specific question change, they will stop considering that question in the scores. Furthermore, they will not use that question in future tests. A change in performance patterns may occur for many reasons. However, the most common problem occurs when new research either disproves an old theory or supports a new explanation of behavior. Thus, the GRE Psychology Test is updated to reflect changes in our knowledge about psychological phenomena.
How Useful is the GRE Psychology Test?
Psychology departments consider many kinds of information when they evaluate applicants for admission to graduate school. They generally emphasize grade point average (GPA), GRE General Test scores, GRE Psychology Test scores, and letters of recommendation. They may also examine information such as the student's research experience, the prestige of the student's undergraduate institution, and the match between student and faculty interests (Keith-Spiegel, Tabachnick, & Spiegel, 1994).
The GRE Board (1999a) decided to investigate the correlation between several quantitative measures and a student's first year grades in graduate school. The Board collected quantitative data from 110 graduate departments of psychology for the years 1986 through 1990; the study was based on 1,151 students who had pursued graduate work. The data included the student's undergraduate GPA, GRE General Test scores (verbal, quantitative, and analytical), and scores on the GRE Psychology Test. Of these measures, which one would you guess correlates best with first-year grades in graduate school?
As Table 1 shows, both the GRE Psychology Test and the undergraduate GPA are correlated .37 with first-year graduate grades. This equivalence is especially interesting, because the Psychology Test is based on students' performance on just one examination, whereas the undergraduate GPA is established over four years of college courses. Table 1 also shows that scores on the GRE Psychology Test are more highly correlated with first-year graduate grades than are any of the GRE General Test measures, alone or in combination.
Some researchers have argued that first-year grades may not be an appropriate measure of success in graduate school (Sternberg & Williams, 1997). However, we have no evidence that other measures of success—such as faculty evaluations of graduate students—are reliable or valid (Thayer & Kalat, 1998). Also, first-year grades are important because they can determine a graduate student's future. At many universities, graduate students who fail to reach some minimum criterion in their first-year grades, such as a B average, are dismissed from the program or denied financial aid.
Kuncel, Hezlett, and Ones (1998) examined data on graduate students in all academic fields, not just psychology. In their meta-analysis of 201 studies of 27,039 graduate students, the GRE Subject Test was correlated .46 with first-year graduate GPA, .48 with scores on comprehensive examinations in graduate school, .53 with faculty ratings, and .21 with completion of a graduate degree. With the exception of degree completion, the GRE Subject Tests correlated more strongly with these measures than either the GRE General Test or the undergraduate GPA.
Future Directions of the GRE Psychology Test
Several years ago the GRE Board considered offering the Subject Tests in a computer-based format, consistent with the current GRE General Test. However, constructing the computer-based GRE General Test required adding an enormous number of new items to the test pool. The cost of constructing a similar number of new items for the Psychology Test pool would be prohibitively expensive. In other words, you can count on the paper-and-pencil format for the foreseeable future!
The GRE Psychology Committee has been more concerned about another issue, the nature of the test questions. Many people—both students taking the GRE Psychology Test and the committee members themselves—have complained that the test may overemphasize rote recall of facts, some of them not especially interesting or important. Items of that kind are the easiest to write, and they usually produce decent item statistics. However, they do not reflect what most psychologists consider the best side of psychology.
In recent years, the GRE Psychology Committee has tried to reduce the number of items that require simple recognition of terms or psychologists' names. We have replaced these items with more conceptual questions. However, students often obtain commercially available preparation books for the Subject Test. Some of these books contain numerous "name that psychologist" items as well as outdated "factual" questions. Students may be misled by their scores on the practice tests in these books. The books will also encourage students to use inappropriate review strategies in preparing for the GRE Psychology Test. These strategies will be especially counterproductive as the GRE Psychology Committee increases the number of conceptual questions.
The official guide—GRE: Practicing to Take the Psychology Test—will give you better information and more realistic practice items. You can order this resource from the GRE website at www.gre.org or by calling 1-800-537-3160 or (609) 771-7243.
Since the mid-1990s, new editions of the test have included some "analysis-of-evidence sets" among the methods items. For these questions, you will read a paragraph describing a hypothetical research study and its results. Then you'll answer two to five questions about proper conclusions to be drawn, potential weaknesses of the study as described, and ways to improve the research design or to follow up with additional research. So far, students' performance on these item sets has correlated well with the rest of the test. Future forms of the GRE Psychology Test will continue to use such items. These analysis-of-evidence items attempt to assess critical thinking, consistent with the increasing emphasis on critical thinking in psychology (e.g., Nunmedal & Halpern, 1995).
In summary, the GRE Psychology Test continues to evolve. With fewer factual items and a larger number of conceptual and critical-thinking items, the test will have greater content validity. As a result, the test will assess more completely those skills that psychologists value most highly in their graduate students.
Graduate Record Examinations Board. (1999a). Guide to the use of scores: Graduate Records Examinations, 1999-2000. Princeton, NJ: Educational Testing Service.
Graduate Record Examinations Board. (1999b). Psychology Test descriptive booklet, 1999-2001. Princeton, NJ: Educational Testing Service.
TKalat, J. W., & Matlin, M. W. (2000). The GRE Psychology Test: A useful but poorly understood test. Teaching of Psychology, 27, 23-26.
Keith-Spiegel, P., Tabachnick, B. G., & Spiegel, G. B. (1994). When demand exceeds supply: Second-order criteria used by graduate school selection committees. Teaching of Psychology, 21, 79-81.
Kuncel, N. R., Hezlett, S. A., & Ones, D. S. (1998, April). The predictive validity of the Graduate Record Examination: A meta-analysis. In J. P. Campbell & D. S. Ones (Chairs), Selection Into I-O programs: Focus on GRE validity. Symposium conducted at the 13th annual conference of the Society for Industrial and Organizational Psychology, Dallas, TX.
Norcross, J. C., Hanych, J. M., & Terranova, R. D. (1996). Graduate study in psychology: 1992-1993. American Psychologist, 51, 631-643.
Nummedal, S. G., & Halpern, D. F. (1995). Introduction: Making the case for "psychologists teach critical thinking." Teaching of Psychology, 22, 4-5.
Sternberg, R. J., & Williams, W. M. (1997). Does the Graduate Record Examination predict meaningful success in the graduate training of psychologists? A case study. American Psychologist, 52, 630-641.
Thayer, P. W., & Kalat, J. W. (1998). Questionable criteria. American Psychologist, 53, 566.
1 The sidebar on page 25 lists Educational Testing Service publications available online at www.gre.org/codelst.html#sidata.