Print Page   |   Contact Us   |   Sign In   |   Register
Psi-Chi-ology Lab
Blog Home All Blogs
Search all posts for:   


View all (17) posts »

All About Psi Chi's New Call for Abstracts!

Posted By Psi Chi Central Office, Wednesday, June 7, 2017

All About Psi Chi's
New Call for Abstracts!

To encourage open and reliable research practices, Psi Chi Journal of Psychological Research is proud to announce a Call for Abstracts (due June 30) for potential manuscripts to be published in a special issue. In today’s post, Invited Editor Dr. Steven V. Rouse (Pepperdine University) shares about this very unique opportunity.

What is the special issue about?

In the last couple of years, there has been growing interest in making research more transparent. It used to be the case that researchers would only share their materials or data with people who directly asked for it, and then only when they could justify a compelling reason. The problem is that this made it hard to replicate studies, so it was hard to know whether the results were simply a statistical fluke or really represented a consistently observable finding. So, several journals have started awarding Open Practice Badges to articles that have followed some of the contemporary best practices in research.

What are Open Practices Badges?

The Center for Open Science encouraged journals to begin awarding three badges, which you can see at The first is called Open Data, which simply means that the researchers have agreed to post their data online at an open-access site, allowing anyone to download the data (provided that it doesn’t violate confidentiality or any other aspect of the IRB approval).

The second is Open Materials, which is given to an article if the authors post their surveys and other research materials on a freely accessible website (except, of course, for anything that is protected by copyright or test security restrictions).

The third is Preregistration. This means that the researchers took the time (ideally prior to data collection, but at least prior to data analysis) to specify their research plans. This would include details like the number of subjects that would be included, the specific procedures of the study, the hypotheses, and the statistical analyses that would be performed. Really, almost all of us think through these questions before starting a study anyway because these are the kinds of questions that researchers have to answer for IRB approval. The difference is that the researchers agree to have these details “frozen” on a publicly accessible website before gathering or analyzing their data.

In addition to those three badges created by the Center for Open Science, Psi Chi Journal also created its own fourth badge: Replication. Because Psi Chi Journal and the Psi Chi Research Advisory Committee believe that replication is an important part of the scientific process (as explained HERE), we wanted to recognize articles that serve this important role.

Beyond that quick overview of the four badges, I wrote an editorial that explains each of them in more depth.

Why should a researcher want to be published in this special issue?

I truly believe that the field of empirical psychology is at a turning point that will be historic. We are moving in a direction that is more open and transparent, more collaborative, and more intentional. And I think this change is a positive one. It excites me to know that Psi Chi Journal is at the forefront of this change, along with some of the flagship journals of the Association for Psychological Science. And it excites me to know that our authors—especially the authors who are already doing high-quality empirical research in their undergraduate years—will be among the earliest psychological researchers to earn Open Practice Badges.

Who is eligible to submit an abstract?

Any Psi Chi member—undergrad, grad, or faculty member—can submit an empirical research article to be considered for publication in Psi Chi Journal. For this special issue, though, there are a couple of specific criteria. First, the manuscript needs to qualify for one or more of these Open Practice Badges. But, second, the project needs to be far enough along—either completed or in advanced stages—for us to be able to expect the completed manuscript submission later this fall. So if you are working on your IRB proposal this summer and plan to collect data when the fall arrives, why not send in a proposal abstract?

How easy or difficult is it to earn a badge?

The nice thing about the badges is that researchers can choose for themselves how much they want to jump into this new approach—do they want to stick their toes in the water to try it out, or do they want to jump all the way in. For me personally, when I realized that this was the direction that our science is moving in, I just jumped right in, and I found out it was a lot easier than I expected. In part, this is because of an amazing website called the Open Science Framework (, which allows researchers to create free accounts that they can use to store their data, their materials, and to preregister their studies.

But, realistically, some badges would be easier to earn than others. The Open Materials badge would be the easiest. After all, this just means that the authors have posted the surveys and other research materials on OSF for anyone to access. As long as this doesn’t violate copyright laws or test security ethical principles, I think most studies should be able to earn this badge. In fact, every time I conduct a survey study now, I post the survey on my OSF site:

People who are willing to make a bigger commitment could also explore the Open Data badge. It’s still a very easy process—authors simply upload Excel files or SPSS files or R files onto OSF. This is a bit trickier than the Open Materials Badge, though, because authors need to make sure this doesn’t violate privacy. If an author didn’t indicate on their IRB proposal that they were going to upload the data, they should probably check with their IRB representatives to make sure that this wouldn’t be a violation of their approval. But now, for example, whenever I submit an IRB proposal, I inform my IRB that the data will be stripped of any identifying information and then will be posted permanently on OSF; I also explain this on the Informed Consent Forms.

A Preregistration requires a greater commitment, but it’s become so helpful to me personally that I can’t imagine ever doing a study without preregistering it. When you create a project on OSF, there’s a link that lets you preregister it; it leads you through a set of questions about your methodology, hypotheses, and planned analyses, and then when you are satisfied with your answers, it freezes the preregistration. In other words, the system puts a time-stamp on your preregistration and prevents you from making any further changes. Then, when you actually conduct the study, you follow the steps you already laid out in advance (unless you have a reason to make a change, in which case you simply explain the reason for the change. These are called Transparent Changes, which still allow you to earn a badge).

Probably the greatest commitment is a Replication, because everything about your study is guided by the article that you are trying to replicate. In some ways, this seems like it should be easier, because you are following in someone else’s footsteps. However, in order to be a true replication, an author needs to be conscientious to step exactly in those existing footprints or to be aware of any deviations. So everything about the study must perfectly duplicate the original study or clearly explain what factors they changed and why. After all, if the results of a replication differ from those of the original study, we need to be able to come up with hypotheses for why the findings weren’t consistent.

Would my article be eligible for the $1,000,000 Preregistration Challenge?

This is really exciting. The Center for Open Science has a $1,000,000 fund to award prizes of $1,000 to 1,000 researchers who preregistered their studies and then published them in approved journals. Learn more.

However, there are two important details. First, when preregistering on OSF, you have to specify that you want to be eligible for the Preregistration Challenge and answer a specific set of questions. You see, when you preregister, OSF provides you with a few different options of preregistration questions to answer. The simplest set of questions is the AsPredicted form, which I used for a project preregistered as seen HERE. As shown, this is simply a set of eight very basic questions. However, this wouldn’t qualify for the challenge. Another option is called the Preregistration Challenge form, which I used for a project I preregistered HERE. As you can see, this is much more in-depth, with 26 questions to answer. Then, when you submit it, a researcher at OSF reviews it and sends you an e-mail if there are details that you need to clarify. So this is more time-consuming, but I have found it to be really helpful in getting feedback about my plans.

Second, even if a project has been preregistered, it can only win the award if it gets published in a journal that qualifies for Preregistration badges. If you visit HERE, you can see that more than 3,000 journals are now awarding badges. However, only a few dozen of these are in psychology—the rest are in other scientific disciplines. So you have to be sure to submit it for publication in one of the journals on this list. Because Psi Chi Journal was one of the early adopters of the Open Practice Badges, our manuscripts meet this qualification requirement for the award.

What were your thoughts when you were asked to lead this special issue?

When Dr. Debi Brannon, the editor of Psi Chi Journal, asked me to serve as the guest editor, I was really excited. After all, I really believe that this is an important new change in the field of empirical psychology, and I look forward to the day when it’s more common to see preregistrations, replications, and open posting of data and materials. But then it struck me how unique this idea is. You see, every time I’ve ever seen a Special Issue of any psychological research journals, all of the articles are unified by a certain theme or a certain topic, like when the journal Current Directions in Psychological Science had a special issue all about the cognitive ability of dogs (Vol. 25, Issue 5). This is completely different. This issue won’t be unified by a topic area. Instead, the articles will be similar in that they will have all taken the steps necessary to earn one or more of these badges. Psi Chi Journal has never before had a special issue, and I think this is the best way to start—bringing awareness to this new set of best practices in psychological research.

Conduct an Lab Experiment

Psi Chi members, do you have questions about the special issue, our journal, or its badges? We would like to hear from you in the comment section below.

Tags:  Conducting Research  Psi Chi Related 

Share |
Permalink | Comments (0)

Phone: (423) 756-2044 | Fax: (423) 265-1529 | Certified member of the Association of College Honor Societies
Membership Software Powered by YourMembership  ::  Legal