hostile priming effects seem to be robust

In 1979, Srull & Wyer published a study wherein participants were presented with a series of 4 words from which they had to construct grammatically correct 3-word phrases. Some phrases described aggressive behaviors (i.e., break his leg). Later, participants read a story about the day in the life of a man named Donald. In this story, Donald performed ambiguously aggressive behaviors (e.g., argued with his landlord). Finally, participants provided their judgments of Donald by rating him on a series of traits that were combined into a measure of hostility (e.g., hostile, unfriendly, dislikable, kind (r), considerate (r), and thoughtful (r)). Participants who completed more aggressive phrases in the first task subsequently rated Donald as more hostile. Exposure to hostile-relevant stimuli that subsequently affects some type of a subsequent hostile-relevant impression is generically referred to as a “hostile priming” effect.*

So, is there strong evidence for the robustness of a “hostile priming” effect?** If you asked me 6 years ago I would have said “yes.” Why? First, DeCoster and Claypool (2004) performed a meta-analysis on cognitive priming effects that used an impression formation outcome variable and found that, overall, there is an effect of about 1/3 of a standard deviation in the predicted direction (i.e., k = 45, N = 4794, d = 0.35, 95% CI[0.30, 0.41]). Further, several of the studies included in the DeCoster and Claypool meta-analysis primed the construct of “hostility” and had an outcome variable that was relevant to the construct “hostility.” Second, the hostile priming effect has been demonstrated in dozens of published studies.

However, in 2016, I feel there are a few reasons to question the robustness of the hostile priming effect. First, DeCoster and Claypool didn’t investigate the presence of publication bias. That is not a knock on their excellent meta-analysis. But I believe that everybody in 2016 is more cognizant of the potential problems of publication bias than 12+ years ago. And we currently have more tools to detect publication bias than we did 12+ years ago. Second, it seems that cognitive phenomena labeled as a “priming” effect are currently viewed more skeptically. Fair or not, that is my belief about the current perceptions of cognitive priming effects. Third, many of the studies in the DeCoster and Claypool meta-analysis were authored by Diederik Stapel. Obviously, we should interpret the studies authored by Stapel differently in 2016 than in 2004. 

In addition to being interested in the hostile priming effect, I also wanted to force myself to learn some new tools. (This is probably a good place to note that this exercise was mostly a way for me to practice using some new tools, so there may be errors involved or I may describe something in a slightly incorrect way. If you find an error, help me learn.*** Please and thank you!)

I gathered what I believe to be a comprehensive list of all of the publications with (a) an assimilative hostile priming manipulation and (b) some type of a hostile-relevant impression formation task. I found 27 publications with 38 individual studies (please let me know if you are aware of studies I missed).

First, I did a p-curve analysis on all of the studies. Here is a link to the p-curve disclosure table (https://mfr.osf.io/render?url=https://osf.io/ar5cf/?action=download%26mode=render). The analysis reveals these studies contain evidential value, z = -6.02, p < .001. The p-curve analysis estimated the average power of the studies to be 65%.

I was pretty liberal with my inclusion criteria for this first p-curve analysis. I next winnowed the studies down to ones that I believed were most focused on the effect of interest. First, I took out the Stapel studies (the continuous p-curve analyses implied “evidentiary” value in Stapel’s studies for those who are interested, z = -3.85, p = .0001).****  Next, there were some studies that were not really interested in the effects of hostile priming on impression formation per se.  Instead, some studies were interested in the relation between some construct they believed was associated with the construct of “hostility”, and these studies used an impression formation outcome variable to demonstrate this hypothesized relation.  For example, DeWall and Bushman (2009) proposed that people hold mental associations between the construct of “hot temperatures” and “hostility.”  This proposition was demonstrated by exposing individuals to words associated with hot temperatures prior to having them report their judgments of Donald.  Thus, the emphasis in studies of this ilk are not so much on impression formation, but these studies use impression formation tasks as a tool to test their other hypotheses.  This led me to take out studies that primed “hostility” with hot temperatures (e.g., DeWall & Bushman, 2009; McCarthy, 2014), alcohol (e.g., Bartholow & Heinz, 2006; Pederson et al., 2014), sex (Mussweiler & Damisch, 2008; Mussweiler & Forster, 2000), and aggressive sports (e.g., Wann & Branscombe, 1990). (collectively, these omitted studies did not have evidential value, z = -0.71, p = .24). 

A p-curve analysis on the remaining 18 effects still revealed evidentiary value for an effect, z = -4.91, p < .001. The p-curve analysis estimated the average power of the remaining studies to be 74%. 

So, this is just a first pass on examining the evidentiary value within these studies.  And, based solely on these p-curve analyses, it looks like there is evidence for a hostile priming effect on subsequent impressions of hostility.  I plan on working through a few other tests such as Test for Insufficient Variance, meta-analysis of effect sizes, etc.  Again, this is my way of forcing myself to learn, possibly get some free feedback, identify obvious errors, etc.. 

* “Hostile priming” needn’t be limited to subsequent impression formation tasks. It could, for example, include a behavioral outcome measure (e.g., Carver et al., 1983). However, for the present purposes, I limit the discussion to outcome variables that broadly fall into the class of impression formation tasks. This choice is purely based on my personal interests and not on any theoretical justification.

** In this blog post I am only referring to an “assimilation effect”: That is, priming effects that cause subsequent judgments to possess more of the primed construct.  There are situations when the primed constructs cause subsequent judgments to possess less of the primed construct. These latter effects are referred to as “contrast effects” and are not discussed herein. Again, this choice is purely based on my personal interests and not on any theoretical justification.

***If you find an error, please find me at the next conference and say "hi." You are entitled to one free beer/coffee from me (either one at any time of the day/night, I don't judge). 

**** As Alanis Morissette says “isn’t it ironic”