Lest anyone forget, we’re in the run up to Christmas. The already annoying adverts transform into a parade of even more annoying adverts (one good thing about living outside the UK is I never see the sodding Argos aliens), flogging overpriced goodies through any means necessary. A lot more time seems dedicated to expensive luxury items modelled by beautiful specimens of man- and woman-hood! And hey, everybody knows that sex sells, right?
A new study in Psychological Science purports to find that, actually, sex doesn’t always sell: women find sexy adverts a turn off, and more so when the goods are cheap. So sometimes, ads can be just *too* sexy.
The study, “The Price Had Better Be Right: Women’s Reactions to Sexual Stimuli Vary With Market Factors”, by Vohs, Sengupta, & Dahl, garnered media attention (Sexy adverts turn women off, research shows, The Independent), as these things are wont to do, and you can read the press release here.
A while back, Rolf Zwaan deconstructed another article from Psychological Science, asking 50 questions about clean rooms and messy data. Funnily enough, an author of that article was also an author of this article. And I have quite a few questions for them about this one too.
Let’s have a quick run through of the logic here. According to Vohs, Sengupta, and Dahl, women typically react negatively to sexual images. For women, sex should be portrayed as rare, special, and infrequent in order to maintain it as something of high value, and properly reflect the higher costs of sex and reproduction for women and their overall lower sexual desire. Vohs in particular adheres to the theory (her own, along with Baumeister) of sexual economics, which apparently predicts that women should want sex to take place only when they get something out of it other than sex itself. Thus, associating sex with something cheap should produce a more negative reaction than associating it with something expensive, because it debases the value of sex. Men shouldn’t care whether sex is used to sell something cheap or expensive, because they don’t value sex as highly as women.
I don’t find this terribly convincing, and the back-up evidence was largely produced by the authors. But, let’s move swiftly on to the meat and potatoes of the experiments. Just like Rolf, I’ve a few questions which should be pretty straightforward, and I’ll follow the structure of the paper. I won’t quite get to 50, but let’s see…
Eighty-seven undergraduates (47 women) participated in a 2 (gender) × 2 (price:cheap vs. expensive) between-subjects design that used a sexy ad to promote a product. Two additional conditions (n = 46) assessed women’s attitudes to nonsexual ads that varied in price (cheap vs. expensive).
(1) How old were the participants, and where were they from? The authors are in marketing departments at Universities in three different countries (US, Canada, Hong Kong). This strikes me as something that might have huge cross-cultural differences.
(2) There were two additional control conditions with 46 subjects. Were there 46 subjects in each of the two conditions or 23 in each? Were the subjects different from those who took part in the other conditions? Why were these controls only run on women and not on men?
(3) Doesn’t ~22 participants per condition seem on the low side? How big an effect are we expecting here, like, King Kong bellyflopping in the Hudson River after leaping from the Empire State Building effect sized? I mean that’s a pretty decent leap to start with.
All participants were subjected to a cognitive load while they perused the ads, so that the ads would elicit spontaneous reactions (Gilbert, Krull, & Pelham, 1988).
(4) Why is this in the participants section?
To induce cognitive load, we had participants silently rehearse 10 digits while viewing three ads for 20 s each. The second ad promoted Chaumet women’s watches.
(5) The participants saw three adverts for 20 seconds each, with the target advert the second of the three. What were the other adverts? Is having an additional cognitive load task really going to ensure you only tap participants initial reactions to the adverts when they had 20 seconds to view them?
(6) Why Chaumet watches? Why watches specifically? Why only one product? If the participants knew the brand, why would they believe one would be available for 10$ anyway?
Participants in the sexual condition saw explicit sexual imagery taking up the majority of the ad,
with an image of the product in the bottom corner (see Sengupta & Dahl, 2008, for the images).
(7) Why aren’t pictures of the stimuli included in this paper, and instead only available in one from 5 years ago, in a different journal? And honestly, explicit? It’s maybe just about post-watershed.
(8) Why do the graphs start from zero when the rating scales were from 1 to 7? It’s hard for me to tell me what the actual ratings were, and this would be really quite useful.
A 2 (gender) × 2 (price) analysis of variance (ANOVA) of the data from the sexual condition revealed the predicted interaction, F(1, 83) = 4.97, p < .05, hp 2 = .056,
(9) Phew, that was close (p = .03)! Is the effect size partial eta squared? Anyway, looks like the authors’ ability to predict crossover interactions, ably noted by Rolf Zwaan, extends to ordinal interactions too! Kudos. I can’t really interpret the “control” non-sexual conditions on the right since no direct statistical comparisons are made between them and any of the other conditions.
(10) Why assess the results of the two control conditions using an F-test and not a t-test? More to the point, why assess the two control conditions separately from the experimental conditions to which you compare them? There should be an interaction test here. Note that they repeat the same mistake throughout the paper. It’s hard to draw firm conclusions without such tests.
A second hypothesis was that the Gender × Price interaction would predict self-reported negative affect. This hypothesis was supported, F(1, 82) = 3.68, p = .059,
(11) So, it wasn’t supported, then?
As expected, men’s self-reported negative emotions did not vary with price condition (F <1), whereas women showed the predicted pattern of stronger negative emotions after viewing a sexual ad promoting a cheap rather than expensive watch, F(1, 82) = 5.43, p <.05, hp 2 = .088 (Fig. 2).”
(12) but the interaction wasn’t significant, so…
(13) Was a 60 second viewing followed by a quick set of ratings really all the participants did? Were they paid/compensated for their time?
(14) Can the authors really draw firm conclusions from a single stimulus? What if it was just something peculiar to the images used in this study?
(15) What were the ratings calibrated to? What would be considered a really good, likeable advert which would elicit positive emotions? What would be considered a really bad, horrible advert that would elicit negative emotions? I can think of a few.
(16) The authors state that this experiment was intended to replicate exp 1 with the addition of a second watch to the advert, a men’s watch, to test for the possibility that men found the product irrelevant in Exp 1 because it was a women’s watch. Why not just replace the women’s watch with a men’s watch instead of presenting both, so that you can test what happens when it becomes irrelevant to women instead, as well as what happens when it becomes relevant to men?
(17) Why assume that the women’s watch was irrelevant to men? Don’t men buy gifts? Is that really the only possible explanation the authors could come up with? What if it was some other product altogether? Perfume? Power tools? You know, something that’s always annoyingly gender stereotyped in adverts. Did the reviewers really think this was a good control?
Participants were 212 adults (107 female). They participated in a 2 (gender) × 2 (price:cheap vs. expensive) design with 2 hanging control conditions in which nonsexual ads (cheap vs. expensive) were viewed only by women.
(18) 212 participants in this one, but again those two weird unclear control conditions, where it’s not clear if they were given to the same subjects or not. Also, again, lacking critical information about the participants. These participants are described only as adults, leading me to believe it’s a somewhat different population to experiment 1. How old were they? How were they recruited? How were they compensated? Where were they from?
After reporting the digits and thus releasing the cognitive load, participants moved sliding scales (equating to 100 points)
(19) Wait, I thought this was a direct replication with the exception of the change of stimuli? Why change to a sliding 0-100 scale from a 1-7 Likert type scale?
As predicted, and as in Experiment 1, gender and price had an interactive effect on the two key outcomes in the sexual condition—ad attitudes: F(1, 115) = 3.83, p = .05, hp2 =.032;
(20) .052, and not significant, again.
(21) Hang on, F(1, 115)? there are 212 participants, four conditions (excluding the two weird control conditions, because I don’t know who actually does them). Why are there only 115 df here? Shouldn’t it be 208? Honestly, the results section here is really confusing, because it’s not always clear what actual tests were done.
A few other random questions:
(22) Isn’t anybody curious how every hypothesis was as predicted, and was unchallenged even when the stats weren’t significant?
(23) Did participants even notice how much the watches cost? Why not just ask them?
(24) Do the authors/reviewers/editor not think, perhaps, that given there is basically no research on this topic conducted by anyone other than the authors, it might look a little odd that everything comes out exactly the way it is predicted?
These findings have several* implications. One is that women can be swayed to tolerate sexual imagery, as long as it comports with their preferences regarding when and why sex is used. A second, more profound implication is that women’s reactions to sexual images reveal their preferences about how sex should be understood. *Two.
(25) Don’t the authors/reviewers/editor think that these conclusions are just a little strident, given that we have virtually no idea who the participants were, have no information about the sexual preferences of the participants, and only a single product (a watch) was ever presented? Aren’t there any alternative possibilities, questions left open, limitations of the current research?
(26) Do the authors really think that women’s sexual preferences can be so simply identified with perception of financial value? Why claim that women can be swayed to tolerate sexual imagery as if by default they do not? Is there any evidence for this claim that the authors could provide that isn’t in a paper they wrote?
(27) Do they really want to claim that reactions to an advert for a watch is a good way of assessing women’s valuation of sex? REALLY?
And then, a few more questions, which just occurred to me while quickly going over Sengupta & Dahl (2008). Just a few, you know, slight concerns.
(28) Why don’t the authors of the current paper mention that their experiments are a replication of the experiments in Sengupta & Dahl?
(29) Why do the authors think that Sengupta & Dahl, with almost exactly the same set-up, find that men’s attitude towards sexual adverts was better than towards non-sexual adverts? Why didn’t they predict that for the current study, on the basis of their previous results?
(30) Why did Sengupta & Dahl actually predict the different pattern of results that they found in their original paper, but make different predictions here, which the results, yet again, perfectly matched?
(31) Didn’t the reviewers and/or editor even look at Sengupta & Dahl 2008, despite it being an absolutely critical paper for the interpretation of Vohs et al.?