In my last essay, I made a joke about clocking in at the gender slop factory because I think it is funny how, despite my best attempts to be a Serious Academic, I keep writing about housework like a Ladies Home Journal feature writer from the ‘80s. However, my slop comment turned out to be a little too on-the-nose. This week, I had a career first: my own research showed up in an AI slop summary challenging one of my essays!
A reader posted that she had asked an AI to help parse one of the claims in my essay from last week about whether women are “naturally” better at housework than men. In the essay, I wrote that women may have a slightly lower disgust threshold, probably as an adaptation to protect the pregnant body from pathogens, but since housework tasks range widely in grossness, I did not think that was a big driver of gender differences in housework time.* Because the specifics of household tasks have changed so much over the centuries, essentialist gender role arguments have limited explanatory power. Wanting to evaluate that claim, the commenter asked AI, “does clutter bother women more than men?”
The AI generated a perfectly reasonable-sounding response challenging my point and citing some research about stress, cortisol, and household clutter. As I read, the summary started to sound uncannily familiar, so much so that I scrolled down and clicked on all the citations. Much to my disquiet, six out of the nine links referenced a study I had conducted myself. I got cogsplained by a bot! But the weirdest part is that the AI clearly hadn’t read the actual study, because it got all the details wrong. When I tried to figure out where it had gotten the details, the story got even weirder. Way weirder.
Let me back up and tell you about this study, because it’s a personal cautionary tale of mine about the telephone game that can happen when you talk to the media about research. I conducted this study as a fledgling graduate student cranking out my dissertation. It used data from UCLA’s Center for the Everyday Lives of Families study, or CELF. CELF was led by an eccentric but loveable group of researchers, helmed by anthropologist Eleanor Ochs, that included psychologists, anthropologists, linguists, and even a few archaeologists. We sought to understand how the “typical” dual-income, middle-class family operated. We embedded ourselves in families’ homes for a “week in the life” in which we tracked families’ locations, videotaped their interactions, asked them to fill out mood and mental health questionnaires, and sampled their spit. For the family observations, we adopted a primate research technique called “scan sampling” in which you record what everyone in the group is doing at ten minute intervals. Just as you might observe which gorilla is picking nits off another gorilla, we recorded when family members were arguing, when they were watching TV, and which they were washing dishes. Some really interesting projects came out of the study, including one by an archaeological anthropologist that involved indexing all the refrigerator magnets in families’ homes. It turns out that the average number of objects on a typical fridge is 55, and there is a lot of variability between plain stainless steel and whatever this is:
The CELF study yielded a coffee table book cataloguing the clutter in the families’ homes, and was heavily referenced in Jennifer Senior’s (excellent) book about the travails of parenting, All Joy and No Fun. It was also featured in a New York Times story quoting my friend Anthony Graesch (the same person who counted the refrigerator magnets) that the home observations were “The very purest form of birth control ever devised.” (He went on to have two kids, just like me, so the birth control failed for both of us).
I focused my dissertation research on the spit we collected as part of the study. That is, I looked at cortisol, a stress hormone that can be detected in saliva. We asked family members to drool into test tubes multiple times a day for several days, measured the level of cortisol in each saliva sample, and then tested whether their cortisol levels were correlated with our observations of family life.
Like many other hormones, cortisol has a diurnal rhythm – that is, it changes across the day. Cortisol typically peaks within the first hour after you wake up, when your body is getting mobilized for the day, and it drops across the day, reaching its nadir at night. The slope of cortisol looks like this:
Because of this diurnal slope, it’s not useful to talk about cortisol levels being “high” or “low.” Without knowing the time of day that you sampled your cortisol, those levels don’t mean much. (If you’ve ever heard a wellness guru saying that high cortisol is universally bad, they’re wrong). The value of a single sample matters less than the pattern of how cortisol rises and falls across the day. Under conditions of chronic stress and burnout, the diurnal slope flattens out – that is, cortisol levels start out low and fail to decline much across the day.
In my first study of cortisol within the CELF families, I found that women who were unhappier in their marriages had flatter daily slopes of cortisol. Men’s marital satisfaction was not linked with their cortisol patterns. It’s long been known that men show an overall health benefit from marriage—married men live longer than single men—whereas women are more sensitive to the quality of their marriage. Women in poor quality marriages are less healthy than single women, whereas women in high quality marriages are more healthy. Our cortisol finding might help to explain this differential health benefit by showing that for women, a bad relationship showed up in the body as chronic stress.
This study piqued my curiosity about why a bad marriage might be a chronic stressor for women, and I tested a few possibilities in my dissertation. In one study, I used the scan sampling data to calculate how much time couples were spending in at-home activities like housework and leisure. I found that, just as I’ve been telling my husband for years, chores are bad for your health; spending more time doing chores was linked with less recovery in cortisol levels at the end of the day, for both men and women. Since women were spending a larger proportion of their day doing housework (even though both spouses were employed outside the home), this might chip away at their health.
The clutter study built on the time use study. We asked CELF families to conduct self guided tours of their homes: they were given a camcorder (it was the early aughts!) and asked to walk around and describe the spaces and objects that mattered to them. I was curious about this data because it was naturalistic—that is, we were capturing open-ended speech, rather than numbers on a standardized questionnaire—so I thought we might be able to capture more insight about how the participants felt about their lives at home.
Since the CELF study was so interdisciplinary, I learned a lot from the material-culture anthropologists during our weekly team meetings, including my magnet-counting friend. The team frequently marveled at how much stuff our families possessed, so much so that it was spilling into garage spaces and storage sheds. Many of the anthropologists had done fieldwork in hunter-gatherer societies or excavated the remains of bygone societies, so they had a sense of how many possessions families had, both globally and historically. Our American families represented an extreme outlier: their homes were exploding with possessions.
Another theme of our weekly meetings was the uniquely American lust for renovation and renewal. Our families often had home improvement projects planned or underway. No one was content with the home as-is. (As I said, this was the early aughts, so my hunch is that renovation lust has only accelerated since then, given the glut of home remodeling TV shows that have emerged in the last decade).
Since we were interested in patterns of cortisol recovery from the workday, we wanted to see whether the home felt restorative – that is, a haven within which to unwind after a stressful day – or another source of stress in its own right. What might make the home feel more stressful? We hypothesized that being overwhelmed by clutter or a to-do list of unfinished projects might amp up ambient stress at home.
We used the home tour transcripts to ferret out these themes. In order to do that, I worked with a software program called LIWC – Linguistic Inquiry and Word Count. This is a program that takes a block of text (a conversation, a blog post, a diary) and looks at how frequently words are used in different categories. One of my favorite LIWC studies, done by the rad Sarah Pressman, found that when people use more social words in their autobiographies, they live longer. Other LIWC studies have found that happier couples use the pronoun “we” more often. It’s an interesting but admittedly pretty crude way to analyze text, because it’s just looking at frequency and not context; my colleague Morteza Dehghani has come up with much more sophisticated methods for investigating natural language.
In any case, LIWC didn’t have pre-loaded categories for constructs like “clutter” and “renovation,” so we created some custom categories by making a dictionary of words like “mess,” “junk,” “disorder” and “sloppy” to capture “clutter” and words like “unfinished,” “repair,” “fix,” “remodel” and “renovation” to capture the “home unfinished” category. We also created a “restful” category with words like “relax,” “calm,” “comforting” and “soothing,” and a “nature” category with words like “plant,” “tree,” “yard” and “patio.”
So what did we found? The average home tour was about 15-20 minutes long, and people used about four or five “clutter” words per tour. The word “mess” came up in 40 home tours. People used our dictionary words pretty often in the tours; the average tour used about eight of our target words, and 58 of the 60 tours included at least one target word. We looked separately at how each category linked up with patterns of mood and cortisol, but the paper got very long, so we ended up doing a factor analysis to condense the “clutter” and “renovation” categories into one “stressful home” variable and the “restful” and “nature” categories into our “restorative home” variable. (The results didn’t change much after we collapsed the four categories into two).
Home tour language linked up with daily cortisol patterns in the CELF wives, such that “stressful home” language predicted flatter cortisol slopes (the “chronic stress” pattern) and “restorative home” language predicted steeper cortisol slopes that seemed to reflect healthier adapation to everyday stress. This graph shows all the cortisol values in our sample; the white diamonds, women above the median for stressful home language use, had lower morning cortisol and less of a dropoff across the day.
The cortisol results was echoed by daily mood patterns: wives who used more “stressful home” language had increasing levels of depressed mood as the day went on, and decreasing depressed moods when they used more “restorative home.” We didn’t get any results for the men in the study—maybe because they’re not socialized to care as much about their home spaces, or maybe just because there were other factors that would explain their cortisol better.
I wrote up this study with my advisor, Rena Repetti, and it ended getting published in Personality and Social Psychology Bulletin, a pretty good but not Nature-caliber journal. (At the time, getting a paper published in JPSP, the Journal of Personality and Social Psychology, was a big calling card for landing an academic job. I knew this paper wasn’t going to get into JPSP, but I figured that at least I could get it into a journal with most of the same words in the title so that people who scanned my CV quickly would be impressed).
I consider this to be the weakest study from my dissertation (one of our other studies, which I’m much prouder of, actually did make it into JPSP). It had a small sample, we used a weird method in which we basically made up our own variables, and we were using a brief snippet of naturalistic data—the home tours—and extrapolating wildly from it. Around the time we published this paper in 2010, the replication crisis was starting to bubble up in psychology. In 2015, the Open Science Collaboration released a large-scale replication attempt of 100 psychology studies, finding that many of them did not hold up. That created a real reckoning for the field and many researchers realized that our previously wild west approach to research—measure a lot of different things and then see what predicts what—creates a real risk of capitalizing on chance. Replicability concerns have made me even more wary of our study. I think we’d need to try our approach again with a bigger sample to interpret the findings with real confidence.
Here’s the irony, though: this study has gotten more attention from journalists than any other research I’ve published in my career. Every few months, I get a call from a reporter who is doing a story on spring cleaning and wants me to provide a quote on how clutter is bad for your health. It’s been covered in the New York Times, TIME Magazine, Vogue, Everyday Health, and lots of other outlets. That’s not reflection of the quality of our study, but of the paucity of research on household clutter. It’s an evergreen topic that magazines love to assign their writers to cover, but very few researchers have investigated. At first, I was thrilled to get inquiries from journalists; I love magazines, and loved getting quoted. But in the last few years, I’ve started to get frustrated with the direction of the questioning, which is often more about finding fun facts to prop up an existing thesis (decluttering is good for you!) rather than trying to translate research accurately. I’ve also gotten frustrated with the whole goal of stories like these, always trying to find the “one weird trick” that can fix your life. I think the clutter research is sticky because it highlights an individual rather than structural target of change. We can’t necessarily make our work lives less stressful or our home lives more equitable, but we can take some bags to the dump.
Plus, my study didn’t even show that clutter is bad for your health. It showed that subjective perceptions of clutter (and unfinished household projects) are linked with the daily rhythms of stress hormones. We didn’t objectively measure how our families’ homes looked, but rather how people talked about their homes in an open-ended way. That’s important, because it means that clutter is in the eye of the beholder. I can have an objectively messy house and feel fine about it. I can also have a spotless house and talk incessantly about how much work it needs. If I’m in the latter group, spring cleaning isn’t helpful.
That brings me back to AI. Over the years of having the clutter study featured in newspapers, blogs, and magazines, it’s taken on a telephone-game life of its own, getting translated in less and less accurate terms. The night before I discovered the AI summary of the study, I was floating in an inner tube at a pool party and talking about housework when someone said, “Oh yeah, wasn’t there a study that found clutter increases your stress hormones?” Like the pedantic nerd I am, I told her that in fact, clutter did not increase cortisol, but rather, clutter language may affect the diurnal slope of cortisol in a very small sample of WEIRD adults.
Most scientific journals are paywalled, and university library systems pay big bucks for subscriptions. So when AI trawls the internet to get details on research studies, it isn’t unlocking the actual studies. Instead, the AI summary of my research was actually an amalgam of articles about the study, many of which were a little off. Of the seven references linked at the end of the summary, one was a Psychology Today blog post by a couples therapist that described our study, one was to the New York Times feature about our study. Both were pretty accurate descriptions, but then things got weirder.
One link went to a Yahoo News feature that didn’t describe on our study directly, but rather reported on an Instagram post from an account called Girls about our study. The Instagram post said, “A UCLA study found that women living in cluttered homes had significantly higher levels of cortisol, the hormone linked to chronic stress. The more stuff visible in their space, the more overwhelmed they felt—and the longer those stress levels stayed elevated throughout the day.” Nope! We didn’t find “significantly higher levels of cortisol,” we didn’t focus on specifically women who were “living in cluttered homes,” we didn’t measure the “stuff visible in their space,” and we didn’t measure how “overwhelmed” they were. Another link pointed to a Facebook story by an account called The Female Quotient. The link had a graphic of a stressed-out looking Kate Winslet and a caption that read, “A UCLA study found women living in cluttered homes had higher cortisol levels, the hormone linked to chronic stress. The more mess around them, the more drained and overwhelmed they felt. And that stress stuck around. Men in the same homes? Not nearly as affected.” The Female Quotient story seemed like a slightly watered-down (and even less accurate) version of the Instagram post by Girls. Yet another link pointed to an Instagram reel that read, “Why does clutter spike your stress…but not his? The UCLA study every man needs to hear. HER cortisol rises when the house is messy. HIS stays the same. It’s not nagging. The study proves—it’s biology.” What!? This reel takes about 10 steps of extrapolation away from our original findings to essentialize about gender differences (and we never showed that “his cortisol stayed the same.” If we did, men would have flat diurnal slopes and we would have written about that!).
One of the weirdest links took me to a LinkedIn page that seemed to be an AI-written summary of our study by someone named Elena Sandru, a “Leadership Programs Engineer.” Here’s a sample sentence from the article: “In this article, I will explore compelling scientific evidence regarding clutter's impact on women's focus, productivity, and overall wellbeing, drawing from authoritative sources such as "No Place Like Home" by the UCLA Mindful Awareness Research Center and various medical reports.” “No Place Like Home” is, in fact, the title of my study, but it had nothing to do with the UCLA Mindful Awareness Research Center. The article goes on to say, “One astounding finding from the UCLA study is that a cluttered environment can lead to up to a 25% decrease in focus and productivity. When the mind is constantly grappling with visual stimuli, it's challenging to allocate mental resources effectively.” WHAT? We did not even look at focus and productivity, and where did the 25% figure come from? After a bunch of nonsense, the article then says, “The relationship between clutter and stress is well-documented…The "No Place Like Home" study from UCLA highlights that clutter can elevate cortisol levels, the body's primary stress hormone, by up to 20%.” WHAAAAAT!? In what universe did we claim that clutter highlights cortisol by up to 20%? We used some pretty fancy statistical models in our study, but that number comes out of thin air.
In short, digging into the sources for the AI investigation of whether clutter bothers women took me into a funhouse mirror of information, with claims that did not come directly from our actual study but rather from social media posts and reels that themselves were based on inaccurate summaries of our study, and then whole AI-generated articles that seemed to be themselves created from the social media posts and reels, with an extra soupcon of made-up statistics layered on top. It was slop upon slop upon slop, a house of cards made out of slop with an actual research study blinking somewhere way out in the distance.
Of course, if I wasn’t the actual investigator who recognized details from my own study, I would not have dug so deep into these sources. If I were a harried student trying to hastily assemble a research paper or a journalist writing up an quickie advice story, I would have taken it at face value—clutter is bad for women’s health!—and moved on. I was already wary of asking AI to help generate sources, after seeing how bowdlerized the results can be, but now I wonder whether even asking it common-sense questions can also lead people astray. It’s only as good as its sources, and if its sources are themselves inaccurate AI summaries of inaccurate social media clickbait, it’s not good for very much at all (unless you’re asking it questions like, “What do fact-free Instagram reels say about Issue X?”).
There is one bright spot in all of this. When I posted in Notes about this experience, one commenter got curious and tried asking the same question with a more sophisticated approach, the SIFT Toolbox, which was developed to more skeptically interpret research. The commenter prompted Claude.ai to specifically look at my research (so some pre-existing knowledge would be needed to most effectively use this tool), and the resulting summary was actually pretty accurate. It got even more accurate after the commenter shared the published study with Claude.ai, suggesting that even a smarter and more skeptical AI isn’t necessarily looking at the actual research it’s interpreting.
I would like to think that AIs that are used to explain the world will move more in the direction of approaches like SIFT, which evaluates claims cautiously. But based on my own pre-AI experience of having my study get misinterpreted over the years, I’m not sure the incentives line up in that direction. In an attention economy, people want fast, easy clicks, not carefully vetted knowledge. We should all be worried about that. And for the record—as I finish writing this post in my heavily cluttered workspace—clutter probably doesn’t affect your health all that much.
*Speaking of disgust, it’s interesting to think about how our hygiene standards have changed over the centuries. Back when people bathed once a week and refrigerators did not yet exist, the average household was dramatically more disgusting than it is now. I often think that if I took a time machine to 1600s England, I would immediately faint from the stench. The Ren Faire folks are rarely going for veracity on the smells, or the PortaPotty line would be even more noxious.
MUSIC RECOMMENDATION OF THE WEEK
Earlier recs: Broncho, Alvvays, Capitol Years
Emmett Kelly/The Cairo Gang/Hard Quartet
I was watching a romcom on the plane last year when I heard an absolutely perfect pop song playing over the end credits. I thought it was a British invasion outtake I’d never heard before, maybe a forgotten Who B-side. I couldn’t Shazam it because I was on a plane without wifi, so I tried to remember as many lyrics as I could and then googled it after landing. Turns out it was a 2015 release by a band called The Cairo Gang, named after a 1920s group of British spies. The production is timeless—it could be from any era between the 1960s and the present—and it’s two minutes of perfection.
The internet told me that the Cairo Gang is led by Emmett Kelly, a guitar-playing wordsmith who seems to be well-connected. He played in the band The Double with the drumstick-spinning Jim White of The Dirty Three, and I found a video of him jamming with Ty Segall on the drums.
Kelley is so well-connected that he landed in the ultimate supergroup for ‘90s indie rock fans, the Hard Quartet, which also includes the aforementioned Jim White, Steven Malkmus from Pavement, and Matt Sweeney from Chavez. They’re like the Traveling Wilburys for Gen X scenesters. Here’s my current favorite of their songs, and it happens to have Emmett Kelly singing lead. You know you’re a guitar hero when you relegate Malkmus to the bass:
Excellent article.
I have a bugbear about Substack notes where someone claims “research shows x” without citing a study. They are far more popular than they should be.
Clutter stress might be linked to personality types. Those women and men with OCD tendencies would have more stress living in a clutter filled home, whereas "creative" types would only be bothered by the mess if they can't find items like keys. No control for this difference in any of the studies. As well, people with significant depression lack the energy to clean and may find the mess even more depressing as it is just another failure.