Of Love & Lasers: Perceptions of Narratives by AI Versus Human Authors

Here's the abstract:

Artificial Intelligence (AI) programs can produce narratives. However, readers’ preconceptions about AI may influence their response to these narratives, and furthermore, AI-generated writing may differ from human writing. Genre may also be relevant for readers’ attitudes regarding AI. This study tests the effects of actual AI versus human authorship, stated (labeled) authorship, and genre on perceptions of narratives and narrative engagement. Participants were randomly assigned within a 2 (actual author: human or AI) X 2 (stated author: human or AI) X 2 (genre: romance or science fiction) design, across two studies. In Study 1, actual AI narratives were perceived as more enjoyable, but human narratives were more appreciated. Furthermore, participants enjoyed actual AI-written sci-fi more than human-written sci-fi. Study 2 found that actual AI stories were rated more highly, particularly in appreciation, transportation, character identification, and future engagement. However, stated human authorship led to higher ratings for romance, but not for sci-fi. An interaction was observed such that for the sci-fi condition, stated human writing was perceived as more likely to be actually AI-written. Future research could expand upon these findings across more genres, as well as examining the determinants of preferences for stated human content.

---

I have to note that the samples of writing shown to readers were not whole novels, and I think that the difference between a short work and a long one might affect reader responses (I've often seen writers state that short story writing requires some different skills from novel-writing). This possibility is acknowledged by the authors of the paper:

[a] limitation of the studies is the length of the stories (600 and 1,000 words); it is possible that the effects might differ for longer narratives. This length was chosen to accommodate the token limit of the AI, as they could not generate longer stories with a single prompt. As AI algorithms improve, this study can be replicated with stories of greater length.

Some additional factors that might affect the extent to which the findings are applicable to all romance novels and their readers are:

• it is unclear how many of the readers were well-versed in the two genres. The authors of the study do state that "questions included participants’ general preferences for genres" but, they add, that "will not be discussed here." People who regularly read romance may respond to romance texts differently due to their greater awareness of the conventions of romance fiction or be primed in some way to respond differently to them than non-romance readers.
• I read the human-written texts which can be found at the repository and I'm not sure whether I'd classify all of the human-written "romance" as romance.  One of the pieces in Study 2 is a "800 - 1,000 word romance excerpt from a story about a rainy night chat under an umbrella" but it appears to be the protagonist remembering the chat they had with a partner who is now deceased, or at least, that's what I assumed when I read this part:

Iana didn’t smell like rubbing alcohol.

Not even when, later, I tried to clean the cuts on his face with gauze pads.

Not even when he was at the hospital, breathing through a tube.

But this was later.

Later, when I came home and I smelled Iana.

He wasn’t there.

I would never open the door again, see him cooking in the kitchen.

I would never fall asleep with my books, his hand on my back.

I would never find another example of his smell.

At least, I didn’t think so at the time.

And now, years later, I still don’t.

Another of the pieces in Study 2 is "an 800 - 1,000 word romance excerpt from a story about a first date at a cafe". The protagonists discuss shoplifting, one of them shoplifts in the cafe and then has to run away, so there is little romantic interaction. By contrast, all of the AI-generated romance texts promised a happily-ever-after for the protagonists.

This possibly ties in with the observation by the study's authors that

Examining the actual stories created by our AI and human authors, we anecdotally observe that the AI stories were more formulaic than human stories, drawing more on traditional tropes of romances or science fiction. The human-written stories in Study 2, created by individuals in a creative writing graduate program, were more unusual and potentially challenging for readers.

This suggests that a lack of adherence to genre conventions may have been an issue for readers if they were asked, or thought they were being asked, to assess the pieces as works of genre romance. In that context, rather than being "potentially challenging for readers," the readers may have understood the texts perfectly well and just felt that they failed, or didn't entirely work, as genre romance. However, I couldn't work out if the pieces were explicitly described in terms of genre when they were presented to the participants.

• Given that in Study 1 the human author was "a freelance writer with a creative writing degree" and in Study 2 the authors were "individuals in a creative writing graduate program" it seems unlikely that all of the human authors were experienced romance writers. This might mean that the results would differ if longer works by experienced romance authors were compared to text produced by AI. The authors of the study do acknowledge that "a limitation of Study 2 is that the graduate student authors may not have as much experience as professional writers, and so the study bears replicating with more experienced writers" but they do not mention whether or not that experience should be in writing in general or in one or both of the specific genres discussed.