Digital Digest: Algorithms for What to Read Next

When Rolling Stone journalist Randall Jennings published Untouchable: The Strange Life and Tragic Death of Michael Jackson with Grove Press last fall, he unwittingly unleashed the ire of a group of fans who, upset over the book’s handling of the deceased pop star’s private life, clamored on Facebook and Twitter for a decidedly new-media sort of revenge—single-star book reviews on Amazon. More remarkable was the response prompted by New York Times columnist David Streitfeld’s January 2013 story about the attacks: Within days, scores of five-star reviews had sprouted on Amazon as miffed users, apparently motivated more by opposition to the principles of the campaign than by any familiarity with the book itself, sought to counterbalance its effects. The result has been more than three hundred polarized reviews for a book that, according to Nielsen BookScan, has seen only modest sales.

What those hordes of mostly anonymous reviewers recognize is something that publishers have been struggling to come to terms with, particularly now that nearly a third of book-buying dollars are driven through Amazon (according to market research firm Bowker). While it’s always been commonplace for best-seller status to feed sales, that kind of crowd cachet is increasingly generated online through reviews and recommendations on literary and retail sites, and it has become a major factor in publishers’ obsession with “discovery”—how readers find new titles and decide what to buy next. At a panel on the topic during New York’s Digital Book World conference this past January, Matthew Baldacci, associate publisher of St. Martin’s Press, called for “more powerful book reviewers online,” saying that publishers are increasingly expecting amateur commentators to take over “the role that booksellers used to take” in spurring sales.

So important are online reviews—up to 70 percent of consumers trust them, according to a recent Nielsen study—that a niche industry has sprung up to churn out endorsements on demand. Through message boards, or on sites such as Amazon’s Mechanical Turk and the “small services” marketplace Fiverr, marketers can connect with reviewers-for-hire willing to wax enthusiastic about anything from books to hotel rooms. Estimates about the proportion of phony reviews to the overall total run as high as 30 percent, with Gartner research predicting that paid endorsements (deemed illegal by the Federal Trade Commission unless disclosed) will account for 10 to 15 percent of product feedback by 2014. Writers get in on the deception game too, using fake online personae to tout their own works—a practice known as “sock puppetry” that was condemned in a widely circulated petition last fall, initiated by an anonymous coalition of writers and signed by a host of authors including Michael Connelly and Laura Lippman. Amazon went on to conduct a late-year purge of several thousand book reviews, ostensibly to remove comments tainted by an “interest in the product,” such as reviews by authors’ friends and family members. 

Concern over the trustworthiness of online reviews has resulted in a number of proposals, with Jeff Bercovici of Forbes suggesting that Amazon adopt a moderated commentary system that foregrounds vetted remarks, and researchers at MIT developing a program to authenticate user identities across different online platforms. By analyzing the language used in large samples of both fake and genuine reviews, a team at Cornell University was able to develop an algorithm capable of detecting fraudulence 90 percent of the time. Once such review-parsing programs make it out of the lab and into the marketplace they’ll join the hive of algorithms already driving reader “discovery,” such as those used on Amazon, Netflix, Goodreads, and Facebook. Each platform has its own algorithm for data collection and recommendation, from simple search enhancements (other works by the same author or in the same genre) to customer clustering (where users are profiled and grouped into discrete categories). But the most prominent approach is what’s known as “collaborative filtering,” which generates recommendations by comparing users who show a similar record of choices (purchases and “likes,” for example). Amazon enjoys an edge in this department by running the comparison across items instead of just customers—an advantage it can leverage along with the data its users voluntarily supply about which books they already own, like, or want.

Incorporating that sort of reader feedback in a more nuanced way is part of the promise behind Bookish, launched this past February by publishing giants Simon & Schuster, Penguin, and Hachette. The site factors professional reviews and awards into its book recommendations, drawing titles from sixteen different publishers (self-published books are excluded), and plans to take into account the details of user responses. Meanwhile, with the release of BookScout in January, Random House is backing its own discovery engine: The Facebook app lets users share favorite titles and offers suggestions based on the preferences evinced in readers’ timelines—a data stream into which Bookish can also be plugged.

But Bookish doesn’t just recommend books—it also sells them. Alongside each title is an option for direct purchase from the usual lineup of online retailers (although the Kindle is absent so far from the list of supported e-book formats). And it’s this marriage of discovery and retail that may be the platform’s most salient feature. At Digital Book World, industry watchers spent as much energy fretting over how readers buy new books as over how they find them, with some suggesting that post-merger giant Random House Penguin could have the clout to challenge Amazon. At issue is whether publishers’ focus on discovery will beget new sales models or whether it will simply funnel more dollars through Amazon. 

Adrian Versteegh is a PhD candidate at New York University, where he teaches literature and writes about insomnia. He lives in Brooklyn, New York, and Berlin.