Hello Neighbor: How a Local Newspaper Builds Community via Online Comments
By: Sameer Syed
Local newspapers have the important role of helping people know about and act on local issues. Most readers consume their local news online, but still expect news providers to show a genuine connection to the community, like reporters personally engaging with the area or showing an understanding of the community’s history.
The Southeast Missourian serves an active group of readers who care about what’s going on in the community — and they aren’t shy about sharing their opinions. Readers post tens of thousands of online comments each year and almost every reader (86%) discusses articles they read online with friends, family, and community members. This digital townhall builds a sense of community, encourages neighbors to get to know each other and find out how local affairs will affect their daily lives. To keep the comments free of toxicity and harassment, the Southeast Missourian spent the last year experimenting with Perspective API, Jigsaw’s machine learning-powered moderation tool, and updating its own moderation policy. Today the paper announced a revamped moderation and comment policies that includes a return to usernames and continued use of Perspective.
‘We were on the brink of shutting down comments’
Publisher Jon K. Rust was ready to turn off online commentary. “It was becoming too expensive and time consuming to effectively moderate content. And too often we had unacceptable delays between the time a user posted a comment that didn’t adhere to our community standards and when it was flagged by another user and then reviewed by a member of our moderation team.”
Reader comments following an update on Perspective’s performance
Jon and his team turned to Perspective to manage the flow and speed of the moderation process. Perspective works by giving a “score” (a number between 0.0 and 1.0) that indicates how confident the algorithm is that a comment is similar to toxic comments it has seen in the past. Using that score as a guide, Southeast Missourian moderators can make the final decision to remove or not post a comment.
In the first three months of the test, comments that needed moderator review after posting dropped in frequency by 59%, and the number of comments flagged as highly toxic (>.9) dropped to virtually zero (0.04%). Almost a year into the test and the positive results continue: user reported comments dropped by almost half compared to last year, toxic comments are edging towards zero (0.03%), and moderators are more efficient at identifying questionable content, removing approximately three times the number of comments compared to 2018. Readers are noticing the shift and more than one-third rated the commentary on the site as “civil” or “highly civil,” and almost all (91.8%) rated commentary on the site as about the same or more civil than other websites they use.
In addition to Perspective helping moderators, the Southeast Missourian also asked readers to gauge the perceived toxicity of their comments asking in real-time “are you sure your comment is civil?” The prompt convinced 7% of authors to revise a comment.
‘Requiring real names did not encourage civil discourse’
In April 2017, the Southeast Missourian forbade anonymous commenting, believing real names would decrease the number of toxic comments and help readers connect with each other. Using Perspective, they saw that the average toxicity of comments was identical between commenters using real names and usernames. The policy change also decreased the number of people commenting by more than half. Today the paper is bringing back usernames in addition to new moderation policies.
“What did dramatically improve the tone of discussion was [Perspective], combined with clearer messaging and quicker moderator follow-up. This revelation probably doesn’t surprise anyone…witnessing the coarsening of comments [on social media even where] identities are largely known,” Rust writes in his column announcing the change.
It’s been a year of learning and this is just the latest update from the team at the Southeast Missourian. They will continue to experiment with moderation and how best to engage readers because according to Jon, “a community positively and robustly engaged is a better place to live.”
Sameer Syed is a Partnerships lead at Jigsaw, where he builds strategic relationships to make sure everyone has access to impactful technology.
