Toxic Comments: Hosting better news discussions
– Als ich diesen Projektvorschlag für den Google DNI Fund eingereicht habe, war ich so zuversichtlich – und meinte: „Das ist wichtig, das kann gar nicht abgelehnt werden.“ Leider doch!
Project title: Toxic Comments: Hosting better news discussions
Brief overview:
The internet is broken and we have to fix it. Recent years have seen an alarming increase in "toxic" communication in comments sections all around.
People wishing to express themselves are leaving the conversation because of abusive, hateful or otherwise anti-social speech. Several prominent news sites have shut down their discussions forums entirely, seemingly having given up on thoughtful audience feedback altogether.
I'd like to develop a tool for publishers that helps like a spam filter so that the civil exchange of ideas and different opinions becomes possible yet again.
Detailed description:
Todays tone and behaviour in news discussions is reaching toxic levels, ultimately driving intelligent people away. This happens at a great loss for societal progress because the story and discussion around the fireplace is an age-old tradition of our species.
The assumption being tested with this prototype is that the careful application of the so-called "broken windows theory" supported by a bag of matching moderation tools could help restore our values in online conversation. According to the broken windows theory, "visible signs of crime, anti-social behavior and civil disorder create an urban environment that encourages further crime and disorder […]. The theory thus suggests that policing methods that target minor crimes […] help to create an atmosphere of order and lawfulness […]." (source: Wikipedia)
Carefully moderated comments sections offer rich debate and useful information for both journalists and readers. And who is not pleasently surprised when the comments below an article turn out to be even more useful than the original article? Even though content moderation poses substantial time investment on the publishers' side, it saves a multitude of this time on the consumers' side to sort through comment columns. Equipped with better tool support, moderators could keep these democratically so important discussion spaces alive and at a lower cost, but not so lively that they have to be turned off. As we know, the dosage makes the poison.
The prototype will do a job comparable to your spam filter and predict the probability that a user comment demonstrates abusive or in other regards offensive language. Flagged comments could then be sorted in ranked order according to "toxicity" indicators or trending topic to point moderators to the most controversial or fastest developing poisonous comment threads that deserve their attention first. Nobody would dispense with their email spam filter but when it comes to content moderation we could do with much more and better tool support as well.
The same functionality could be provided to readers and commenters on the news site. Readers could choose themselves via slider interface to filter out comments above a certain threshold of profanity, while commenters would get immediate feedback on the consequences of their writing.
Comments that would likely register at toxic levels would include those that contain hate speech, are off-topic, contain name-calling, are without substance or fit other offensive criteria for being sent to moderation. Perspective API (see below) currently recommends that no comment be automatically banned by the algorithm. Human judgement should be the basis for the final decision. The tool would guide the moderator by ranking the comments in order of attention needed, and the moderator would guide the tool by making the final call that the tool can learn from for scoring future comments.
What makes your project innovative?
This project will supply a prototype implementation for identifying, classifying and managing toxic comments.
Because I am not a content provider myself this project would be transformative to me if I could become a technology provider in the respected area. Through the web crawl corpus that my service rivva.de has archived over the last 11+ years I'm in the unique position to compare and contrast conversational tonality and behaviour online over long periods of time. My goal is to sample this corpus of comments from news sites, blogs, forums, Twitter and Facebook to create the initial dataset for experimentation. Ideally the community would submit and flag even more training examples. I could include the possibility to label comments and therefore create a valuable dataset directly on rivva.de. It would fit. The site's focus has always been about the aggregation of different perspectives on the news. This repository of data then could establish a baseline and playground to evaluate the ideas described and implicated.
The technology built for this project will mainly include several natural language processing and understanding tasks.
Since language is highly ambiguous, advanced natural language understanding will be most crucial to the problem. This project would help me familiarize myself more with state-of-the-art deep learning models like convolutional and long-short-term-memory neural networks.
Key deliverables will be a new dataset of manually labelled comments for the machine learning task of classifying user generated content into different bins of toxicity as well as an API for easy integration of this task into benefitting systems.
It's essential that we account for multiple degrees of toxicity and possibly overlapping classes. What types of user comments are appropriate on a given site is usually highly dependent on its content and intended audience.
Although a team led by Google (Jigsaw) is experimenting on the very same frontier, the German language poses some unique problems in text analysis and therefore bears overlapping efforts. Especially since Germany has spearheaded the effort by its new law against hate speech.
How will your Project support and stimulate innovation in digital news journalism? Why does it have an impact?
Better tool support for the task of content moderation allows to host better conversations online.
When comments sections can be managed more effectively and more efficiently, more stories can be opened up to comments in less time. When commenters can directly get feedback on the comment they are producing, tonality and behaviour online should improve, because they want their thoughts to be published, not rejected. When readers can themselves sort comments by their toxic level, better comments sections translate into more user engagement and therefore more advertising revenue. All in all, trust in news organizations can be regained.
But most importantly, discussion spaces are way too important to leave to the big social networks. They are one of the pillars of the internet and we should fight for (almost) everyone of them. They act as a medium for change. Just think of Apple's famous "Think different" campaign: "Because they change things. They push the human race forward."
Competition:
Perspective API by Jigsaw and Google's Counter Abuse Technology team is part of a collaborative research project called Conversation AI. Its alpha version is currently being used by Wikipedia, The New York Times, The Economist, The Guardian and the Coral Project. A Kaggle data science competition on the subject and sponsored by Jigsaw has just ended.
Communities that have been operating on manual moderation with quite a success and for quite some time, often assisted by karma points, up- and down-voting systems and community ground rules, include MetaFilter, Hacker News and Reddit.