Detecting Online Hate Speech Using Context Aware Models
Extracted source text · arXiv 1710.07395
\documentclass[11pt,a4paper]{article}
\usepackage[hyperref]{acl2017}
%\usepackage[draft]{hyperref}
\usepackage{times}
\usepackage{latexsym}
\usepackage{graphicx}
\usepackage{url}
\aclfinalcopy % Uncomment this line for the final submission
%\def\aclpaperid{***} % Enter the acl Paper ID here
%\setlength\titlebox{5cm}
% You can expand the titlebox if you need extra space
% to show all the authors. Please do not make the titlebox
% smaller than 5cm (the original size); we will check this
% in the camera-ready version and ask you to change it back.
\newcommand\BibTeX{B{\sc ib}\TeX}
\title{Detecting Online Hate Speech Using Context Aware Models}
\author{Lei Gao \\
Texas A\&M University \\
{\tt sjtuprog@tamu.edu} \\\And
Ruihong Huang \\
Texas A\&M University \\
{\tt huangrh@cse.tamu.edu} \\}
\date{}
\begin{document}
\maketitle
\begin{abstract}
In the wake of a polarizing election, the cyber world is laden with hate speech. Context
accompanying a hate speech text is useful for identifying hate speech, which however
has been largely overlooked in existing datasets and hate speech detection models. In this paper, we provide an annotated corpus of hate speech
with context information well kept. Then we propose two types of hate speech detection models that incorporate context information, a logistic regression model with context features and a neural network model with learning components for context. Our evaluation shows that both models outperform a strong baseline by around 3\% to 4\% in F1 score and combining these two models further improve the performance by another 7\% in F1 score.
\end{abstract}
\section{Introduction}
Following a turbulent election season, 2016's cyber world is awash with hate speech. Automatic detection of hate speech has become an urgent need since human supervision is unable to deal with large quantities of emerging texts.
Context information, by our definition, is the text, symbols or any other kind of information related to the original text.
%For instance, the contextual information of a tweet can be tweets it replied to, its hashtags or even the trending topic when the tweet was posted. As to an online news comment, the contextual information can be the other comments in the same discussion thread, the news article the comment was written for, etc.
%Despite the fact that current hate speech detection systems can achieve good performance on their dataset \cite{chen2012detecting, burnap2014hate,hosseinmardi2015detection, waseem2016hateful,nobata2016abusive},
While intuitively, context accompanying hate speech is useful for detecting hate speech,
context information of hate speech has been overlooked in existing datasets and automatic detection models.
Online hate speech tends to be subtle and creative, which makes context especially important for automatic hate speech detection. For instance,
\vspace{0.05in}
\noindent (1) {\it barryswallows: Merkel would never say NO}
\vspace{0.05in}
This comment is posted for the News titled by "German lawmakers approve 'no means no' rape law after Cologne assaults". With context, it becomes clear that this comment is a vicious insult towards female politician.
%For another example, Here ``say No'' means saying no to those attackers.
%\vspace{0.05in}
%\noindent (2){\it ozart: @endpcinamerica What do you mean evolved? They stillare primates....}
%\vspace{0.05in}
%The title of the news article that this comment was written for is ``Fury as feminist blames toddler alligator death on white entitlement''. The discussion of the thread containing this comment is concentrated on ``feminists''. By looking at the context, we understand that this user uses ``they'' in lieu of ``feminists'' and describe them as ``primates'', which makes the comment hateful.
However, almost all the publicly available hate speech annotated datasets do not contain context information.\citet{waseem2016hateful,waseem2016you,wulczyn2016ex,ross2017measuring}.
We have created a new dataset consisting of 1528 Fox News user comments, which were taken from 10 complete discussion threads for 10 widely read Fox News articles.
%in Fox News website.
It is different from previous datasets from the following two perspectives.
First, it preserves rich context information for each comment, including its user screen name, all comments in the same thread and the news article the comment is written for.
%are in this corpus.
Second, there is no biased data selection
%when annotating.
and all comments in each news comment thread were annotated.
%The comments were ordered temporal order between these comments are kept as origin.
%Second, the dataset is full of implicit hate speech, which %makes hateful comments in the dataset
%are difficult to recognize without knowing %their actual meaning.
%requires understanding the compositional meanings of the full comment.
%Third, the dataset is full of implicit and
%A baseline model using character level n-gram features can only achieve F1 score of around 50\%.
%to previous corpus.
%Here we give 2 examples of implicit hate speech in this corpus.
%The following is one example of implicit hate speech comments. It shows creativeness in creating hate speech by using ``abortion'' as a way of insult.
%\vspace{0.05in}
%\noindent (2) {\it VivianLee: @fatboyhd2k I think the parents had a failed abortion--only it's brain was aborted--the body stayed. That's why it's the way it is.}
%\vspace{0.05in}
%\vspace{0.05in}
%\noindent (4){\it Wyerd: American feminists are about as useful as soundless Velcro. Grew up with daddy issues no doubt.}
%\vspace{0.05in}
%This comment uses metaphor to degrade feminists by describing them as ``soundless Velcro''.
%After preparing corpus, we start to think about the suitable model for hate speech detection with contextual information. However, current models of hate speech detection does not take contextual information into account.
In this paper, we explored two types of models, feature-based logistic regression models and neural network models, in order to incorporate context information in automatic hate speech detection. First,
%
%\cite{burnap2014hate,hosseinmardi2015detection,waseem2016hateful,wulczyn2016ex} implemented logistic regression model with character-level n-grams or word level n-grams as features. \cite{nobata2016abusive} built a very complicated logistic regression model using all kinds of features including n-gram features, linguistic features, syntactic features and comment embedding features.
%
%Logistic regression model seems to be well studied. However, neural network model, despite its popularity in all kinds of NLP tasks and its good performance in text classification \cite{yang2016hierarchical}, still needs to be explored in hate speech detection. Compared to logistic regression model, neural network model has the potential to capture the compositional meaning of a document.
%
%
logistic regression models have been used in several prior hate speech detection studies \citet{chen2012detecting,burnap2014hate,van2015detection,hosseinmardi2015detection,burnap2015cyber,waseem2016hateful,wulczyn2016ex,nobata2016abusive} and various features have been tried including character-level and word-level n-gram features, syntactic features, linguistic features, and comment embedding features. However, all the features were derived from the to-be-classified text itself. In contrast, we experiment with logistic regression models using features extracted from context text as well.
Second, neural network models \citet{zhang2015character,tang2015document,yang2016hierarchical} have the potential to capture compositional meanings of text, but they have not been well explored for online hate speech detection until recently \citet{pavlopoulos2017deep}.
%We experiment with varied Long Short-term Memory neural nets (LSTMs) \cite{} and
%Specifi
We experiment with neural net models containing separate learning components that model compositional meanings of context information.
%Thereby we implemented two kinds of models, a logistic regression model and a neural network model.
Furthermore, recognizing unique strengths of each type of models, we build ensemble models of the two types of models.
Evaluation shows that context-aware logistic regression models and neural net models outperform their counterparts that are blind with context information.
Especially, the final ensemble models outperform a strong baseline system by around 10\% in F1-score.
%We incorporated different strategies in these models to utilize contextual information. Specifically, we extract features from context in logistic regression model and we add different neural network components in our neural network model. The evaluations of these two models shows that they both improves 4\% F1 score compared to the baseline system using characer level n-gram features.
%Besides, we further examined the ensemble models of logistic regression model and neural network model and find that the ensemble model improves 10\% F1 score compared to baseline, which shows that our logistic regression model and neural network model are working differently in detection hate speech.
\section{Related Works}
Recently, a few datasets with human labeled hate speech have been created, however, most of existing datasets do not contain context information. Due to the sparsity of hate speech in everyday posts, researchers tend to sample candidates from bootstrapping instead of random sampling, in order to increase the chance of seeing hate speech.
Therefore, the collected data instances are likely to be from distinct contexts.
%in a biased manner
%because hate speech is rare in randomly sampled texts.
%due to sparsity of hate speech in randomly sampled texts.
%data collection usually relies on biased manual selection owing to sparsity of hate speech.
For instance, in the Primary Data Set described in \citet{djuric2015hate} and later used by \citet{nobata2016abusive}, 10\% of the dataset is randomly selected while the remaining consists of comments tagged by users and editors.
%Later on, \cite{sood2012profanity,djuric2015hate,nobata2016abusive} have worked on corpus collected on Yahoo. Specifically, the Primary Data Set is composed of 10\% randomly selected comments and other "reported" comments from users and editors. It is unknown whether contextual information is kept within this corpus.
%
%\cite{kwok2013locate, burnap2014hate, waseem2016hateful} worked on hateful tweets.
\citet{kwok2013locate} built a balanced data set of 24.5k tweets by selecting from Twitter accounts that claimed to be racist or were deemed racist using their followed news sources. \citet{burnap2014hate} collected hateful tweets related to the murder of Drummer Lee Rigby in 2013. \citet{waseem2016hateful} provided a corpus of 16k annotated tweets in which 3.3k are labeled as sexist and 1.9k are labeled as racist. They created this corpus by bootstrapping from certain key words ,specific hashtags and certain prolific users.
\citet{warner2012detecting} created a dataset of $9000$ human labeled paragraphs
%from certain selected URLs. They collected these paragraphs by
that were collected using regular expression matching in order to find hate speech targeting Judaism and Israel.
\citet{hosseinmardi2015detection} extracted data instances from instagram that were associated with certain user accounts.
%They first sampled a large amount of users, then used profanity words to filter and find users who have a high potential to post hate speech.
%The corpus is collected from those users.
\citet{wulczyn2016ex} presented a very large corpus containing over 115k Wikipedia comments that include around 37k randomly sampled comments and the remaining 78k comments were selected from Wikipedia blocked comments.
%which were blocked by other users
%is from random sampling and 78k is selected from comments which were blocked by other users.
%Contextual information is not included in this corpus.
%Current hate speech detection models can not make use of contextual information. Also, neural network model is only used as a generator of embedding features rather than as a classifier.
Most of existing hate speech detection models are feature-based and use features derived from the target text itself.
\citet{burnap2014hate} experimented with different classification methods including Bayesian Logistic Regression, Random Forest Decision Trees and SVMs, using features such as n-grams, reduced n-grams, dependency paths, and hateful terms.
%which improves n-gram baseline of 3\% compared to baseline n-gram SVM model.
\citet{waseem2016hateful} proposed a logistic regression model using character n-gram features.
%The evaluation shows using gender as feature gives an improvement of 0.04\% F1 score.
\citet{djuric2015hate} used the paragraph2vec for joint modeling of comments and words, then the generated embeddings were used as feature in a logistic regression model.
%They improved the AUC score of about 1\% compared to bag of words feature.
\citet{nobata2016abusive} experimented with various syntactic, linguistic and distributional semantic features including word length, sentence length, part of speech tags, and embedding features, in order to improve performance of logistic regression classifiers.
%and distributional semantics features. They also proposed a comment2vec model in which every comment is mapped into a unique vector in a matrix representing comments and every word is mapped into a matrix representing words. The comment vector and word vector are concatenated to predict the next word in a context. The authors combined those features and feed them to a logistic regression classifier.
%The evaluations on WWW2015 corpus shows that all features combined model successfully improved F1 score of 0.9\% and AUC score of 0.2\% compared to character level feature baseline model.
Recently, \citet{schmidt2017survey} surveyed current approaches for hate speech detection,
%and raised a concern on the lack of a benchmark data set for hate speech detection. More importantly,
which interestingly also called to attention on modeling context information for resolving difficult hate speech instances.
%Lastly, it is worth mentioning that \cite{schmidt2017survey} made a survey of hate speech detection using NLP technique. They well summarized current approaches for hate speech detection and raises concern on the lack of a benchmark data set for hate speech detection. More importantly, they also called for attention on contextual knowledge for some difficult instances of hate speech.
\section{The Fox News User Comments corpus}
\subsection{Corpus Overview}
The Fox News User Comments corpus consists of 1528 annotated comments (435 labeled as hateful) that were posted by 678 different users in 10 complete news discussion threads in the Fox News website. The 10 threads were manually selected and represent popular discussion threads during August 2016. All of the comments included in these 10 threads were annotated.
%There is no data selection when annotating.
The number of comments in each of the 10 threads is roughly equal. Rich context information was kept for each comment, including its user screen name, the comments and their nested structure and the original news article. The data corpus along with annotation guidelines is posted on github\footnote{https://github.com/sjtuprog/fox-news-comments}.
%On average, each comment we annotated has 27 words and 145 characters which is longer compared to the tweets corpus created by \cite{waseem2016hateful}, where each tweet has 16 words and 95 characters on average.
%The average number of words in the annotated comments is 27 and the average length of characters is 145.
%Compared to the tweets corpus created by \cite{waseem2016hateful}, where each tweet has 16 words and 95 characters on average, our annotated comments are relatively longer. Especially, 11\% of comments in our dataset have more than 50 words each.
%as well as source code
%will be made publicly available once this paper was published.
%The source code in this paper is also available on \footnote{https://github.com/sjtuprog/hateful-speech-detection}
\subsection{Annotation Guidelines}
Our annotation guidelines are similar to the guidelines used by \citet{nobata2016abusive}. We define
hateful speech to be the language which explicitly or implicitly threatens or demeans a person or a group based upon a facet of their identity such as gender, ethnicity, or sexual orientation. %Specifically, employs stereotypes, exemplifies xenophobia/hateful patriotism, makes a threat towards a specific person/group, bad screen names, using "They" in lieu of identifying group, minority identity implied through context, implies or states inferiority or “uncivillized” nature, generalizing about a group, “Othering”, characterizes groups as crazy or irrational, expresses fear of a power grab by women or minorities, dehumanization, infantilization can be all considered hateful.
The labeling of hateful speech in our corpus is binary. A comment will be labeled as hateful or non-hateful.
\subsection{Annotation Procedure}
We identified two native English speakers for annotating online user comments. The two annotators first discussed and practices before they started annotation.
%We first asked the two annotators to thoroughly discuss over the annotation guidelines and conduct practice annotations for several iterations.
%Then in order to measure inter-agreements, we asked to annotate comments from four discussion threads that contain 648 comments in total.
They achieved a surprisingly high Kappa score \citet{cohen1960coefficient} of 0.98 on 648 comments from 4 threads. %
We think that thorough discussions in the training stage is the key for achieving this high inter-agreement.
For those comments which annotators disagreed on, we label them as hateful as long as one annotator labeled them as hateful.
%After discussion over the annotation guidelines, two amateur who has good knowledge of linguistic and US politics participated in annotation individually.
%The two annotators reached a Kappa score \cite{cohen1960coefficient} of 0.98 on 648 comments in 4 news threads.
Then one annotator
%who was available later finished annotation of
continued to annotate the remaining 880 comments from the remaining six discussion threads.
%For those comments which annotators disagree on, we label them as hateful as long as one annotator labels it as hateful.
\subsection{Characteristics in Fox News User Comments corpus}
Hateful comments in
the Fox News User Comments Corpus is often subtle, creative and implicit. Therefore, context information is necessary in order to accurately identify such hate speech.
%contains a significantly high proportion of implicit hateful speech. The language used in this corpus is extremely diverse and creative.
%In this section,
%By careful observation,
%we present several unique properties of this dataset.
%in those implicit hate speech.
\subsubsection{Context Dependent Comments}
The hatefulness of many comments depended on understanding their contexts. For instance,
\vspace{0.05in}
\noindent (3) {\it mastersundholm: Just remember no trabjo no cervesa}
\vspace{0.05in}
This comment is posted for the news "States moving to restore work requirements for food stamp recipients". This comment implies that Latino immigrants abuse the usage of food stamp policy, which is clearly a stereotyping.
%\vspace{0.05in}
%\noindent (6){\it jmgandolfo: @SFgunrmn ALL of them are.}
%\vspace{0.05in}
%This comment is proceeded by the comment ``what a efen loon, but most femanazis are.''. Because the previous comment expresses hate against feminists and this comment supports it, therefore this comment is also hateful.
\subsubsection{Implicit and creative language}
Many hateful comments use implicit and subtle language, which contain no clear hate indicating word or phrase. In order to recognize such hard cases, we hypothesize that neural net models are more suitable by capturing overall composite meanings of a comment. For instance, the following comment is a typical implicit stereotyping against women.
\vspace{0.05in}
\noindent (4) {\it MarineAssassin: Hey Brianne - get in the kitchen and make me a samich. Chop Chop}
\vspace{0.05in}
%This comment is a typical implicit stereotyping against women. It describes women as householders and shows no respect.
%\vspace{0.05in}
%\noindent (8) {\it PostApocalypticHero: Expect more and more women to be asking .. "why are men no longer interested in me"! We're not going touch you until you pull our pants down!}
%\vspace{0.05in}
%This comment contains an implicit insult to women.
\subsubsection{Long Comments with Regional Focus of hatefulness}
11\% of our annotated comments have more than 50 words each. In such long comments, the hateful indicators usually appear in a small region of a comment while the majority of the comment is neutral. For example,
\vspace{0.05in}
\noindent (5) {\it TMmckay: I thought ...115 words... \textbf{Too many blacks winning, must be racist and needs affirmative action to make whites equally win!} }
\vspace{0.05in}
%In this comment, almost only the bold last sentence contain hate indicators.
\subsubsection{Disrespectful screen names}
Certain user screen names indicate hatefulness, which imply that comments posted by these users are likely to contain hate speech. In the following example, commie is a slur for communists.
\vspace{0.05in}
\noindent (6){\it nocommie11: Blah blah blah. Israel is the only civilized nation in the region to keep the unwashed masses at bay.}
\vspace{0.05in}
%This screen name indicates that this user is likely to post hateful comments.
%\vspace{0.05in}
%\noindent (11){\it LibtardTroller: Good. If I have to work for money and food then so should the leeches of society. Food stamps should really be eliminated totally. Then you'll see unemployment truly drop. If you have kids you can't afford, have no family or friends for help, and need the government to survive, you made poor choices and deserve to live with the consequences.}
%\vspace{0.05in}
%Libtard is a derogatory term for liberals. Troller is a self-identification of being malicious.
\section{Context-aware Online Hate Speech Detection Models}
%We explore both feature-based logistic regression models and neural net models in order to incorporate context information for hate speech detection.
\subsection{Logistic Regression Models}
In logistic regression models, we extract four types of features, word-level and character-level n-gram features as well as two types of lexicon derived features.
We extract these four types of features from the target comment first. Then we extract these features from two sources of context texts, specifically the title of the news article that the comment was posted for and the screen name of the user who posted the comment.
%c3 different kinds of content, which is the comment, the title of the news article which the comment posted for and the screen name of the user who posted the comment, i.e. username.
%Specifically, the usernames and news title are not encoded into user IDs or news title IDs. They are regarded as a content and while we are using them, we extract same set of features as the comment.
For logistic regression model implementation, we use l2 loss. We adopt the balanced class weight as described in Scikit learn\footnote{http://scikit-learn.org/stable/modules/generated/ \\
sklearn.linear\_model.LogisticRegression.html}.
Logistic regression model with character-level n-gram features is presented as a strong baseline for comparison since it was shown very effective. \cite{waseem2016hateful,nobata2016abusive}
\subsubsection{Word-level and Character-level N-gram Features}
For character level n-grams, we extract character level bigrams, tri-grams and four-grams. For word level n-grams, we extract unigrams and bigrams.
\subsubsection{LIWC Feature}
Linguistic Inquiry and Word Count, also called LIWC, has been proven useful for text analysis and classification \citet{pennebaker2001linguistic}. In the LIWC dictionary, each word is labeled with several semantic labels. In our experiment, we use the LIWC 2015 dictionary which contain 125 semantic categories.
%A text is encoded into a 125 dimension LIWC feature vector.
Each word is converted into a 125 dimension LIWC vector, one dimension per semantic category. The LIWC feature vector for a comment or its context is a 125 dimension vector as well, which is the sum of all its words' LIWC vectors.
\subsubsection{NRC Emotion Lexicon Feature}
NRC emotion lexicon contains a list of English words that were labeled with eight basic emotions (anger, fear, anticipation, trust, surprise, sadness, joy, and disgust) and sentiment polarities (negative and positive)\cite{mohammad2013nrc}. We use NRC emotion lexicon to capture emotion clues in text.
%A text is encoded into a 10 dimension NRC emotion vector.
Each word is converted into a 10 dimension emotion vector, corresponding to eight emotion types and two polarity labels. The emotion vector for a comment or its context is a 10 dimension vector as well, which is the sum of all its words' emotion vectors.
% emotion vector. NRC emotion vector is the sum of all its words' NRC emotion labels.
%\subsection{Neural Network Model}
%\begin{figure}[t]
% \centering
%\includegraphics[width=6.3cm,height=8cm,keepaspectratio]{nnmodel.jpeg}
%\caption{Our Neural Network model}
%\label{model}
%\end{figure}
%As shown in figure \ref{model},
\subsection{Neural Network Models}
Our neural network model mainly consists of three parallel LSTM \citet{hochreiter1997long} layers. It has three different inputs, including the target comment, its news title and its username.
Comment and news title are encoded into a sequence of word embeddings. We use pre-trained word embeddings in word2vec\footnote{https://code.google.com/archive/p/word2vec/}. Username is encoded into a sequence of characters. We use one-hot encoding of characters.
Comment is sent into a bi-directional LSTM with attention mechanism. \cite{bahdanau2014neural}. News title and username are sent into a bi-directional LSTM. Note that we did not apply attention mechanism to the neural network models for username and news title because both types of context are relatively short and attention mechanism tends to be useful when text input is long.
The three LSTM output layers are concatenated, then connected to a sigmoid layer, which outputs predictions.
%Note that the three inputs are not always necessary. In our experiments, we will also evaluate the model with some inputs disabled.
The number of hidden units in each LSTM used in our model is set to be 100. The recurrent dropout rate of LSTMs is set to 0.2. In addition, we use binary cross entropy as the loss function and a batch size
of 128. The neural network models are trained for 30 epochs.
\subsection{Ensemble Models}
%As described above, the logistic regression model and neural network model are performing differently in identifying hate speech.
To study the difference of logistic regression model and neural network model and potentially get performance improvement, we will build and evaluate ensemble models.
\section{Evaluation}
We evaluate our model by 10 fold cross validation using our newly created Fox News User Comments Corpus. Both types of models use the exact same 10 folds of training data and test data.
%Different evaluation metrics have been employed in prior studies of online hate speech detection For easy comparisons,
We report experimental results using multiple metrics, including accuracy, precision/recall/F1-score, and accuracy area under curve (AUC).
\subsection{Experimental Results}
\subsubsection{Logistic Regression Models}
\begin{table*}[h]
\begin{center}
\resizebox{1.8\columnwidth}{!}{
\begin{tabular}{|l|c|c|c|c|c|c|}
\hline \bf Features &\bf Input Contents&\bf Accuracy & \bf Precision & \bf Recall & \bf F1 & \bf AUC\\ \hline
char (baseline) & comment& 0.738 & 0.549 & 0.469 & 0.504 & 0.733\\
+word & comment & 0.735 & 0.548 & 0.443 & 0.488 & 0.736\\
%char+NRC & comment & 0.735 & 0.543 & 0.462 & 0.498 & 0.729\\
%char+LIWC & comment & 0.735 & 0.544 & 0.469 & 0.503 & 0.742 \\
+LIWC+NRC & comment & 0.732 & 0.533 & 0.465 & 0.495 & 0.740\\
%char+word+LIWC & comment & 0.742 & 0.559 & 0.467 & 0.508 & 0.747\\
%char+word+NRC & comment & 0.740 & 0.557 & 0.452 & 0.498 & 0.736\\
+word+LIWC+NRC & comment & \textbf{0.747} & \textbf{0.568} & \textbf{0.476} & \textbf{0.517} & \textbf{0.750}\\
\hline
%\multicolumn{7}{|c|}{Context-aware Logistic Regression Models} \\ \hline
& + username & 0.747 & \textbf{0.576} & 0.474 & 0.518 & 0.765\\
& + title & 0.745 & 0.558 & 0.496 & 0.523 & 0.761\\
& + title+ username {\bf(Best)} & \textbf{0.750} & 0.572 & \textbf{0.516} & \textbf{0.542} & \textbf{0.778}\\
\hline
\end{tabular}
}
\end{center}
\caption{\label{lr} Performance of Logistic Regression Models }
\end{table*}
\begin{table*}[ht]
\begin{center}
\resizebox{1.8\columnwidth}{!}{
\begin{tabular}{|l|c|c|c|c|c|c|}
\hline \bf Model &\bf Input Contents & \bf Accuracy & \bf Precision & \bf Recall & \bf F1 & \bf AUC\\ \hline
LSTM & comment & 0.726 & 0.524 & 0.398 & 0.450 & 0.678\\
bi-LSTM & comment & 0.720 & 0.513 & \textbf{0.440} & 0.473 & 0.682\\
bi-LSTM with attention & comment & \textbf{0.750} & \textbf{0.591} & 0.437 & \textbf{0.499} & \textbf{0.735} \\
\hline
& + username & 0.742 & 0.566 & 0.437 & 0.489 & 0.748\\
& + title {\bf (best)} & \textbf{0.766} & \textbf{0.614} & \textbf{0.499} & \textbf{0.548} & 0.760\\
& + title + username & 0.755 & 0.589 & 0.496 & 0.532 & \textbf{0.766}\\
\hline
\end{tabular}
}
\end{center}
\caption{\label{nn} Performance of Neural Network Models }
\end{table*}
Table \ref{lr} shows the performance of logistic regression models.
%In this experiment, to find the best possible features of logistic regression model, we first fix input contents to be the comment only. Then we select character n-gram as the base feature and keeps adding new features to searched for best combinations of features. After finding the best combination of features we then use this best feature extractor to apply on different contents to find the best combinations of contents.
The first section of table \ref{lr} shows the performance of logistic regression models using
%different
features extracted from a target comment only. The result shows that the logistic regression model was improved in every metric after adding both word-level n-gram features and lexicon derived features. However, the improvements are moderate.
%uses comment as the only input.
%The first row shows performance of a strong baseline,
%The logistic regression model using character-level n-gram features has been demonstrated as a strong baseline for online hate speech detection in prior research.\cite{waseem2016hateful,nobata2016abusive}
%The first row of table \ref{lr} shows the performance of the baseline model. The other rows in this section shows the baseline model with additional features. The fourth row shows that the logistic regression model was improved in every metric after adding both word-level n-gram features and lexicon derived features. However, the improvements are moderate.
%char stands for the baseline model using character level n-gram features. Its input content is the comment itself.
%+word
%The second row shows performance for the logistic regression model using both character level n-gram features and word level n-gram features.
%Compared to the baseline, the AUC score improved by 0.3\%.
%The third row shows that the logistic regression model obtained a small improvement in AUC score after adding two types of lexicon derived features on top of character-level and word-level n-gram features.
%+LIWC+NRC stands for the model model using character level n-gram features, LIWC and NRC features. Compared to baseline, the AUC score improved 0.7\%.
%The fourth row shows that the logistic regression model was improved using all the three metrics, accuracy, F1-score and AUC, after adding both word-level n-gram features and lexicon derived features.
%were improved after a +word+LIWC+NRC stands for the model using character level n-gram features, word level n-gram features, LIWC and NRC features. Compared to baseline, the AUC score improved 1.7\%.
%The first section shows that by using lexicon features, which are LIWC feature and NRC feature, improves the performance of logisitc regression model.
The second section shows the performance of logistic regression models using the four types of features extracted from both a target comment and its contexts%, which include its username and the title of the news article the comment was posted for.
%context while using same set of features.
%The first row shows the logistic regression model performance using features extracted from both a comment and its username.
%comment and username as contents.
%We can see that compared to the model which only uses features extracted from the comment, the AUC score improved by 1.5\%.
%The second row shows that features extracted from the news article title help to improve hate speech detection performance by 1.1\% in AUC score.
The result shows that the logistic regression model using features extracted from a comment and both types of context achieved the best performance and obtained improvements of 2.8\% and 2.5\% in AUC score and F1-score respectively.
%Compared to the model that only uses features extracted from the comment, the AUC score improved by 2.8\%. In addition, this model achieved the best F1-score as well.
%Compared to baseline, the AUC score improved 4.5\%.
%
%The second section shows that by
%Therefore, using context information clearly improves hate speech detection performance of logistic regression models.
%The improvement from more content is large than that of more features.
%In column Features, char stands for character level n-gram features. + stands for add features. word, LIWC, NRC stand for the corresponding features as described in section 4.1.
%In column Input Contents, comment stands for the comment itself excluding the user and mentioned user. Title stands for the news title to which the comment replies. Username stands for the screen name of that user.
\subsubsection{Neural Network Models}
Table \ref{nn} shows the performance of neural network models.
%Column model stands for the neural network model for comment input.
%As shown in figure \ref{model}, our neural network model has up to three inputs. It is flexible that we can disable some of the inputs and evaluate on different inputs. Similar to evaluations on logistic regression model, we first fix comment as the only input and evaluate on different models, then we look for the best combinations of inputs.
The first section of table \ref{nn} shows the performance of several neural network %variation
models that use comments as the only input. %Essentially, we removed the other two learning components in Figure \ref{model}, that use usernames and news article titles as input.
The model names are self-explanatory.
%The models LSTM, bi-LSTM and bi-LSTM with attention are short for LSTM neural network, bi-directional LSTM and bi-directional LSTM with attention mechanism respectively.
We can see that %bi-directional LSTM slightly improved the performance compared with the initial LSTM model while
the attention mechanism coupled with the bi-directional LSTM neural net greatly improved the online hate speech detection, by 5.7\% in AUC score.
%is for using bi-directional LSTM. Compared to LSTM, the AUC score improves 0.04\%. Row bi-LSTM with attention stands for bi-directional LSTM with attention mechanism. Comapred to LSTM, the AUC score improves 5.7\%.
The second section of table \ref{nn} shows performance of the best neural net model (bi-directional LSTM with attention) %mechanism
after adding additional learning components that take context as input. The results show that adding username and news title can both improve model performance. Using news title gives the best F1 score while using both news title and username gives the best AUC score.
%performance of models using different input contents.
%The first row shows that adding the username based learning component slightly improved the AUC score by 1.3\%, compared with the system using a target comment as input only.
%The second row shows that adding the news title based learning component clearly improved the AUC score by 3.4\% and achieved the best F1-score as well.
%Row + title stands for enabling news title as inputs. Compared to only use comment, the AUC score improves 2.5\%.
%The third row shows that adding both context based learning components improved the AUC score by 3.1\%.
%Row + title + username stands for enabling news title and username as inputs. Compared to only use comment, the AUC score improves 3.1\%.
%Therefore compared with the neural network model that only considers the content of a comment, using context information, especially the news article title a comment was posted for, clearly improved hate speech detection performance of neural net models.
\subsubsection{Ensemble Models}
\begin{table*}[ht]
\begin{center}
\resizebox{1.8\columnwidth}{!}{
\begin{tabular}{|l|c|c|c|c|c|}
\hline \bf Model & \bf Accuracy & \bf Precision & \bf Recall & \bf F1 & \bf AUC\\
\hline
Char (Baseline) & 0.738 & 0.549 & 0.469 & 0.504 & 0.733 \\
\hline
Best Neural Network Model & 0.766 & 0.614 & 0.499 & 0.548 & 0.760\\
Best Logistic Regression Model & 0.750 & 0.572 & 0.516 & 0.542 & 0.778\\
\hline
%Min Score Ensemble & 0.774 & 0.735 & 0.337 & 0.459 & 0.789 \\
Max Score Ensemble & 0.740 & 0.539 & \textbf{0.678} & \textbf{0.600} & 0.794 \\
Average Score Ensemble & \textbf{0.779} & \textbf{0.650} & 0.496 & 0.560 & \textbf{0.804} \\
\hline
\end{tabular}
}
\end{center}
\caption{\label{ensemble} Performance of Ensemble Models }
\end{table*}
Table \ref{ensemble} shows performance of ensemble models by combining prediction results of the best context-aware logistic regression model and the best context-aware neural network model.
%Here baseline means the logistic regression model with character level n-grams as features. Neural Network Model is the Row bi-LSTM with attention with comment and title as input contents as shown in table \ref{nn}. Logistic Regression Model is the model which uses all combined feature and all combined input contents as shown in table \ref{lr}.
We used two strategies in combining prediction results of two types of models.
Specifically, the Max Score Ensemble model made the final decisions based on %score of a comment is
the maximum of two scores assigned by the two separate models; instead, the Average Score Ensemble model %means
used the average score to make final decisions.
%, we use the maximum score of the two scores assigned by the two models for each comment in order to determine the prediction label. In Average Score Ensemble,
%of ensemble model.
% Min Score Ensemble, we pick the minimum score and for
%In addition, we use the average score of the two predicted scores in order to determine the prediction label for each instance. Specifically, if the average score for a comment is greater than 0.5, the comment is labeled as hateful, otherwise, the comment is labeled as non-hateful.
We can see that both ensemble models further improved hate speech detection performance compared with using one model only and achieved the best classification performance. Compared with the logistic regression baseline, the Max Score Ensemble model improved the recall by more than 20\% with a comparable precision and improved the F1 score by around 10\%, in addition, the Average Score Ensemble model improved the AUC score by around 7\%.
\section{Analysis}
\subsection{Logistic Regression Models}
As shown in table \ref{lr}, given comment as the only input content, the combination of character n-grams, word n-grams, LIWC feature and NRC feature achieves the best performance. It shows that in addition to character level features, adding more features can improve hate speech detection performance.
%Additionally, the LIWC and NRC features are lexicon based features, which implies that this kind of feature is a good fit for this task.
However, the improvement is limited. Compared with baseline model, the F1 score only improves 1.3\%.
In contrast, when context information was taken into account, the performance greatly improved. Specifically, after incorporating features extracted from the news title and username, the model performance was improved by around 4\% in both F1 score and AUC score. This shows that using additional context based features in logistic regression models is useful for hate speech detection.
\subsection{Neural Network Models}
As shown in table \ref{nn}, given comment as the only input content, the bi-directional LSTM model with attention mechanism achieves the best performance.
%From single directional LSTM to bi-directional LSTM, from no attention mechanism to with attention mechanism, the performance improves as the model become more complex.
%The reason why attention mechanism works is probably because of the regional concentration of hatefulness as mentioned in Section 3.4.4. %should be ref
Note that the attention mechanism significantly improves the hate speech detection performance of the bi-directional LSTM model. We hypothesize that this is because hate indicator phrases are often concentrated in a small region of a comment, which is especially the case for long comments.
%It is worth noting that there is a big difference in forward LSTM and backward LSTM. The forward LSTM has very poor performance. This is probably because in implicit hate speech, the indicating part usually lies near the end rather than the beginning.
%When combined with username and news title, the neural network model has different performance compared to logistic regression. When news title is in the inputs, accuracy, precision, recall, F1 and AUC scores they all improved. But when username is included in inputs, it can only improve AUC score.
\subsection{Ensemble Models}
\begin{figure}[t]
\centering
\includegraphics[width=4.6cm,height=6cm,keepaspectratio]{pie.jpeg}
%\caption{Distribution of hateful comments in Max Score Ensemble model}
\caption{System Prediction Results of Comments that were Annotated as Hateful}
\label{pie}
\end{figure}
As shown in table \ref{ensemble}, both ensemble models significantly improved hate speech detection performance.
%the Max Score Ensemble model of neural network and logistic regression greatly improves the performance of our model. Compared to baseline system, the recall improves over 20\%.
%The Average Ensemble Model achieves the best accuracy, precision and AUC score while the Max Ensemble Model achieves the best recall and F1 score.
%Since Max Score Ensemble can be regarded as a union of predictions from two models, we can go into details of actual contributions of each model when they are combined.
Figure \ref{pie} shows the system prediction results of comments that were labeled as hateful in the dataset.
%we studied the distribution of hateful comments in the Max Score Ensemble model which reveals why the recall has a great improvement. For each prediction of hateful comment, we label it by the model which successfully predicts it.
%In total, there are 435 hateful comments in the dataset, the max ensemble model recognized 294 of them correctly. Among the 294 true positive cases, 146 comments were labeled by both the neural net model and the logistic regression model while 71 and 77 comments were only recognized by the neural net model and the logistic regression model respectively.
It can be seen that the two models perform differently. We further examined predicted comments and find that both types of models have unique strengths in identifying certain types of hateful comments.
\subsubsection{Strengths of Logistic Regression Models}
The feature-based logistic regression models are capable of making good use of character-level n-gram features, which are powerful in identifying hateful comments that contains OOV words, capitalized words or misspelled words.
%By looking into those predicted samples, we find that the logistic regression is able to capture character level features, which greatly helps discover all capitalized rage comments and misspelled hateful content. Misspelled words, which is also OOV words, cannot be make good use of by neural net model.
We provide two examples from the hateful comments that were only labeled by the logistic regression model:
\vspace{0.05in}
\noindent (7){\it kmawhmf:FBLM.}
\vspace{0.05in}
Here FBLM means fuck Black Lives Matter. This hateful comment contains only character information which can exactly be made use of by our logistic regression model.
%\vspace{0.05in}
%\noindent (13){\it wulfeman007 @Spyderia BARRACK INSANE OBAMA HAS ALREADY HAD HIS NUTS CUT FOR HIS TRANSFORMATION AFTER HE LEAVES OFFICE}
%\vspace{0.05in}
%This comment conveys its hatefulness by using all capitalized words.
\vspace{0.05in}
\noindent (8){\it SFgunrmn: what a efen loon, but most femanazis are.}
\vspace{0.05in}
This comment deliberately misspelled feminazi for femanazis, which is a derogatory term for feminists. It shows that logistic regression model is capable in dealing with misspelling.
\subsubsection{Strengths of Neural Network Models}
The LSTM with attention mechanism are suitable for identifying specific small regions indicating hatefulness in long comments. In addition, the neural net models are powerful in capturing implicit hateful language as well. The following are two hateful comment examples that were only identified by the neural net model: %In the following example, The second one shows implicit hateful comments.
\vspace{0.05in}
\noindent (9){\it freedomscout: @LarJass \textbf{Many religions are poisonous to logic and truth, that much is true}...and human beings still remain fallen human beings even they are Redeemed by the Sacrifice of Jesus Christ. So there's that. But the fallacies of thinking cannot be limited or attributed to religion but to error inherent in human motivation, the motivation to utter self-centeredness as fallen sinful human beings. \textbf{Nearly all of the world's many religions are expressions of that utter sinful nature}...Christianity and Judaism being the sole exceptions.}
\vspace{0.05in}
This comment is expressing the stereotyping against religions which are not Christian or Judaism. The hatefulness is concentrated within the two bolded segments.
%, which is usually the first argument and the last argument that best summarizes major points in this long comment.
%\vspace{0.05in}
%\noindent (16){\it freedomscout: The ones harmed the most are the children who are taught in government schooling that "transgenderism" is a valid description of a true reality. First, the very fact that "transgenderism" is not a true reality in itself but a condition of mind makes the claim of validity false...thus being deceptive for the children, youth, adolescents and adults who are deceived by it. Deception about anything critically important in the learning, growth, self-understanding and self-appreciation, and view of the civil society is evil and must be refused and refuted. To confuse children and youth, especially on the issues of their own nature is inexcusable and abusive to them. Because the falsehoods regarding "transgenderism" are being foisted on the culture by intention and willing complicity, the evil effects are widespread and become repeated in social media of all kinds.The result for stability of family and civil society is degradation of both.}
%\vspace{0.05in}
%This comment is a typical case of long complex comment. It is composed of several arguments against transgenderism. Instead of expressing hate using emotional words or derogatory slurs, this kind of comments calmly argues against equality of sexual orientation, which is a hateful ideology.
\vspace{0.05in}
\noindent (10){\it mamahattheridge: blacks Love being victims.}
%\vspace{0.05in}
In this comment, the four words themselves are not hateful at all. But when combined together, it is clearly hateful against black people.
%\vspace{0.05in}
%\noindent (18){\it MothraSaveUs: There is no law that will civilize the uncivilized.}
%\vspace{0.05in}
%This comment is posted under the news of German Cologne assaults. It implies refugees to be "the uncivilized".
\section{Conclusion}
We demonstrated the importance of utilizing context information for online hate speech detection.
We first presented a corpus of hateful speech consisting of full threads of online discussion posts. %It aims to call for attention on contextual information which has long been neglected by researchers in hate speech study community.
In addition, we presented two types of models, feature-based logistic regression models and neural network models, in order to incorporate context information for improving hate speech detection performance.
%that can make use of context information as well as improving the performance.
Furthermore, we show that ensemble models leveraging strengths of both types of models achieve the best performance for automatic online hate speech detection.
%logistic regression model and neural network model achieved 10\% improvement of F1 score compared to baseline system of hateful speech detection.
\bibliography{acl2017}
\bibliographystyle{acl_natbib}
\end{document}
\begin{thebibliography}{}
\expandafter\ifx\csname natexlab\endcsname\relax\def\natexlab#1{#1}\fi
\bibitem[{Bahdanau et~al.(2014)Bahdanau, Cho, and Bengio}]{bahdanau2014neural}
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014.
\newblock Neural machine translation by jointly learning to align and
translate.
\newblock {\em arXiv preprint arXiv:1409.0473\/} .
\bibitem[{Burnap and Williams(2015)}]{burnap2015cyber}
Pete Burnap and Matthew~L Williams. 2015.
\newblock Cyber hate speech on twitter: An application of machine
classification and statistical modeling for policy and decision making.
\newblock {\em Policy \& Internet\/} 7(2):223--242.
\bibitem[{Burnap and Williams(2014)}]{burnap2014hate}
Peter Burnap and Matthew~Leighton Williams. 2014.
\newblock Hate speech, machine classification and statistical modelling of
information flows on twitter: Interpretation and communication for policy
decision making.
\newblock In {\em Proceedings of the Internet, Politics, and Policy
conference\/}.
\bibitem[{Chen et~al.(2012)Chen, Zhou, Zhu, and Xu}]{chen2012detecting}
Ying Chen, Yilu Zhou, Sencun Zhu, and Heng Xu. 2012.
\newblock Detecting offensive language in social media to protect adolescent
online safety.
\newblock In {\em Privacy, Security, Risk and Trust (PASSAT), 2012
International Conference on and 2012 International Confernece on Social
Computing (SocialCom)\/}. IEEE, pages 71--80.
\bibitem[{Cohen(1960)}]{cohen1960coefficient}
Jacob Cohen. 1960.
\newblock A coefficient of agreement for nominal scales.
\newblock {\em Educational and psychological measurement\/} 20(1):37--46.
\bibitem[{Djuric et~al.(2015)Djuric, Zhou, Morris, Grbovic, Radosavljevic, and
Bhamidipati}]{djuric2015hate}
Nemanja Djuric, Jing Zhou, Robin Morris, Mihajlo Grbovic, Vladan Radosavljevic,
and Narayan Bhamidipati. 2015.
\newblock Hate speech detection with comment embeddings.
\newblock In {\em Proceedings of the 24th International Conference on World
Wide Web\/}. ACM, pages 29--30.
\bibitem[{Hochreiter and Schmidhuber(1997)}]{hochreiter1997long}
Sepp Hochreiter and J{\"u}rgen Schmidhuber. 1997.
\newblock Long short-term memory.
\newblock {\em Neural computation\/} 9(8):1735--1780.
\bibitem[{Hosseinmardi et~al.(2015)Hosseinmardi, Mattson, Rafiq, Han, Lv, and
Mishra}]{hosseinmardi2015detection}
Homa Hosseinmardi, Sabrina~Arredondo Mattson, Rahat~Ibn Rafiq, Richard Han, Qin
Lv, and Shivakant Mishra. 2015.
\newblock Detection of cyberbullying incidents on the instagram social network.
\newblock {\em arXiv preprint arXiv:1503.03909\/} .
\bibitem[{Kwok and Wang(2013)}]{kwok2013locate}
Irene Kwok and Yuzhou Wang. 2013.
\newblock Locate the hate: Detecting tweets against blacks.
\newblock In {\em AAAI\/}.
\bibitem[{Mohammad and Turney(2013)}]{mohammad2013nrc}
Saif~M Mohammad and Peter~D Turney. 2013.
\newblock Nrc emotion lexicon.
\newblock Technical report, NRC Technical Report.
\bibitem[{Nobata et~al.(2016)Nobata, Tetreault, Thomas, Mehdad, and
Chang}]{nobata2016abusive}
Chikashi Nobata, Joel Tetreault, Achint Thomas, Yashar Mehdad, and Yi~Chang.
2016.
\newblock Abusive language detection in online user content.
\newblock In {\em Proceedings of the 25th International Conference on World
Wide Web\/}. International World Wide Web Conferences Steering Committee,
pages 145--153.
\bibitem[{Pavlopoulos et~al.(2017)Pavlopoulos, Malakasiotis, and
Androutsopoulos}]{pavlopoulos2017deep}
John Pavlopoulos, Prodromos Malakasiotis, and Ion Androutsopoulos. 2017.
\newblock Deep learning for user comment moderation.
\newblock {\em arXiv preprint arXiv:1705.09993\/} .
\bibitem[{Pennebaker et~al.(2001)Pennebaker, Francis, and
Booth}]{pennebaker2001linguistic}
James~W Pennebaker, Martha~E Francis, and Roger~J Booth. 2001.
\newblock Linguistic inquiry and word count: Liwc 2001.
\newblock {\em Mahway: Lawrence Erlbaum Associates\/} 71(2001):2001.
\bibitem[{Ross et~al.(2017)Ross, Rist, Carbonell, Cabrera, Kurowsky, and
Wojatzki}]{ross2017measuring}
Bj{\"o}rn Ross, Michael Rist, Guillermo Carbonell, Benjamin Cabrera, Nils
Kurowsky, and Michael Wojatzki. 2017.
\newblock Measuring the reliability of hate speech annotations: The case of the
european refugee crisis.
\newblock {\em arXiv preprint arXiv:1701.08118\/} .
\bibitem[{Schmidt and Wiegand(2017)}]{schmidt2017survey}
Anna Schmidt and Michael Wiegand. 2017.
\newblock A survey on hate speech detection using natural language processing.
\newblock {\em SocialNLP 2017\/} page~1.
\bibitem[{Tang et~al.(2015)Tang, Qin, and Liu}]{tang2015document}
Duyu Tang, Bing Qin, and Ting Liu. 2015.
\newblock Document modeling with gated recurrent neural network for sentiment
classification.
\newblock In {\em EMNLP\/}. pages 1422--1432.
\bibitem[{Van~Hee et~al.(2015)Van~Hee, Lefever, Verhoeven, Mennes, Desmet,
De~Pauw, Daelemans, and Hoste}]{van2015detection}
Cynthia Van~Hee, Els Lefever, Ben Verhoeven, Julie Mennes, Bart Desmet, Guy
De~Pauw, Walter Daelemans, and V{\'e}ronique Hoste. 2015.
\newblock Detection and fine-grained classification of cyberbullying events.
\newblock In {\em International Conference Recent Advances in Natural Language
Processing (RANLP)\/}. pages 672--680.
\bibitem[{Warner and Hirschberg(2012)}]{warner2012detecting}
William Warner and Julia Hirschberg. 2012.
\newblock Detecting hate speech on the world wide web.
\newblock In {\em Proceedings of the Second Workshop on Language in Social
Media\/}. Association for Computational Linguistics, pages 19--26.
\bibitem[{Waseem(2016)}]{waseem2016you}
Zeerak Waseem. 2016.
\newblock Are you a racist or am i seeing things? annotator influence on hate
speech detection on twitter.
\newblock In {\em Proceedings of the 1st Workshop on Natural Language
Processing and Computational Social Science\/}. pages 138--142.
\bibitem[{Waseem and Hovy(2016)}]{waseem2016hateful}
Zeerak Waseem and Dirk Hovy. 2016.
\newblock Hateful symbols or hateful people? predictive features for hate
speech detection on twitter.
\newblock In {\em Proceedings of NAACL-HLT\/}. pages 88--93.
\bibitem[{Wulczyn et~al.(2016)Wulczyn, Thain, and Dixon}]{wulczyn2016ex}
Ellery Wulczyn, Nithum Thain, and Lucas Dixon. 2016.
\newblock Ex machina: Personal attacks seen at scale.
\newblock {\em arXiv preprint arXiv:1610.08914\/} .
\bibitem[{Yang et~al.(2016)Yang, Yang, Dyer, He, Smola, and
Hovy}]{yang2016hierarchical}
Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy.
2016.
\newblock Hierarchical attention networks for document classification.
\newblock In {\em Proceedings of NAACL-HLT\/}. pages 1480--1489.
\bibitem[{Zhang et~al.(2015)Zhang, Zhao, and LeCun}]{zhang2015character}
Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015.
\newblock Character-level convolutional networks for text classification.
\newblock In {\em Advances in neural information processing systems\/}. pages
649--657.
\end{thebibliography}
\begin{thebibliography}{}
\expandafter\ifx\csname natexlab\endcsname\relax\def\natexlab#1{#1}\fi
\bibitem[{Bahdanau et~al.(2014)Bahdanau, Cho, and Bengio}]{bahdanau2014neural}
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014.
\newblock Neural machine translation by jointly learning to align and
translate.
\newblock {\em arXiv preprint arXiv:1409.0473\/} .
\bibitem[{Burnap and Williams(2015)}]{burnap2015cyber}
Pete Burnap and Matthew~L Williams. 2015.
\newblock Cyber hate speech on twitter: An application of machine
classification and statistical modeling for policy and decision making.
\newblock {\em Policy \& Internet\/} 7(2):223--242.
\bibitem[{Burnap and Williams(2014)}]{burnap2014hate}
Peter Burnap and Matthew~Leighton Williams. 2014.
\newblock Hate speech, machine classification and statistical modelling of
information flows on twitter: Interpretation and communication for policy
decision making.
\newblock In {\em Proceedings of the Internet, Politics, and Policy
conference\/}.
\bibitem[{Chen et~al.(2012)Chen, Zhou, Zhu, and Xu}]{chen2012detecting}
Ying Chen, Yilu Zhou, Sencun Zhu, and Heng Xu. 2012.
\newblock Detecting offensive language in social media to protect adolescent
online safety.
\newblock In {\em Privacy, Security, Risk and Trust (PASSAT), 2012
International Conference on and 2012 International Confernece on Social
Computing (SocialCom)\/}. IEEE, pages 71--80.
\bibitem[{Cohen(1960)}]{cohen1960coefficient}
Jacob Cohen. 1960.
\newblock A coefficient of agreement for nominal scales.
\newblock {\em Educational and psychological measurement\/} 20(1):37--46.
\bibitem[{Djuric et~al.(2015)Djuric, Zhou, Morris, Grbovic, Radosavljevic, and
Bhamidipati}]{djuric2015hate}
Nemanja Djuric, Jing Zhou, Robin Morris, Mihajlo Grbovic, Vladan Radosavljevic,
and Narayan Bhamidipati. 2015.
\newblock Hate speech detection with comment embeddings.
\newblock In {\em Proceedings of the 24th International Conference on World
Wide Web\/}. ACM, pages 29--30.
\bibitem[{Hochreiter and Schmidhuber(1997)}]{hochreiter1997long}
Sepp Hochreiter and J{\"u}rgen Schmidhuber. 1997.
\newblock Long short-term memory.
\newblock {\em Neural computation\/} 9(8):1735--1780.
\bibitem[{Hosseinmardi et~al.(2015)Hosseinmardi, Mattson, Rafiq, Han, Lv, and
Mishra}]{hosseinmardi2015detection}
Homa Hosseinmardi, Sabrina~Arredondo Mattson, Rahat~Ibn Rafiq, Richard Han, Qin
Lv, and Shivakant Mishra. 2015.
\newblock Detection of cyberbullying incidents on the instagram social network.
\newblock {\em arXiv preprint arXiv:1503.03909\/} .
\bibitem[{Kwok and Wang(2013)}]{kwok2013locate}
Irene Kwok and Yuzhou Wang. 2013.
\newblock Locate the hate: Detecting tweets against blacks.
\newblock In {\em AAAI\/}.
\bibitem[{Mohammad and Turney(2013)}]{mohammad2013nrc}
Saif~M Mohammad and Peter~D Turney. 2013.
\newblock Nrc emotion lexicon.
\newblock Technical report, NRC Technical Report.
\bibitem[{Nobata et~al.(2016)Nobata, Tetreault, Thomas, Mehdad, and
Chang}]{nobata2016abusive}
Chikashi Nobata, Joel Tetreault, Achint Thomas, Yashar Mehdad, and Yi~Chang.
2016.
\newblock Abusive language detection in online user content.
\newblock In {\em Proceedings of the 25th International Conference on World
Wide Web\/}. International World Wide Web Conferences Steering Committee,
pages 145--153.
\bibitem[{Pavlopoulos et~al.(2017)Pavlopoulos, Malakasiotis, and
Androutsopoulos}]{pavlopoulos2017deep}
John Pavlopoulos, Prodromos Malakasiotis, and Ion Androutsopoulos. 2017.
\newblock Deep learning for user comment moderation.
\newblock {\em arXiv preprint arXiv:1705.09993\/} .
\bibitem[{Pennebaker et~al.(2001)Pennebaker, Francis, and
Booth}]{pennebaker2001linguistic}
James~W Pennebaker, Martha~E Francis, and Roger~J Booth. 2001.
\newblock Linguistic inquiry and word count: Liwc 2001.
\newblock {\em Mahway: Lawrence Erlbaum Associates\/} 71(2001):2001.
\bibitem[{Ross et~al.(2017)Ross, Rist, Carbonell, Cabrera, Kurowsky, and
Wojatzki}]{ross2017measuring}
Bj{\"o}rn Ross, Michael Rist, Guillermo Carbonell, Benjamin Cabrera, Nils
Kurowsky, and Michael Wojatzki. 2017.
\newblock Measuring the reliability of hate speech annotations: The case of the
european refugee crisis.
\newblock {\em arXiv preprint arXiv:1701.08118\/} .
\bibitem[{Schmidt and Wiegand(2017)}]{schmidt2017survey}
Anna Schmidt and Michael Wiegand. 2017.
\newblock A survey on hate speech detection using natural language processing.
\newblock {\em SocialNLP 2017\/} page~1.
\bibitem[{Tang et~al.(2015)Tang, Qin, and Liu}]{tang2015document}
Duyu Tang, Bing Qin, and Ting Liu. 2015.
\newblock Document modeling with gated recurrent neural network for sentiment
classification.
\newblock In {\em EMNLP\/}. pages 1422--1432.
\bibitem[{Van~Hee et~al.(2015)Van~Hee, Lefever, Verhoeven, Mennes, Desmet,
De~Pauw, Daelemans, and Hoste}]{van2015detection}
Cynthia Van~Hee, Els Lefever, Ben Verhoeven, Julie Mennes, Bart Desmet, Guy
De~Pauw, Walter Daelemans, and V{\'e}ronique Hoste. 2015.
\newblock Detection and fine-grained classification of cyberbullying events.
\newblock In {\em International Conference Recent Advances in Natural Language
Processing (RANLP)\/}. pages 672--680.
\bibitem[{Warner and Hirschberg(2012)}]{warner2012detecting}
William Warner and Julia Hirschberg. 2012.
\newblock Detecting hate speech on the world wide web.
\newblock In {\em Proceedings of the Second Workshop on Language in Social
Media\/}. Association for Computational Linguistics, pages 19--26.
\bibitem[{Waseem(2016)}]{waseem2016you}
Zeerak Waseem. 2016.
\newblock Are you a racist or am i seeing things? annotator influence on hate
speech detection on twitter.
\newblock In {\em Proceedings of the 1st Workshop on Natural Language
Processing and Computational Social Science\/}. pages 138--142.
\bibitem[{Waseem and Hovy(2016)}]{waseem2016hateful}
Zeerak Waseem and Dirk Hovy. 2016.
\newblock Hateful symbols or hateful people? predictive features for hate
speech detection on twitter.
\newblock In {\em Proceedings of NAACL-HLT\/}. pages 88--93.
\bibitem[{Wulczyn et~al.(2016)Wulczyn, Thain, and Dixon}]{wulczyn2016ex}
Ellery Wulczyn, Nithum Thain, and Lucas Dixon. 2016.
\newblock Ex machina: Personal attacks seen at scale.
\newblock {\em arXiv preprint arXiv:1610.08914\/} .
\bibitem[{Yang et~al.(2016)Yang, Yang, Dyer, He, Smola, and
Hovy}]{yang2016hierarchical}
Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy.
2016.
\newblock Hierarchical attention networks for document classification.
\newblock In {\em Proceedings of NAACL-HLT\/}. pages 1480--1489.
\bibitem[{Zhang et~al.(2015)Zhang, Zhao, and LeCun}]{zhang2015character}
Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015.
\newblock Character-level convolutional networks for text classification.
\newblock In {\em Advances in neural information processing systems\/}. pages
649--657.
\end{thebibliography}