A Reddit Dataset for the Russo-Ukrainian Conflict in 2022
Extracted source text · arXiv 2206.05107
\documentclass[conference]{IEEEtran}
\IEEEoverridecommandlockouts
% The preceding line is only needed to identify funding in the first footnote. If that is unneeded, please comment it out.
\usepackage{cite}
\usepackage{amsmath,amssymb,amsfonts}
\usepackage{algorithmic}
\usepackage{graphicx}
\usepackage{textcomp}
\usepackage{xcolor}
%##### imported packages
\usepackage{cite}
\usepackage{amsmath,amssymb,amsfonts}
\usepackage{algorithmic}
\usepackage{graphicx}
\usepackage{textcomp}
\usepackage{xcolor}
\usepackage{graphicx}
\usepackage{multirow}
\usepackage[normalem]{ulem}
\useunder{\uline}{\ul}{}
\usepackage{caption}
\usepackage{subcaption}
\usepackage{csquotes}
\usepackage{cleveref}
\usepackage{float}
\usepackage[hyphens]{url}
\usepackage{csquotes}
\newcommand{\todo}[1]{{\textcolor{red}{#1}}}
\newcommand{\james}[1]{\textcolor{black}{#1}}
\newcommand{\ehsan}[1]{{\textcolor{black}{#1}}}
\newcommand{\noteeh}[1]{{\textcolor{cyan}{#1}}}
\newcommand{\gareth}[1]{{\textcolor{blue}{#1}}}
\def\BibTeX{{\rm B\kern-.05em{\sc i\kern-.025em b}\kern-.08em
T\kern-.1667em\lower.7ex\hbox{E}\kern-.125emX}}
\begin{document}
\title{A Reddit Dataset for the Russo-Ukrainian Conflict in 2022}
\author{
\IEEEauthorblockN{
Yiming Zhu\IEEEauthorrefmark{1},
Ehsan-Ul Haq\IEEEauthorrefmark{1},
Lik-Hang Lee\IEEEauthorrefmark{2},
Gareth Tyson\IEEEauthorrefmark{1},
% TBD\IEEEauthorrefmark{1},
and Pan Hui\IEEEauthorrefmark{1}\IEEEauthorrefmark{3}}
\IEEEauthorblockA{\IEEEauthorrefmark{1}Hong Kong University of Science and Technology}
\IEEEauthorblockA{\IEEEauthorrefmark{2}Korea Advanced Institute of Science and Technology}
\IEEEauthorblockA{\IEEEauthorrefmark{3}University of Helsinki, Helsinki}
Email: \{yzhucd, euhaq\}@connect.ust.hk \quad likhang.lee@kaist.ac.kr
\quad \{gtyson, panhui\}@ust.hk
}
\maketitle
\begin{abstract}
Reddit consists of sub-communities that cover a focused topic. This paper provides a list of relevant subreddits for the ongoing Russo-Ukrainian crisis. We perform an exhaustive subreddit exploration using keyword search and shortlist 12 subreddits as potential candidates that contain nominal discourse related to the crisis. These subreddits contain over 300,000 posts and 8 million comments collectively. We provide an additional categorization of content into two categories, \enquote{R-U Conflict}, and \enquote{Military Related}, based on their primary focus. We further perform content characterization of those subreddits. The results show a surge of posts and comments soon after Russia launched the invasion. \enquote{Military Related} posts are more likely to receive more replies than \james{\enquote{R-U Conflict} posts}.
% \gareth{Think one of the enquotes is wrong :) I assume one is supposed to be R-U Conflict?}
Our textual analysis shows an apparent preference for the Pro-Ukraine stance in \enquote{R-U Conflict}, while \enquote{Military Related} retain a neutral stance.
\end{abstract}
\begin{IEEEkeywords}
social media, dataset, Reddit, the Russo-Ukrainian Conflict, characterization analysis
\end{IEEEkeywords}
\section{Introduction}
On 24 February 2022, Russia launched what they refer to as a \emph{special military operation} into Eastern Ukraine after their announcement to recognize the Donetsk and Luhansk people's republics.\footnote{\url{https://edition.cnn.com/2022/02/23/europe/russia-ukraine-putin-military-operation-donbas-intl-hnk/index.html}} This led to the advancement of Russian military forces, attracting global attention.
Various organizations and individuals worldwide have been sharing their opinions on the crisis.
Frequently, these expressions of diversified opinion carry support for either of the parties. However, such expressions are sometimes put forward with the objective of opinion engineering~\cite{haq2022weaponising}. Social media and online discussion forums can help researchers to explore essential research questions, including general opinion and sentiment on the topic, identifying and counteracting fake news, propaganda campaigns, as well as population movements.~\cite{kouloumpis2011twitter,shu2017fake,haq2020survey}
Such studies rely on social media datasets as a fundamental part of observing the questions mentioned above. Previously, researchers have collected and shared such datasets related to various social events, such as COVID-19 or Elections~\cite{zarei2020first,allcott2017social}. There are also already several datasets for the Russo-Ukrainian crisis from different social media, such as Twitter~\cite{haq2022twitter}, and Weibo~\cite{fung2022weibo}. Here, we present the first dataset from Reddit, a globally used discussion forum. We aim to present the key subreddits relevant to the crisis and describe the data contained within.
Unlike Twitter and Facebook, where people mostly submit short texts, Reddit users can submit longer posts containing more content. Furthermore, Reddit is themed around topic-specific communities called subreddits. Users can interact with posts by commenting and voting, making Reddit an ideal source to explore and study social information about crisis events \cite{bunting2021socially,wasike2011framing}. Therefore, to facilitate further related analysis, we present an open Reddit dataset relevant to the ongoing Russo-Ukrainian Conflict in 2022. Our dataset can support studies covering research areas such as social opinion analysis, politics characterization, and modeling of user relationships. We will keep collecting data from Reddit and updating this dataset.\footnote{\url{https://github.com/James-ZYM/RussiaUkraineConflict_Dataset}}
\section{Related Work}
\textbf{Online Social Networks datasets.}
Researchers across various areas utilize Online Social Networks (OSNs) to study social and computing-related research questions such as polarisation and sentiment analysis. Several researchers share such datasets. Some of them aim to purely collect complete live data on social media, like Pushshift \cite{baumgartner2020pushshift}: a large-scale Reddit dataset including more than 600 million posts and 5 billion comments since June 2005; Snap \cite{snap}: a Twitter dataset containing 467 million Twitter posts from 20 million users covering seven months from 1 June 2009.
In contrast, other OSNs datasets focus on specific topics or events, e.g., presidential elections \cite{elect1,elect2,allcott2017social}, Syrian War \cite{war1,war2}, and COVID-19 \cite{banda2021large,zarei2020first}. As for the Russo-Ukrainian Conflict, Cremisini released a dataset containing 4,538 news articles in English and performed bias detection for Pro-Ukraine and Pro-Russia stances \cite{cremisini2019challenging}.
Most closely related are a small set of social media datasets pertaining to the Russo-Ukrainian Conflict, including Twitter \cite{chen2022tweets, haq2022twitter, pohl2022twitter}, VKontakte \cite{park2022voynaslov}, and Weibo \cite{fung2022weibo}. Reddit differs from these platforms due to its unique niche-oriented communities. Such a structure allows us to organize data in a more topic-centric way, supporting studies like language analysis \cite{reddit1}, user modeling \cite{haq2022short}, and sentiment analysis \cite{reddit2}. To our knowledge, there are no other Reddit datasets covering the Russo-Ukrainian Conflict. We present our dataset here to address this gap.
\textbf{Data analysis on crisis events.} Various researchers have studied crisis communication on OSNs in recent years. Saroj and Pal presented a systematic survey on the effect of emergencies on social media \cite{saroj2020use}. Their work discusses how post data can help crisis management. Nazan performed a comparative sentiment analysis on tweets related to the Syrian refugee crisis, revealing correlations between tweets' sentiment and the language of content \cite{ozturk2018sentiment}.
For the COVID-19 crisis, Cinelli provided a comparative analysis of users’ activity on five different social media platforms and built a model to analyze information spread \cite{cinelli2020covid}. Regarding the ongoing Russo-Ukrainian Conflict, Hanley has analyzed Russian disinformation narratives utilizing semantic search \cite{hanley2022happenstance}. Such works depend on reliable and comprehensive datasets. In this work, we also perform a data characterization. We believe our results can serve as a reference for future studies in this area.
\section{Dataset}
\subsection{Reddit Data}
On Reddit, users can submit a post to start a new conversation on a specific topic by sharing text, links, or other multimedia. Accordingly, other users can comment or vote up/down the post. On Reddit, users subscribe to topic-specific communities called subreddits. Therefore, to collect data on a specific topic, several previous Reddit studies target particular subreddits \cite{weninger2013exploration,singer2014evolution}.
\subsection{Subreddit Selection}
\renewcommand{\arraystretch}{1.4}
\begin{table}[]
\centering
\resizebox{0.7\columnwidth}{!}{%
\begin{tabular}{|l|c|}
\hline
\textbf{Classes} & \textbf{Key Words} \\ \hline
\textbf{Country} & \textit{\begin{tabular}[c]{@{}c@{}}Ukraine, Ukrainian, Russia, \\ Russian, Russo-Ukrainian\end{tabular}} \\ \hline
\textbf{Activity} & \textit{war, conflict, combat, defense} \\ \hline
\textbf{Leader} & \textit{Putin, Zelenskyy} \\ \hline
\end{tabular}%
}
\caption{List of key words for searching subreddits.}
\label{tab:keywords}
\end{table}
\renewcommand{\arraystretch}{1}
We select relevant subreddits to collect posts and comments related to the conflict. As detailed in Table~\ref{tab:keywords}, we first create a list of keywords related to the Russo-Ukrainian Conflict.
We only consider English keywords because Reddit limits the names of subreddits to Latin characters, digits, and underscore.\footnote{\url{https://github.com/reddit-archive/reddit/blob/master/r2/r2/lib/validator/validator.py\#L512}}
In practice, we input each keyword into the search bar and record all the results in the \emph{Communities} section with at least 10,000 subscribers. In addition, we manually filter non-related subreddits based on their names and descriptions. We performed the search on 29 May 2022, discovering 67 subreddits potentially related to the Russo-Ukrainian Conflict.
%We note that we did not perform further exploration and update the results.
Note that any subreddits formed after 29 May 2022 are not included. Eventually, a subreddit is selected as a candidate once it belongs to any one of the following categories:
\begin{enumerate}
\item \textbf{R-U Conflict:} Subreddits directly focusing on the Russo-Ukrainian Conflict. These subreddits' names or community descriptions must include elements of the Russo-Ukrainian Conflict, e.g., Ukraine, Russia, and aliases for the Russo-Ukrainian Conflict. Posts and comments submitted on these subreddits focus on the crisis.
\item \textbf{Military Related topic:} Subreddits containing general discussions on defense and military-related content, while recent posts were relevant to the Russo-Ukrainian Conflict. We manually check the first 50 posts in \emph{Hot}, \emph{Top}, and \emph{Rising} order respectively. If there are more than 30\% of posts collectively related to the conflict, this subreddit is viewed as relevant to the Russo-Ukrainian Conflict and selected.
% \noteeh{how do you do this particular step?}
\end{enumerate}
Finally, we select 12 subreddits, as listed in Table~\ref{tab:seleted_subreddits}. We have also tried adding other potential topics that could generate new categories, e.g., politics and news. However, we find that the two categories above cover all the major potential subreddits. Other topics do not facilitate us getting any new related results.
\renewcommand{\arraystretch}{1.4}
\begin{table}[]
\resizebox{\columnwidth}{!}{%
\begin{tabular}{|l|l|}
\hline
\textbf{Category} & \textbf{Subreddit Name} \\ \hline
\multirow{4}{*}{\textbf{R-U Conflict}} & \textit{ukraine, ukraina, UkraineConflict} \\
& \textit{RussiaUkraineWar2022, UkrainianConflict} \\
& \textit{UkraineWarReports, UkraineInvasionVideos} \\
& \textit{UkraineWarVideoReport} \\ \hline
\multirow{2}{*}{\textbf{Military Related}} & \textit{war, CombatFootage, CredibleDefense} \\
& \textit{geopolitics} \\ \hline
\end{tabular}%
}
\caption{Selected subreddits for each category (Results on 29 May 2022).}
\label{tab:seleted_subreddits}
\end{table}
\renewcommand{\arraystretch}{1}
\subsection{Data Collection}
\ehsan{We utilize pushshift.io API\footnote{\url{https://github.com/pushshift/api}} for data collection.} We performed collection on 30 May 2022 and downloaded data from our 12 selected subreddits starting from 24 February 2022 to 29 May 2022. Additionally, to protect data integrity, we keep the data format consistent with the Pushshift dataset's \cite{baumgartner2020pushshift}, and preserve all columns in our final publicized dataset.
As a result, by 30 May,\footnote{30 May 2022 0:00:00 UTC} we have collected 306,482 posts and 8,751,705 comments from the selected subreddits. Figure~\ref{fig:cnt_cat} shows the distribution of daily data volume for each category. Around 95,349 items are contributed to all these subreddits each day.
\begin{figure}[tp]
\centering
\includegraphics[width=\columnwidth]{figs/count_cat.png}
\caption{Daily volume of posts and comments for each category; (a): R-U Conflict; (b): Military Related.}
\label{fig:cnt_cat}
\end{figure}
\section{Data Analysis}
\subsection{Descriptive Analysis}
This section depicts the metadata in each category.
% The analysis results reflect a general situation of subscribers' activity in different subreddits.
Table~\ref{tab:overall_cnt} shows overall frequencies.
\renewcommand{\arraystretch}{1.3}
\begin{table*}[]
\resizebox{\textwidth}{!}{%
\begin{tabular}{|llccccccc|}
\hline
\multicolumn{1}{|l|}{\textbf{Category}} &
\textbf{Subreddit Name} &
\textbf{Created Date} &
\textbf{\#Subscribers(k)} &
\textbf{\#Active Users} &
\textbf{\#Total Posts} &
\textbf{\#Total Comments} &
\textbf{Avg \#Replies/Post} &
\textbf{Avg Scores/Post} \\ \hline
\multicolumn{1}{|l|}{\multirow{9}{*}{\textbf{R-U Conflict}}} &
\textit{ukraine} &
Dec 23, 2008 &
731 &
269,612 &
125,008 &
3,800,366 &
0.72 &
1.58 \\
\multicolumn{1}{|l|}{} &
\textit{ukraina} &
Mar 14, 2014 &
119 &
23,481 &
11,058 &
100,572 &
0.36 &
1.12 \\
\multicolumn{1}{|l|}{} &
\textit{RussiaUkraineWar2022} &
Feb 24, 2022 &
105 &
37,546 &
15,788 &
275,837 &
0.54 &
1.14 \\
\multicolumn{1}{|l|}{} &
\textit{UkrainianConflict} &
Feb 20, 2014 &
354 &
122,278 &
56,086 &
1,437,688 &
0.49 &
1.45 \\
\multicolumn{1}{|l|}{} &
\textit{UkraineConflict} &
Mar 2, 2014 &
26 &
10,801 &
8,899 &
54,020 &
0.06 &
1.05 \\
\multicolumn{1}{|l|}{} &
\textit{UkraineWarVideoReport} &
Feb 24, 2022 &
428 &
150691 &
34,989 &
1,394,659 &
0.56 &
1.36 \\
\multicolumn{1}{|l|}{} &
\textit{UkraineWarReports} &
Feb 24, 2022 &
55.6 &
16,134 &
7,242 &
77,787 &
0.61 &
1.09 \\
\multicolumn{1}{|l|}{} &
\textit{UkraineInvasionVideos} &
Feb 24, 2022 &
51.1 &
16,119 &
10,110 &
91,716 &
0.02 &
1.06 \\
\multicolumn{1}{|l|}{} &
\textbf{Total} &
\textbf{-} &
\textbf{1750.7} &
\textbf{467,352} &
\textbf{269,180} &
\textbf{7,232,645} &
\textbf{0.57} &
\textbf{1.43} \\ \hline
\multicolumn{1}{|l|}{\multirow{5}{*}{\textbf{Military Related}}} &
\textit{war} &
Mar 19, 2008 &
92.5 &
18,018 &
9,167 &
97,119 &
0.45 &
1.03 \\
\multicolumn{1}{|l|}{} &
\textit{CombatFootage} &
Sep 10, 2012 &
1,035 &
127,638 &
24,765 &
1,287,740 &
1.10 &
1.26 \\
\multicolumn{1}{|l|}{} &
\textit{CredibleDefense} &
Aug 9, 2013 &
58.6 &
6,780 &
1,090 &
91,536 &
0.65 &
1.03 \\
\multicolumn{1}{|l|}{} &
\textit{geopolitics} &
Mar 26, 2008 &
435 &
8,407 &
2,280 &
42,665 &
1.48 &
1.03 \\
\multicolumn{1}{|l|}{} &
\textbf{Total} &
\textbf{-} &
\textbf{1621.1} &
\textbf{150,489} &
\textbf{37,302} &
\textbf{1,519,060} &
\textbf{0.95} &
\textbf{1.18} \\ \hline
\multicolumn{2}{|c|}{\textbf{All}} &
\textbf{-} &
\textbf{2936.8} &
\textbf{546,593} &
\textbf{306,482} &
\textbf{8,751,705} &
\textbf{0.62} &
\textbf{1.40} \\ \hline
\end{tabular}%
}
\caption{Frequencies of metadata for the selected subreddits (Results are based on data collected from 24 Feb 2022 to 29 May 2022. We performed the data collection on 30 May 2022). The \enquote{Active Users} refer to those users who ever sent at least one post(s) or comment(s) during the data collection period.}
\label{tab:overall_cnt}
\end{table*}
\renewcommand{\arraystretch}{1}
\subsubsection{Daily volume of posts and comments}
To understand the trend of subscribers' activity during this crisis event, Figure~\ref{fig:cnt_cat}. presents the time series of each category's submitted posts and comments.
We observe that posts and comments in \enquote{R-U Conflict} constitute the vast majority of the total volume. As for the distribution, the two categories' volumes of posts and comments spiked to more than 10,000 and 230,000 per day during the first week after the crisis. After that, the volumes decreased dramatically during the second week and gradually flattened out in the end. This suggests that the Russo-Ukrainian Conflict spread rapidly in these communities after the war broke out. The attention of those subscribers then shifted away quickly after two weeks.
\subsubsection{Engagement of posts}
\begin{figure}[tp]
\centering
\includegraphics[width=\columnwidth]{figs/cdf_replies.png}
\caption{(a): CDF plot for normalized replies number per \emph{engaging post}, replies number divided by the size of subscribers for each subreddit; (b): CDF plot for number of \emph{engaging posts} per \emph{engaging} poster (who has submitted at least one \emph{engaging posts} before).}
\label{fig:cdf_replies}
\end{figure}
We further examine the engagement of both categories. Prior studies have revealed a high correlation between the topic continuity and engagement of posts \cite{zeng2021modeling,pethe2019trumpiest}. Especially when evaluating the engagement between users, the number of replies (comments generated under a post) works as a significant predictive indicator.
More replies mean more engagement among users, indicating the topic will continue in further discussion \cite{haq2022short, wang2022successful}.
In addition, we conjecture that the number of subscribers in a subreddit might contribute to the likelihood of a post receiving replies. Here, we use the normalized number of replies to evaluate the engagement between the poster and commenters (i.e., we divide the number of replies by the number of subscribers in a subreddit). We first notice a large number of posts with no replies, introducing a significant bias when examining the distribution of replies. Overall, only 27\% of posts have received replies during the collection period, implying that a small amount of posts generates the most engagement in each category. Therefore, to concentrate our analysis on those posts driving users' engagement, the calculation and descriptive statistics are only based on posts with at least one reply. We refer such posts as \emph{engaging posts}.
Figure~\ref{fig:cdf_replies}(a) shows the distribution of their normalized number of replies. Although \enquote{Military Related} subreddits have fewer posts in total, according to the statistics of normalized replies number, their engaging posts receive more replies on average ($min=0.010, max=61.749, \mu=0.361, median=0.023$, and $\sigma = 1.865$) than those in \enquote{R-U Conflict} ($min=0.014, max=29.904, \mu=0.057, median=0.014$, and $\sigma = 0.295$), reflecting more substantial engagement between posters and commenters. In addition, we also find that replies of engaging posts for \enquote{R-U Conflict} follow a heavy-tailed distribution.
Previous study shows that discussions in a subreddit with high engagement are mainly generated by certain groups of users \cite{choi2015characterizing}. \james{%Inspired by their results,
Thus, we align with the prior approach and also refer the submitters of engaging posts as \emph{engaging posters}. Furthermore, we analyze the distribution of the number of engaging posts they have submitted, as shown in Figure~\ref{fig:cdf_replies}(b).
% Compared to \enquote{Military Related} ($min=1, max=37, \mu=1.58, median=1.00$, and $\sigma = 2.03$), engaging posters in \enquote{R-U Conflict} have submitted more engaging posts on average ($min=1, max=1857, \mu=3.45, median=1.00$, and $\sigma = 20.09$) during the collection period.
}
% \gareth{Could we add a brief description of this figure. What does the X-axis mean by 'interactive posts'? Does this mean engaging posts?}
Furthermore, only \james{12\%}
% \gareth{Minor point - would be nice to avoid the word 'around'. It takes up space and conveys an impression of fuzziness. Instead, we could just say 10\% (or 10.x\% if we want to be more precise.}
of users in this category have ever generated engaging posts, significantly lower than \enquote{R-U Conflict} (36\%).
\subsubsection{Rank of posts}
\begin{figure}[]
\centering
\includegraphics[width=\columnwidth]{figs/cdf_score.png}
\caption{(a): CDF plot for normalized scores in \emph{ranked posts}, score divided by the size of subscribers for each subreddit; (b): CDF plot for number \emph{ranked posts} per \emph{ranked poster} (who has submitted at least one \emph{ranked posts} before).}
\label{fig:cdf_score}
\end{figure}
We next explore posts with a high rank in \james{the observed subreddits, an aggregated evaluation of posts' priority on the \emph{Hot} section. Posts' ranks are influenced by vote scores.}
% \gareth{What do we mean by category here? The two groups of subreddits?}
After a post is submitted, viewers can up-vote or down-vote it. Generally, posts with higher vote scores (up-votes - down-votes) are ranked higher than posts with lower vote scores. Several previous studies have utilized vote scores in works like user behavior studies \cite{weninger2015random} and discussion threads analysis \cite{weninger2013exploration}.
Here we investigate the distribution of highly ranked posts' scores. Since Reddit automatically makes submitters up-vote their posts, the default score of all newly submitted posts is 1. Similarly, we also notice a large number of posts with default scores, introducing a strong bias in distribution analysis. \james{Similar to the previous section, to concentrate on posts with a higher rank, we refer those posts with score $>$ 1 as \emph{ranked posts} and their submitters as \emph{ranked posters}. We also draw the distribution plots for normalized score of ranked posts (i.e., we divide the vote score by the number of subscribers in a subreddit) and the number of such posts for their ranked posts, see Figure~\ref{fig:cdf_score}. According to the statistics of normalized score,
% both two categories present similar distribution of scores received by ranked posts.
compared to \enquote{Military Related} ($min=0.019, max=15.652, \mu=0.372, median=0.039$, and $\sigma = 1.159$), ranked posts are more likely to receive higher vote scores in \enquote{R-U Conflict} ($min=0.027, max=58.22, \mu=0.742, median=0.056$, and $\sigma = 2.716$).
% Furthermore, ranked posters in \enquote{R-U Conflict} ($min=1, max=82, \mu=1.91, median=1.00$, and $\sigma = 3.93$) have also submitted more ranked posts than those in \enquote{Military Related} ($min=1, max=5, \mu=1.20, median=1.00$, and $\sigma = 0.64$).
}
% \gareth{Anything explicit to flag here?}
\subsection{Text Content}
We next explore the characteristics of users' language usage and opinions. Table~\ref{tab:overall_txt} shows the average of linguistic features in our dataset.
\renewcommand{\arraystretch}{1.1}
\begin{table*}[]
\resizebox{\textwidth}{!}{%
\begin{tabular}{|llccccc|}
\hline
\multicolumn{1}{|l|}{\textbf{Category}} &
\textbf{Subreddit Name} &
\textbf{Title Length/Post} &
\textbf{Text Length/Post} &
\textbf{Text Length/Comment} &
\textbf{\#Emojis/Post} &
\textbf{\#Emojis/Comment} \\ \hline
\multicolumn{1}{|l|}{\multirow{9}{*}{\textbf{R-U Conflict}}} &
\textit{ukraine} &
13.39 &
21.76 &
29.17 &
0.115 &
0.060 \\
\multicolumn{1}{|l|}{} &
\textit{ukrain} &
13.24 &
18.05 &
24.29 &
0.171 &
0.152 \\
\multicolumn{1}{|l|}{} &
\textit{RussiaUkraineWar2022} &
14.31 &
9.94 &
28.98 &
0.177 &
0.106 \\
\multicolumn{1}{|l|}{} &
\textit{UkrainianConflict} &
15.91 &
0.32 &
30.40 &
0.055 &
0.043 \\
\multicolumn{1}{|l|}{} &
\textit{UkraineConflict} &
15.05 &
3.95 &
30.22 &
0.102 &
0.085 \\
\multicolumn{1}{|l|}{} &
\textit{UkraineWarVideoReport} &
12.96 &
10.72 &
25.27 &
0.061 &
0.064 \\
\multicolumn{1}{|l|}{} &
\textit{UkraineWarReports} &
13.08 &
10.52 &
27.79 &
0.091 &
0.077 \\
\multicolumn{1}{|l|}{} &
\textit{UkraineInvasionVideos} &
13.51 &
4.47 &
21.97 &
0.093 &
0.128 \\
\multicolumn{1}{|l|}{} &
\textbf{Total} &
\textbf{13.96} &
\textbf{12.96} &
\textbf{28.45} &
\textbf{0.098} &
\textbf{0.061} \\ \hline
\multicolumn{1}{|l|}{\multirow{5}{*}{\textbf{Military Related}}} &
\textit{war} &
11.71 &
0.67 &
26.23 &
0.065 &
0.063 \\
\multicolumn{1}{|l|}{} &
\textit{CombatFootage} &
11.87 &
3.41 &
25.72 &
0.025 &
0.024 \\
\multicolumn{1}{|l|}{} &
\textit{CredibleDefense} &
16.15 &
9.70 &
62.42 &
0.009 &
0.006 \\
\multicolumn{1}{|l|}{} &
\textit{geopolitics} &
12.15 &
3.63 &
62.56 &
0.022 &
0.008 \\
\multicolumn{1}{|l|}{} &
\textbf{Total} &
\textbf{11.97} &
\textbf{2.92} &
\textbf{29.08} &
\textbf{0.034} &
\textbf{0.025} \\ \hline
\multicolumn{2}{|c|}{\textbf{All}} &
\textbf{13.72} &
\textbf{11.80} &
\textbf{28.56} &
\textbf{0.091} &
\textbf{0.055} \\ \hline
\end{tabular}%
}
\caption{Average of linguistic features for the selected subreddits, (based on data collected from 24 Feb 2022 to 29 May 2022).}
\label{tab:overall_txt}
\end{table*}
\renewcommand{\arraystretch}{1}
\subsubsection{Term frequency}
% word cloud
Several related works on Reddit show that term frequency is an essential factor in content analysis, e.g., semantic content analysis related to weight loss \cite{pappa2017factors} and politics characterization for Reddit communities \cite{soliman2019characterization}. Term frequency also reflects the distribution of language usage, revealing the trend of users' topics, opinions, or attitudes.
For each subreddit, examine the term frequency (i.e., the count of a term divided by the total number of terms in the subreddit). Then, we select the top 1K terms with the highest frequency in each subreddit and create a union set for them. We define the terms for each union set as the most used terms in each category. After that, we calculate the normalized term frequency (i.e., for a term in one union set, the average value of all its term frequencies for subreddits in this category). Figure~\cref{fig:wc_ru,fig:wc_mr} show the word clouds of these terms weighted by the normalized term frequency after lemmatization for nouns and stop word removal.
We further calculate the relative difference of the normalized term frequency \cite{soliman2019characterization} in the two categories. For \enquote{R-U Conflict}, we subtract the normalized term frequency of relative words in \enquote{Military Related}, and vice versa.
This allows us to discern the terms used more frequently in one category than the other. \james{For each category, we refer such terms as its \emph{characteristic terms}.} We utilize relative differences to weight characteristic terms in word clouds, see \cref{fig:wc_ru_key,fig:wc_mr_key}.
% \gareth{How do we formally define characteristic terms? I think the previous sentence might need to be rewritten.}
According to the plots, the two categories show different stances toward the ongoing war. In \enquote{R-U Conflict}, users tend to use terms with higher negative connotations, e.g., \emph{orcs}, \emph{evil}, \emph{idiot}, \emph{troll}, and F-words.
We consider that such patterns for characteristic terms, compared to \enquote{Military Related}, reveal a preference for the Pro-Ukraine stance in \enquote{R-U Conflict}. In contrast, \enquote{Military Related} subscribers prefer terms highly related to global politics and the military. These terms contain much more normal connotations, indicating a neutral stance in this category compared to \enquote{R-U Conflict}.
\begin{figure*}[]
\centering
\begin{subfigure}[b]{0.4\textwidth}
\centering
\includegraphics[width=\textwidth]{figs/wc_ru.png}
\caption{R-U Conflict (most used terms)}
\label{fig:wc_ru}
\end{subfigure}
% \hfill
\begin{subfigure}[b]{0.4\textwidth}
\centering
\includegraphics[width=\textwidth]{figs/wc_mr.png}
\caption{Military Related (most used terms)}
\label{fig:wc_mr}
\end{subfigure}
% \hfill
\begin{subfigure}[b]{0.4\textwidth}
\centering
\includegraphics[width=\textwidth]{figs/wc_ru_key.png}
\caption{R-U Conflict (characteristic terms)}
\label{fig:wc_ru_key}
\end{subfigure}
% \hfill
\begin{subfigure}[b]{0.4\textwidth}
\centering
\includegraphics[width=\textwidth]{figs/wc_mr_key.png}
\caption{Military Related (characteristic terms)}
\label{fig:wc_mr_key}
\end{subfigure}
\caption{Most used and characteristic terms of categories; In (a) and (b) we plot the union sets for top 1,000 most used terms of each subreddis in \enquote{R-U Conflict} and \enquote{Military Related} respectively; In (c) and (d) we plot the differences of normalized term frequency for words in each category's union set compared to the other.
opposed subreddit}
\label{fig:wc_all}
\end{figure*}
\subsubsection{Linguistic features}
Stylometric properties, including text length and emojis use, have been widely applied in user profiling and accounts classification tasks \cite{lagutina2019survey,kosmajac2020twitter}. To understand the distribution of these properties in users’ discussions, we measure the average text length and number of emojis for the main text (both posts and comments).
% According to Table~\ref{tab:overall_txt}, we observe that, instead of the categories, the statistics values are highly related to the properties of subreddits, e.g., topic choices and community rules for the post format. For example, \emph{UkraineWarVideoReport} and \emph{UkraineInvasionVideos} both belong to \enquote{R-U Conflict}, but the latter has a shorter text length on average for the popularity of posting videos in this community.
In the online discussion on social platforms, specific patterns of widely used emoji phrases can reflect the user behaviors and public attitudes to ongoing topics, e.g., \includegraphics[width=1.1em]{figs/clapping-hands.png}: Applause and agreement on the topic, or \includegraphics[width=1.1em]{figs/face-with-rolling-eyes.png}: Moderate disdain, disapproval, frustration. Aiming to gain an overview of the emojis usage patterns in the whole dataset, we present the usage count for the most popular emojis in Figure~\ref{fig:emoji}. We notice that there is a strong bias toward the Pro-Ukraine stance in emoji usage, implied by frequently used phrases like \emph{\enquote{Ukrainian heart}} (\includegraphics[width=1.1em]{figs/flag-ukraine.png}\includegraphics[width=1.1em]{figs/red-heart.png} or \includegraphics[width=1.1em]{figs/blue-heart.png}\includegraphics[width=1.1em]{figs/yellow-heart.png}), \emph{\enquote{sunflower seed}} (\includegraphics[width=1.1em]{figs/sunflower.png})\footnote{\url{https://www.scmp.com/news/world/russia-central-asia/article/3168742/how-sunflowers-became-symbol-ukraines-resistance}} and \emph{\enquote{Russian clown}} (\includegraphics[width=1.1em]{figs/flag-russia.png}\includegraphics[width=1.1em]{figs/clown-face.png}). For context, here is an overt example of a post using emoji phrases in the Pro-Ukraine stance:
\begin{displayquote}
\emph{\#ukrainestrong! SLAVA UKRAINE! \includegraphics[width=1.1em]{figs/blue-heart.png}\includegraphics[width=1.1em]{figs/yellow-heart.png} if any aggressors see her!! well you now a sunflower!!! \includegraphics[width=1.1em]{figs/blue-heart.png}\includegraphics[width=1.1em]{figs/yellow-heart.png}\includegraphics[width=1.1em]{figs/sunflower.png}}
\end{displayquote}
% \begin{figure*}[]
% \centering
% \begin{subfigure}[b]{0.85\textwidth}
% \centering
% \includegraphics[width=\textwidth]{figs/lin_sta.png}
% \caption{Average number of text length and emojis for subreddits.}
% \label{fig:lin_sta}
% \end{subfigure}
% \hfill
% \begin{subfigure}[b]{0.85\textwidth}
% \centering
% \includegraphics[width=\textwidth]{figs/emoji.png}
% \caption{Top 20 popular emojis in the whole dataset.}
% \label{fig:emoji}
% \end{subfigure}
% \caption{Statistics of text length and emojis; (a) measures the average count for length and emojis number of text contents, both posts and comments, in each selected subreddits; (b) shows the top 20 most frequently used emojis in the whole dataset. }
% \label{fig:lin_all}
% \end{figure*}
\begin{figure*}[]
\centering
\includegraphics[width=.75\textwidth]{figs/emoji.png}
\caption{\james{Count} of the top 20 most frequently used emojis in the whole dataset.
% \gareth{I guess this caption is wrong unless you are adding another graph? Also, what is the meaning of the colour code in the graph? If the colours have no meaning, I would set all the bars to the same colour.}
}
\label{fig:emoji}
\end{figure*}
% \subsubsection{Domain}
% leave this part first, since no very valuable results found.
% top popular domain (need to exclude Reddit itself?)
\subsubsection{Sentiment}
Finally, we investigate the changes in users' sentiment in posts. We utilize the VADER sentiment analysis tool \cite{hutto2014vader} to classify posts' text (including the title and the corresponding main text). We perform the sentiment classification based on VADER compound scores, an aggregated score reflecting the sentiment bias. A related sentiment analysis for Twitter text has come up with an effective standard \cite{elbagir2019twitter}, and we here apply the same strategy: \textbf{Positive:} VADER compound score $>$ 0.001; \textbf{Negative:} VADER compound score $<$ 0.001; \textbf{Neutral:} Otherwise.
We calculate the normalized frequency for each sentiment class in every category by date. We first examine the relative frequency for each sentiment class in every subreddit (the ratio of the number of posts in a class vs.\ the total number of posts in a subreddit). Then, the normalized frequency for a class in each category is the average value of all its relative frequencies.
Figure~\ref{fig:sentiment} shows the distribution of daily sentiment based on the normalized frequency. We observe that \enquote{R-U Conflict} ($min=0.37, max=0.56, \mu=0.47, median=0.47$, and $\sigma = 0.04$) and \enquote{Military Related} ($min=0.35, max=0.73, \mu=0.50, median=0.50$, and $\sigma = 0.07$) have a similar ratio of \emph{negative} class, but subreddits in \enquote{Military Related} ($min=0.09, max=0.30, \mu=0.18, median=0.18$, and $\sigma = 0.04$) contains relatively fewer \emph{positive} posts than in \enquote{R-U Conflict} ($min=0.18, max=0.27, \mu=0.22, median=0.22$, and $\sigma = 0.02$). In addition, \enquote{Military Related} subreddits carry larger variations on its distribution of \emph{negative} posts ratio, especially after April 21.
% We try associating this trend with the ongoing war's timeline, and reveal a potential correlation between ratio changes and events, e.g., \enquote{canyon} before April 21 (Battle of Donbas\footnote{\url{https://en.wikipedia.org/wiki/Battle_of_Donbas_(2022)}}), surges after April 28 (2022 Transnistria attacks\footnote{\url{https://en.wikipedia.org/wiki/2022_Transnistria_attacks}}).
\begin{figure*}[]
\centering
\includegraphics[width=.7\textwidth]{figs/sentiment.png}
\caption{Distribution of daily users' sentiment in posts; (a) Sentiment distribution of posts in \enquote{R-U Conflict}; (b) Sentiment distribution of posts in \enquote{Military Related}.}
\label{fig:sentiment}
\end{figure*}
\section{Discussion \& Conclusion}
Social media datasets play an essential role in exploring research questions related to society. We shortlist 12 prominent subreddits relevant to the Russo-Ukrainian Conflict via an exhaustive subreddit exploration. Our dataset has collected more than 300K posts and 8M comments from these selected subreddits.
To shed light on the dataset's features, we have also performed several characterization analyses on daily volumes, posts' engagement, rank, term frequency, linguistic features, and the sentiment of text. The ongoing Russo-Ukrainian Conflict has been discussed widely on Reddit since the first week of the war. After that, the daily volume decreased significantly and flattened out within one month, implying a shift away in Reddit users' attention. Furthermore, we also notice a bias toward the Pro-Ukraine stance in our dataset. This is revealed by certain widely used words and specific emojis phrases with negative connotations toward the Russian invasion. We believe this general finding can support further studies on our dataset, e.g., misinformation detection for crisis events or social opinion analysis during the war.
\textbf{Limitations.}
% Our dataset is limited by the loss of previously deleted posts and comments. We are not able to collect those posts and comments that have already been deleted before Pushshift scrapes them. The pushshift.io API collects data for users by performing queries in the Pushshift dataset \cite{baumgartner2020pushshift}. All collected data through this API actually has been scraped and stored in the Pushshift dataset previously. Therefore, only before the posts or comments are deleted can Pushshift scrapes and stores their data. Otherwise, the queries only return results showing \enquote{[deleted]} or \enquote{[removed]}. As a result, our dataset has also missed the contents of such deleted posts and comments for this limitation.
\james{Our dataset is limited by the loss of previously deleted posts and comments. Since our collection is based on pushshift.io API, we are not able to collect those posts and comments that have already been deleted before Pushshift scrapes them. In details, the API queries for such posts and comments only return results showing \enquote{[deleted]} or \enquote{[removed]}. As a result, our dataset can not collect or store these deleted contents.}
% \gareth{Can I suggest removing the above paragraph? It sounds like we are complaining about pushshift :) Instead, we could keep it simple and say that we miss posts that are deleted?}
In addition, our dataset does not include the two major Russia-specific subreddits, \emph{russia} and \emph{russiapolitics}. This is because Reddit has quarantined these communities due to the high volume of misinformation,\footnote{\url{https://www.reddit.com/r/russia/}} and Pushshift does not record any data from these subreddits.
% Utilizing the Reddit API\footnote{\url{https://www.reddit.com/dev/api/}} or PRAW API\footnote{\url{https://github.com/praw-dev/praw}} can collect part of this data from quarantined subreddits. However, there are restrictions on the number of results that can be returned. Because of this limitation, these two APIs are not suitable for building a longitudinal dataset.
% https://github.com/praw-dev/praw
% As a result, the data of these communities will not appear and can only be collected by Reddit offical API\footnote{\url{https://www.reddit.com/dev/api/}}. However, this set of APIs has limitations on the number of results and searching times, which makes it impractical for building a longitudinal dataset in our work.
% In addition, as mentioned in previous sections, our dataset is also limited by not being able to collect data from quarantined major Russian subreddits. A recent work manages to get 15-day data from \emph{r/Russia} for semantic search by PRAW API \cite{hanley2022happenstance}. This set of APIs, however, works on the base of Reddit official API and also contains the identical limitations. Different from their need for short-period data in single subreddit, we aim to build a longitudinal dataset with comprehensive data source from multiple subreddits. Therefore, PRAW is still not enough for our work. A method to collect complete data from quarantined Russian subreddits is needed to extent our dataset.
% \section*{Acknowledgment}
% The preferred spelling of the word ``acknowledgment'' in America is without
% an ``e'' after the ``g''. Avoid the stilted expression ``one of us (R. B.
% G.) thanks $\ldots$''. Instead, try ``R. B. G. thanks$\ldots$''. Put sponsor
% acknowledgments in the unnumbered footnote on the first page.
\bibliographystyle{IEEEtran}
\bibliography{dataset}
% \section{Prepare Your Paper Before Styling}
% Before you begin to format your paper, first write and save the content as a
% separate text file. Complete all content and organizational editing before
% formatting. Please note sections \ref{AA}--\ref{SCM} below for more information on
% proofreading, spelling and grammar.
% Keep your text and graphic files separate until after the text has been
% formatted and styled. Do not number text heads---{\LaTeX} will do that
% for you.
% \subsection{Abbreviations and Acronyms}\label{AA}
% Define abbreviations and acronyms the first time they are used in the text,
% even after they have been defined in the abstract. Abbreviations such as
% IEEE, SI, MKS, CGS, ac, dc, and rms do not have to be defined. Do not use
% abbreviations in the title or heads unless they are unavoidable.
% \subsection{Units}
% \begin{itemize}
% \item Use either SI (MKS) or CGS as primary units. (SI units are encouraged.) English units may be used as secondary units (in parentheses). An exception would be the use of English units as identifiers in trade, such as ``3.5-inch disk drive''.
% \item Avoid combining SI and CGS units, such as current in amperes and magnetic field in oersteds. This often leads to confusion because equations do not balance dimensionally. If you must use mixed units, clearly state the units for each quantity that you use in an equation.
% \item Do not mix complete spellings and abbreviations of units: ``Wb/m\textsuperscript{2}'' or ``webers per square meter'', not ``webers/m\textsuperscript{2}''. Spell out units when they appear in text: ``. . . a few henries'', not ``. . . a few H''.
% \item Use a zero before decimal points: ``0.25'', not ``.25''. Use ``cm\textsuperscript{3}'', not ``cc''.)
% \end{itemize}
% \subsection{Equations}
% Number equations consecutively. To make your
% equations more compact, you may use the solidus (~/~), the exp function, or
% appropriate exponents. Italicize Roman symbols for quantities and variables,
% but not Greek symbols. Use a long dash rather than a hyphen for a minus
% sign. Punctuate equations with commas or periods when they are part of a
% sentence, as in:
% \begin{equation}
% a+b=\gamma\label{eq}
% \end{equation}
% Be sure that the
% symbols in your equation have been defined before or immediately following
% the equation. Use ``\eqref{eq}'', not ``Eq.~\eqref{eq}'' or ``equation \eqref{eq}'', except at
% the beginning of a sentence: ``Equation \eqref{eq} is . . .''
% \subsection{\LaTeX-Specific Advice}
% Please use ``soft'' (e.g., \verb|\eqref{Eq}|) cross references instead
% of ``hard'' references (e.g., \verb|(1)|). That will make it possible
% to combine sections, add equations, or change the order of figures or
% citations without having to go through the file line by line.
% Please don't use the \verb|{eqnarray}| equation environment. Use
% \verb|{align}| or \verb|{IEEEeqnarray}| instead. The \verb|{eqnarray}|
% environment leaves unsightly spaces around relation symbols.
% Please note that the \verb|{subequations}| environment in {\LaTeX}
% will increment the main equation counter even when there are no
% equation numbers displayed. If you forget that, you might write an
% article in which the equation numbers skip from (17) to (20), causing
% the copy editors to wonder if you've discovered a new method of
% counting.
% {\BibTeX} does not work by magic. It doesn't get the bibliographic
% data from thin air but from .bib files. If you use {\BibTeX} to produce a
% bibliography you must send the .bib files.
% {\LaTeX} can't read your mind. If you assign the same label to a
% subsubsection and a table, you might find that Table I has been cross
% referenced as Table IV-B3.
% {\LaTeX} does not have precognitive abilities. If you put a
% \verb|\label| command before the command that updates the counter it's
% supposed to be using, the label will pick up the last counter to be
% cross referenced instead. In particular, a \verb|\label| command
% should not go before the caption of a figure or a table.
% Do not use \verb|\nonumber| inside the \verb|{array}| environment. It
% will not stop equation numbers inside \verb|{array}| (there won't be
% any anyway) and it might stop a wanted equation number in the
% surrounding equation.
% \subsection{Some Common Mistakes}\label{SCM}
% \begin{itemize}
% \item The word ``data'' is plural, not singular.
% \item The subscript for the permeability of vacuum $\mu_{0}$, and other common scientific constants, is zero with subscript formatting, not a lowercase letter ``o''.
% \item In American English, commas, semicolons, periods, question and exclamation marks are located within quotation marks only when a complete thought or name is cited, such as a title or full quotation. When quotation marks are used, instead of a bold or italic typeface, to highlight a word or phrase, punctuation should appear outside of the quotation marks. A parenthetical phrase or statement at the end of a sentence is punctuated outside of the closing parenthesis (like this). (A parenthetical sentence is punctuated within the parentheses.)
% \item A graph within a graph is an ``inset'', not an ``insert''. The word alternatively is preferred to the word ``alternately'' (unless you really mean something that alternates).
% \item Do not use the word ``essentially'' to mean ``approximately'' or ``effectively''.
% \item In your paper title, if the words ``that uses'' can accurately replace the word ``using'', capitalize the ``u''; if not, keep using lower-cased.
% \item Be aware of the different meanings of the homophones ``affect'' and ``effect'', ``complement'' and ``compliment'', ``discreet'' and ``discrete'', ``principal'' and ``principle''.
% \item Do not confuse ``imply'' and ``infer''.
% \item The prefix ``non'' is not a word; it should be joined to the word it modifies, usually without a hyphen.
% \item There is no period after the ``et'' in the Latin abbreviation ``et al.''.
% \item The abbreviation ``i.e.'' means ``that is'', and the abbreviation ``e.g.'' means ``for example''.
% \end{itemize}
% An excellent style manual for science writers is \cite{b7}.
% \subsection{Authors and Affiliations}
% \textbf{The class file is designed for, but not limited to, six authors.} A
% minimum of one author is required for all conference articles. Author names
% should be listed starting from left to right and then moving down to the
% next line. This is the author sequence that will be used in future citations
% and by indexing services. Names should not be listed in columns nor group by
% affiliation. Please keep your affiliations as succinct as possible (for
% example, do not differentiate among departments of the same organization).
% \subsection{Identify the Headings}
% Headings, or heads, are organizational devices that guide the reader through
% your paper. There are two types: component heads and text heads.
% Component heads identify the different components of your paper and are not
% topically subordinate to each other. Examples include Acknowledgments and
% References and, for these, the correct style to use is ``Heading 5''. Use
% ``figure caption'' for your Figure captions, and ``table head'' for your
% table title. Run-in heads, such as ``Abstract'', will require you to apply a
% style (in this case, italic) in addition to the style provided by the drop
% down menu to differentiate the head from the text.
% Text heads organize the topics on a relational, hierarchical basis. For
% example, the paper title is the primary text head because all subsequent
% material relates and elaborates on this one topic. If there are two or more
% sub-topics, the next level head (uppercase Roman numerals) should be used
% and, conversely, if there are not at least two sub-topics, then no subheads
% should be introduced.
% \subsection{Figures and Tables}
% \paragraph{Positioning Figures and Tables} Place figures and tables at the top and
% bottom of columns. Avoid placing them in the middle of columns. Large
% figures and tables may span across both columns. Figure captions should be
% below the figures; table heads should appear above the tables. Insert
% figures and tables after they are cited in the text. Use the abbreviation
% ``Fig.~\ref{fig}'', even at the beginning of a sentence.
% \begin{table}[htbp]
% \caption{Table Type Styles}
% \begin{center}
% \begin{tabular}{|c|c|c|c|}
% \hline
% \textbf{Table}&\multicolumn{3}{|c|}{\textbf{Table Column Head}} \\
% \cline{2-4}
% \textbf{Head} & \textbf{\textit{Table column subhead}}& \textbf{\textit{Subhead}}& \textbf{\textit{Subhead}} \\
% \hline
% copy& More table copy$^{\mathrm{a}}$& & \\
% \hline
% \multicolumn{4}{l}{$^{\mathrm{a}}$Sample of a Table footnote.}
% \end{tabular}
% \label{tab1}
% \end{center}
% \end{table}
% \begin{figure}[htbp]
% \centerline{\includegraphics{fig1.png}}
% \caption{Example of a figure caption.}
% \label{fig}
% \end{figure}
% Figure Labels: Use 8 point Times New Roman for Figure labels. Use words
% rather than symbols or abbreviations when writing Figure axis labels to
% avoid confusing the reader. As an example, write the quantity
% ``Magnetization'', or ``Magnetization, M'', not just ``M''. If including
% units in the label, present them within parentheses. Do not label axes only
% with units. In the example, write ``Magnetization (A/m)'' or ``Magnetization
% \{A[m(1)]\}'', not just ``A/m''. Do not label axes with a ratio of
% quantities and units. For example, write ``Temperature (K)'', not
% ``Temperature/K''.
% \section*{Acknowledgment}
% The preferred spelling of the word ``acknowledgment'' in America is without
% an ``e'' after the ``g''. Avoid the stilted expression ``one of us (R. B.
% G.) thanks $\ldots$''. Instead, try ``R. B. G. thanks$\ldots$''. Put sponsor
% acknowledgments in the unnumbered footnote on the first page.
% \section*{References}
% Please number citations consecutively within brackets \cite{b1}. The
% sentence punctuation follows the bracket \cite{b2}. Refer simply to the reference
% number, as in \cite{b3}---do not use ``Ref. \cite{b3}'' or ``reference \cite{b3}'' except at
% the beginning of a sentence: ``Reference \cite{b3} was the first $\ldots$''
% Number footnotes separately in superscripts. Place the actual footnote at
% the bottom of the column in which it was cited. Do not put footnotes in the
% abstract or reference list. Use letters for table footnotes.
% Unless there are six authors or more give all authors' names; do not use
% ``et al.''. Papers that have not been published, even if they have been
% submitted for publication, should be cited as ``unpublished'' \cite{b4}. Papers
% that have been accepted for publication should be cited as ``in press'' \cite{b5}.
% Capitalize only the first word in a paper title, except for proper nouns and
% element symbols.
% For papers published in translation journals, please give the English
% citation first, followed by the original foreign-language citation \cite{b6}.
% \begin{thebibliography}{00}
% \bibitem{b1} G. Eason, B. Noble, and I. N. Sneddon, ``On certain integrals of Lipschitz-Hankel type involving products of Bessel functions,'' Phil. Trans. Roy. Soc. London, vol. A247, pp. 529--551, April 1955.
% \bibitem{b2} J. Clerk Maxwell, A Treatise on Electricity and Magnetism, 3rd ed., vol. 2. Oxford: Clarendon, 1892, pp.68--73.
% \bibitem{b3} I. S. Jacobs and C. P. Bean, ``Fine particles, thin films and exchange anisotropy,'' in Magnetism, vol. III, G. T. Rado and H. Suhl, Eds. New York: Academic, 1963, pp. 271--350.
% \bibitem{b4} K. Elissa, ``Title of paper if known,'' unpublished.
% \bibitem{b5} R. Nicole, ``Title of paper with only first word capitalized,'' J. Name Stand. Abbrev., in press.
% \bibitem{b6} Y. Yorozu, M. Hirano, K. Oka, and Y. Tagawa, ``Electron spectroscopy studies on magneto-optical media and plastic substrate interface,'' IEEE Transl. J. Magn. Japan, vol. 2, pp. 740--741, August 1987 [Digests 9th Annual Conf. Magnetics Japan, p. 301, 1982].
% \bibitem{b7} M. Young, The Technical Writer's Handbook. Mill Valley, CA: University Science, 1989.
% \end{thebibliography}
% \vspace{12pt}
% \color{red}
% IEEE conference templates contain guidance text for composing and formatting conference papers. Please ensure that all template text is removed from your conference paper prior to submission to the conference. Failure to remove the template text from your paper may result in your paper not being published.
\end{document}
% Generated by IEEEtran.bst, version: 1.14 (2015/08/26)
\begin{thebibliography}{10}
\providecommand{\url}[1]{#1}
\csname url@samestyle\endcsname
\providecommand{\newblock}{\relax}
\providecommand{\bibinfo}[2]{#2}
\providecommand{\BIBentrySTDinterwordspacing}{\spaceskip=0pt\relax}
\providecommand{\BIBentryALTinterwordstretchfactor}{4}
\providecommand{\BIBentryALTinterwordspacing}{\spaceskip=\fontdimen2\font plus
\BIBentryALTinterwordstretchfactor\fontdimen3\font minus
\fontdimen4\font\relax}
\providecommand{\BIBforeignlanguage}[2]{{%
\expandafter\ifx\csname l@#1\endcsname\relax
\typeout{** WARNING: IEEEtran.bst: No hyphenation pattern has been}%
\typeout{** loaded for the language `#1'. Using the pattern for}%
\typeout{** the default language instead.}%
\else
\language=\csname l@#1\endcsname
\fi
#2}}
\providecommand{\BIBdecl}{\relax}
\BIBdecl
\bibitem{haq2022weaponising}
E.-U. Haq, G.~Tyson, T.~Braud, and P.~Hui, ``Weaponising social media for
information divide and warfare,'' in \emph{Proceedings of the 33rd ACM
Conference on Hypertext and Social Media (HT ’22), June 28- July 1, 2022,
Barcelona, Spain}, 2022.
\bibitem{kouloumpis2011twitter}
E.~Kouloumpis, T.~Wilson, and J.~Moore, ``Twitter sentiment analysis: The good
the bad and the omg!'' in \emph{Proceedings of the international AAAI
conference on web and social media}, vol.~5, no.~1, 2011, pp. 538--541.
\bibitem{shu2017fake}
K.~Shu, A.~Sliva, S.~Wang, J.~Tang, and H.~Liu, ``Fake news detection on social
media: A data mining perspective,'' \emph{ACM SIGKDD explorations
newsletter}, vol.~19, no.~1, pp. 22--36, 2017.
\bibitem{haq2020survey}
E.~U. Haq, T.~Braud, Y.~D. Kwon, and P.~Hui, ``A survey on computational
politics,'' \emph{IEEE Access}, vol.~8, pp. 197\,379--197\,406, 2020.
\bibitem{zarei2020first}
K.~Zarei, R.~Farahbakhsh, N.~Crespi, and G.~Tyson, ``A first instagram dataset
on covid-19,'' \emph{arXiv preprint arXiv:2004.12226}, 2020.
\bibitem{allcott2017social}
H.~Allcott and M.~Gentzkow, ``Social media and fake news in the 2016
election,'' \emph{Journal of economic perspectives}, vol.~31, no.~2, pp.
211--36, 2017.
\bibitem{haq2022twitter}
E.-U. Haq, G.~Tyson, L.-H. Lee, T.~Braud, and P.~Hui, ``Twitter dataset for
2022 russo-ukrainian crisis,'' \emph{arXiv preprint arXiv:2203.02955}, 2022.
\bibitem{fung2022weibo}
Y.~R. Fung and H.~Ji, ``A weibo dataset for the 2022 russo-ukrainian crisis,''
\emph{arXiv preprint arXiv:2203.05967}, 2022.
\bibitem{bunting2021socially}
A.~M. Bunting, D.~Frank, J.~Arshonsky, M.~A. Bragg, S.~R. Friedman, and
N.~Krawczyk, ``Socially-supportive norms and mutual aid of people who use
opioids: An analysis of reddit during the initial covid-19 pandemic,''
\emph{Drug and alcohol dependence}, vol. 222, p. 108672, 2021.
\bibitem{wasike2011framing}
B.~S. Wasike, ``Framing social news sites: An analysis of the top ranked
stories on reddit and digg.'' \emph{Southwestern Mass Communication Journal},
vol.~27, no.~1, 2011.
\bibitem{baumgartner2020pushshift}
J.~Baumgartner, S.~Zannettou, B.~Keegan, M.~Squire, and J.~Blackburn, ``The
pushshift reddit dataset,'' in \emph{Proceedings of the international AAAI
conference on web and social media}, vol.~14, 2020, pp. 830--839.
\bibitem{snap}
J.~Yang and J.~Leskovec, ``Patterns of temporal variation in online media,'' in
\emph{Proceedings of the fourth ACM international conference on Web search
and data mining}, 2011, pp. 177--186.
\bibitem{elect1}
B.~Heredia, J.~Prusa, and T.~Khoshgoftaar, ``Exploring the effectiveness of
twitter at polling the united states 2016 presidential election,'' in
\emph{2017 IEEE 3rd International Conference on Collaboration and Internet
Computing (CIC)}.\hskip 1em plus 0.5em minus 0.4em\relax IEEE, 2017, pp.
283--290.
\bibitem{elect2}
J.~Golbeck, M.~Mauriello, B.~Auxier, K.~H. Bhanushali, C.~Bonk, M.~A.
Bouzaghrane, C.~Buntain, R.~Chanduka, P.~Cheakalos, J.~B. Everett
\emph{et~al.}, ``Fake news vs satire: A dataset and analysis,'' in
\emph{Proceedings of the 10th ACM Conference on Web Science}, 2018, pp.
17--21.
\bibitem{war1}
F.~K.~A. Salem, R.~Al~Feel, S.~Elbassuoni, M.~Jaber, and M.~Farah, ``Fa-kes: A
fake news dataset around the syrian war,'' in \emph{Proceedings of the
International AAAI Conference on Web and Social Media}, vol.~13, 2019, pp.
573--582.
\bibitem{war2}
D.~Guha-Sapir, B.~Schl{\"u}ter, J.~M. Rodriguez-Llanes, L.~Lillywhite, and
M.~H.-R. Hicks, ``Patterns of civilian and child deaths due to war-related
violence in syria: a comparative analysis from the violation documentation
center dataset, 2011--16,'' \emph{The Lancet Global Health}, vol.~6, no.~1,
pp. e103--e110, 2018.
\bibitem{banda2021large}
J.~M. Banda, R.~Tekumalla, G.~Wang, J.~Yu, T.~Liu, Y.~Ding, E.~Artemova,
E.~Tutubalina, and G.~Chowell, ``A large-scale covid-19 twitter chatter
dataset for open scientific research—an international collaboration,''
\emph{Epidemiologia}, vol.~2, no.~3, pp. 315--324, 2021.
\bibitem{cremisini2019challenging}
A.~Cremisini, D.~Aguilar, and M.~A. Finlayson, ``A challenging dataset for bias
detection: the case of the crisis in the ukraine,'' in \emph{International
Conference on Social Computing, Behavioral-Cultural Modeling and Prediction
and Behavior Representation in Modeling and Simulation}.\hskip 1em plus 0.5em
minus 0.4em\relax Springer, 2019, pp. 173--183.
\bibitem{chen2022tweets}
E.~Chen and E.~Ferrara, ``Tweets in time of conflict: A public dataset tracking
the twitter discourse on the war between ukraine and russia,'' \emph{arXiv
preprint arXiv:2203.07488}, 2022.
\bibitem{pohl2022twitter}
J.~Pohl, M.~V. Seiler, D.~Assenmacher, and C.~Grimme, ``A twitter streaming
dataset collected before and after the onset of the war between russia and
ukraine in 2022,'' \emph{Available at SSRN}, 2022.
\bibitem{park2022voynaslov}
C.~Y. Park, J.~Mendelsohn, A.~Field, and Y.~Tsvetkov, ``Voynaslov: A data set
of russian social media activity during the 2022 ukraine-russia war,''
\emph{arXiv preprint arXiv:2205.12382}, 2022.
\bibitem{reddit1}
X.~Ferrer, T.~van Nuenen, J.~M. Such, and N.~Criado, ``Discovering and
categorising language biases in reddit,'' in \emph{International AAAI
Conference on Web and Social Media (ICWSM 2021)(forthcoming)}, 2020.
\bibitem{haq2022short}
E.-U. Haq, T.~Braud, L.-H. Lee, A.~K. Vallapuram, Y.~Yu, G.~Tyson, and P.~Hui,
``Short, colorful, and irreverent! a comparative analysis of new users on
wallstreetbets during the gamestop short-squeeze,'' in \emph{In Companion
Proceedings of the Web Conference 2022 (WWW’22 Companion), April 25–29,
2022, Virtual Event, Lyon, France.}, 2022.
\bibitem{reddit2}
C.~Long, B.~M. Lucey, and L.~Yarovaya, ``I just like the stock" versus" fear
and loathing on main street": The role of reddit sentiment in the gamestop
short squeeze,'' \emph{SSRN Electronic Journal}, vol.~31, pp. 1--37, 2021.
\bibitem{saroj2020use}
A.~Saroj and S.~Pal, ``Use of social media in crisis management: A survey,''
\emph{International Journal of Disaster Risk Reduction}, vol.~48, p. 101584,
2020.
\bibitem{ozturk2018sentiment}
N.~{\"O}zt{\"u}rk and S.~Ayvaz, ``Sentiment analysis on twitter: A text mining
approach to the syrian refugee crisis,'' \emph{Telematics and Informatics},
vol.~35, no.~1, pp. 136--147, 2018.
\bibitem{cinelli2020covid}
M.~Cinelli, W.~Quattrociocchi, A.~Galeazzi, C.~M. Valensise, E.~Brugnoli, A.~L.
Schmidt, P.~Zola, F.~Zollo, and A.~Scala, ``The covid-19 social media
infodemic,'' \emph{Scientific reports}, vol.~10, no.~1, pp. 1--10, 2020.
\bibitem{hanley2022happenstance}
H.~W. Hanley, D.~Kumar, and Z.~Durumeric, ``Happenstance: Utilizing semantic
search to track russian state media narratives about the russo-ukrainian war
on reddit,'' \emph{arXiv preprint arXiv:2205.14484}, 2022.
\bibitem{weninger2013exploration}
T.~Weninger, X.~A. Zhu, and J.~Han, ``An exploration of discussion threads in
social news sites: A case study of the reddit community,'' in \emph{2013
IEEE/ACM international conference on advances in social networks analysis and
mining (ASONAM 2013)}.\hskip 1em plus 0.5em minus 0.4em\relax IEEE, 2013, pp.
579--583.
\bibitem{singer2014evolution}
P.~Singer, F.~Fl{\"o}ck, C.~Meinhart, E.~Zeitfogel, and M.~Strohmaier,
``Evolution of reddit: from the front page of the internet to a
self-referential community?'' in \emph{Proceedings of the 23rd international
conference on world wide web}, 2014, pp. 517--522.
\bibitem{zeng2021modeling}
X.~Zeng, J.~Li, L.~Wang, and K.-F. Wong, ``Modeling global and local
interactions for online conversation recommendation,'' \emph{ACM Transactions
on Information Systems (TOIS)}, vol.~40, no.~3, pp. 1--33, 2021.
\bibitem{pethe2019trumpiest}
C.~Pethe and S.~Skiena, ``The trumpiest trump? identifying a subject's most
characteristic tweets,'' \emph{arXiv preprint arXiv:1909.04002}, 2019.
\bibitem{wang2022successful}
L.~Wang, J.~Li, X.~Zeng, and K.-F. Wong, ``Successful new-entry prediction for
multi-party online conversations via latent topics and discourse modeling,''
in \emph{Proceedings of the ACM Web Conference 2022}, 2022, pp. 1663--1672.
\bibitem{choi2015characterizing}
D.~Choi, J.~Han, T.~Chung, Y.-Y. Ahn, B.-G. Chun, and T.~T. Kwon,
``Characterizing conversation patterns in reddit: From the perspectives of
content properties and user participation behaviors,'' in \emph{Proceedings
of the 2015 acm on conference on online social networks}, 2015, pp. 233--243.
\bibitem{weninger2015random}
T.~Weninger, T.~J. Johnston, and M.~Glenski, ``Random voting effects in
social-digital spaces: A case study of reddit post submissions,'' in
\emph{Proceedings of the 26th ACM conference on hypertext \& social media},
2015, pp. 293--297.
\bibitem{pappa2017factors}
G.~L. Pappa, T.~O. Cunha, P.~V. Bicalho, A.~Ribeiro, A.~P.~C. Silva,
W.~Meira~Jr, and A.~M.~R. Beleigoli, ``Factors associated with weight change
in online weight management communities: a case study in the loseit reddit
community,'' \emph{Journal of medical Internet research}, vol.~19, no.~1, p.
e5816, 2017.
\bibitem{soliman2019characterization}
A.~Soliman, J.~Hafer, and F.~Lemmerich, ``A characterization of political
communities on reddit,'' in \emph{Proceedings of the 30th ACM conference on
hypertext and Social Media}, 2019, pp. 259--263.
\bibitem{lagutina2019survey}
K.~Lagutina, N.~Lagutina, E.~Boychuk, I.~Vorontsova, E.~Shliakhtina,
O.~Belyaeva, I.~Paramonov, and P.~Demidov, ``A survey on stylometric text
features,'' in \emph{2019 25th Conference of Open Innovations Association
(FRUCT)}.\hskip 1em plus 0.5em minus 0.4em\relax IEEE, 2019, pp. 184--195.
\bibitem{kosmajac2020twitter}
D.~Kosmajac and V.~Keselj, ``Twitter user profiling: bot and gender
identification,'' in \emph{International Conference of the Cross-Language
Evaluation Forum for European Languages}.\hskip 1em plus 0.5em minus
0.4em\relax Springer, 2020, pp. 141--153.
\bibitem{hutto2014vader}
C.~Hutto and E.~Gilbert, ``Vader: A parsimonious rule-based model for sentiment
analysis of social media text,'' in \emph{Proceedings of the international
AAAI conference on web and social media}, vol.~8, no.~1, 2014, pp. 216--225.
\bibitem{elbagir2019twitter}
S.~Elbagir and J.~Yang, ``Twitter sentiment analysis using natural language
toolkit and vader sentiment,'' in \emph{Proceedings of the international
multiconference of engineers and computer scientists}, vol. 122, 2019, p.~16.
\end{thebibliography}