Expressivity of Emergent Language is a Trade-off between Contextual Complexity and Unpredictability
Extracted source text ยท arXiv 2106.03982
\documentclass{article}
% if you need to pass options to natbib, use, e.g.:
% \PassOptionsToPackage{numbers, compress}{natbib}
% before loading neurips_2021
% ready for submission
% \usepackage{neurips_2021}
% to compile a preprint version, e.g., for submission to arXiv, add add the
% [preprint] option:
\usepackage[preprint]{neurips_2021}
% to compile a camera-ready version, add the [final] option, e.g.:
% \usepackage[final]{neurips_2021}
% to avoid loading the natbib package, add option nonatbib:
% \usepackage[nonatbib]{neurips_2021}
\usepackage[utf8]{inputenc} % allow utf-8 input
\usepackage[T1]{fontenc} % use 8-bit T1 fonts
\usepackage{hyperref} % hyperlinks
\usepackage{url} % simple URL typesetting
\usepackage{booktabs} % professional-quality tables
\usepackage{amsfonts} % blackboard math symbols
\usepackage{nicefrac} % compact symbols for 1/2, etc.
\usepackage{microtype} % microtypography
\usepackage{xcolor} % colors
% added by Shawn
\usepackage{xcolor}
\usepackage{caption}
\usepackage{subcaption}
\usepackage{graphicx}
\usepackage{wrapfig}
\usepackage{amssymb}
\usepackage{amsmath}
\usepackage{amsthm}
\usepackage{bm}
\theoremstyle{definition}
\newtheorem{definition}{Definition}[section]
\usepackage[ruled,vlined]{algorithm2e}
\usepackage{verbatim}
\usepackage{lipsum}
\usepackage{booktabs}
\usepackage{multirow}
\usepackage{mathabx}
\usepackage{hyperref}
\newcounter{quotecount}
\newcommand{\MyQuote}[1]{\vspace{0.01cm}\addtocounter{quotecount}{1}%
\parbox{13cm}{\em #1}\hspace*{0.5cm}[\arabic{quotecount}]\\[0.01cm]}
\newcommand{\yicomment}[1]{\textcolor{blue}{#1}}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\title{Expressivity of Emergent Language is a Trade-off between Contextual Complexity and Unpredictability}
% The \author macro works with any number of authors. There are two commands
% used to separate the names and addresses of multiple authors: \And and \AND.
%
% Using \And between authors leaves it to LaTeX to determine where to break the
% lines. Using \AND forces a line break at that point. So, if LaTeX puts 3 of 4
% authors names on the first line, and the last on the second line, try using
% \AND instead of \And before the third author name.
\author{%
Shangmin Guo \\ % \thanks{Use footnote for providing further information
%about author (webpage, alternative address)---\emph{not} for acknowledging
%funding agencies.} \\
School of Informatics\\
University of Edinburgh\\
\texttt{s.guo@ed.ac.uk} \\
% examples of more authors
\And
Yi Ren\\
Department of Computer Science\\
University of British Columbia\\
\texttt{renyi.joshua@gmail.com}
\And
Kory Mathewson \\
DeepMind \\
% Address \\
\texttt{korymath@google.com} \\
\AND
Simon Kirby \\
Centre for Language Evolution\\
School of Philosophy, Psychology and Language Sciences \\
University of Edinburgh \\
\texttt{simon.kirby@ed.ac.uk} \\
\And
Stefano V. Albrecht \\
School of Informatics\\
University of Edinburgh\\
\texttt{s.albrecht@ed.ac.uk} \\
\And
Kenny Smith \\
Centre for Language Evolution\\
School of Philosophy, Psychology and Language Sciences \\
University of Edinburgh \\
\texttt{kenny.smith@ed.ac.uk} \\
}
\begin{document}
\maketitle
\begin{abstract}
Researchers are now using deep learning models to explore the emergence of language in various language games, where simulated agents interact and develop an emergent language to solve a task.
Although it is quite intuitive that different types of language games posing different communicative challenges might require emergent languages which encode different levels of information, there is no existing work exploring the \emph{expressivity} of the emergent languages.
In this work, we propose a definition of \emph{partial order} between expressivity based on the generalisation performance across different language games.
We also validate the hypothesis that expressivity of emergent languages is a trade-off between the complexity and unpredictability of the context those languages are used in.
Our second novel contribution is introducing contrastive loss into the implementation of referential games.
We show that using our contrastive loss alleviates the collapse of message types seen using standard referential loss functions.
\end{abstract}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Introduction}
\label{sec:intro}
The concept of ``language game'' was first introduced by \cite{wittgenstein2009philosophical} to explore the acquisition of meanings of language utterances as results of ``rules'' of games being played.
Instantiating this concept with the signalling game design from \cite{Lewis_refgame} enables linguists to explore the emergence of linguistic structure \citep{kirby2001spontaneous, kirby2002emergence} where artificial languages are represented as logic or symbolic systems.
The great success of deep learning (DL) models on complicated cognitive tasks \citep{ krizhevsky2012imagenet, lecun2015deep, silver2016mastering} then inspired researchers to apply DL-based models to language games such that agents could invent communication protocols from scratch. % , using paradigms where pairs or groups of DL agents in the course of completing tasks that require communication.
%This paradigm enables researchers to explore aspects such as:
%1) language emergence \citep[e.g.][]{lazaridou2018emergence, graesser2019emergent};
%2) the benefits from language-like communication for robotics \citep[e.g.][]{zhang2019efficient, kajic2020learning}, natural language processing tasks \citep{lee2017emergent}, and human computer interaction \citep{yuan2020emergence}.
% In language games implemented by the existing works, there are usually two types of agents:
In language games implemented by the existing works \citep[e.g.][]{lazaridou2018emergence, graesser2019emergent, zhang2019efficient, lee2017emergent, yuan2020emergence}, there are usually two types of agents:
1) \emph{speakers} that emit messages based on their observations (e.g. a concatenation of two 5-digits-long one-hot vectors in Figure~\ref{fig:diagram_recon}); and
2) listeners that receive messages and take actions accordingly (e.g. reconstructing the input in Figure~\ref{fig:diagram_recon}, or selecting the input among two samples in Figure~\ref{fig:diagram_refer}).
%To enable DL-based agents to invent communication protocols from scratch, following the sequence-to-sequence (Seq2Seq) architecture from \cite{sutskever2014sequence}, a common practice is to introduce a sequence generator into speaking agents and a sequence encoder into listening agents \citep{ivan2017nips}, which is illustrated in Figure~\ref{fig:diagram_both_games}.
%In addition to the communication modules, the speakers also have perception modules which encode the inputs into continuous embedding vectors,
%and the listeners also have action modules that decode the embeddings of messages and generate outputs required by different games, e.g. n-channel classification in referential games \citep{ivan2017nips}, or sequence generators in reconstruction games \citep{chaabouni2020compositionality}.
Based on the goals for listeners, we can categorise most of the games to the following three types:
1) referential games where listeners need to select the object observed by the speaker among a set of candidates \citep[e.g.][]{lazaridou2017multi, ren2020compositional};
2) reconstruction games where listeners need to reconstruct the speaker's observation \citep[e.g.][]{chaabouni2020compositionality, kharitonov2020entropy}; and
3) navigation games where listeners need to go to the specified locations \citep[e.g.][]{lowe2017multi, kajic2020learning}.
We focus on referential and reconstruction games, illustrated in Figure~\ref{fig:diagram_both_games}, which have also been investigated in the linguistic community \citep[e.g.][]{winters2018contextual, kirby2014iterated}.
\begin{figure}[t]
\centering
\begin{subfigure}[t]{0.49\textwidth}
\centering
\includegraphics[width=0.9\textwidth]{recon_game_onehot.pdf}
\caption{Reconstruction game}
\label{fig:diagram_recon}
\end{subfigure}
\hfill
\begin{subfigure}[t]{0.49\textwidth}
\centering
\includegraphics[width=0.9\textwidth]{refer_game_onehot.pdf}
\caption{Referential game}
\label{fig:diagram_refer}
\end{subfigure}
\caption{Illustration of the two types of games we investigate in this work.
% In the reconstruction game (a), the speaker is presented with an observation (a concatenation of two 5-digits-long one-hot vectors in this illustration) and generates a message which is passed to the listener; the listener's task is to reconstruct the speaker's observation (which they do successfully in this illustration).
% In the referential game (b) the listener's task is to select which of several (in the illustration, which of two) possible observations the speaker is describing.
}
\label{fig:diagram_both_games}
% \vspace{-0.15in}
\end{figure}
Intuitively, the speaker's messages must convey different levels of information for listeners to complete their task in these two different types of games.
The reconstruction game requires speakers to express all the details about the original observation such that listeners can perfectly reconstruct it.
Listeners in referential games, however, can differentiate the observation from distractors as long as messages from speakers encode some unique feature of it.
Experiments with human participants show that emergent languages are sensitive to the requirements of the communicative tasks for which they are used, with languages developing in which only necessary information is encoded \citep{winters2018contextual}.
Since languages are used to complete tasks \citep{wittgenstein2009philosophical}, we argue that it is necessary to explore how to facilitate an emergent language that is universally helpful for completing various types of games.
For the purposes of the paper, we refer to such ability to encode task-relevant information about input space (e.g. structure, dimensionality, distribution of samples) as expressivity.
In this work, we are going to explore this concept under the framework of DL-based language games.
As an initial step, we first propose a definition of partial orders between expressivity of emergent languages (Section~\ref{ssec:define_expressivity}), and we also explore the determining factors for expressivity.
It has been shown by \citet{winters2018contextual} that the predictability of the context shapes the degree of signal autonomy\footnote{The degree of \textit{signal autonomy} refers to the capacity for \textit{utterances} used in a natural language to be interpreted without contextual knowledge \citep{wray2007consequences}.} in human experiments, which means that the information which can be inferred out from an emergent language does not need to be encoded by speakers.
% Therefore, we hypothesise that the expressivity of languages that emerge in simulations with DL-based agents would also differ depending on the games.
% In particular, we propose a definition of the partial order between expressivity of different emergent languages based on their generalisability across various kinds of games. % , inspired by transfer learning techniques which measure the generalisation performance of models on a target domain which is different from the source domains used in the training \citep{pan2009survey}.
% Moreover, inspired by \cite{winters2018contextual}, we propose the following hypothesis about the determining factors for expressivity:
Therefore, in DL-based language games, we propose the following hypothesis about the determining factors for expressivity:
\MyQuote{
The expressivity of emergent languages is a trade-off between the complexity and the unpredictability of context in language games.
\label{hyp:tradeoff_hypothesis}
}
\vspace{-0.1in}
Contextual complexity and unpredictability are explained in more details in Section~\ref{ssec:expressivity_hypothesis}, but informally relate to the difficulty of distinguishing the referent from the context on any one trial (complexity\footnote{Complexity: for instance, the reconstruction game is more complex than the referential game, since the output space in is much larger than the candidate set.}),
and the variability of context on a series of such trials (unpredictability\footnote{Unpredictability: in this sense the context of referential game has higher unpredictability than reconstruction game, since the candidates constituted context would change in referential games whereas the output sapce for reconstruction keeps the same.}).
%the amount of information the message must encode to enable the listener to identify the target referent on any one trial (complexity\footnote{Complexity: for instance, the reconstruction game is more complex than the referential game, since the output space in is larger than the candidate set.}), and the extent to which a series encoding the same information is required across a series of such trials (unpredictability\footnote{Unpredictability: in this sense the context of referential game has higher unpredictability than reconstruction game, since the strategy that works on one trial in a simple referential game, e.g. specifying the value of the observation's first bit, might be successful on one trial but not on the next.}).
Through experiments controlling the complexity and unpredictability of context in the language games, we validate the above hypothesis.
Given the hypothesis, we argue that the language games should be carefully designed in order to facilitate more expressive emergent languages.
Furthermore, to overcome the technical limitations imposed by GPU memory size on large-scale games, another contribution of us is that we modify the contrastive loss function proposed by \cite{chen2020simple} and apply it into referential games.
We also show that our contrastive loss is equivalent to the loss function applied by most of the existing works, \citep[e.g][]{ivan2017nips, li2019ease, ren2020compositional, guo2019emergence} while its space complexity (in the algorithmic sense) is lower.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{comment}
\section{Background}
\label{sec:background}
\textbf{Language Emergence Games:} By applying the ideas of language game \citep{wittgenstein2009philosophical} and signalling game \citep{Lewis_refgame}, linguists have done great work on exploring the emergence of linguistic structure with artificial languages, \citep[e.g.][]{kirby2002emergence, kirby2001spontaneous}, where the languages are represented as logic and symbolic systems.
Since DL-based models have shown their capability to handle complicated cognitive tasks \citep{lecun2015deep, silver2016mastering}, there is a renewed interests in simulating language emergence with DL-based models.
To enable DL-based agents to invent communication protocols from scratch, following the Seq2Seq architecture from \cite{sutskever2014sequence}, a common practice is to introduce a sequence generator into speaking agents and a sequence encoder into listening agents \citep{ivan2017nips, lee2017emergent}, such that the two types of agents can communicate through a discrete channel.
Except the communication modules, the speakers also have perception modules which could encode the inputs into continuous embedding vectors,
and the listeners also have action modules that could decode the embedding vectors of messages and predict outputs required by different games, e.g. n-channel classification in referential games \citep{ivan2017nips}, or sequence generators in reconstruction game \citep{chaabouni2020compositionality}.
% Besides, although researches gained lots of progress, it is still controversial that how to facilitate important properties of natural languages such as compositionality in such architecture of games \citep{kottur2017natural, mordatch2018emergence, ren2020compositional}.
\textbf{Signal Autonomy:} The degree of \textit{signal autonomy} refers to the capacity for utterances used in a language to be interpreted without contextual knowledge \citep{wray2007consequences}.
Considering that language systems would adapt to their contextual niche \citep{winters_kirby_smith_2015}, it is quite straightforward to follow that context in the language games would affect the signal autonomy.
More specifically, with language games played by human participants, \cite{winters2018contextual} showed that the predictability of the context shapes the degree of signal autonomy, and crucially, that means that the stuff that can be inferred does not need to be encoded by the speakers. % i.e. listeners heavily rely on the specific context to interpret the meanings of utterances when the context is highly predictable.
It has also been shown by \cite{gil2005repetition} that there is a significant cross-linguistics variation of signal autonomy degree between different natural languages.
Considering the objectives of different language games and the corresponding context for listeners, it is quite straightforward and intuitive to reason that reconstruction games would raise a higher level of signal autonomy, thus a higher degree of expressivity, than referential games where there is a context, candidate set, for listeners to interpret the meaning of utterances.
\end{comment}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Methodology}
\label{sec:method}
All the simulation models we report here are illustrated in Figure~\ref{fig:modules_in_games}. %, and the two types of game are also illustrated in the caption. % a speaker agent makes an observation, encodes that observation in the message, the message is passed to the listener who either reconstructs the observation (reconstruction game) or attempts to select the observation from a context of possibilities (referential game).
The shared elements of the method are described in Section~\ref{ssec:environment} and the differences between the games in Section~\ref{ssec:games}.
Section~\ref{ssec:define_expressivity} then gives our definition of expressivity in terms of this experimental paradigm, and Section~\ref{ssec:expressivity_hypothesis} specifies our trade-off hypothesis (given informally above) with reference to this modelling framework.
In Section~\ref{ssec:exp_lan_transfer}, we illustrate the experiment used to verify the hypothesis.
\subsection{Elements and Configurations of Our Games}
\label{ssec:environment}
\textbf{Universal components}: As shown in Figure~\ref{fig:modules_in_games}, for whichever type of language game, there are $4$ universally essential components:
1) \textbf{input space} $\mathcal{X}$ from which the speaker's observations are drawn, which consists of $10,000$ samples\footnote{According to the results from \cite{chaabouni2020compositionality}, an input space containing $10,000$ samples is large enough for neural models to achieve $>99\%$ generalisation accuracy.} where each $\bm{x}\in\mathcal{X}$ is a concatenation of $4$ one-hot vectors whose length is $10$;
2) \textbf{message space} $\mathcal{M}$ consists of $10^6$ 6-tokens-long sequences $\bm{m}$ (called as a \emph{message}) where the size of the token inventory is $10$; note that the tokens are initially meaningless, and meaning emerges through the negotiation of a communication protocol between speaker and listener;
3) \textbf{speaker} $S$ consists of both a multi layer perceptron (MLP) for encoding the $40$-digits-long input $\bm{x}$ onto a $256$-digits-long embedding $\bm{h}^S$, and a decoder for generating a message $\bm{m}$ based on long-short term memory (LSTM) network \citep{hochreiter1997long};
iv) \textbf{listener} $L$ constituted by a LSTM-based sequence \emph{encoder} which encodes $\bm{m}$ into a $256$-digits-long embedding $\bm{h}^L$, and a game-specific action module which we will define for different games in Section~\ref{ssec:games}.
\begin{figure}[t]
\centering
\includegraphics[width=\textwidth]{both_game_illustration.pdf}
\caption{Illustration of the architecture of agents we implement for referential and reconstruction games.
Blue modules are components of speaker, and green modules belong to listener.
Orange and grey indicate the vectors generated by agents and input samples respectively.
Red text indicates for which type of game the listener's action module is used.
}
\label{fig:modules_in_games}
% \vspace{-0.25in}
\end{figure}
\textbf{Definition of an emergent language}: Following e.g. \cite{ren2020compositional}, we define an emergent language $\mathcal{L}$ as a mapping function from the input space $\mathcal{X}$ to the message space $\mathcal{M}$, i.e. $\mathcal{L}: \mathcal{X}\mapsto\mathcal{M}$, and it can be represented as $\left\{(\bm{x}_i,\mathcal{L}(\bm{x}_i)) | \forall \bm{x}_i\in\mathcal{X} \right\}$.
Note that $\mathcal{L}$ could be a non-injective mapping function, i.e. it is possible for two different inputs to be mapped to an identical message.
To distinguish between the messages emitted by a speaker in one trial (which might recur) and the unique message instances in $\mathcal{M}$, we refer to a unique message instance as a ``\emph{message type}''.
% Besides, sequence generators \citep[such as][]{sutskever2014sequence, vaswani2017attention} in speakers would assign probabilities to every token $m^k_i\in\bm{m}_i$ by applying a Softmax function to the output layer, based on which we could calculate the probability mass of $\bm{m}_i$ conditioned on $\bm{x}_i$, i.e. $p(\bm{m}_i|\bm{x}_i)$. %\yicomment{Here the i for xi is sample index, but for mi, it might be confused with the i-th dimension of message.}
\begin{comment}
\textbf{Information flow}: In each run of the game, the speaker $S$ would first generate a message $\bm{m}$ based on an input $\bm{x}$, i.e. the information first flows from input to message.
The message then passes through the noiseless and lossless discrete communication channel to the listener $L$.
After receiving and encoding the message $\bm{m}$, the action of listener $L$ would vary according to different objectives in different types of game.
In summary, the flow of the information is: $\bm{x}\rightarrow S \rightarrow \bm{m} \rightarrow L \rightarrow \mbox{game-specific module}$.
\end{comment}
\textbf{Optimisation}: To overcome the discrete channel issue, we apply the Gumbel-Softmax trick proposed by \cite{gumbel} to back-propagate the gradients from $L$ to $S$, and the temperature hyper-parameter $\tau$ is set to $1.0$.
As for updating the parameters, we use the Adam algorithm introduced by \cite{kingma2014adam}, and the learning rate is set to $10^{-4}$.
Our implementation\footnote{Codes are provided in the supplementary materials.} of games bases on the framework \textit{EGG} developed by \cite{Kharitonov2019Egg} under the MIT license.
The experiments cost $1,519$ hours on Nvidia Tesla P100.
\subsection{Different Types of Games}
\label{ssec:games}
\subsubsection{Referential Game}
\label{ssec:refer_game}
In addition to the four universal components, referential games \citep[following e.g.][]{ivan2017nips} include another component: a candidate set $D\subseteq\mathcal{X}$ consists of one target input $\bm{x}_t$ and $|D|-1$ distractors $\{\bm{x}_{d}^1, \bm{x}_{d}^2, \dots, \bm{x}_{d}^{|D|-1}\}$ sampled uniformly at random from $\mathcal{X}$ without replacement.
The action module of listener is a $|D|$-channel classifier where the probability of choosing a candidate, i.e. $c_i\in\{1,2,\dots,|D|\}$, is defined by the following equation:
\begin{equation}
p(c_i|\bm{x}_i,\bm{m})\propto \exp{E(\bm{h}_{\bm{m}}^{L}, f(\bm{x}_i))} = \exp{\left({\bm{h}_{\bm{m}}^{L}}^\top\cdot f(\bm{x}_i) \right)}
% p(c_i|\bm{x}_i,\bm{m}) = \frac{\exp{\left({\bm{h}_{\bm{m}}^{L}}^\top\cdot f(\bm{x}_i) \right)}}{\sum_{j=1}^{|D|}\exp{\left({\bm{h}_{\bm{m}}^{L}}^\top\cdot f(\bm{x}_j) \right)}}
\label{eq:traditional_prob_predict_refer}
\end{equation}
where $\bm{h}_{\bm{m}}^{L}$ is the embedding for the message $\bm{m}$ obtained from the sequence encoder of listener $L$, and $f(\cdot)$ is the MLP-based encoding function of $L$ for candidates $\bm{x}\in D$.
Since the messages are interpreted together with a candidate set, we refer to the candidate sets as the \emph{context} in referential games.
In the same way that synonymous words have closer embeddings \citep{mikolov2013efficient}, if a candidate has an embedding closer to message $\bm{m}$, the probability mass for predicting it would then be higher.
Hence, the predicated probability mass is inversely proportional to the distances, e.g. the dot produce in the Equation~\ref{eq:traditional_prob_predict_refer} above, between the embeddings of $\bm{m}$ and all candidates.
By comparing the observation above with the contrastive loss function proposed by \cite{hadsell2006dimensionality}, we find that the aim of the embeddings in referential games is the same as with contrastive loss, i.e. to make the distance between positive pairs (targets and their corresponding messages) closer while keeping the embeddings for negative pairs (distractors and the messages) as far apart as possible.
% While negative sampling has been deeply investigated in the self-supervised learning (SSL) community, \citep[e.g.][]{chen2020simple, chen2020mocov2, grill2020bootstrap}, it has not been discussed in the emergent communication community yet.
% Besides, a significant difference between this work and SSL is that language games contain discrete communication channels to imitate natural language properties % , whereas the information flows are all continuous in SSL.
To effectively validate our hypothesis about the determining factors of expressivity, we need to include the whole input space $\mathcal{X}$ as the candidates set $D$ in which case the data would become too large to be stored in GPU memory using an implementation like \citep{ivan2017nips}\footnote{The space complexity of referential loss given by \cite{ivan2017nips} is $\mathcal{O}(|D|^2)$} when the batch size is large.
To overcome these memory issues, based on the analogy between conventional referential game implementation and contrastive loss, we apply a contrastive loss similar to \citep{chen2020simple} in referential games.
Suppose that we have a batch of input samples $B=\{\bm{x}_1, \bm{x}_2, \dots, \bm{x}_i,\dots, \bm{x}_{|B|}\}$ and the corresponding messages from the speaker are $\{\bm{m}_1, \bm{m}_2, \dots, \bm{m}_i, \dots, \bm{m}_{|B|}\}$ which would be further encoded into $\{\bm{h}_{\bm{m}_1}^L, \bm{h}_{\bm{m}_2}^L, \dots, \bm{h}_{\bm{m}_i}^L, \dots, \bm{h}_{\bm{m}_{|B|}}^L \}$ by the listener $L$.
Then the loss function for a sample $\bm{x}_i$ is defined based on a Softmax function:
\begin{equation}
\ell_i = -\log \frac{ \exp{\left({\bm{h}_{\bm{m}_i}^L}^\top \cdot f^L(\bm{x}_i)\right)} }{ \sum_{j=1}^{|B|} \exp{ \left( {\bm{h}_{\bm{m}_i}^L}^\top \cdot f^L(\bm{x}_j) \right) }}
\label{eq:refer_game_loss}
\end{equation}
where $f^L(\cdot)$ is the encoding function of listener $L$ for candidates.
% Since $\exp{}()$ is a non-negative as well as monotonically increasing function, and the maximum of fraction inside the $\log()$ function is $1$ \footnote{The numerator is the exponential of dot product between embeddings for the message and the target, and denominator is a sum of the exponential of the dot products between embeddings for the message and \emph{all} inputs in the same batch.}, minimising this function is going to: i) maximise the dot product between the message and its corresponding input; ii) minimise the dot product between the message and the other inputs.
In this way, the number of candidates $|D|$ is exactly the batch size $|B|$ during training.
\subsubsection{Reconstruction Game}
\label{ssec:recon_game}
The reconstruction game is constituted by only the four universal components introduced in Section~\ref{ssec:environment}.
Although there is also an LSTM-based sequence encoder in the listener $L$ in reconstruction game, the action module is a generator instead of a classifier, as shown in Figure~\ref{fig:modules_in_games}.
In this work, since our inputs $\bm{x}$ are all $40$-digits-long binary vectors, the action module is set to be an MLP whose output layer consists of $40$ neurons, thus the reconstructed $\hat{\bm{x}}$ is a vector consisting of $40$ digits whose values are all in the range $(0,1)$.
The binary cross entropy loss function for $\hat{\bm{x}}$ is then calculated as:
\begin{equation}
\ell = \frac{1}{40} \sum_{k=1}^{K=40} x^k\log(\hat{x}^k) + (1-x^k)\log(1-\hat{x}^k)
\label{eq:recon_game_loss}
\end{equation}
where $\hat{x}_k = \sigma(g^k(\bm{h}^L_{\bm{x}}))$, $\sigma$ is the Sigmoid function, and $g(\cdot)$ represents the MLP for reconstruction inside $L$ where $g^k(\cdot)$ is the $k$-th dimension of its output.
Since the messages are interpreted within the whole output space, we refer to the space of all possible outputs ($\mathcal{O}$) as the \emph{context} in reconstruction game.
\subsection{Definition of Expressivity in Language Games}
\label{ssec:define_expressivity}
Following the intuition that the different types of games require messages to convey different amount of information about input space $\mathcal{X}$, mutual information (MI) may seem to be a good measurement for it.
We, however, argue in this work that even for two emergent languages that are both deterministic and bijective mapping functions, i.e. they have the same MI value $\log (|\mathcal{X}|)$, their expressivity could still be different.
Further discussion about MI can be found in Appendix~\ref{appsec:MI_is_insufficient}.
Since it is more challenging to directly measure the expressivity of emergent languages, we instead focus on the partial orders between them in this work.
Inspired by a common practice in the transfer learning community \citep{pan2009survey}, where models are trained on a source domain but evaluated on different domains, here we propose a definition of the partial order between the expressivity of two emergent languages $\mathcal{L}_A$ and $\mathcal{L}_B$, i.e. $\mathfrak{E}_{\mathcal{L}_A}$ and $\mathfrak{E}_{\mathcal{L}_B}$:
\theoremstyle{definition}
\begin{definition}[$\mathfrak{E}_{\mathcal{L}_A}^{\mathcal{G}} >
\mathfrak{E}_{\mathcal{L}_B}^{\mathcal{G}}$]
\label{def:expressivity}
Let $g$ be an instance of language game, and $\mathcal{G}$ be a set of language game instances.
Then, given $\mathcal{G}$,
if the generalisation performance of $\mathcal{L}_A$ is better than $\mathcal{L}_B$ on some language games $\mathcal{G'}$, i.e. $\forall g \in \mathcal{G'},\ \mathfrak{P}_{\mathcal{L}_A}^{g} > \mathfrak{P}_{\mathcal{L}_B}^{g}$,
where $\varnothing \subset \mathcal{G'} \subseteq \mathcal{G}$ and $\mathfrak{P}$ denotes the generalisation performance,
while there is no statistically significant difference between the converged generalisation performance of $\mathcal{L}_A$ and $\mathcal{L}_B$ on the remaining games, i.e. $\mathfrak{P}_{\mathcal{L}_A}^{g'} \approx \mathfrak{P}_{\mathcal{L}_B}^{g'} \forall g'\in \mathcal{G} \setminus \mathcal{G'}$,
we then say that expressivity of $\mathcal{L}_A$ is higher than $\mathcal{L}_B$ on $\mathcal{G}$, i.e. $\mathfrak{E}_{\mathcal{L}_A}^{\mathcal{G}} > \mathfrak{E}_{\mathcal{L}_B}^{\mathcal{G}}$.
The metric of generalisation performance could be varied according to different types of games.
\end{definition}
Briefly speaking, if one language performs better than the other on some games while they perform approximately the same on all games, we then say the expressivity of that language is higher than the other.
Following this definition, there are several possible relationships between the expressivity of two languages $\mathcal{L}_A$ and $\mathcal{L}_B$:
i) $\mathfrak{E}_{\mathcal{L}_A}^{\mathcal{G}} > \mathfrak{E}_{\mathcal{L}_B}^{\mathcal{G}}$, defined as above;
ii) $\mathfrak{E}_{\mathcal{L}_A}^{\mathcal{G}} = \mathfrak{E}_{\mathcal{L}_B}^{\mathcal{G}}$, if $\mathfrak{P}_{\mathcal{L}_A}^{g} \approx \mathfrak{P}_{\mathcal{L}_B}^{g},\ \forall g\in \mathcal{G}$;
iii) $\mathfrak{E}_{\mathcal{L}_B}^{\mathcal{G}} > \mathfrak{E}_{\mathcal{L}_A}^{\mathcal{G}}$, defined as above;
iv) $\mathfrak{E}_{\mathcal{L}_A}^{\mathcal{G}}$ is \emph{not comparable} to $\mathfrak{E}_{\mathcal{L}_B}^{\mathcal{G}}$ ($\mathfrak{E}_{\mathcal{L}_A}\neq\mathfrak{E}_{\mathcal{L}_A}$), if propositions $\exists g' \in \mathcal{G},\ s.t.\ \mathfrak{P}_{\mathcal{L}_A}^{g'} > \mathfrak{P}_{\mathcal{L}_B}^{g'}$ and $\exists g'' \in \mathcal{G},\ s.t.\ \mathfrak{P}_{\mathcal{L}_B}^{g''} > \mathfrak{P}_{\mathcal{L}_A}^{g''}$ are both true.
\subsection{Trade-off Hypothesis of Expressivity}
\label{ssec:expressivity_hypothesis}
As illustrated in Section~\ref{ssec:refer_game} and~\ref{ssec:recon_game}, in this work we restrict the concept of \emph{context} to:
i) the candidate set $D$ for listeners in referential games; and
ii) space of all possible outputs, i.e. $\mathcal{O} = \left\{ \left[o_1, o_2, \dots, o_{40}\right], s.t.\ o_i\in\{0,1\} \right\}$, for listeners in reconstruction game.
For referential games, it is then straightforward to observe that both complexity and unpredictability of context can be related to the size of candidate set $|D|$, following:
i) the fact that it requires less information to select $x_t$ from $\{x_t, x_{d_1}\}$ than $\{x_t, x_{d_1}, x_{d_2}, \dots, x_{d_{999}}\}$, as it is more probable that the latter set contains an $x_d$ that is more similar to $x_t$, i.e. the complexity is a \emph{monotonously increasing} function of $|D|$; and
ii) the fact that an $x_d$ is more probable to appear again in the next round of game since the sampling procedure of distractors follows a Bernoulli distribution whose mean is $\frac{|D|}{|\mathcal{X}|}$, i.e. unpredictability of context is a \emph{monotonously decreasing} function of $|D|$.
As for reconstruction game, although its context's complexity is very high, i.e. $|\mathcal{O}|=2^{40}$, the predictability of its context is also very high since the inputs $\mathcal{X}$ are identical in every epoch.
Note that although the inputs are also identical for every epoch in referential games, the listener would face different context in every batch since the candidates are randomly sampled. % from $\mathcal{X}$.
Following the references of contextual complexity and unpredictability above as well as \hyperref[def:expressivity]{Definition 2.1} proposed in Section~\ref{ssec:define_expressivity}, given \hyperref[hyp:tradeoff_hypothesis]{our hypothesis}, we can expect a $|D|\in (2, |\mathcal{X}|) \subset \mathbb{Z}^{+}$ such that a referential game with $|D|$ candidates can facilitate an emergent language whose expressivity is higher than the ones from the reconstruction game or referential games with most complex ($|D|=|\mathcal{X}|$) or most unpredictable ($|D|=2$) context.
%a suitable range of $|D|$ for referential game, because when $|D|$ gradually increase, the contextual complexity would decrease while the contextual unpredictability would increase.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Language Transfer Experiment}
\label{ssec:exp_lan_transfer}
To verify \hyperref[hyp:tradeoff_hypothesis]{our hypothesis}, we transfer the language which emerges in one game to the other, of which the procedure is illustrated in Experiment~\ref{al:exp_design}.
The results will be shown and discussed in Section~\ref{sec:results}.
\begin{algorithm}
% \small
\SetAlgoLined
\SetAlgorithmName{Experiment}{}
\KwIn{Input: A set of source game $\mathcal{G}_s$, a set of target game $\mathcal{G}_t$} \\
\For{every game $g_s^i$ in $\mathcal{G}_s$} {
1. initialise a new speaker and listener for $g_s^i$, and train them to play $g_s^i$ with the whole $\mathcal{X}$; \\
2. after the agents covnerge on $g_s^i$, record $\mathcal{L} = (\bm{x}, \bm{m})\ \forall \bm{x}\in\mathcal{X}$; \\% where $\bm{m}$ is the message for $\bm{m}$ from speaker; \\
3. randomly shuffle and split $\mathcal{L}$ into 2 disjoint sets $\mathcal{L}_{train}$ and $\mathcal{L}_{test}$ where $|\mathcal{L}_{train}|=90\% \cdot |\mathcal{L}|$; \\
4. \For{every game $g_t^j$ in $\mathcal{G}_t$}{
1. initialise a new listener for $g_t^j$; \\
2. train the listener with $\mathcal{L}_{train}$ to complete $g_t^j$;\\
3. record the performance of listener on $\mathcal{L}_{test}$ as the \textbf{generalisation performance of $g_s^i$ on $g_t^j$};
\\
}
}
\caption{Procedure of the language transfer experiment}
\label{al:exp_design}
\end{algorithm}
Since our hypothesis concerns the size of $D$, we propose to experiment on a reconstruction game (``recon'') and a series of referential games with varying candidate set size $|D|$ (``referX'' where X$=|D|$).
To be more specific, we set $\mathcal{G}_s$ and $\mathcal{G}_t$ identical to each other, i.e. $\mathcal{G}_s = \mathcal{G}_t = \mathcal{G} = \{\mbox{``recon''},\allowbreak \mbox{``refer2''},\allowbreak \mbox{``refer10''},\allowbreak \mbox{``refer100''},\allowbreak \mbox{``refer1000''},\allowbreak \mbox{``refer2500''},\allowbreak \mbox{``refer5000''},\allowbreak \mbox{``refer7500''},\allowbreak \mbox{``refer10000''}\}$.
The emergent language whose source game is a $g\in \mathcal{G}$ is also named by the type of its source, e.g. ``refer10 language'' represents the language which emerges in referential games whose $|D|=10$.
The language transfer procedure in our implementation is to train new listener to: i) reconstruct the inputs given only the messages; ii) select the inputs among the candidates sets given only the messages.
As for the performance metrics, we use accuracy in referential games, and binary cross entropy subtracted by 1 in reconstruction game.
% Beyond the transfer performance, we also record the number of message types for every training epoch in the language transfer experiment, where ``message type'' is used to indicate the unique message instance.
% For example, a speaker may emit one single message for two different inputs, in this case we would define the number of message type as $1$ although the message appears twice.
\begin{comment}
\subsection{Contrastive Loss v.s. Referential Loss}
\label{ssec:exp_contrastive_loss}
As we use contrastive loss instead of the conventional referential loss used in the previous works \citep{ivan2017nips, li2019ease, ren2020compositional, guo2019emergence, guo2019dissertation}, another experiment we need is to verify that both have similar results on the core experiment.
Therefore, we also run the language transfer experiment with referential loss, but limit the set of games to $\mathcal{G}_s' = \mathcal{G}_t' = \{\mbox{``recon''},\allowbreak \mbox{``refer2''},\allowbreak \mbox{``refer10''},\allowbreak \mbox{``refer100''},\allowbreak\}$ as larger candidate sets would cause out of memory problem, and the results are shown in Section~\ref{ssec:results_contrastive_loss}.
\end{comment}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Results \& Discussion}
\label{sec:results}
To alleviate the effect from randomness as much as possible, all the following results are obtained with 6 different random seeds.
All values are averaged across the seeds, and the corresponding standard deviation (SD) are also provided.
\subsection{Expressivity is a trade-off between contextual complexity and unpredictability}
\label{ssec:results_language_transfer}
The raw results of language transfer experiment are given in Table~\ref{tab:lan_transfer_raw_data} in Appendix~\ref{appsec:lan_transfer_results}.
% To better illustrate the partial order between the expressivity of emergent languages from different source games, w
We provide two views of these results as follows:
1) how the generalisation performance of a language from \emph{a source} game changes over \emph{different target} games in Figure~\ref{fig:result_source_on_targets}; and
2) how the generalisation performance of languages from \emph{different source games} change on the \emph{same target} game in Figure~\ref{fig:result_target_on_sources}.
\begin{figure}[h]
\centering
\includegraphics[width=0.9\textwidth]{result_source_vs_target.png} %{acc_source_vs_target.pdf}
\caption{
Generalisation performance of language from the same source game over different target games, where each line represents a \emph{source game}.
Since the plot would become not readable if we plot error bars here, we provide SD in Table~\ref{tab:lan_transfer_raw_data} in Appendix~\ref{appsec:lan_transfer_results}.
The x-axis indicates the type of target games.
Meanwhile, the left y-axis is the accuracy on referential games and the right y-axis is the binary cross entropy loss on reconstruction game (subtracted from 1).
If a line lies above the other over all the target games, it means that the corresponding source language has a higher expressivity than the other.
On the other hand, if two lines intersect, e.g. `recon' and `refer1000' above, then the expressivity of these languages is \emph{not comparable}.
Overall, the partial order between the emergent languages is:
$\mbox{`recon'}\neq\mbox{`refer1000'}\approx\mbox{`refer2500'}\approx\mbox{`refer5000'} > \mbox{`refer7500'}\approx\mbox{`refer10000'} > \mbox{`refer100'} > \mbox{`refer10'} > \mbox{`refer2'}$.
}
\label{fig:result_source_on_targets}
\end{figure}
We first illustrate the partial order between the expressivity of different languages in Figure~\ref{fig:result_source_on_targets}.
Although recon language performs consistently worse than some referential languages, e.g. `refer1000' and `refer2500', \emph{on referential games}, its performance on reconstruction game is the best, thus its expressivity is \emph{not comparable} with the others according to our definition of expressivity, as defined in Section~\ref{ssec:define_expressivity}.
For referential languages, it is very clear that the `refer2', `refer10', and `refer100' performs worse than the others on referential games with more candidates than their sources.
Combining the previous fact with the fact that `refer7500' and `refer10000' perform worse than `refer1000', `refer2500', and `refer5000' shows that increasing the size of candidate set $|D|$ would not always benefit the expressivity.
As for `refer1000', `refer2500', and `refer5000', there is no statistically significant difference between their performance on the simpler referential games, and they intersect on some games, thus their expressivity are approximately the same.
As shown in Figure~\ref{fig:result_target_on_sources}, on whichever type of game, the peak of the curves is always among `refer1000', `refer2500', or `refer5000', which means that the best expressivity always emerges in these 3 types of games.
Considering that the complexity of context in referential game is a \emph{monotonously increasing} function of the $\frac{|D|}{|\mathcal{X}|}$ while the contextual unpredictability is a \emph{monotonously decreasing} of it, the results shown in Figure~\ref{fig:result_target_on_sources} support our hypothesis, i.e. the expressivity is a trade-off between the contextual complexity and unpredictability.
To be specific, although context of `refer2', `refer10', and `refer100' have high unpredictability, their complexity is low thus the languages have lower expressivity.
On the other hand, context of `refer7500' and `refer10000' have high complexity, but expressivity of neither language is the highest due to the high predictability of the context in these games.
We argue here that unpredictability of context could help to incentivise the agents to encode more information into the messages such that the messages could overcome the uncertainty of contexts and be interpreted in various contexts.
Regarding the expressivity of reconstruction language, we argue that its expressivity suffers from a relatively low complexity, i.e. $\frac{|\mathcal{X}|}{|\mathcal{O}|}$\footnote{$\frac{|\mathcal{X}|}{|\mathcal{O}|} \ll \frac{|\mathcal{D}|}{|\mathcal{X}|}$, where $|\mathcal{O}|=2^{40}$, $|\mathcal{X}|=10000$, and $|\mathcal{D}|\in [2,10000]$.}, thus has actually not been maximised during the training and performs worse than some referential languages on referential games.
However, an important fact about `recon' is that although its expected generalisation performance of recon is not as good as some referential languages, its standard deviation is quite large as shown in Table~\ref{tab:lan_transfer_raw_data}, which indicates that the transferring procedure of `recon' to referential games is not very stable.
Thus, it is possible to obtain a `recon' which is more expressive than referential languages in some cases.
\begin{figure}
\centering
\includegraphics[width=0.9\textwidth]{acc_target_vs_source.pdf}
\caption{Generalisation performance of language from different source games on the same target game.
Each line represents a kind of \emph{target game}, and x-axis indicates the type of source games.
The two y-axis have the same meaning as the ones in Figure~\ref{fig:result_source_on_targets}.
Note that when target game is recon, i.e. the red line, the values should be read from the right y-axis.
In all target games except `recon', the best generalisation performance is always from `refer1000', `refer2500' or `refer5000', but not the most predictable or complex games.
}
\label{fig:result_target_on_sources}
\end{figure}
\subsection{Degree of degeneracy affects but does not fully determine the expressivity}
\label{ssec:result_degenerate_degree}
To further explore the determining factors for the expressivity, inspired by \cite{kirby2015compression}, the first factor we explore is the \emph{degree of degeneracy} (i.e. the degree of ambiguity) of the languages.
As a language is defined as a mapping function $\mathcal{L}: \mathcal{X} \mapsto \mathcal{M}$, the degree of its degeneracy could be measured by the number of distinct message types.
As shown in the Table~\ref{tab:msg_num_dist}, in referential languages, the degeneracy degree decreases rapidly with the increase of the contextual complexity of the source games, and reaches a ceiling around $|D|\geq 1,000$.
There is a positive correlation between the number of message types and expressivity for the languages with low complexity, which indicates that degree of degeneracy influences the expressivity of these languages.
However, the fact that the average number of message types on refer10000 is higher than refer1000, while the expressivity of `refer1000' is higher than `refer10000', i.e. emergent languages having lower level of degeneracy does not necessarily have higher expressivity, indicates that degeneracy degree of one language cannot fully determine its expressivity.
\begin{table}[h]
\scriptsize
\begin{tabular}{cccccccccc}
\hline
Game & refer2 & refer10 & refer100 & refer1000 & refer2500 & refer5000 & refer7500 & refer10000 & recon \\ \hline
\multicolumn{1}{c|}{Mean} &
\multicolumn{1}{c|}{2872.18} &
\multicolumn{1}{c|}{8565.69} &
\multicolumn{1}{c|}{9547.58} &
\multicolumn{1}{c|}{9801.60} &
\multicolumn{1}{c|}{9839.33} &
\multicolumn{1}{c|}{9841.27} &
\multicolumn{1}{c|}{9840.34} &
\multicolumn{1}{c|}{9856.96} &
9425.08 \\ \hline
\multicolumn{1}{c|}{$25$th-percentile} &
\multicolumn{1}{c|}{2629.0} &
\multicolumn{1}{c|}{8325.0} &
\multicolumn{1}{c|}{9501.5} &
\multicolumn{1}{c|}{9778.75} &
\multicolumn{1}{c|}{9825.0} &
\multicolumn{1}{c|}{9837.5} &
\multicolumn{1}{c|}{9819.0} &
\multicolumn{1}{c|}{9845.0} &
9445.5 \\ \hline
\multicolumn{1}{c|}{$75$th-percentile} &
\multicolumn{1}{c|}{3195.0} &
\multicolumn{1}{c|}{8795.5} &
\multicolumn{1}{c|}{9639.25} &
\multicolumn{1}{c|}{9823.25} &
\multicolumn{1}{c|}{9854.25} &
\multicolumn{1}{c|}{9864.25} &
\multicolumn{1}{c|}{9866.0} &
\multicolumn{1}{c|}{9871.0} &
9611.25 \\ \hline
\end{tabular}
\caption{Mean, $25$th-percentile, and $75$th-percentile of the distributions of message types numbers from different types of source game.}
\label{tab:msg_num_dist}
\vspace{-0.2in}
\end{table}
It is interesting that the degeneracy degree of `recon' is similar to `refer100', and is obviously lower than the more complicated referential languages, while its expressivity is the best when the target game is a reconstruction game.
However, like its language transfer performance, reconstruction language's degeneracy degree also has a higher variance, which is indicated by its $25/75$-th percentiles in Table~\ref{tab:msg_num_dist}.
One possible explanation to both phenomena is that the relatively low complexity ($\frac{|\mathcal{X}|}{|\mathcal{O}|}=\frac{10^4}{2^{40}}$) makes the training procedure on reconstruction game more flexible thus suffers from high variance.
% One possible explanation is that reconstruction languages convey all the information about the speaker's observation, but it is over-informative thus a listener $L$ from referential games has difficulty in extracting the task-relevant part for completing referential games.
% We will leave further exploration of this phenomenon to future works.
\begin{comment}
\begin{figure}
\centering
\includegraphics[width=0.8\textwidth]{msg_num_dist.pdf}
\caption{Distribution of number of message types from different types of source game. The numbers appearing on the x-axis indicate the types of source games, and the black bars correspond to the $[25,75]$ percentiles.}
\label{fig:msg_num_dist}
\end{figure}
\end{comment}
Our results also indicate that it is necessary to establish a referential game with candidate sets containing enough many distractors to guarantee that the emergent language could be as unambiguous as possible.
As far as we know, the existing works do not investigate the effects of the size of candidate sets, e.g. \cite{lee2017emergent} set $|D|=2$, \cite{li2019ease} set $|D|=5$, and $|D|\leq 20$ is used by \cite{lazaridou2018emergence}.
Given our results, our heuristic is that $|D|$ should be higher than either $1000$ for large input spaces or $\frac{1}{10}|\mathcal{X}|$ for smaller ones.
\subsection{Structure degree of mappings of different languages}
\label{ssec:result_lan_topo_sim}
Since the degenerate degree is not the only factor determining expressivity, another factor to be considered is how the mappings of a language $\mathcal{L}$ structured after the training procedure of agents on source games.
According to the cardinality of the mappings of $\mathcal{L}$, we can divide them into two disjoint sets:
i) \emph{one-to-one} mappings, i.e. an input is mapped to a unique message; and
ii) \emph{many-to-one} mappings (a.k.a degenerate components), i.e. several inputs are mapped to the same message instance.
%Between this two types, the structure of one-to-one mappings has been explored under the view of compositionality in the previous works, e.g. \citep{comp_measure_toposim, comp_measure_tre}.
%However, as illustrated by \cite{chaabouni2020compositionality}, a good generalisability does \emph{not} require a good compositionality, which means that measurements for compositionality are not good predictors for expressivity either.
%Therefore, limited by the scope of this work,
In this work, we only focus on the \emph{many-to-one} mappings.
Given the objectives of games, for the degenerate components in an emergent language, a message $\bm{m}$ being mapped to several closely-related inputs could imply that it captures common features among these inputs.
Suppose that several inputs $X_K = \left\{\bm{x}_1, \dots, \bm{x}_K\right\}$ are mapped to the same message $\bm{m}$ in a converged language $\mathcal{L}$, we hypothesise that \emph{the average distances between every pair of messages}, e.g. $\frac{1}{|X_K|}\sum_{i,j\in\{1,\dots,K\}} d(\bm{x}_i, \bm{x}_j)$, would become smaller after training, compared with their random initial values.
Therefore, we compare the averaged distances between meanings of degenerate components as well as the frequency of degenerate components at the beginning and end of training.
Results on `recon' and `refer1000' are shown in Figure~\ref{fig:result_avg_meaning_dis}.
\begin{figure}[h]
\centering
\begin{subfigure}[t]{0.5\textwidth}
\centering
\includegraphics[width=0.99\textwidth]{refer1000_msgnum_change.pdf}
\caption{`Refer1000'}
\label{fig:result_avg_meaning_dis_refer1000}
\end{subfigure}\hfill
\begin{subfigure}[t]{0.5\textwidth}
\centering
\includegraphics[width=0.99\textwidth]{recon_msgnum_change.pdf}
\caption{`Recon'}
\label{fig:result_avg_meaning_dis_recon}
\end{subfigure}
\caption{
Average distances between the meanings and statistics about frequency of \emph{Top-10} degenerate components at the beginning/end of training on `refer1000' and `recon'.
The bars indicate the \emph{number of meanings} one message type corresponds to, i.e. the frequency of degenerate components, and the black lines indicate the corresponding SD.
The numbers on the top of bars, however, are the \emph{mean distances between those meanings} with their SD.}
\label{fig:result_avg_meaning_dis}
\end{figure}
By comparing the data of beginning and end of training in Figure~\ref{fig:result_avg_meaning_dis}, we can easily observe that \emph{the averaged distance} decreased a lot (shown by the small numbers on red bars) throughout the training in both games, which indicates that the degenerate components actually became more although not fully structured.
Furthermore, it can also be observed that the averaged distances at the end of training on `recon' is smaller than on `refer1000', while `recon' has a higher degree of degeneracy (shown by that bars in Figure~\ref{fig:result_avg_meaning_dis_recon} are higher than the ones in Figure~\ref{fig:result_avg_meaning_dis_refer1000}).
Combining this observation with the result from Section~\ref{ssec:result_degenerate_degree}, it is reasonable to hypothesise that reconstruction game requires more structured mappings.
Another observation we have limited understanding of is that `refer1000' becomes more ambiguous after training, although its accuracy does improve to $~100\%$.
% However, limited by the scope of this work, we will leave further exploration of it to future works.
\subsection{Contrastive loss alleviates the collapse of message types}
\label{ssec:results_contrastive_loss}
As we use our contrastive loss instead of the conventional referential loss applied by \citep[e.g.][]{ivan2017nips, ren2020compositional}, we also compare the behaviours of both loss functions on $\mathcal{G}_s' = \mathcal{G}_t' = \{\mbox{``recon''},\allowbreak \mbox{``refer2''},\allowbreak \mbox{``refer10''},\allowbreak \mbox{``refer100''},\allowbreak\}$.
We observe that contrastive loss is more efficient on removing degenerate components in large-scale games (where $|D|\geq 100$), and can better avoid the collapse of the number of message types.
To be specific, as shown in Figure~\ref{fig:msg_num_curvers_refer_loss}, the numbers of message types collapses to less than $500$ on both reconstruction and referential games with conventional referential loss.
They, however, would only decrease to roughly $5000$ on `refer10' and `refer1000' with contrastive loss as shown in Figure~\ref{fig:msg_num_curvers_contrastive_loss}.
The contrastive loss can also make the training of referential game more stable, as illustrated by the smaller variance in Figure~\ref{fig:msg_num_curvers_contrastive_loss}.
% Besides, for referential games whose $|D| > 100$, contrastive loss could always alleviate the collapse of message types.
\begin{figure}[h]
\centering
\begin{subfigure}{0.5\textwidth}
\centering
\includegraphics[width=0.99\textwidth]{msg_num_curves.pdf}
\caption{With contrastive loss}
\label{fig:msg_num_curvers_contrastive_loss}
\end{subfigure}\hfill
\begin{subfigure}{0.5\textwidth}
\centering
\includegraphics[width=0.99\textwidth]{msg_num_curves_refer_loss.pdf}
\caption{With conventional referential loss}
\label{fig:msg_num_curvers_refer_loss}
\end{subfigure}
\caption{How number of message types change over training epochs with different loss functions.
The lines are the means over 6 runs and shadow areas indicate the corresponding standard deviation.}
\label{fig:results_contrastive_vs_referential}
\end{figure}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Conclusion \& Future Work}
\label{sec:conclusion}
According to our knowledge, this is the first work exploring the \emph{expressivity} of emergent languages from different types of language games involving DL-based agents.
We first defined the partial order between expressivity of different emergent languages based on the generalisation performance across a set of games, and proposed a hypothesis about the factors influencing the expressivity of emergent languages.
Through a series of experiments on a reconstruction game and referential games with varying numbers of candidates, we then validated that the expressivity at first increases and then decreases as a function of the size of candidate sets, which indicates that expressivity is indeed a trade-off between the complexity and unpredictability of context in language games.
After analysing the data on various target games, we also pointed out that the contextual complexity of language games, especially referential games, should be large enough such that we could facilitate more expressive languages.
We further explored the degeneracy and structure degree of the mappings constituting different languages, and found that both degree are influenced by the types of games.
To overcome the memory inefficiency issue caused by the conventional referential loss function, we introduced a contrastive loss into our implementation, and showed its effectiveness at avoiding the collapse of number of message types.
Since this is a step towards defining and understanding the expressivity of emergent languages, there are still some questions unanswered by this work.
%Although we propose some hypotheses about why the expressivity of reconstruction language is the best on reconstruction while it underperforms the most expressive referential language on referential games, this problem has not yet been fully explored; this finding is puzzling since the reconstruction game requires information about all details of the original inputs, which should be sufficient for agents to complete referential games.
For example, although we show that neither of complexity nor unpredictability can fully determine the expressivity of emergent languages.
We still do not understand how they affect the expressivity, and there is no quantitative measurement for such effects.
%Furthermore, some properties of our emergent languages are surprising given the simplicity of some of our referential games.
%In the simplest setting of referential game, we see emergent languages with more than $2,000$ message types, whereas an optimal strategy requires a language consists of only $40$ unique messages.\footnote{
%Considering that the inputs are all concatenations of four 10-digits-long vectors, there are only $40$ different values for all properties.
%Since any two members of the context $D$ will always differ on at least one property, $40$ messages are enough for referring to one specific value of the target given another distractor.
%}
%Thus, it needs further exploration of how redundancy helps during the communication between agents.
More importantly, we lack understanding of the structure of mappings constituting a language, especially when the languages are generated or encoded by neural networks.
Last but not least, since harder tasks don't always lead to higher expressivity, it may imply that we need to carefully design:
i) the pretraining task for neural language models in order to facilitate the best performance of models on downstream tasks;
ii) curriculum learning for multi-agent reinforcement learning algorithms enhanced by emergent communication techniques \citep[e.g.][]{evtimova2017emergent}.
We argue that the expressivity of languages in the above scenarios still needs further exploration in order to maximise the performance of the aforementioned systems.
%\newpage
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{comment}
\begin{ack}
Use unnumbered first level headings for the acknowledgements. All acknowledgements
go at the end of the paper before the list of references. Moreover, you are required to declare
funding (financial activities supporting the submitted work) and competing interests (related financial activities outside the submitted work).
More information about this disclosure can be found at: \url{https://neurips.cc/Conferences/2021/PaperInformation/FundingDisclosure}.
Do {\bf not} include this section in the anonymized submission, only in the final paper. You can use the \texttt{ack} environment provided in the style file to autmoatically hide this section in the anonymized submission.
\end{ack}
\end{comment}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\bibliographystyle{apalike}
\bibliography{neurips_2021}
\newpage
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{comment}
\section*{Checklist}
%%% BEGIN INSTRUCTIONS %%%
The checklist follows the references. Please
read the checklist guidelines carefully for information on how to answer these
questions. For each question, change the default \answerTODO{} to \answerYes{},
\answerNo{}, or \answerNA{}. You are strongly encouraged to include a {\bf
justification to your answer}, either by referencing the appropriate section of
your paper or providing a brief inline description. For example:
\begin{itemize}
\item Did you include the license to the code and datasets? \answerYes{See Section~2.}
\item Did you include the license to the code and datasets? \answerNo{The code and the data are proprietary.}
\item Did you include the license to the code and datasets? \answerNA{}
\end{itemize}
Please do not modify the questions and only use the provided macros for your
answers. Note that the Checklist section does not count towards the page
limit. In your paper, please delete this instructions block and only keep the
Checklist section heading above along with the questions/answers below.
%%% END INSTRUCTIONS %%%
\begin{enumerate}
\item For all authors...
\begin{enumerate}
\item Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
\answerYes{See abstract and Section~\ref{sec:intro}}
\item Did you describe the limitations of your work?
\answerYes{See Section~\ref{sec:conclusion}}
\item Did you discuss any potential negative societal impacts of your work?
\answerNA{}
\item Have you read the ethics review guidelines and ensured that your paper conforms to them?
\answerYes{}
\end{enumerate}
\item If you are including theoretical results...
\begin{enumerate}
\item Did you state the full set of assumptions of all theoretical results?
\answerNA{}
\item Did you include complete proofs of all theoretical results?
\answerNA{}
\end{enumerate}
\item If you ran experiments...
\begin{enumerate}
\item Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
\answerYes{See Section~\ref{ssec:environment}}
\item Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
\answerYes{See Section~\ref{ssec:environment}.}
\item Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
\answerYes{See e.g. Figure~\ref{fig:result_avg_meaning_dis} and~\ref{fig:results_contrastive_vs_referential}.}
\item Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
\answerYes{See Section~\ref{ssec:environment}.}
\end{enumerate}
\item If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
\begin{enumerate}
\item If your work uses existing assets, did you cite the creators?
\answerYes{See Section~\ref{ssec:environment}}
\item Did you mention the license of the assets?
\answerYes{See Section~\ref{ssec:environment}}
\item Did you include any new assets either in the supplemental material or as a URL?
\answerYes{See supplemental material, we still use MIT license.}
\item Did you discuss whether and how consent was obtained from people whose data you're using/curating?
\answerNA{}
\item Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
\answerNA{}
\end{enumerate}
\item If you used crowdsourcing or conducted research with human subjects...
\begin{enumerate}
\item Did you include the full text of instructions given to participants and screenshots, if applicable?
\answerNA{}
\item Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
\answerNA{}
\item Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
\answerNA{}
\end{enumerate}
\end{enumerate}
\end{comment}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\appendix
\section*{Appendices}
\section{Language Transfer Experiment Results}
\label{appsec:lan_transfer_results}
In Figure~\ref{fig:result_source_on_targets} and~\ref{fig:result_target_on_sources}, we provide two views of the generalisation performance of all game instances on each other.
Here, we give the precise values in the following Table~\ref{tab:lan_transfer_raw_data}.
All the results are obtained by multiple runs, and both mean and standard deviation ($\sigma$) are given.
\begin{table}[h!]
\scriptsize
\begin{tabular}{@{}ccccccccccc@{}}
\toprule
Source &
Statistics &
refer2 &
refer10 &
refer100 &
refer1000 &
refer2500 &
refer5000 &
refer7500 &
refer10000 &
recon \\ \midrule
\multicolumn{1}{|c|}{\multirow{2}{*}{refer2}} &
\multicolumn{1}{c|}{mean} &
\multicolumn{1}{c|}{0.9936} &
\multicolumn{1}{c|}{0.9543} &
\multicolumn{1}{c|}{0.6043} &
\multicolumn{1}{c|}{0.1203} &
\multicolumn{1}{c|}{0.0668} &
\multicolumn{1}{c|}{0.0616} &
\multicolumn{1}{c|}{0.0592} &
\multicolumn{1}{c|}{0.0565} &
\multicolumn{1}{c|}{0.7248} \\ \cmidrule(l){2-11}
\multicolumn{1}{|c|}{} &
\multicolumn{1}{c|}{$\sigma$} &
\multicolumn{1}{c|}{0.0014} &
\multicolumn{1}{c|}{0.0077} &
\multicolumn{1}{c|}{0.0394} &
\multicolumn{1}{c|}{0.0146} &
\multicolumn{1}{c|}{0.0105} &
\multicolumn{1}{c|}{0.0078} &
\multicolumn{1}{c|}{0.0090} &
\multicolumn{1}{c|}{0.0047} &
\multicolumn{1}{c|}{0.0105} \\ \midrule
\multicolumn{1}{|c|}{\multirow{2}{*}{refer10}} &
\multicolumn{1}{c|}{mean} &
\multicolumn{1}{c|}{0.9980} &
\multicolumn{1}{c|}{0.9909} &
\multicolumn{1}{c|}{0.9076} &
\multicolumn{1}{c|}{0.4896} &
\multicolumn{1}{c|}{0.3434} &
\multicolumn{1}{c|}{0.3318} &
\multicolumn{1}{c|}{0.3282} &
\multicolumn{1}{c|}{0.3188} &
\multicolumn{1}{c|}{0.7559} \\ \cmidrule(l){2-11}
\multicolumn{1}{|c|}{} &
\multicolumn{1}{c|}{$\sigma$} &
\multicolumn{1}{c|}{0.0007} &
\multicolumn{1}{c|}{0.0036} &
\multicolumn{1}{c|}{0.0245} &
\multicolumn{1}{c|}{0.0833} &
\multicolumn{1}{c|}{0.0809} &
\multicolumn{1}{c|}{0.0801} &
\multicolumn{1}{c|}{0.0778} &
\multicolumn{1}{c|}{0.0752} &
\multicolumn{1}{c|}{0.0147} \\ \midrule
\multicolumn{1}{|c|}{\multirow{2}{*}{refer100}} &
\multicolumn{1}{c|}{mean} &
\multicolumn{1}{c|}{0.9986} &
\multicolumn{1}{c|}{0.9978} &
\multicolumn{1}{c|}{0.9812} &
\multicolumn{1}{c|}{0.8298} &
\multicolumn{1}{c|}{0.7162} &
\multicolumn{1}{c|}{0.7121} &
\multicolumn{1}{c|}{0.7078} &
\multicolumn{1}{c|}{0.7001} &
\multicolumn{1}{c|}{0.7870} \\ \cmidrule(l){2-11}
\multicolumn{1}{|c|}{} &
\multicolumn{1}{c|}{$\sigma$} &
\multicolumn{1}{c|}{0.0007} &
\multicolumn{1}{c|}{0.0009} &
\multicolumn{1}{c|}{0.0035} &
\multicolumn{1}{c|}{0.0171} &
\multicolumn{1}{c|}{0.0299} &
\multicolumn{1}{c|}{0.0312} &
\multicolumn{1}{c|}{0.0293} &
\multicolumn{1}{c|}{0.0350} &
\multicolumn{1}{c|}{0.0089} \\ \midrule
\multicolumn{1}{|c|}{\multirow{2}{*}{refer1000}} &
\multicolumn{1}{c|}{mean} &
\multicolumn{1}{c|}{0.9989} &
\multicolumn{1}{c|}{0.9979} &
\multicolumn{1}{c|}{0.9881} &
\multicolumn{1}{c|}{0.9003} &
\multicolumn{1}{c|}{0.8256} &
\multicolumn{1}{c|}{0.8139} &
\multicolumn{1}{c|}{0.8153} &
\multicolumn{1}{c|}{0.8033} &
\multicolumn{1}{c|}{0.8094} \\ \cmidrule(l){2-11}
\multicolumn{1}{|c|}{} &
\multicolumn{1}{c|}{$\sigma$} &
\multicolumn{1}{c|}{0.0007} &
\multicolumn{1}{c|}{0.0009} &
\multicolumn{1}{c|}{0.0015} &
\multicolumn{1}{c|}{0.0143} &
\multicolumn{1}{c|}{0.0151} &
\multicolumn{1}{c|}{0.;0165} &
\multicolumn{1}{c|}{0.0226} &
\multicolumn{1}{c|}{0.0183} &
\multicolumn{1}{c|}{0.0117} \\ \midrule
\multicolumn{1}{|c|}{\multirow{2}{*}{refer2500}} &
\multicolumn{1}{c|}{mean} &
\multicolumn{1}{c|}{0.9988} &
\multicolumn{1}{c|}{0.9981} &
\multicolumn{1}{c|}{0.9892} &
\multicolumn{1}{c|}{0.9036} &
\multicolumn{1}{c|}{0.8235} &
\multicolumn{1}{c|}{0.8178} &
\multicolumn{1}{c|}{0.8160} &
\multicolumn{1}{c|}{0.8059} &
\multicolumn{1}{c|}{0.8047} \\ \cmidrule(l){2-11}
\multicolumn{1}{|c|}{} &
\multicolumn{1}{c|}{$\sigma$} &
\multicolumn{1}{c|}{0.0006} &
\multicolumn{1}{c|}{0.0009} &
\multicolumn{1}{c|}{0.0012} &
\multicolumn{1}{c|}{0.0121} &
\multicolumn{1}{c|}{0.0099} &
\multicolumn{1}{c|}{0.0158} &
\multicolumn{1}{c|}{0.0148} &
\multicolumn{1}{c|}{0.0197} &
\multicolumn{1}{c|}{0.0093} \\ \midrule
\multicolumn{1}{|c|}{\multirow{2}{*}{refer5000}} &
\multicolumn{1}{c|}{mean} &
\multicolumn{1}{c|}{0.9990} &
\multicolumn{1}{c|}{0.9975} &
\multicolumn{1}{c|}{0.9890} &
\multicolumn{1}{c|}{0.8991} &
\multicolumn{1}{c|}{0.8212} &
\multicolumn{1}{c|}{0.8141} &
\multicolumn{1}{c|}{0.8125} &
\multicolumn{1}{c|}{0.8013} &
\multicolumn{1}{c|}{0.8016} \\ \cmidrule(l){2-11}
\multicolumn{1}{|c|}{} &
\multicolumn{1}{c|}{$\sigma$} &
\multicolumn{1}{c|}{0.0004} &
\multicolumn{1}{c|}{0.0004} &
\multicolumn{1}{c|}{0.0022} &
\multicolumn{1}{c|}{0.0050} &
\multicolumn{1}{c|}{0.0140} &
\multicolumn{1}{c|}{0.0095} &
\multicolumn{1}{c|}{0.0117} &
\multicolumn{1}{c|}{0.0140} &
\multicolumn{1}{c|}{0.0046} \\ \midrule
\multicolumn{1}{|c|}{\multirow{2}{*}{refer7500}} &
\multicolumn{1}{c|}{mean} &
\multicolumn{1}{c|}{0.9990} &
\multicolumn{1}{c|}{0.9975} &
\multicolumn{1}{c|}{0.9890} &
\multicolumn{1}{c|}{0.8893} &
\multicolumn{1}{c|}{0.7963} &
\multicolumn{1}{c|}{0.7895} &
\multicolumn{1}{c|}{0.7906} &
\multicolumn{1}{c|}{0.7812} &
\multicolumn{1}{c|}{0.7969} \\ \cmidrule(l){2-11}
\multicolumn{1}{|c|}{} &
\multicolumn{1}{c|}{$\sigma$} &
\multicolumn{1}{c|}{0.0005} &
\multicolumn{1}{c|}{0.0004} &
\multicolumn{1}{c|}{0.0034} &
\multicolumn{1}{c|}{0.0156} &
\multicolumn{1}{c|}{0.0206} &
\multicolumn{1}{c|}{0.0185} &
\multicolumn{1}{c|}{0.0223} &
\multicolumn{1}{c|}{0.0215} &
\multicolumn{1}{c|}{0.0079} \\ \midrule
\multicolumn{1}{|c|}{\multirow{2}{*}{refer10000}} &
\multicolumn{1}{c|}{mean} &
\multicolumn{1}{c|}{0.9988} &
\multicolumn{1}{c|}{0.9980} &
\multicolumn{1}{c|}{0.9887} &
\multicolumn{1}{c|}{0.8916} &
\multicolumn{1}{c|}{0.8044} &
\multicolumn{1}{c|}{0.7930} &
\multicolumn{1}{c|}{0.7940} &
\multicolumn{1}{c|}{0.7828} &
\multicolumn{1}{c|}{0.7943} \\ \cmidrule(l){2-11}
\multicolumn{1}{|c|}{} &
\multicolumn{1}{c|}{$\sigma$} &
\multicolumn{1}{c|}{0.0007} &
\multicolumn{1}{c|}{0.0005} &
\multicolumn{1}{c|}{0.0023} &
\multicolumn{1}{c|}{0.0077} &
\multicolumn{1}{c|}{0.0099} &
\multicolumn{1}{c|}{0.0143} &
\multicolumn{1}{c|}{0.0127} &
\multicolumn{1}{c|}{0.0122} &
\multicolumn{1}{c|}{0.0036} \\ \midrule
\multicolumn{1}{|c|}{\multirow{2}{*}{recon}} &
\multicolumn{1}{c|}{mean} &
\multicolumn{1}{c|}{0.9988} &
\multicolumn{1}{c|}{0.9963} &
\multicolumn{1}{c|}{0.9924} &
\multicolumn{1}{c|}{0.8616} &
\multicolumn{1}{c|}{0.7733} &
\multicolumn{1}{c|}{0.7653} &
\multicolumn{1}{c|}{0.7663} &
\multicolumn{1}{c|}{0.7565} &
\multicolumn{1}{c|}{0.9504} \\ \cmidrule(l){2-11}
\multicolumn{1}{|c|}{} &
\multicolumn{1}{c|}{$\sigma$} &
\multicolumn{1}{c|}{0.0012} &
\multicolumn{1}{c|}{0.0052} &
\multicolumn{1}{c|}{0.0350} &
\multicolumn{1}{c|}{0.1348} &
\multicolumn{1}{c|}{0.1589} &
\multicolumn{1}{c|}{0.1596} &
\multicolumn{1}{c|}{0.1581} &
\multicolumn{1}{c|}{0.1687} &
\multicolumn{1}{c|}{0.0356} \\ \bottomrule
\end{tabular}
\caption{Mean and standard deviation $\sigma$ of generalisation performance of language games on each other.}
\label{tab:lan_transfer_raw_data}
\end{table}
\begin{figure}[h!]
\centering
\begin{subfigure}[t]{\dimexpr0.45\textwidth+20pt\relax}
\makebox[20pt]{\raisebox{80pt}{\rotatebox[origin=c]{90}{recon}}}%
\includegraphics[width=\dimexpr\linewidth-20pt\relax]
{recon_msgvis_1.pdf}
\makebox[20pt]{\raisebox{80pt}{\rotatebox[origin=c]{90}{refer2}}}%
\includegraphics[width=\dimexpr\linewidth-20pt\relax]
{refer2_msgvis_1.pdf}
\makebox[20pt]{\raisebox{80pt}{\rotatebox[origin=c]{90}{refer1000}}}%
\includegraphics[width=\dimexpr\linewidth-20pt\relax]
{refer1000_msgvis_1.pdf}
\caption{Beginning}
\end{subfigure}
~
\begin{subfigure}[t]{0.45\textwidth}
\includegraphics[width=\textwidth]
{recon_msgvis_500.pdf}
\includegraphics[width=\textwidth]
{refer2_msgvis_500.pdf}
\includegraphics[width=\textwidth]
{refer1000_msgvis_500.pdf}
\caption{End}
\end{subfigure}
\caption{Visualisation of the distances between meanings of same messages by t-SNE.
The left column is the beginning of training, and the right column is the end of training.
The messages are given in the legend, and meanings of same message are plotted in the same colour.
}
\label{fig:msg_vis_by_tsne}
\end{figure}
\section{Message-meaning mappings before and after training}
\label{appsec:msg_meaning_maps}
Although we show the average distances between the meanings of top-10 degenerate components in different games in Figure~\ref{fig:result_avg_meaning_dis}, we here provide some further visualisation to illustrate it.
We first map all the meanings in $\mathcal{X}$ to a 2D space by t-SNE proposed by \cite{van2008visualizing}.
Then, for the meanings corresponding to the top-10 degenerate messages, we plot them with the same colour.
Limited to the space, we choose the most typical language games, i.e. `recon' (the only reconstruction language), `refer2' (most ambiguous one), and `refer1000' (most expressive referential language), and the results are shown in Figure~\ref{fig:msg_vis_by_tsne}.
The distances at the beginning and end of the training are given in the left and right column respectively.
By carefully observing the distances between the dots with same colour, we can see that they become closer to each other after the training procedure.
Among the three types of languages, number of dots in `refer2' is the most, which indicates that it is the most ambiguous language, and number of dots in `refer1000' is the least, which indicates that it is the most unambiguous language.
This also matches with the quantitative results we show in Table~\ref{tab:msg_num_dist}.
Furthermore, we can also see that the distances between the same coloured dots in `recon'-end is the least, which also matches our result shown in Figure~\ref{fig:result_avg_meaning_dis}.
% \section{Effectiveness of contrastive loss}
\section{Mutual information is insufficient}
\label{appsec:MI_is_insufficient}
In Section~\ref{ssec:define_expressivity}, we mentioned that MI cannot completely reflect the expressivity of an emergent language.
Here, we provide more detailed explanations.
On one hand, higher MI value does not necessarily mean more task-relevant information, e.g. encoding the colours in messages is unnecessary for tasks involving only the shapes of inputs although it provides more information.
On the other hand, not only the determinacy of the mappings matters for generalising them, but also the structures, e.g. it would be easier for humans to extrapolate compositional languages to novel concepts than holistic ones although they are both unambiguous.
% Therefore, MI is inappropriate for measuring the expressivity of an emergent language.
In the following paragraphs, we will give more reasoning and empirical evidence to support our claim.
Following the mapping function definition of emergent languages illustrated in Section~\ref{ssec:environment}, suppose that we get a message type collection $M\subset\mathcal{M}$ in some training epoch, the calculation of the mutual information between $M$ and $\mathcal{X}$ is given as follows:
\begin{equation}
\begin{aligned}
I(M|X) & = \sum_{x\in X} \sum_{m \in M} p(x,m) \log \frac{p(x,m)}{p(x)p(m)}
\stackrel{\text{p(x,m)=$\frac{1}{|\mathcal{X}|}$}}{=} \sum_{x\in \mathcal{X}} \sum_{m \in \mathcal{M'}} \frac{1}{|\mathcal{X}|} \log \frac{p(x,m)}{p(x)p(m)} \\
& = \frac{1}{|\mathcal{X}|} \sum_{x\in X} \sum_{m \in M} \log \frac{p(x,m)}{p(x)p(m)}
= \frac{1}{|\mathcal{X}|} \sum_{x\in X} \sum_{m \in M} \left(\log p(x,m) - \log p(x) - \log p(m) \right) \\
& \stackrel{\text{p(x)=$\frac{1}{|\mathcal{X}|}$}}{=} \frac{1}{N} \sum_{x\in X} \sum_{m \in M} \left(\log \frac{1}{|\mathcal{X}|} - \log \frac{1}{|\mathcal{X}|} - \log p(m) \right) = \frac{1}{|\mathcal{X}|} \sum_{x\in \mathcal{X}} \sum_{m \in M} \left(- \log p(m) \right) \\
& = - \sum_{m \in M} \log p(m)
= - \sum_{m \in M} \log \frac{f(m)}{|M|}
= - \sum_{m \in M} \left( \log f(m) - \log |M| \right) \\
& = \sum_{m \in M} \left( \log |M| - \log f(m) \right)
\end{aligned}
\end{equation}
where $p(x) = \frac{1}{|\mathcal{X}|}$ as $x$ follows a uniform distribution over the input space, and $p(x,m) = \frac{1}{|\mathcal{X}|}$ since every $x$ is mapped to a message.
By observing the above equation, it is straightforward to see that the MI value is decided by the number of message types and the degeneracy degree of the ambiguous messages.
Therefore, if two emergent languages have approximately the same number of messages types (as shown in Table~\ref{tab:msg_num_dist} and degeneracy degree, the MI values for these two languages should also be the same.
However, as we illustrate in Section~\ref{ssec:result_degenerate_degree} that degree of degeneracy cannot fully determine the expressivity of emergent languages, we can then deduce that MI cannot fully measure the expressivity of emergent languages.
To provide more empirical evidence to support our claim, we tracked the MI values over training epochs, and the results are shown in the following Figure~\ref{fig:MI_curves}.
As we can see, there is no statistically significant difference between the MI values of `referX' (where X$\geq 1,000$) while we can show that their expressivity is different in Section~\ref{ssec:exp_lan_transfer}.
\begin{figure}[h]
\centering
\includegraphics[width=\textwidth]{MI.pdf}
\caption{Mutual information curves over training epochs in different types of games. The converged values are provided in Table~\ref{fig:MI_values_last_epoch}}
\label{fig:MI_curves}
\end{figure}
\begin{table}[h]
\footnotesize
\begin{tabular}{|c|c|c|c|c|c|c|c|c|}
\hline
Game & refer10 & refer100 & refer1000 & refer2500 & refer5000 & refer7500 & refer10000 & recon \\ \hline
Mean & 78269.22 & 86823.66 & 89823.16 & 90447.69 & 90504.40 & 90378.89 & 90533.06 & 85982.79 \\ \hline
$\sigma$ & 3760.05 & 1464.69 & 443.12 & 213.76 & 412.37 & 306.85 & 139.42 & 3155.99 \\ \hline
\end{tabular}
\caption{Converged MI values of different types of games. Both mean and standard deviation ($\sigma$) are given.}
\label{fig:MI_values_last_epoch}
\end{table}
% \answerTODO{\textcolor{red}{Fully probabilistic version of MI}}
\section{Varying the capacity of channel and network}
To verify that our findings are robust to the capacity of the communication channel and agents, we also run some extra experiments on the following different configurations of referential games:
\begin{enumerate}
\item \emph{larger channel capacity}: set the length of messages to $8$ and size of token inventory to $20$, thus the size of message space becomes $20^8$;
\item \emph{larger agent capacity}: set the hidden layer size to $128$, thus the capacity of both speaker and listener become larger than the original setting ($256$);
\item \emph{larger channel\&agent capacity}: do the above two changes at the same time.
\end{enumerate}
Since our key findings are about the generalisation performance between recon1000, recon2500, and recon10000, we just run these extra experiments on a game set $\mathcal{G}_{extra}=\left\{\mbox{`recon1000'}, \mbox{`recon2500'}, \mbox{`recon10000'} \right\}$ with multiple runs.
The results of configuration 1, 2, and 3 are given in Figure~\ref{fig:results_on_larger_channel},~\ref{fig:results_on_larger_agent}, and~\ref{fig:results_on_both_large} respectively.
It can be observed in all figures that `recon1000' or `recon2500' would always perform better than `recon10000' on all kinds of target games, i.e. `recon1000' or `recon2500' always perform better than `recon10000' on all of the above configurations.
Therefore, we believe the key observation from Figure~\ref{fig:result_source_on_targets} hold for varying configurations of communication channel and capacity of agents, which means that our conclusions are robust to the configuration of language games.
\begin{figure}[h]
\centering
\includegraphics[width=0.8\textwidth]{acc_from_targets_large_channel.pdf}
\caption{Results of language transfer with larger channel capacity.
Error bars indicate the standard deviation of the mean values.}
\label{fig:results_on_larger_channel}
\end{figure}
\begin{figure}[h]
\centering
\includegraphics[width=0.8\textwidth]{acc_from_targets_large_agent.pdf}
\caption{Results of language transfer with larger agent capacity.
Error bars indicate the standard deviation of the mean values.}
\label{fig:results_on_larger_agent}
\end{figure}
\begin{figure}[h]
\centering
\includegraphics[width=0.8\textwidth]{acc_from_targets_both_large.pdf}
\caption{Results of language transfer with larger channel and agent capacity.
Error bars indicate the standard deviation of the mean values.}
\label{fig:results_on_both_large}
\end{figure}
\end{document}
\begin{thebibliography}{}
\bibitem[Chaabouni et~al., 2020]{chaabouni2020compositionality}
Chaabouni, R., Kharitonov, E., Bouchacourt, D., Dupoux, E., and Baroni, M.
(2020).
\newblock Compositionality and generalization in emergent languages.
\newblock {\em arXiv preprint arXiv:2004.09124}.
\bibitem[Chen et~al., 2020]{chen2020simple}
Chen, T., Kornblith, S., Norouzi, M., and Hinton, G. (2020).
\newblock A simple framework for contrastive learning of visual
representations.
\newblock In {\em International conference on machine learning}, pages
1597--1607. PMLR.
\bibitem[Evtimova et~al., 2018]{evtimova2017emergent}
Evtimova, K., Drozdov, A., Kiela, D., and Cho, K. (2018).
\newblock Emergent communication in a multi-modal, multi-step referential game.
\newblock In {\em International Conference on Learning Representations}.
\bibitem[Graesser et~al., 2019]{graesser2019emergent}
Graesser, L., Cho, K., and Kiela, D. (2019).
\newblock Emergent linguistic phenomena in multi-agent communication games.
\newblock {\em arXiv preprint arXiv:1901.08706}.
\bibitem[Guo et~al., 2020]{guo2019emergence}
Guo, S., Ren, Y., Havrylov, S., Frank, S., Titov, I., and Smith, K. (2020).
\newblock The emergence of compositional languages for numeric concepts through
iterated learning in neural agents.
\newblock {\em EvoLang}.
\bibitem[Hadsell et~al., 2006]{hadsell2006dimensionality}
Hadsell, R., Chopra, S., and LeCun, Y. (2006).
\newblock Dimensionality reduction by learning an invariant mapping.
\newblock In {\em 2006 IEEE Computer Society Conference on Computer Vision and
Pattern Recognition (CVPR'06)}, volume~2, pages 1735--1742. IEEE.
\bibitem[Havrylov and Titov, 2017]{ivan2017nips}
Havrylov, S. and Titov, I. (2017).
\newblock Emergence of language with multi-agent games: Learning to communicate
with sequences of symbols.
\newblock In {\em Advances in Neural Information Processing Systems}.
\bibitem[Hochreiter and Schmidhuber, 1997]{hochreiter1997long}
Hochreiter, S. and Schmidhuber, J. (1997).
\newblock Long short-term memory.
\newblock {\em Neural computation}, 9(8):1735--1780.
\bibitem[Jang et~al., 2017]{gumbel}
Jang, E., Gu, S., and Poole, B. (2017).
\newblock Categorical reparameterization with gumbel-softmax.
\bibitem[Kaji{\'c} et~al., 2020]{kajic2020learning}
Kaji{\'c}, I., Ayg{\"u}n, E., and Precup, D. (2020).
\newblock Learning to cooperate: Emergent communication in multi-agent
navigation.
\newblock {\em arXiv preprint arXiv:2004.01097}.
\bibitem[Kharitonov et~al., 2019]{Kharitonov2019Egg}
Kharitonov, E., Chaabouni, R., Bouchacourt, D., and Baroni, M. (2019).
\newblock {EGG}: a toolkit for research on {E}mergence of lan{G}uage in
{G}ames.
\newblock In {\em Proceedings of the 2019 Conference on Empirical Methods in
Natural Language Processing and the 9th International Joint Conference on
Natural Language Processing (EMNLP-IJCNLP): System Demonstrations}.
Association for Computational Linguistics.
\bibitem[Kharitonov et~al., 2020]{kharitonov2020entropy}
Kharitonov, E., Chaabouni, R., Bouchacourt, D., and Baroni, M. (2020).
\newblock Entropy minimization in emergent languages.
\newblock In {\em International Conference on Machine Learning}, pages
5220--5230. PMLR.
\bibitem[Kingma and Ba, 2014]{kingma2014adam}
Kingma, D.~P. and Ba, J. (2014).
\newblock Adam: A method for stochastic optimization.
\newblock {\em arXiv preprint arXiv:1412.6980}.
\bibitem[Kirby, 2001]{kirby2001spontaneous}
Kirby, S. (2001).
\newblock Spontaneous evolution of linguistic structure-an iterated learning
model of the emergence of regularity and irregularity.
\newblock {\em IEEE Transactions on Evolutionary Computation}, 5(2):102--110.
\bibitem[Kirby et~al., 2014]{kirby2014iterated}
Kirby, S., Griffiths, T., and Smith, K. (2014).
\newblock Iterated learning and the evolution of language.
\newblock {\em Current opinion in neurobiology}, 28:108--114.
\bibitem[Kirby and Hurford, 2002]{kirby2002emergence}
Kirby, S. and Hurford, J.~R. (2002).
\newblock The emergence of linguistic structure: An overview of the iterated
learning model.
\newblock In {\em Simulating the evolution of language}, pages 121--147.
Springer.
\bibitem[Kirby et~al., 2015]{kirby2015compression}
Kirby, S., Tamariz, M., Cornish, H., and Smith, K. (2015).
\newblock Compression and communication in the cultural evolution of linguistic
structure.
\newblock {\em Cognition}, 141:87--102.
\bibitem[Krizhevsky et~al., 2012]{krizhevsky2012imagenet}
Krizhevsky, A., Sutskever, I., and Hinton, G.~E. (2012).
\newblock Imagenet classification with deep convolutional neural networks.
\newblock {\em Advances in neural information processing systems},
25:1097--1105.
\bibitem[Lazaridou et~al., 2018]{lazaridou2018emergence}
Lazaridou, A., Hermann, K.~M., Tuyls, K., and Clark, S. (2018).
\newblock Emergence of linguistic communication from referential games with
symbolic and pixel input.
\newblock {\em arXiv preprint arXiv:1804.03984}.
\bibitem[Lazaridou et~al., 2017]{lazaridou2017multi}
Lazaridou, A., Peysakhovich, A., and Baroni, M. (2017).
\newblock Multi-agent cooperation and the emergence of (natural) language.
\newblock In {\em International Conference on Learning Representations}.
\bibitem[LeCun et~al., 2015]{lecun2015deep}
LeCun, Y., Bengio, Y., and Hinton, G. (2015).
\newblock Deep learning.
\newblock {\em nature}, 521(7553):436--444.
\bibitem[Lee et~al., 2018]{lee2017emergent}
Lee, J., Cho, K., Weston, J., and Kiela, D. (2018).
\newblock Emergent translation in multi-agent communication.
\newblock In {\em International Conference on Learning Representations}.
\bibitem[Lewis, 1969]{Lewis_refgame}
Lewis, D. (1969).
\newblock {\em Convention: A philosophical study}.
\newblock John Wiley \& Sons.
\bibitem[Li and Bowling, 2019]{li2019ease}
Li, F. and Bowling, M. (2019).
\newblock Ease-of-teaching and language structure from emergent communication.
\newblock {\em arXiv preprint arXiv:1906.02403}.
\bibitem[Lowe et~al., 2017]{lowe2017multi}
Lowe, R., Wu, Y., Tamar, A., Harb, J., Abbeel, P., and Mordatch, I. (2017).
\newblock Multi-agent actor-critic for mixed cooperative-competitive
environments.
\newblock {\em arXiv preprint arXiv:1706.02275}.
\bibitem[Mikolov et~al., 2013]{mikolov2013efficient}
Mikolov, T., Chen, K., Corrado, G., and Dean, J. (2013).
\newblock Efficient estimation of word representations in vector space.
\newblock {\em arXiv preprint arXiv:1301.3781}.
\bibitem[Pan and Yang, 2009]{pan2009survey}
Pan, S.~J. and Yang, Q. (2009).
\newblock A survey on transfer learning.
\newblock {\em IEEE Transactions on knowledge and data engineering},
22(10):1345--1359.
\bibitem[Ren et~al., 2020]{ren2020compositional}
Ren, Y., Guo, S., Labeau, M., Cohen, S.~B., and Kirby, S. (2020).
\newblock Compositional languages emerge in a neural iterated learning model.
\newblock {\em ICLR}.
\bibitem[Silver et~al., 2016]{silver2016mastering}
Silver, D., Huang, A., Maddison, C.~J., Guez, A., Sifre, L., Van Den~Driessche,
G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M.,
et~al. (2016).
\newblock Mastering the game of go with deep neural networks and tree search.
\newblock {\em nature}, 529(7587):484--489.
\bibitem[Van~der Maaten and Hinton, 2008]{van2008visualizing}
Van~der Maaten, L. and Hinton, G. (2008).
\newblock Visualizing data using t-sne.
\newblock {\em Journal of machine learning research}, 9(11).
\bibitem[Winters et~al., 2018]{winters2018contextual}
Winters, J., Kirby, S., and Smith, K. (2018).
\newblock Contextual predictability shapes signal autonomy.
\newblock {\em Cognition}, 176:15--30.
\bibitem[Wittgenstein, 1954]{wittgenstein2009philosophical}
Wittgenstein, L. (1954).
\newblock {\em Philosophical investigations}.
\newblock John Wiley \& Sons.
\bibitem[Wray and Grace, 2007]{wray2007consequences}
Wray, A. and Grace, G.~W. (2007).
\newblock The consequences of talking to strangers: Evolutionary corollaries of
socio-cultural influences on linguistic form.
\newblock {\em Lingua}, 117(3):543--578.
\bibitem[Yuan et~al., 2020]{yuan2020emergence}
Yuan, L., Fu, Z., Shen, J., Xu, L., Shen, J., and Zhu, S.-C. (2020).
\newblock Emergence of pragmatics from referential game between theory of mind
agents.
\newblock {\em arXiv preprint arXiv:2001.07752}.
\bibitem[Zhang et~al., 2019]{zhang2019efficient}
Zhang, S.~Q., Zhang, Q., and Lin, J. (2019).
\newblock Efficient communication in multi-agent reinforcement learning via
variance based control.
\newblock {\em arXiv preprint arXiv:1909.02682}.
\end{thebibliography}
\begin{thebibliography}{}
\bibitem[Chaabouni et~al., 2020]{chaabouni2020compositionality}
Chaabouni, R., Kharitonov, E., Bouchacourt, D., Dupoux, E., and Baroni, M.
(2020).
\newblock Compositionality and generalization in emergent languages.
\newblock {\em arXiv preprint arXiv:2004.09124}.
\bibitem[Chen et~al., 2020]{chen2020simple}
Chen, T., Kornblith, S., Norouzi, M., and Hinton, G. (2020).
\newblock A simple framework for contrastive learning of visual
representations.
\newblock In {\em International conference on machine learning}, pages
1597--1607. PMLR.
\bibitem[Evtimova et~al., 2018]{evtimova2017emergent}
Evtimova, K., Drozdov, A., Kiela, D., and Cho, K. (2018).
\newblock Emergent communication in a multi-modal, multi-step referential game.
\newblock In {\em International Conference on Learning Representations}.
\bibitem[Graesser et~al., 2019]{graesser2019emergent}
Graesser, L., Cho, K., and Kiela, D. (2019).
\newblock Emergent linguistic phenomena in multi-agent communication games.
\newblock {\em arXiv preprint arXiv:1901.08706}.
\bibitem[Guo et~al., 2020]{guo2019emergence}
Guo, S., Ren, Y., Havrylov, S., Frank, S., Titov, I., and Smith, K. (2020).
\newblock The emergence of compositional languages for numeric concepts through
iterated learning in neural agents.
\newblock {\em EvoLang}.
\bibitem[Hadsell et~al., 2006]{hadsell2006dimensionality}
Hadsell, R., Chopra, S., and LeCun, Y. (2006).
\newblock Dimensionality reduction by learning an invariant mapping.
\newblock In {\em 2006 IEEE Computer Society Conference on Computer Vision and
Pattern Recognition (CVPR'06)}, volume~2, pages 1735--1742. IEEE.
\bibitem[Havrylov and Titov, 2017]{ivan2017nips}
Havrylov, S. and Titov, I. (2017).
\newblock Emergence of language with multi-agent games: Learning to communicate
with sequences of symbols.
\newblock In {\em Advances in Neural Information Processing Systems}.
\bibitem[Hochreiter and Schmidhuber, 1997]{hochreiter1997long}
Hochreiter, S. and Schmidhuber, J. (1997).
\newblock Long short-term memory.
\newblock {\em Neural computation}, 9(8):1735--1780.
\bibitem[Jang et~al., 2017]{gumbel}
Jang, E., Gu, S., and Poole, B. (2017).
\newblock Categorical reparameterization with gumbel-softmax.
\bibitem[Kaji{\'c} et~al., 2020]{kajic2020learning}
Kaji{\'c}, I., Ayg{\"u}n, E., and Precup, D. (2020).
\newblock Learning to cooperate: Emergent communication in multi-agent
navigation.
\newblock {\em arXiv preprint arXiv:2004.01097}.
\bibitem[Kharitonov et~al., 2019]{Kharitonov2019Egg}
Kharitonov, E., Chaabouni, R., Bouchacourt, D., and Baroni, M. (2019).
\newblock {EGG}: a toolkit for research on {E}mergence of lan{G}uage in
{G}ames.
\newblock In {\em Proceedings of the 2019 Conference on Empirical Methods in
Natural Language Processing and the 9th International Joint Conference on
Natural Language Processing (EMNLP-IJCNLP): System Demonstrations}.
Association for Computational Linguistics.
\bibitem[Kharitonov et~al., 2020]{kharitonov2020entropy}
Kharitonov, E., Chaabouni, R., Bouchacourt, D., and Baroni, M. (2020).
\newblock Entropy minimization in emergent languages.
\newblock In {\em International Conference on Machine Learning}, pages
5220--5230. PMLR.
\bibitem[Kingma and Ba, 2014]{kingma2014adam}
Kingma, D.~P. and Ba, J. (2014).
\newblock Adam: A method for stochastic optimization.
\newblock {\em arXiv preprint arXiv:1412.6980}.
\bibitem[Kirby, 2001]{kirby2001spontaneous}
Kirby, S. (2001).
\newblock Spontaneous evolution of linguistic structure-an iterated learning
model of the emergence of regularity and irregularity.
\newblock {\em IEEE Transactions on Evolutionary Computation}, 5(2):102--110.
\bibitem[Kirby et~al., 2014]{kirby2014iterated}
Kirby, S., Griffiths, T., and Smith, K. (2014).
\newblock Iterated learning and the evolution of language.
\newblock {\em Current opinion in neurobiology}, 28:108--114.
\bibitem[Kirby and Hurford, 2002]{kirby2002emergence}
Kirby, S. and Hurford, J.~R. (2002).
\newblock The emergence of linguistic structure: An overview of the iterated
learning model.
\newblock In {\em Simulating the evolution of language}, pages 121--147.
Springer.
\bibitem[Kirby et~al., 2015]{kirby2015compression}
Kirby, S., Tamariz, M., Cornish, H., and Smith, K. (2015).
\newblock Compression and communication in the cultural evolution of linguistic
structure.
\newblock {\em Cognition}, 141:87--102.
\bibitem[Krizhevsky et~al., 2012]{krizhevsky2012imagenet}
Krizhevsky, A., Sutskever, I., and Hinton, G.~E. (2012).
\newblock Imagenet classification with deep convolutional neural networks.
\newblock {\em Advances in neural information processing systems},
25:1097--1105.
\bibitem[Lazaridou et~al., 2018]{lazaridou2018emergence}
Lazaridou, A., Hermann, K.~M., Tuyls, K., and Clark, S. (2018).
\newblock Emergence of linguistic communication from referential games with
symbolic and pixel input.
\newblock {\em arXiv preprint arXiv:1804.03984}.
\bibitem[Lazaridou et~al., 2017]{lazaridou2017multi}
Lazaridou, A., Peysakhovich, A., and Baroni, M. (2017).
\newblock Multi-agent cooperation and the emergence of (natural) language.
\newblock In {\em International Conference on Learning Representations}.
\bibitem[LeCun et~al., 2015]{lecun2015deep}
LeCun, Y., Bengio, Y., and Hinton, G. (2015).
\newblock Deep learning.
\newblock {\em nature}, 521(7553):436--444.
\bibitem[Lee et~al., 2018]{lee2017emergent}
Lee, J., Cho, K., Weston, J., and Kiela, D. (2018).
\newblock Emergent translation in multi-agent communication.
\newblock In {\em International Conference on Learning Representations}.
\bibitem[Lewis, 1969]{Lewis_refgame}
Lewis, D. (1969).
\newblock {\em Convention: A philosophical study}.
\newblock John Wiley \& Sons.
\bibitem[Li and Bowling, 2019]{li2019ease}
Li, F. and Bowling, M. (2019).
\newblock Ease-of-teaching and language structure from emergent communication.
\newblock {\em arXiv preprint arXiv:1906.02403}.
\bibitem[Lowe et~al., 2017]{lowe2017multi}
Lowe, R., Wu, Y., Tamar, A., Harb, J., Abbeel, P., and Mordatch, I. (2017).
\newblock Multi-agent actor-critic for mixed cooperative-competitive
environments.
\newblock {\em arXiv preprint arXiv:1706.02275}.
\bibitem[Mikolov et~al., 2013]{mikolov2013efficient}
Mikolov, T., Chen, K., Corrado, G., and Dean, J. (2013).
\newblock Efficient estimation of word representations in vector space.
\newblock {\em arXiv preprint arXiv:1301.3781}.
\bibitem[Pan and Yang, 2009]{pan2009survey}
Pan, S.~J. and Yang, Q. (2009).
\newblock A survey on transfer learning.
\newblock {\em IEEE Transactions on knowledge and data engineering},
22(10):1345--1359.
\bibitem[Ren et~al., 2020]{ren2020compositional}
Ren, Y., Guo, S., Labeau, M., Cohen, S.~B., and Kirby, S. (2020).
\newblock Compositional languages emerge in a neural iterated learning model.
\newblock {\em ICLR}.
\bibitem[Silver et~al., 2016]{silver2016mastering}
Silver, D., Huang, A., Maddison, C.~J., Guez, A., Sifre, L., Van Den~Driessche,
G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M.,
et~al. (2016).
\newblock Mastering the game of go with deep neural networks and tree search.
\newblock {\em nature}, 529(7587):484--489.
\bibitem[Van~der Maaten and Hinton, 2008]{van2008visualizing}
Van~der Maaten, L. and Hinton, G. (2008).
\newblock Visualizing data using t-sne.
\newblock {\em Journal of machine learning research}, 9(11).
\bibitem[Winters et~al., 2018]{winters2018contextual}
Winters, J., Kirby, S., and Smith, K. (2018).
\newblock Contextual predictability shapes signal autonomy.
\newblock {\em Cognition}, 176:15--30.
\bibitem[Wittgenstein, 1954]{wittgenstein2009philosophical}
Wittgenstein, L. (1954).
\newblock {\em Philosophical investigations}.
\newblock John Wiley \& Sons.
\bibitem[Wray and Grace, 2007]{wray2007consequences}
Wray, A. and Grace, G.~W. (2007).
\newblock The consequences of talking to strangers: Evolutionary corollaries of
socio-cultural influences on linguistic form.
\newblock {\em Lingua}, 117(3):543--578.
\bibitem[Yuan et~al., 2020]{yuan2020emergence}
Yuan, L., Fu, Z., Shen, J., Xu, L., Shen, J., and Zhu, S.-C. (2020).
\newblock Emergence of pragmatics from referential game between theory of mind
agents.
\newblock {\em arXiv preprint arXiv:2001.07752}.
\bibitem[Zhang et~al., 2019]{zhang2019efficient}
Zhang, S.~Q., Zhang, Q., and Lin, J. (2019).
\newblock Efficient communication in multi-agent reinforcement learning via
variance based control.
\newblock {\em arXiv preprint arXiv:1909.02682}.
\end{thebibliography}