Transformers Can Do Bayesian Inference
Extracted source text · arXiv 2112.10510
\documentclass[dvipsnames]{article} % For LaTeX2e
\usepackage{iclr2022_conference,times}
\usepackage[ruled,vlined]{algorithm2e}
\usepackage{amsmath}
\usepackage{tikz}
\usetikzlibrary{shapes,decorations,arrows,calc,arrows.meta,fit,positioning}
\usetikzlibrary{positioning,calc}
\usetikzlibrary{backgrounds,fit}
\usetikzlibrary{arrows.meta}
\usetikzlibrary{arrows}
\usetikzlibrary{decorations.pathreplacing,calligraphy}
\usepackage{microtype} % microtypography
\usepackage{booktabs}
\usepackage{amssymb}
\usepackage{array}
\newcolumntype{H}{>{\setbox0=\hbox\bgroup}c<{\egroup}@{}}
\usepackage{xcolor}
\usepackage{multirow}
\usepackage{listings}
%\usepackage{todonotes}
\newcommand{\numsamples}{n}
%\newcommand{\frank}[1]{\todo[inline,color=yellow]{Frank: #1}}
%\newcommand{\frankout}[1]{\todo[color=yellow]{Frank: #1}}
%\newcommand{\sam}[1]{\todo[inline,color=orange]{Sam: #1}}
%\newcommand{\noah}[1]{\todo[inline,color=orange]{Noah: #1}}
%\newcommand{\sebastian}[1]{\todo[inline,color=orange]{Sebastian: #1}}
%\newcommand{\sebastianout}[1]{\todo[color=orange]{Sebastian: #1}}
%\newcommand\todo[1]{\textcolor{red}{#1}}
%\newcommand\note[1]{\textcolor{blue}{#1}}
% Optional math commands from https://github.com/goodfeli/dlbook_notation.
\input{math_commands.tex}
\usepackage{hyperref}
\usepackage{url}
%\usepackage[dvipsnames]{xcolor}
\usepackage{pgfplots}
\pgfplotsset{
compat=1.15,
BarStyle/.style={
ybar interval,
fill=white!80!black,draw=black
},
every axis/.append style={label style={font=\tiny},
tick label style={font=\tiny},
legend style={fill opacity=0.8, draw opacity=1, text opacity=1, draw=white!80!black, font=\tiny},
},
}
%\tikzstyle{every node}=[font=\tiny]
\usepackage{caption}
\usepackage{subcaption}
\captionsetup[subfigure]{font={bf,small}, skip=1pt, singlelinecheck=false}
\usepackage{amsthm}
\newtheorem{theorem}{Theorem}
\newtheorem{insight}{Insight}
\newtheorem{corollary}{Corollary}[insight]
\title{Transformers Can Do Bayesian Inference}%\\ in a single Forward Propagation}
% Put Deep Learning and maybe even Transformers in the Title, Bayesian Inference instead of Posterior Approximation
% Authors must not appear in the submitted version. They should be hidden
% as long as the \iclrfinalcopy macro remains commented out below.
% Non-anonymous submissions will be rejected without review.
%%%%%% TODO %%%%%%
% change riemann distr. to piece-wise constant distr.
\author{%
Samuel M\"uller \textsuperscript{1}~\; Noah Hollmann \textsuperscript{2}~\; Sebastian Pineda \textsuperscript{1}~\; Josif Grabocka \textsuperscript{1}~\; Frank Hutter \textsuperscript{1,3}\\
\textsuperscript{1} University of Freiburg, \textsuperscript{2} Charit\'e Berlin, \textsuperscript{3} Bosch Center for Artificial Intelligence\\
{Correspondence to Samuel M\"uller: \texttt{muellesa@cs.uni-freiburg.de}}}
% For a paper whose authors are all at the same institution,
% omit the following lines up until the closing ``}''.
% Additional authors and addresses can be added with ``\and'',
% just like the second author.
% To save space, use either the email address or home page, not both
\iffalse
\author{%
Samuel M\"uller\\
University of Freiburg
%\texttt{muellesa@cs.uni-freiburg.de}
\And
Noah Hollmann\\
Charit\'e Berlin
%{\texttt noah.hollmann@charite.de}
\And
Sebastian Pineda Arango\\
University of Freiburg
%{\texttt pineda@cs.uni-freiburg.de}
\And
Josif Grabocka\\
University of Freiburg
%{\texttt grabocka@informatik.uni-freiburg.de}
\And
Frank Hutter\\
University of Freiburg \& Bosch Center for AI
%{\texttt fh@cs.uni-freiburg.de}\\
}
\fi
% examples of more authors
% \And
% Coauthor \\
% Affiliation \\
% Address \\
% \texttt{email} \\
% \AND
% Coauthor \\
% Affiliation \\
% Address \\
% \texttt{email} \\
% \And
% Coauthor \\
% Affiliation \\
% Address \\
% \texttt{email} \\
% \And
% Coauthor \\
% Affiliation \\
% Address \\
% \texttt{email} \\
% The \author macro works with any number of authors. There are two commands
% used to separate the names and addresses of multiple authors: \And and \AND.
%
% Using \And between authors leaves it to \LaTeX{} to determine where to break
% the lines. Using \AND forces a linebreak at that point. So, if \LaTeX{}
% puts 3 of 4 authors names on the first line, and the last on the second
% line, try using \AND instead of \And before the third author name.
\newcommand{\fix}{\marginpar{FIX}}
\newcommand{\new}{\marginpar{NEW}}
\newcommand{\indep}{\perp \!\!\! \perp}
\newcommand{\newmathterm}[1]{\textcolor{PineGreen}{#1}}
\newcommand{\xy}{$(x,y)$}
\iclrfinalcopy % Uncomment for camera-ready version, but NOT for submission.
\begin{document}
\maketitle
\newcommand{\priordatafitting}{PF}
\newcommand{\priorfittednetworks}{PFNs}
\newcommand{\priorfittednetwork}{PFN}
\begin{abstract}
Currently, it is hard to reap the benefits of deep learning for Bayesian methods, which allow the explicit specification of prior knowledge and accurately capture model uncertainty.
We present \emph{Prior-Data Fitted Networks (\priorfittednetworks{})}. \priorfittednetworks{} leverage large-scale machine learning techniques to approximate a large set of posteriors.
The only requirement for \priorfittednetworks{} to work is the ability to sample from a prior distribution over supervised learning tasks (or functions).
Our method restates the objective of posterior approximation as a supervised classification problem with a set-valued input:
it repeatedly draws a task (or function) from the prior, draws a set of data points and their labels from it, masks one of the labels and learns to make probabilistic predictions for it based on the set-valued input of the rest of the data points.
Presented with a set of samples from a new supervised learning task as input, \priorfittednetworks{} make probabilistic predictions for arbitrary other data points in a single forward propagation, having learned to approximate Bayesian inference.
We demonstrate that \priorfittednetworks{} can near-perfectly mimic Gaussian processes and also enable efficient Bayesian inference for intractable problems, with over 200-fold speedups in multiple setups compared to current
%Bayesian inference
methods.
We obtain strong results in very diverse areas such as Gaussian process regression, Bayesian neural networks,
classification for small tabular data sets, and few-shot image classification, demonstrating the generality of \priorfittednetworks{}.
Code and trained \priorfittednetworks{} are released at \url{https://github.com/automl/TransformersCanDoBayesianInference}.
%and Bayesian optimization.
%, alongside studies showing the superior posterior approximation ability of our method on prior data.
%but also compute posteriors
%and thereby presents an alternative to current methods for Bayesian inference.
%Our method is able to gain over 200-fold speedups in multiple setups compared to current Bayesian inference methods.
%Additionally, we show our method\textquotesingle{s} ability to make use of meta-datasets (sets of datasets), to further improve its performance by transferring knowledge from similar datasets.
\end{abstract}
\iffalse
\begin{figure}[b]
\centering
\input{curves}
\caption{This plot visualizes a set of functions sampled from a prior. Given this batch of functions, the objective of our model is to predict $y$ for the point $(x, ?)$ given other pairs of points $\{(x_1, y_1),\dots,(x_5, y_5)\}$.}
\label{fig:curves}
\end{figure}
\fi
\newcommand{\ppd}{PPD}
\section{Introduction}
In the last decade, supervised machine learning (ML) methods using deep learning architectures have made substantial progress on machine learning tasks where a large amount of training data is available \citep{vaswani-neurips17,he2016resnet,alexnet}.
A very important problem in ML is thus to transfer these successes to smaller-scale setups with less data available.
In this paper, we propose a way to build models that approximate posteriors with flexible and replaceable priors using deep learning models. It makes specifying a prior as simple as defining a sampling scheme of supervised learning tasks.
%in the prior.
%\frankout{We sometimes refer to the number of samples as $n$, sometimes as $K$ (e.g., in Figure 1), and in the code as numsamples. Sam: numsamples vs K is inconsistent, but otherwise I tried to stick to $n$ is the number of samples in a dataset, $m$ the validation samples and $N=n+m$.}
\begin{figure}[h]
\centering
\input{algovis}
\caption{A visualization of Prior-Data Fitted Networks (\priorfittednetworks{}). We sample %(meta-train)
datasets from a prior and fit a PFN on hold-out examples of these datasets. Given an actual %(meta-test)
dataset, we feed it and a test point to the PFN and obtain an approximation to Bayesian inference in a single forward propagation.}
\label{fig:algovis}
\end{figure}
While the success of deep learning on large datasets can be attributed to the capacity of neural networks to approximate any function, there is still a need for encoding prior knowledge, for example through model architecture (e.g., Convolutional Neural Networks \citep{convnet}) or regularizers (e.g., data augmentations \citep{hendrycks2019augmix,cubuk2020randaugment}).
Otherwise, the no free lunch theorems show that there are no good methods to solve the class of prediction problems \citep{wolpert1997nofreelunch}.
Thus, a large number of specialized algorithms have been developed for different small-scale tasks \citep{convnet,kadra2021regularization,xgboost}.
Encoding prior information into a machine learning model can, however, be challenging.
A well-defined way to bias a model is to use Bayesian inference.
The foundation of Bayesian inference is an assumption on the distribution of the data to appear in a real-world application. This assumption gives rise to a prior belief over the probability for the data to follow a particular model.
One might, e.g., implement a prior, encoding the data as created by a neural network (Bayesian Neural Networks, \citep{mackay1992practical}), by a polynomial, the likelihood of a Gaussian mixture or random code in a pre-defined programming language \citep{SOLOMONOFF199773}.
Using Bayesian inference for prediction in supervised learning has the advantages that (i) it has a theoretical foundation that makes it valid in setups where the prior $p(t)$ fits; (ii) it can thus better account for the actual likelihood of different events; (iii) it is well calibrated and (iv) it is interpretable as the prior describes the expectations of the model.
However, retrieving the posterior predictive distribution for a given prior is intractable in most cases~\citep{Blei_2017, mackay1992practical}.
Figure \ref{fig:algovis} outlines Prior-Data Fitted Networks (\priorfittednetworks{}), for approximating such Bayesian models.
We assume a given representative prior distribution over supervised learning tasks (or functions), which provides our inductive bias. To train a PFN, we use supervised learning with set-valued inputs representing entire datasets: we repeatedly sample a meta-train task (or function) from the given prior, draw a set of data points and their labels from it, mask one of the labels and learn to make probabilistic predictions for it based on the set-valued input of the rest of the data points.
%
Given an actual real-world dataset, we feed it together with a test point as inputs to the \priorfittednetwork{} and that outputs its prediction distribution for the test point, conditioned on the dataset. As we will demonstrate, this distribution approximates exact Bayesian posterior prediction. We refer to this step as (Bayesian) inference, as opposed to the training of the \priorfittednetwork{} itself.
%\sam{I separate here into training of the Transformer and (Bayesian) inference on a particular dataset. The latter is called training for some of our baselines, like BNNs. We should probably try to stick to this distinction and refer to Bayesian Inference for a given dataset.}
%We learn to fit new samples given other samples from the sampled datasets, that is we meta-learn on the sampled datasets. At inference time the resulting Prior-data Fitted Networks (\priorfittednetworks{}) are simply given a real-world dataset instead of a sampled dataset. The training procedure leads the \priorfittednetworks{} to do Bayesian Inference and and the output of it yields an approximation of the posterior predictive distribution.
Our \priorfittednetworks{} thus allow us to approximate the posterior predictive distribution for \emph{any} prior from which we are able to sample data.
This is a very weak requirement compared to the standard assumptions of other approximations for Bayesian inference~\citep{hoffman2014no,svi2013hoffman,jordan1999introduction}. %that unnormalized posterior probabilities can be computed, which is made by currently common methods, such as Markov Chain Monte Carlo (MCMC)~\citep{neal-bayes96a,andrieu2003introduction,welling2011bayesian,hoffman2014no}
%and Variational Inference (VI) \citep{jordan1999introduction,wainwright2008graphical,svi2013hoffman}.
This allows a simple approximation of a large set of priors, including priors that are very hard to approximate with currently available tools.
We make the following contributions:
\begin{itemize}
%\item We introduce Prior-Data Fitted Networks (\priorfittednetworks{});
\item We present architectural changes to successfully use Transformers for posterior predictive distribution (PPD) approximation, including a novel predictive distribution for regression tasks. The proposed method is simple, cheap and generally applicable to a large set of priors.
%\item We present a novel way to evaluate PPD approximations without the need of biases through the selection of datasets; \frank{I don't understand why this is one of the contributions; I'd drop it}
% \item We show that via meta-learning on data sets sampled from a prior, \priorfittednetworks{} yield an approximation technique for the PPD ;
\item We demonstrate that \priorfittednetworks{} can approximate the PPD of Gaussian processes and Bayesian neural networks (BNNs) orders of magnitude faster than MCMC with NUTS or SVI with Bayes-by-Backprop \citep{blundell-icml15a}.
\item We demonstrate that PFNs can have an impact on real-world tasks. (i) We implement BNNs with priors over architectures on PFNs that enable tuning free predictions in a single forward pass and outperform all baselines on a large benchmark of small tabular datasets. (ii) Additionally, we show that simple handwriting priors enable few-shot learning on Omniglot \citep{lake2015omniglot}.
% and apply simple handwriting priors, as well as their successful application to few-shot learning and classification for small tabular data.
%and Bayesian Optimization for Hyperparameter Optimization.
\end{itemize}
\newcommand{\dataset}{\mathcal{D}}
\newcommand{\priordatadistribution}{PD}
\newcommand{\bayesnn}{BNN}
\section{Background}
This paper concerns the use of a large amount of data to later yield strong performance on tasks for which only small datasets are available.
In multiple machine learning setups, one can see successes based on this approach, such as fine-tuning unsupervised models on image data \citep{chen2021simsiam,chen2020simclr} or large language corpora \citep{devlin2018bert}.
Another way to transfer knowledge from large corpora is to design prompts for language models~\citep{gpt-j,gpt-3}.
While we use a large amount of data during training, unlike the other approaches, all the data we use is generated artificially.
In meta-learning, or learning-to-learn~\citep{hochreiter2001learning,finn-icml17a}, one tries to generalize learning methods from a training set of datasets to a validation set of datasets.
Similar to our approach, there is a branch of research on building models that can learn to learn in a single forward pass \citep{santoro2016meta,hochreiter2001learning,gordon2018ml-pip}.
More recently (Conditional) Neural Processes \citep{garnelo2018CNP,garnelo2018NP,kim2018ANP} were proposed for meta-learning.
These methods focus on architectural constructions as stochastic processes.
The difference, besides the dissimilar model architectures and dissimilar prediction distributions, to previous meta-learning work is that we set up learning problems from which we sample artificial, on-the-fly-generated datasets such that we can show that our method does not only learn to learn, but that it learns to perform Bayesian inference.
In this work, we are interested in Bayesian posterior prediction for supervised learning problems, i.e., we model the output $y$ for a new input $x$ based on a supervised dataset of arbitrary size $n$, $\dataset{} = \{(x_i,y_i)\}_{i=1}^n$, where $y_i$ is the \textit{output} for $x_i$.
%Generally, we consider predictions based on datasets of arbitrary size $n$, $\dataset{} = \{(x_i,y_i)\}_{i=1}^n$, where $y_i$ is the \textit{output} for $x_i$.
To address the supervised learning problem in a Bayesian way, we consider a prior $p(t)$ over the latent variable $t$, the task.
We can now use Bayes\textquotesingle{} Theorem to define the posterior $p(t|\dataset{})$.
\iffalse
\begin{align}
p(t|\dataset{}) = \frac{p(\dataset{}|t)p(t)}{p(\dataset{})} = \frac{p(\dataset{}|t)p(t)}{\int_t p(\dataset{}|t)}.
\end{align}
\fi
The crucial distribution for prediction, the posterior predictive distribution (PPD), can then be inferred as
\vspace*{-0.1cm}
\begin{align}
p(y|x,\dataset{}) = \int_t p(y|x,t)p(t|\dataset{}). \label{ppd_formula}
\end{align}
\vspace*{-0.3cm}
% this is actually not true if x dependent on D, which is the case for few-shot learning for example, but I don't want to drag the x with all the time..
In some cases, one can arrive at the PPD $p(y|x,\dataset{})$ in closed form (e.g., for Gaussian Processes, \citep{gp2005rasmussen}), but in most cases the PPD can only be approximated.
Traditionally, one approximates the posterior $p(t|\dataset{})$ and based on this approximation infers an approximation of the PPD.
To approximate the posterior, there are two classes of prominent methods:
i) Markov Chain Monte Carlo (MCMC)~\citep{neal-bayes96a,andrieu2003introduction,welling2011bayesian} methods, like NUTS~\citep{hoffman2014no}, allow accurate but sometimes very slow approximations of the posterior; and~ii) Variational Inference (VI) \citep{jordan1999introduction,wainwright2008graphical,svi2013hoffman} methods approximate the posterior by a tractable distribution, such as a factorized normal distribution.
%is used for computational reasons.
%VI is often faster than MCMC but less accurate. %, it is often less accurate.
%In contrast to these methods, we learn to approximate the PPD directly. We do so by training a neural network on samples from the data-prior $p(\dataset{})$.
%For all cases we consider, it is simple and cheap to sample from $p(\dataset{})$, fulfilling the only requirement our method has. MCMC on the other hand requires access to a non-normalized posterior $f(t|D,x) \propto p(t|D,x)$, and VI requires access to $p(t,D)$, both of which can be hard to compute.
%Our method, e.g., allows the easy usage of physical priors built in a lab or the use of simulation software as prior, which is much harder to do for both MCMC and VI.
% THE NEW PART ON SIMULATION-BASED INF
One more related line of research is amortized simulation-based inference \citep{cranmer2020review,chan2018likelihood}, which was previously adapted to general probabilistic programs \citep{ritchie2016deep} and sequences \citep{ortega2019meta}.
In particular relevant to this work is neural posterior estimation \citep{lueckmann2021benchmarking,papamakarios2016fast}. The objective here is to find a model that approximates the posterior $p(t|X)$ by training on samples from $p(t,X)$, where $X$ can be a dataset $\dataset{}$ like above, but typically is a single vector.
%We, on the other hand, approximate the PPD in the supervised learning case, that is, we approximate $p(y|x,\mathcal{D})$ based on samples from $p(\mathcal{D})$.
Previous work in simulation-based inference is focused on simulations for which a specific model underlying the data is known \citep{lueckmann2021benchmarking}.
Our method is in so far similar to simulation-based inference as both solely use samples from the prior. In the next section, we will show how we can model the PPD directly, though, without ever instantiating the posterior, and apply this to large datasets with general priors.
\newcommand{\priornll}{Prior-Data NLL}
\vspace{-10pt}
\begin{algorithm}[b]
\SetAlgoLined
\SetKwInOut{Input}{Input}
\SetKwInOut{Output}{Output}
\Input{A prior distribution over datasets $p(\dataset{})$, from which samples can be drawn and the number of samples $K$ to draw}
\Output{A model $q_\theta$ that will approximate the PPD}
Initialize the neural network $q_\theta$\;
% $p(x_1,\dots,x_{n+m},y_1,\dots,y_{n+m})$\;
\For{$j\gets1$ \KwTo $K$}{
Sample $D \cup \{(x_i,y_i)\}_{i=1}^m\sim p(\dataset)$\; %$x_1,\dots,x_{n+m},y_1,\dots,y_{n+m} \sim p(\dataset{})$\;
Compute stochastic loss approximation $\bar{\ell}_\theta$ = $\sum_{i=1}^{m}(-\log %q_\theta(y_{n+i}|x_{n+i},x_1,\dots,x_{n},y_1,\dots,y_{n}))$\;
q_\theta(y_i|x_i, D))$\;
Update parameters $\theta$ with stochastic gradient descent on $\nabla_\theta \bar{\ell}_\theta$\;
}
\caption{Training a \priorfittednetwork{} model by Fitting Prior-Data}
\label{alg:prior-fitting}
\end{algorithm}
\section{PPD Approximation with \priorfittednetworks{}}
\label{sec:proof}
Let us consider a parameterized model $q_\theta$ that can accept a dataset $D = \{(x_i,y_i)\}_{ i=1}^n$, as well as a query $x$ as input, and which predicts a distribution over possible values of $y$ for the query $x$.
%, where $\theta$ are the model's parameters.
Many neural architectures can be used as such a model; in this paper, we use a variant of Transformers \citep{vaswani-neurips17} as they are a reliable and powerful architecture.
We train this model by cross-entropy over samples drawn from the prior.
Our proposed loss, the \emph{Prior-Data Negative Log-Likelihood (\priornll{})} $\ell_\theta$ is defined as
\begin{align}
%\ell_\theta &= -\int_{D,x,y,f} p(x,y|f) p(D|f) p(f) \log q_\theta(y|x,\dataset{}).
\ell_\theta &= \E_{D \cup \{x,y\} \sim p(\dataset{})}[-\log q_\theta(y|x,D)],
\end{align}
where $D \cup \{x,y\}$ simply is a dataset of size $|D|+1$ sampled from $p(\dataset)$.
\newcommand{\kl}{\textit{KL}}
\priorfittednetworks{} are trained to minimize $\ell_\theta$ like outlined in Algorithm \ref{alg:prior-fitting}:
We draw many datasets from our prior and fit our model to predict a hold-out example correctly for these.
In contrast to VI and MCMC, we learn to approximate the PPD directly from dataset samples.
For all cases we consider, it is simple and cheap to sample from $p(\dataset{})$, fulfilling the only requirement our method has. MCMC on the other hand requires access to a non-normalized posterior $f(t|D,x) \propto p(t|D,x)$, and VI requires access to the density values of $p(t,D)$, both of which can be hard to compute.
Extending standard loss derviations \cite{gordon2018ml-pip} to the meta-level, we can show that minimizing the \priornll{} $\ell_\theta$ yields an approximation of the PPD in terms of cross-entropy and thus \kl{}-Divergence:
\begin{insight}
\label{theorem:loss}
The proposed objective $\ell_\theta$ is equal to the expectation of the cross-entropy between the PPD and its approximation $q_\theta$: $\ell_\theta = \E_{x,D \sim p(\dataset{})}[ \text{H}(p(\cdot | x,D), q_\theta(\cdot | x,D))]$
\end{insight}
\begin{proof}
The above can be shown with a simple derivation. We mark transformations for easy reading.
\begin{align}
\ell_\theta %&= -\int_{D,x,y,f} p(x,y|f) p(D|f) p(f) \log q_\theta(y|x,\dataset{})\\
%&= -\int_{D,x,y,f} \newmathterm{p(x,y,D|f)} p(f) \log q_\theta(y|x,\dataset{})\\
%&= -\int_{D,x,y,f} \newmathterm{p(x,y,D,f)} \log q_\theta(y|x,\dataset{})\\
&= \newmathterm{-\int_{D,x,y} p(x,y,D)} \log q_\theta(y|x,D)
= -\newmathterm{\int_{D,x} p(x,D) \int_{y} p(y|x,D)} \log q_\theta(y|x,D)\\
&= \int_{D,x} p(x,D) \newmathterm{\text{H}(p(\cdot | x,D), q_\theta(\cdot | x,D))}
= \newmathterm{\E_{x,D \sim p(\dataset{})}}[ \text{H}(p(\cdot | x,D), q_\theta(\cdot | x,D))]
%&= \int_{D,x,y,f} p(x) p(y|x,f) \newmathterm{p(f|D) p(D)} \log q_\theta(y|x,D)\\
%&= \int_{D,x,y} p(x) p(D) \newmathterm{ \int_f p(y|x,f) p(f|D)} \log q_\theta(y|x,D)\\
%&= \int_{D,x,y} p(x) p(D) \int_f p(y|x,f) \newmathterm{ \frac{p(D,f)}{p(D)}} \log q_\theta(y|x,D)\\
%\intertext{Using $D,f \indep x$:}
%&= \int_{D,x,y} p(x) p(D) \int_f p(y|x,f) \newmathterm{ \frac{p(D,f|x)}{p(D|x)}} \log q_\theta(y|x,D)\\
%&= \int_{D,x,y} p(x) p(D) \int_f p(y|x,f) \newmathterm{ \frac{p(D|x,f)p(f|x)}{p(D|x)}} \log q_\theta(y|x,D)\\
%\intertext{Using $y \indep D |x,f$}
%&= \int_{D,x,y} p(x) p(D) \int_f \newmathterm{ \frac{p(y,D|x,f)p(f|x)}{p(D|x)}} \log q_\theta(y|x,D)\\
%&= \int_{D,x,y} p(x) p(D) \int_f \newmathterm{ \frac{p(y,D,f|x)}{p(D|x)}} \log q_\theta(y|x,D)\\
%&= \int_{D,x,y} p(x) p(D) \int_f \newmathterm{ p(y,f|D,x)}\log q_\theta(y|x,D)\\
%&= \int_{D,x,y} p(x) p(D) \newmathterm{ p(y|x,D)} \log q_\theta(y|x,D)\\
%&= \int_{D,x,y} \newmathterm{p(y,x,D)} \log q_\theta(y|x,D)
\end{align}
\vspace*{-0.4cm}\qedhere~
\end{proof}
Thus, we have an approximation of the PPD. We can formulate this in terms of $\kl{}$-Divergence, too.
\begin{corollary}
\label{corollary:klrelation}
The loss $\ell_\theta$ equals the expected $\kl{}$-Divergence $\E_{D,x}[ \kl{}(p(\cdot | x,D), q_\theta(\cdot | x,D))]$ between $p(\cdot|x,D)$ and $q_\theta(\cdot|x,D)$ over prior data $x,D$, up to an additive constant. Proof in App.\ \ref{sec:proof_klrelation}.
\end{corollary}
In the following, we consider the optimum of the given optimization problem $\theta^* \in \argmin_{\theta} \ell_\theta$.
\begin{corollary}
\label{corollary:exactness}
If $p$ is a member of the distribution family $q_\theta$ (i.e., there is a $\theta$ such that $q_\theta = p$), we have $q_{\theta^*}(\cdot|x,D) = p(\cdot|x,D)$ for all $x,D$ for which $p(\cdot|x,D)$ is defined. Proof in App.\ \ref{sec:proof_exactness}.
\end{corollary}
%From our analysis, we can see that some existing meta-learning setups that minimize the cross-entropy on hold out data directly, like MAML~\citep{finn-icml17a} or MANN~\citep{santoro2016meta}, are a form of PPD approximation as well.
%The prior-data samples in meta-learning setups are just usually created by humans, e.g., by drawing letters for the Omniglot dataset \citep{lake2015omniglot} and few in number.% and they work only with a small sample from the prior, unlike for \priorfittednetworks{}, where we continuously sample completely novel datasets $\dataset{}$ that are generated cheaply and fully automatically.
%(ii) Second, many self-supervised methods do a similar form of PPD approximation as well. \cite{chen2020big} use a softmax to train on a
\begin{figure}[t]%h]
\centering
\begin{subfigure}[t]{0.42\textwidth}
\centering
\caption{}
\input{transformer_visualization}
\label{fig:transformer_visualization}
\end{subfigure}
\hfill
\begin{subfigure}[t]{0.28\textwidth}
\centering
\caption{}
\input{riemann_2}
\label{fig:riemann_2}
\end{subfigure}
\hfill
\begin{subfigure}[t]{0.28\textwidth}
\centering
\caption{}
\input{riemann_3}
\label{fig:riemann_3}
\end{subfigure}
\caption{(a) A visualization of the Transformer for $n=3$ input pairs and $m=2$ queries. Every bar is the representation of one input and the arrows show what each representation can attend to. (b-c) A visualisation of the Riemann distribution, with and without bounded support.}
\label{fig:riemann}
\vspace{-0em}
\end{figure}
\section{Adapting the Transformer for Bayesian Inference}
In this section, we propose an efficient architecture based on the Transformer encoder \citep{vaswani-neurips17} and
%We use a standard Transformer encoder \citep{vaswani-neurips17} with only a few changes to make it work for our Bayesian inference.
%Transformers were developed with sequence modelling and prediction in mind, but in our application we want to map a set $D = \{(x_i,y_i)\}_{i=1}^n$ of data points $(x_i,y_i)$ and a query $x$ to a distribution $q_\theta(y|x,D)$ over $y$, which in many cases should be continuous.
%We propose an efficient, permutation-invariant, architecture that allows batch-wise training and prediction, as well as
a novel regression head for regression problems.
\iffalse
\begin{figure}[h]
\centering
\input{transformer_visualization}
\caption{A visualization of the Transformer for $n=3$ input pairs and $m=2$ queries. Every bar is the representation of one input and the arrows show what each representation can attend to.}
\label{fig:transformer_visualization}
\end{figure}
\fi
\newcommand{\encx}{\text{enc}_x}
\newcommand{\ency}{\text{enc}_y}
\textbf{An Efficient Architecture}
Our architecture, visualized in Figure \ref{fig:transformer_visualization}, is based on a Transformer encoder without positional encodings, which makes it invariant to permutations in the dataset $D$.
We feed the Transformer with simple linear projections of our inputs and queries.
Our model returns the PPD for each given query $x$, only depending on the dataset $D$ and the query itself.
%See Figure \ref{fig:transformer_visualization} for a visualization.
We sample the number of inputs $n$ and queries $m$ randomly such that $N=m+n$ for some fixed $N$.
See Appendix \ref{sec:appendix_efficient_architecture} for a more detailed description and Appendix \ref{sec:riemannablation} for an ablation of the permutation invariance. For additional practical training information see Appendix \ref{sec:general_training_details}.
\iffalse
\begin{figure}[t]
%\centering
%\input{riemann_distribution}
\centering
\iffalse
\begin{subfigure}[b]{0.32\textwidth}
\centering
\caption{}
\input{riemann_1}
\label{fig:riemann_1}
\end{subfigure}
\hfill
\fi
\begin{minipage}[c]{0.7\textwidth}
\centering
\begin{subfigure}[b]{0.49\textwidth}
\centering
\caption{}
\input{riemann_2}
\label{fig:riemann_2}
\end{subfigure}
\hfill
\begin{subfigure}[b]{0.49\textwidth}
\centering
\caption{}
\input{riemann_3}
\label{fig:riemann_3}
\end{subfigure}
\end{minipage}
\begin{minipage}[c]{0.29\textwidth}
\caption{A visualisation of the Riemann distribution, with and without bounded support.}
\end{minipage}
\label{fig:riemann}
\vspace{-0em}
\end{figure}
\fi
\textbf{Riemann Distribution}
The modelling of continuous distributions is hard with neural networks.
To yield strong performance in modelling PPDs, we used a distribution that works particularly well with neural networks.
Based on the knowledge that neural networks work well for classification and inspired by discretizations in distributional reinforcement learning \citep{bellemare2017distributional}, we made use of a discretized continuous distribution that we call \emph{Riemann Distribution}. %as it is based on a similar idea to approximation as Riemann integrals.
% While I was only inspired by the paper above, AlphaFold uses a similar trick in SI 1.9.6
PDFs of this distribution are bar plots, see Figure \ref{fig:riemann_2}.
\newcommand{\buckets}{\mathbf{B}}
We select the boundaries of the buckets $\buckets{}$ in which we discretize such that they all have the same probability in prior-data: $p(y \in b) = 1/|\buckets{}|, \forall b \in \buckets{}$.
We estimate this using a large prior-data sample.
%The distributions described above are very powerful; e.g., in contrast to most parametric distributions they can also represent multimodal distributions.
%Their downside, however, is that they have finite support: the probability for $y$ to be greater than the highest border is 0. This is a problem whenever one tries to model distributions that have unbounded support.
In experiments with priors that need unbounded support, we replace the last bar on each side with an appropriately scaled half-normal distribution, see Figure \ref{fig:riemann_3}. For a more exact definition, we refer the reader to Appendix \ref{sec:riemmanndefinition}. For an ablation of the Riemann distribution, see Appendix \ref{sec:riemannablation}.
%As we now show, even the finite Riemann distribution can approximate a large class of distributions, including distributions with infinite support, up to arbitrary precision.
\begin{theorem}
\label{theorem:distribution_approximation}
A Riemann distribution $q_{\buckets{}}$ with finite support, where $\buckets{}$ are the buckets, can model any distribution with a finite, almost everywhere continuous (Riemann-integrable) density function and full support up to arbitrary precision. That is, $\forall p \forall \epsilon \exists \buckets{}. \kl{}(q_\buckets{},p) \leq \epsilon$.
Proof in App.\ \ref{sec:proof_distribution_approximation}.
\end{theorem}
\begin{figure}[t]%!h]
\centering
\begin{subfigure}[b]{.49\textwidth}
\subcaption{}\label{fig:heatmap_left}
\centering
\begin{tikzpicture}
\begin{axis}[
width=.8\textwidth, height=.4\textwidth,
ylabel=y,
xlabel=x,
tick align=outside,
tick pos=left,
x grid style={white!69.0196078431373!black},
xmin=-0.0, xmax=1.0,
xtick style={color=black},
y grid style={white!69.0196078431373!black},
ymin=0.1, ymax=1.6,
ytick style={color=black},
scale only axis,
enlargelimits=false,
axis on top]
\addplot graphics[xmin=-0.05,xmax=1.05,ymin=0.1,ymax=1.6] {heatmap.pdf};
\end{axis}
\end{tikzpicture}
\end{subfigure}
%\input{heatmap}
\begin{subfigure}[b]{.49\textwidth}
\subcaption{}\label{fig:heatmap_right}
\centering
\begin{tikzpicture}
\begin{axis}[
xlabel=x,
width=.8\textwidth, height=.4\textwidth,
tick align=outside,
tick pos=left,
x grid style={white!69.0196078431373!black},
xmin=-0.0, xmax=1.0,
xtick style={color=black},
y grid style={white!69.0196078431373!black},
ymin=0.1, ymax=1.6,
ytick style={color=black},
scale only axis,
enlargelimits=false,
axis on top]
\addplot graphics[xmin=-0.05,xmax=1.05,ymin=0.1,ymax=1.6] {heatmap_gp_comp.pdf};
\draw[red,-{Triangle[length=2mm, width=1mm]}] (axis cs:.1-.03,1+.15) -- (axis cs:.1,1.);
\draw[red,-{Triangle[length=2mm, width=1mm]}] (axis cs:.1+.03,1.38+.15) -- (axis cs:.1,1.38);
\end{axis}
\end{tikzpicture}
\end{subfigure}
\begin{subfigure}[b]{1.\textwidth}
\centering
\input{legendforqualitativegpplots}
\end{subfigure}
\caption{(a) The PFN\textquotesingle{s} PPD given two evaluations of the function, highlighting the binning it its predictions (best viewed in full-screen mode). (b) The PFN\textquotesingle{s} and the GP\textquotesingle{s} mean and confidence intervals, which are nearly identical (red arrows mark tiny exemplary differences). For additional comparisons, see Figure \ref{fig:comp_w_gp} in the appendix.}
\label{fig:heatmap}
\end{figure}
\section{Posterior Approximation Studies}
\label{sec:posteriorapproximationstudies}
In our first set of experiments, we study the capability of PFNs to perform Bayesian inference for the tractable case of Gaussian Processes (GPs) with fixed hyperparameters (where we can compare to ground truth data; Section \ref{sec:gpapprox}) and the intractable cases of GPs with unknown hyperparameters (Section \ref{gp_hyper_priors}) and Bayesian Neural Networks (BNNs; Section \ref{sec:BNN_ppd}).
Corollary \ref{corollary:klrelation} shows that the \priornll{} is equal to the \kl{}-Divergence between the exact PPD and the approximation, up to an additive constant.
We thus use it in all following experiments to compare how close methods can match the correct PPD.
%We thus analyze the negative log-likelihood of a method on data generated from the prior over datasets to compare different PPD approximations.
For methods that do not approximate the PPD directly, we approximate the integral of the PPD in Equation \ref{ppd_formula} with a large Monte Carlo integration.
In this section, we report the 95\% confidence interval over different sampled datasets of the \priornll{} for each experiment as shaded areas alongside the mean.
We refer the reader to Appendix \ref{sec:posteriorapproximationstudies_detail} for more information on the setup.
\subsection{Gaussian Process Approximation}
\label{sec:gpapprox}
We begin by approximating GPs to study what \priorfittednetworks{} are capable of; GPs with fixed hyperparameters are convenient for this purpose since the tractability of their PPD allows us to assess how close our approximation is to the exact PPD.
To create the prior for this task,
we sample the $N$ inputs $x_i$ uniformly at random from the unit-cube.
The data-prior of a zero-mean GP with a kernel function $k$, given the $x_i$\textquotesingle{s}, can be described as a multivariate normal distribution $\mathcal{N}(\mathbf{0}, K)$, where $K_{i,j} = k(x_i,x_j)$.
We sample from this normal distribution to yield a dataset $\{(x_1,y_1), \dots, (x_N,y_N)\}$.
%These are non-parametric Bayesian models for which there are exact inference methods, that allow us to see how close our approximation is.
%In this case, our meta-train datasets are functions $f$ sampled from a GP with the given fixed hyperparameters. Since GPs describe conditional distributions given an input $x$, we sample $n+1$ inputs $x_i$ uniformly at random from the unit-cube and evaluate $y_i = f(x_i)$.
As described above, we then train our \priorfittednetworks{} to predict the $y$\textquotesingle{s} for a held-out subset of the dataset. % from the dataset $\{(x_1,y_1), \dots, (x_n,y_n)\}$ and the query point $x_{n+1}$.
The trained \priorfittednetwork{} can then be used to approximate the PPD for new datasets and arbitrary query points in the unit-cube.
%In these cases, we generally use a simple uniform distribution over the unit-cube for $x$.
%In Figure \ref{fig:heatmap} we show the predictions of a transformer approximating a GP for a small set of input samples.
%\frank{Figure 3b desperately needs a legend. We can then also save 2 lines in the caption. It would also help to add little red arrows pointing to areas where the tiny differences in mean / confidence intervals are visible.\\
%For Figure 3a, a legend would also be helpful. Sam: I am on to this.}
Figure \ref{fig:heatmap} visualizes the predictions of such a PFN with two data points and query points $x$ ranging from $0$ to $1$.
We visualize both the estimated PPD as a heat map, as well as confidence bounds, compared to the exact PPD.
The estimated confidence intervals and means are virtually indistinguishable from the ground truth, therefore we marked the most noticeable differences with red arrows.
%Figure \ref{fig:heatmap_left} is a heatmap of the PPD, one can still see the very tiny boxes in which the Riemann distribution is discretized.
%In Figure \ref{fig:heatmap_right}, we compare these predictions to those of a ground-truth GP with the given hyperparameters; both the mean function and the plotted 95\% confidence interval are virtually indistinguishable from this ground truth.
The model learns on its own, completely automatically, to generate smooth distributions without any explicit knowledge of the positions of the bars in $\mathbb{R}$.
We published\footnote{\scriptsize \url{https://huggingface.co/spaces/samuelinferences/\\transformers-can-do-bayesian-inference}} an interactive version of this experiment.
In Figure \ref{fig:comp_w_gp} of the appendix, we show more qualitative examples of the behavior of our approximation together with the exact posterior for a different length scale.
In Figure \ref{fig:comp_w_gp_and_anp}, we additionally compare to Attentive Neural Processes \citep{kim2018ANP}, showing the clear superiority of our method for Bayesian Inference.
Figure \ref{fig:fastgp_main} shows that the qualitative results above generalize to datasets with up to 2000 examples with multiple features.
The tractable GP posterior, which achieves optimal performance by definition, is approximated very closely, and increasingly better so the more meta-train datasets we train the \priorfittednetwork{} for.
%\frank{In the legend of Figure 4a, I would replace ``Transformer (500K samples)'' by ``PFN (500k meta-train datasets)''. Samples is also a term we use for the individual data points, so it's confusing. Indeed, it would be clearer to me if the x-axis in Figure 4a and b would also read ``Number of data points'', rather than ``Number of Examples'', since ``Examples'' can be misunderstood to be an example data set.}
%Additionally, there does not seem to be a trend towards worse approximation with larger datasets.
\pgfplotsset{width=1.\textwidth, height=.6\textwidth, legend style={at={(0.5,-0.4)},anchor=north},legend columns=2}%legend pos=outer south west}
\begin{figure}
\begin{subfigure}[t]{.49\textwidth}
\vspace{0pt}
\subcaption{}
\label{fig:fastgp_main}
\input{overfit_fastgp_1e-4,1.,.6}
\end{subfigure}
\begin{subfigure}[t]{.49\textwidth}
\vspace{0pt}
\subcaption{}
\input{fast_gp_mix_main_plot}
\label{fig:fastgpmix_main}
\end{subfigure}
%\vspace{-1.5em}
\caption{(a) \priornll{} with a fixed GP. While, the PPD of the GP is exact, we can see that our approximations get very close to it. %Hyperparameters: length-scale .6, noise 1e-4, out scalce 1., 5 inputs
(b) \priornll{} with a mix over GP hyperparameters. There is no closed-form solution for this PPD, but we can see that our PFN still gets closest to the true PPD. Our PFN is also more than $200\times$ faster than any of the baselines.}
\label{fig:my_label}
\end{figure}
\iffalse
\begin{figure}[h]
\centering
\input{gp_speed_vs_transformer_512_w_10_features}
\caption{Inference Speed on GPU for a plain GP (without MLE-II) compared to a Transformer, like we use them, with 512 size. Scores are a mean over 10 samples.}
\label{fig:my_label}
\end{figure}
\fi
\subsection{Approximating A GP with Hyper-Priors} \label{gp_hyper_priors}
A common practice in fields applying GPs in the real world is to define distributions over the hyperparameters of these GPs~\citep{rasmussen-book06a,snoek-nips12a}, so-called hyper-priors.
Thus, the prior of these models considers a more diverse set of functions, to model the fact that the correlation between data points, the smoothness of functions and the scale of outputs varies across applications.
%The distributions over these hyperparameters are called hyper-priors.
%Hyper-priors, due to the more varied prior, allow the user to apply GPs to a larger set of tasks.
The downside of using hyper-priors is that it is not possible to compute the PPD of the GP exactly anymore.
There are two common practices to approximate it anyways, and we evaluate against both.
(i) Firstly, MLE-II, the most common approximation. This is a simple special case of variational inference.
Here, suitable hyperparameters are found by maximum a posteriori (MAP) estimation (e.g.,~\cite{rasmussen-book06a}).
We use the fitting setup of BoTorch \citep{balandat2020botorch} which also inspired our hyper-priors. (ii) Secondly, Markov Chain Monte Carlo (MCMC), a method to sample from any distribution for which one can calculate non-normalized probabilities, that is also frequently applied to sample hyperparameters (e.g.,~\cite{snoek-nips12a}).
Here, we use the state-of-the-art algorithm NUTS \citep{hoffman2014no}, which uses gradients to facilitate the Hamiltonian Monte Carlo algorithm.
We plot the performance of the different methods in Figure \ref{fig:fastgpmix_main}.
Our model can approximate the PPD on the prior much closer than both of these methods, and its inference is more than $200\times$ faster than MLE-II and $1\,000\times$ to $8\,000\times$ faster than NUTS. Further details can be found in Appendix \ref{sec:detail_gp_hyper_priors}.
\subsection{Approximating \bayesnn{}s}\label{sec:BNN_ppd}
% \bayesnn{}\textquotesingle{s} differ from plain neural networks in that their weights are assigned a probability distribution instead of a single value or point estimate. Bayesian inference for neural networks calculates the posterior distribution of the weights given the training data,
% $P(w|\dataset{})$ to answer predictive queries
% about unseen data by taking expectations: the predictive
% distribution of an unknown label $\hat{y}$ of a test data item $\hat{x}$,
% is given by $P(\hat{y}|\hat{x}) = \E_{w \sim p(w|\dataset{})}
% [P(\hat{y}|\hat{x}, w)]$. Each possible configuration of the weights, weighted according to
% the posterior distribution, makes a prediction about the unknown label given the test data item $\hat{x}$. Taking an
% expectation over the posterior distribution of the weights is thus
% equivalent to using an ensemble of an infinite number of neural networks, which is intractable for neural networks of any practical size.
Bayesian Neural Networks (\bayesnn{}s) provide a strong ability to model uncertainty
%and strong regularization by inherent ensembling,
due to their Bayesian nature, compared to traditional neural networks.
Recently, posterior approximation methods, such as stochastic variational inference (SVI) and stochastic gradient MCMC methods (such as NUTS) allow for much faster convergence, which has sparked a lot of interest in the area~\citep{bnn_posteriors, fortuin2021bayesian}.
However, computational feasibility still remains a key issue of current BNNs.
%However, these methods
%%also do fail in certain scenarios and
%still converge too slowly for many practical %applications.
\newcommand{\bnnmodel}{\Phi}
In Figure \ref{fig:bayesnnpriornll}, we show that our PFNs can outperform today\textquotesingle{s} default methods for \bayesnn{} inference using a fraction of their compute budget (PFNs are $1\,000\times$ faster than Bayes-by-Backprop SVI and $10\,000\times$ faster than NUTS to achieve the same performance).
The figure shows the \priornll{} of the BNN prior that is used during inference by all solvers.
To generate prior-data for our PFNs, we sampled random weights for a BNN $\bnnmodel{}$ and used normally-distributed i.i.d. feature vectors $X_i$ to obtain $y_i := \bnnmodel{}(X_i)$ with $\mathcal{D} := (X_{1:n}, y_{1:n})$. For the evaluation we sampled 100 datasets, with $m=200$ test data points and $n=100$ revealed data points per dataset.
A detailed description of baselines as well as further experiments are included in Appendix \ref{sec:detail_BNN_ppd}.
%In our experiments, we find that Transformers are able to approximate BNNs orders of magnitude faster than state of the art inference methods such as SVI and MCMC with NUTS at the same accuracy level and without bias as shown in section \ref{sec:tabular_datasets}.\\
%In this section we show empirical results fitting data generated from random BNNs. We also demonstrate a Meta-BNN model, which approximates the application of many BNN architectures sampled from a hyper-prior at once and can eliminate the need for hyperparameter fitting or cross validation in their application.
\begin{figure}[t]
\centering
\begin{subfigure}[b]{.48\textwidth}
\centering
\subcaption{}
\input{bayesian_models_eval_steps_small}
\end{subfigure}
\centering
\begin{subfigure}[b]{.48\textwidth}
\centering
\subcaption{}
\input{bayesian_models_eval_steps_big}
\end{subfigure}
\begin{subfigure}[b]{1.\textwidth}
\centering
\input{bayesian_models_eval_steps_legend}
\end{subfigure}
\caption{Time spent for Bayesian Inference per dataset compared to \priornll{}.
For NUTS we scale the number of warmup and sampling steps, for SVI, we scale the number of training steps and averaged samples.
PFN is evaluated at only one setting (blue dot).
We show two different BNN architectures (a) a BNN with 3 features, 2 layers and a hidden dimensionality of 5 and (b) a BNN with 8 features, 2 layers and a hidden dimensionality of 64.}
\label{fig:bayesnnpriornll}
\end{figure}
% \begin{figure}[h]
% \centering
% \input{bayesian_models_eval_samples_small}
% \caption{Negative Log Likelihood (NLL) of test set samples with the number of training samples indicated over the x-axis. NLL is calculated as the mean over 100 tasks and 95\% confidence intervals over tasks is indicated. Each task contains indicated training samples and 200 evaluation samples.}
% \label{fig:my_label}
% \end{figure}
% [...] The presented method turns the approach to designing classifiers upside down. Instead of maximizing the probability of the predicted class given the data, we design priors on the distribution of features and target classes from which our method generates a suitable method for inference. We also demonstrate how existing classification algorithms can be used to generate priors over the features and target class.
% Talk about the possibility to design priors which weren't solvable by MCMC ..
\section{Application to Tabular Datasets} \label{sec:tabular}
We evaluate Transformer-based classifiers with priors over Gaussian Processes and Bayesian Neural Networks on a range of small real-world tabular datasets.
Surprisingly, the simple priors we describe next outperform a large set of baselines, including XGB \citep{xgboost} and Catboost \citep{catboost}, while greatly surpassing all baselines in uncertainty calibration.
% Details: Number of features variable, Normalization
\textbf{A GP Prior for Tabular Classification} \label{sec:gpclassification}
We build a classification prior based on GPs with a hyper-prior, as described in Section \ref{gp_hyper_priors}.
Before, we used this prior for regression. To make it a classification prior, we find the median over all targets in the dataset $y$ and set the class to one if greater than the median and zero otherwise. %That is a target $y'$ is sampled as $y' \sim \mathbf{B}(\sigma(y))$, where $y$ is the output of the GP.
\textbf{A \bayesnn{} Prior over Architectures} \label{metabayes}
We generalize our \bayesnn{} setup to learn a prior over model architectures.
Fitting a neural network requires finding a suitable architecture, e.g., embedding size, number of layers and activation function.
Commonly, resource-intensive searches need to be employed to find suitable architectures \citep{liu2018darts,zoph2018nasnet}.
The result of this search is not a posterior distribution over architecture choices and their respective parameters, though, but instead only a point-estimate of the architecture choice. This is true for Bayesian optimisation approaches as well \citep{nat_with_bo, bayes_nas, ru2021interpretable}.
Ensembling methods over multiple architectures
can yield a rough approximation to a distribution over architectures,
but this
scales linearly in cost with the number of architectures considered \citep{zaidi2020neural}. Using dropout can be regarded as ensemble learning but is limited to varying connectivity \citep{dropout_ensemble}.
\priorfittednetworks{} allow us to be fully Bayesian, not only about the weights of the \bayesnn{}, but its architecture, too.
By defining a prior probability over the space of model architectures, we remove the need for finding an optimal neural architecture and instead jointly integrate over the space of architectures and their respective weights in a single Bayesian inference. This yields a distribution over architectures in the posterior.
The degrees of freedom of model architectures we considered can be found in Appendix \ref{sec:detail_tabular}. Our prior-data sampling algorithm for learning to predict based on a prior over both weights and architectures iterates the following steps:
\\
\hphantom{~~~~}(1) Sample a model architecture $A \sim p(A)$\\
\hphantom{~~~~}(2) Sample model weights for the given architecture: $W_{i,j} \sim p_w(\cdot)$\\
\hphantom{~~~~}(3) Sample i.i.d.\ features $x_{i,f} \sim \mathcal{N}(0,1)$ for each data point $i$ and each feature $f$\\
%\frank{We probably have priors with more than one feature dimension in X. Was this also sampled? If so, how? I assume the features for all dimensions are sampled independently?}
\hphantom{~~~~}(4) Propagate each $x_i$ forward through $A_W$ yielding $\{(x_i,A_W(x_i))\}_{i=1}^N$\\
%\frank{I find step 4 very unclear. Do you mean ``Take a gradient step for the PFN with set-valued input $D_{train} = \{(x_i,A_W(x_i))\}_{i=1}^n$, test points $x_{n:n+m}$ and loss function $\sum_{j=1}^m\log q_\theta( A_W(x_j) | x_j, D_{train})$?}
We binarized targets like for the GP Prior above. See Appendix \ref{sec:detail_tabular} for more details.
% Sampling from this model is done by sampling independent values for X and Y and evaluating the graph. Activation functions and Gaussian noise are applied at each node.
\begin{figure}[t]
%\centering
%\input{riemann_distribution}
\begin{minipage}[h]{0.62\textwidth}
\centering
\begin{subfigure}[t]{0.45\textwidth}
\centering
\caption{}
\input{tabular/bayesgraph}
\label{fig:bayesgraph}
\end{subfigure}
\hfill
\begin{subfigure}[t]{0.45\textwidth}
\centering
\caption{}
\input{tabular/meta_bnn}
\label{fig:metacausalgraph}
\end{subfigure}
\end{minipage}\hfill
\begin{minipage}[h]{0.35\textwidth}
\caption{(a) A \bayesnn{}. The inputs $x$ are mapped to the output $y$. (b) A prior over BNN architectures. The network considers a distribution over different model architectures at once.}
\end{minipage}
\vspace{-6pt}
\end{figure}
\textbf{Setup of Datasets} \label{sec:tabular_datasets}
We used a large collection of tabular datasets from the open-source OpenML AutoML Benchmark \citep{amlb2019}; we first removed datasets with more than one hundred features or missing values, ending up with 20 datasets that represent a diverse set of classification problems with numerical and categorical features.
We then simplified each dataset to a balanced binary classification problem, and, for each dataset sampled 20 subsets, each including 100 samples. Within each subset we provide labels for the first 30 samples and evaluate on the remaining samples. All baselines and \priorfittednetworks{} use the same datasets and subsets.
We also define a set of six unrelated validation datasets used for optimizing the prior distribution over architectures of the \priorfittednetworks{}. This is similar to setting the range of hyperparameters in a cross-validation grid search and can be reused for all similar problems. See Appendix \ref{sec:detail_tabular} for more details.
\textbf{Setup of Models}
Our baselines comprise two
Gradient Boosted Tree algorithms, XGBoost \citep{xgboost} and Catboost \citep{catboost}, as well as logistic regression, K-Nearest Neighbors, Gaussian Processes (RBF Kernel) and Bayesian Neural Networks (Bayes-by-Backprop SVI with Pyro). Catboost additionally received information on which features are categorical.
We used two \priorfittednetworks{}, one \priorfittednetwork{} for each prior described above. We were able to apply the same model across different feature dimensions.
% Tabnet, a recent neural architecture, missing at the moment because new runs weren't done, TabNet also used cat features, We used DreamQuarks-AIs implementation of TabNet and used early stopping.
%
We used grid search with 5-fold cross-validation to optimize our baselines\textquotesingle{} hyperparameters for each dataset, using the hyperparameter spaces described in Table \ref{table:hyperparameters_tabular_baselines} in the appendix.
% of Appendix \ref{sec:detail_tabular}.
% While our \priorfittednetworks{} do not require per-task hyperparameter optimization, we adapted their priors based on the validation dataset, too.
%The hyperparameter space for cross-validation and the hyper-priors were chosen based on typical parameter ranges and their performance on a separate set of validation datasets (see Appendix \ref{sec:detail_tabular} for more details). The final hyperparameters used for each grid-search can be found in Table \ref{table:hyperparameters_tabular_baselines}.
\textbf{Experimental Results}
Table \ref{table:tabular_auc_table} compares the area under the receiver operating curve (ROC AUC) of PFNs based on BNN and GP priors against the various baselines.
%the Meta-BNN Transformer (T-BNN) (section \ref{fig:bayesnnpriornll}) and Meta-GP-Transformer (T-GP) (section \ref{sec:gpclassification}).
While the PFN-GP demonstrates on-par performance to the best of the evaluated baselines, the PFN-BNN performs strongest in this comparison. The Wilcoxon signed-rank test \citep{wilcox}, a standard
metric for comparing classifiers across multiple datasets is evaluated on each of the 20 subsets of the 21 evaluation datasets separately
%to increase power
and shows a significant performance improvement.
While all baselines make use of cross-validation to find suitable hyperparameters, the \priorfittednetworks{} naturally consider a distribution of hyperparameters and integrate cross-validation and training within the model\textquotesingle{s} Bayesian inference step. As shown in Table \ref{table:tabular_auc_table}, this makes it very fast compared to the baselines: since all it needs to do on a new dataset is a single forward propagation step, when run on a GPU (Nvidia Tesla V100), it
%The proposed method is much faster than previous state-of-the-art methods in the inference of the whole benchmark,
%see Table \ref{table:tabular_auc_table}.
%, this can partly be attributed to removing cross-validation.
%Furthermore, running our methods on
requires as little as 13 seconds for all 20 datasets combined. This is more than $5000 \times$ faster than the 20 hours XGBoost requires, and even faster than
%further increases speed, making it faster than all compared methods, including
K-Nearest Neighbors.
We can also experimentally confirm that our PFNs make well-adjusted uncertainty predictions. We find that on our data they have lower Expected Calibration Error (ECE, using 100 bins \citep{ece}) even than standard BNNs and ensemble methods such as XGBoost, the two baseline methods that best capture uncertainty.
%In Table \ref{table:tabular_auc_table} we measure uncertainty calibration by Expected Calibration Error and
We also provide confidence histograms in Figure \ref{fig:calibration} of the appendix, visually demonstrating the PFN\textquotesingle{s} strong calibration.
\begin{table}[t]
\centering
\input{tabular_auc_light}
\vspace{-2pt}
\caption{Aggregated performance on subsets of datasets with 30 training samples. Our novel PFN-BNN performs strongest overall. See Table \ref{table:tabular_auc_table_full} in the appendix for per-dataset results.\vspace{-4pt}}
\label{table:tabular_auc_table}
\end{table}
% Calibration curves of all other methods can be found in the Appendix
% We evaluate the Transformer in a one-step prediction fashion, that is instead of predicting N samples from the training data, we predict one sample N times.
\section{Application to Few-Shot Learning}
\label{sec:fewshotlearning}
As shown in the previous section, \priorfittednetworks{} can be trained with priors that are not (easily) possible to use in the traditional Bayesian setups.
In this section, we show another example of this, building a prior for hand-written digits and letters and demonstrating strong performance on Omniglot \citep{lake2015omniglot} in combination with fine-tuning of the \priorfittednetwork{} on Omniglot.
As an approximation to hand-written symbols, we create a prior that generates random straight lines.
In each sample from this prior, we consider five different random line assortments as class prototypes.
For each instance of a class, we apply noise to the positions of the lines.
It required just 55 lines of readable code to create this prior (see Figure \ref{fig:strokepriorcode} in the Appendix).
In Figure \ref{fig:strokesamples} of the Appendix we show a sample dataset from this prior.
For our evaluation on Omniglot, we follow the setup of \cite{rothfuss2021pacoh}. We perform 5-shot 5-way classification, i.e.\ we consider five classes and five examples of each class.
We create one task from the first five classes of each of the 30 train and 20 evaluation alphabets in Omniglot.
We train our PFN on the synthetic prior and then fine-tune it on the 30 training tasks created.
This practice is unlike the previous sections.
Our premise is to show the flexibility of our model:
it can even be fine-tuned on similar datasets, if these are available.
We do not use task augmentation, like \cite{rothfuss2021pacoh}, but apply translations to each image, as we only use simple linear layers to encode the images in the Transformer.
In Table \ref{tab:omniglot} one can see that for this very hard version of the Omniglot dataset our simple method yields results on par with the state of the art. For more information on the setup, please see Appendix \ref{sec:section7details}.
\iffalse
\begin{table}[h]
\centering
\footnotesize
%\vspace{-6pt}
\begin{tabular}{l|cH}
\toprule
Method & Accuracy & Calibration error \\ \midrule
Vanilla BNN \citep{liu2016stein}& $0.795 \pm 0.006$ & $0.135 \pm 0.009$ \\
%PACOH-NN-MAP (ours) & $0.866 \pm 0.005$ & $\mathbf{0.075 \pm 0.006}$ \\
%\hline
%MLAP-M \citep{amit2017meta} & $0.635 \pm 0.015$ & $0.108 \pm 0.008$ & $0.119 \pm 0.019$ \\
MLAP \citep{amit2018meta} & $0.700 \pm 0.0135 $ & $0.108 \pm 0.010$ \\
%FO-MAML \citep{nichol2018firstorder} & $0.590 \pm 0.010$ & N/A & N/A \\
MAML \citep{finn-corl17a} & $0.693 \pm 0.013$ & $0.109 \pm 0.011$ \\
BMAML \citep{yoon2018bayesian} & $0.764 \pm 0.025$ & $0.191 \pm 0.018$ \\
PACOH-NN \citep{rothfuss2021pacoh}& $\mathbf{0.885\pm 0.090}$ & $\mathbf{0.091 \pm 0.010}$ \\
%Transformer & 0.7791 & - \\
Transformer & $\mathbf{0.865 \pm 0.019}$ & -\\
\bottomrule
\end{tabular}
\vspace{-0pt}
\caption{Comparison of meta-learning algorithms in terms of test accuracy on the Omniglot dataset. Results within the confidence interval of the best performance are bold-faced.\vspace{-8pt}
}
\label{tab:omniglot}
\end{table}
\fi
% version with two columns
\begin{table}[h]
\centering
\footnotesize
%\vspace{-6pt}
\setlength{\tabcolsep}{0.5em}
\begin{tabular}{l|cH||l|cH}
\toprule
Method & Accuracy & Calibration error & Method & Accuracy & Calibration error\\ \midrule
Vanilla BNN \citep{liu2016stein}& $0.795 \pm 0.006$ & $0.135 \pm 0.009$ & MLAP \citep{amit2018meta} & $0.700 \pm 0.0135 $ & $0.108 \pm 0.010$ \\
MAML \citep{finn-corl17a} & $0.693 \pm 0.013$ & $0.109 \pm 0.011$ & BMAML \citep{yoon2018bayesian} & $0.764 \pm 0.025$ & $0.191 \pm 0.018$ \\
PACOH-NN \citep{rothfuss2021pacoh}& $\mathbf{0.885\pm 0.090}$ & $\mathbf{0.091 \pm 0.010}$ & \priorfittednetwork{} (this work) & $\mathbf{0.865 \pm 0.019}$ & - \\
\bottomrule
\end{tabular}
\vspace{-2pt}
\caption{Comparison of meta-learning algorithms in terms of test accuracy on the Omniglot dataset. Results within the confidence interval of the best performance are bold-faced.\vspace{-8pt}
}
\label{tab:omniglot}
\end{table}
\section{Conclusion \& Future Work}
We presented Prior-Fitted Networks (\priorfittednetworks{}), a novel way to efficiently approximate the posterior predictive distribution using deep neural networks and showed its capability on a set of diverse tasks and priors.
% This opens the possibility for many lines of
% %a multitude of groundbreaking
% future work.
We expect the following future work to be especially fruitful:
(i) Work on finding novel priors that are now feasible to approximate using \priorfittednetworks{}.
(ii) Work on architectures that are well-fit for this task, as we simply used a slight adaption of current Transformer models.
(iii) Work on scaling \priorfittednetworks{} to even larger real-world problems.
(iv) Work on using our model for the amortized simulation-based inference setting.
\iffalse
\section{Ethics Statement}
practices to data set releases: Use open datasets
potentially harmful insights: methodologies and applications: potential conflicts of interest and sponsorship: discrimination/bias/fairness concerns:
privacy and security issues:
legal compliance:
research integrity issues:
\fi
\iffalse
\section{Reproducibility Statement}
\textbf{Code release} In an effort to ensure reproducibility, we release code, including simple notebooks, for all experiment sections, which can be found alongside this submission.
\textbf{Details of training procedures for PFN and baselines} Details shared in the training procedure of all our transformer models can be found in Appendix \ref{sec:general_training_details}. Details for each experiment and subexperiment can be found in Appendices \ref{sec:posteriorapproximationstudies_detail}, \ref{sec:detail_tabular} and \ref{sec:section7details}.
An overview of the hyperparameters used for running our baselines can be found in Table \ref{table:hyperparameters_tabular_baselines}.
\textbf{Complete proof of theoretical results} A detailed proof of Corollary \ref{corollary:klrelation} and Theorem \ref{theorem:distribution_approximation} can be found in appendices \ref{sec:proof_klrelation} and \ref{sec:proof_distribution_approximation}.
\textbf{Availability of datasets} All datasets used in our experiments are either generated by procedures included in our code submission or freely available with downloading procedures included in the submission as well. Further detail on the datasets used can be found in Tables \ref{collection_of_datasets_tabular} and \ref{collection_of_datasets_tabular_valid}.
\fi
\iffalse
\newpage
\subsection{Application to BO for HPO}
In this section, we want to evaluate how PFNs can be used to perform Bayesian Optimization (BO). BO is a method for Hyperparameter Optimization (HPO), where the objective is to find the best hyperparameter configuration $x \in \mathcal{X}$ that minimize a function $f: \mathcal{X} \rightarrow\mathbb{R} $. To optimize the function, BO relies on a probabilistic model $s(\mathbf{x})$, or surrogate, that approximates the function $f$. Given a history of observations $\mathcal{D} = \{(\mathbf{x}_i, y_i)\}^N_i$, the surrogate model provides uncertainty estimations over the domain of $f$, which are used to query the next hyperparameter $x_*$ by maximizing an acquisition function $a(\mathbf{x}|\mathcal{D})$. BO alternates the update of the surrogate and the selection of the next configuration.
A typical choice for surrogate is a Gaussian Process (GP) \citep{rasmussen-book06a}, whose posterior can be computed analitically, using the history of observations $\mathcal{D}$ to create a prior over functions. On the other hand, a common choice for acquisition function is the Expected Improvement (EI), which is defined as:
\begin{align}
a(\mathbf{x}|\mathcal{D})=\mathbb{E}\left[\max\left\{ f(\mathbf{x})-y_{\text{min}}, 0\right\}\right]\,
\end{align}
where $y_{\text{min}}$ is the smallest observed value of $f$.
When using a GP as surrogate in a BO iteration, we update the posterior after a new configuration has been evaluated. If we define a prior distribution over the GP hyperparameters (hyper-priors), we can estimate a suitable set of these hyper-priors by performing maximum a posteriori estimation (MAP), as explained in Section \ref{gp_hyper_priors}. Therefore, the posterior approximation provided by PFNs sets a novel way to speed-up this process. We use as surrogate a Transformer that outputs a Riemann Distribution and provides a discrete probabilistic output $s_b(\mathbf{x})$ for a bucket $b \in \mathbf{B}$. This allows formulating EI as:
\begin{align}
a(\mathbf{x}|\mathcal{D}) = \sum_{b \in \mathbf{B}} \max\Big\{ {\frac{y_b^{\text{max}}+y_b^{\text{min}}}{2}-y_{\text{min}}} ,0 \Big\} \cdot s_b(\mathbf{x}),
\end{align}
where $y_b^{\text{min}}= \min_{y \in b} y$ and $y_b^{\text{max}}= \max_{y \in b} y$ are the borders of the bucket $b$.
\textbf{Setup of Datasets} We evaluate the performance of our proposed model in the meta-validation and meta-test split of HPO-v3 \citep{}, which contains 16 search spaces with pre-computed evaluations of the performance for different hyperparameter configurations. We ran our method and baselines on all the datasets specified by this meta-dataset and repeat the experiments five times using the five random seeds proposed for starting the HPO algorithms. All the BO evaluations were performed for 10 trials.
\textbf{Setup of Our Model} We use a \priorfittednetwork{} (Section \ref{gp_hyper_priors}) with 1000. For training, we sampled inputs from an uniform distribution so that the input matches the search space dimensionality. The corresponding target was generated by sampling a function over the prior defined by GP with Mat\'{e}rn 32 kernel and hyperparameters sampled from a gamma distribution as prior. We tuned three hyper-hyperparameters (hyperparameters of the hyperprior): noise, lengthscale and outputscale concentration). The best hyper-hyperparameters were obtained by selecting the combination with the lowest average rank at the 10-th trial. The best yielded hyperprior configuration was: 1) lengthscale concentration: 10, 2) outputscale concentration: 1, 3) noise concentration 0.001.
\textbf{Setup of Baselines} As baselines we use 1) Random Search \citep{}, a simple yet practical method for HPO, 2) a GP with tuned hyperprior by following the same protocol as for the transformer (and yielding the same best configuration values); 3) DNGO \citep{} a Bayesian Neural Network which uses Bayesian linear regression model to generate a posterior distribution; and 4) BOHAMIANN \citep{}, a Bayesian Neural Networks trained via a stochastic gradient Hamiltonian Monte Carlo. For the BNN, we used the setting proposed by the original authors.
\textbf{Experimental Results}
We show an aggregation of the results for our experimental setup in Figure \citep{fig:mainresults}. Further details on the results per search space are in the Appendix \ref{}. We highlight the close performance of our method (Transformer) compared to the GP and BNN, which is possible to read from the average regret and rank plots. Specially the Crititical Difference diagrams indicate that the difference up to 5 trials is not significant.
\begin{figure}[htb!]
\centering
\begin{subfigure}[b]{0.58\textwidth}
\centering
\includegraphics[width=1.0\textwidth]{bo-plots/regret_and_rank.png}
\end{subfigure}
%\hfill
\begin{subfigure}[b]{0.41\textwidth}
%\vspace{-7cm}
\includegraphics[width=1.0\textwidth]{bo-plots/cd_diagram_1.png}
\medskip
\includegraphics[width=1.0\textwidth]{bo-plots/cd_diagram_5.png}
\medskip
\includegraphics[width=1.0\textwidth]{bo-plots/cd_diagram_10.png}
\end{subfigure}
\caption{Left: Comparisons of normalized regret and mean ranks across all search spaces for Random Search (RS), Gaussian Process (GP), Transformer, DNGO and BOHAMIAN. The bounds indicate the 95\% confidence interval. Right: Critical Diagram for showing the significance on ranks via Wilcoxon Test for 1, 5 and 10 trials. }
\label{fig:mainresults}
\end{figure}
\fi
\newpage
\section{Acknowledgements}
This research was funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under grant number 417962828.
Robert Bosch GmbH is acknowledged for financial support.
We also want to thank the maintainers of PyTorch \citep{NEURIPS2019_9015}.
\bibliography{lib,strings,proc,main}
\bibliographystyle{iclr2022_conference}
\newpage
\appendix
\section{Proof of Corollary \ref{corollary:klrelation}}
\label{sec:proof_klrelation}
\begin{proof}
This can easily be seen by considering the result of Insight \ref{theorem:loss} for the last transformation in line \ref{transformation:th1}.
\begin{align}
&\E_{x,D}[ \kl{}(p(\cdot | x,D), q_\theta(\cdot | x,D))]\\
&= - \E_{x,D}\left[ \int_y p(y|x,D) \log\frac{q_\theta(y|x,D)}{p(y|x,D)}\right]\\
&= - \E_{x,D}\left[ \int_y p(y|x,D) \log q_\theta(y|x,D)\right] + \E_{x,D}\left[ \int_y p(y|x,D) \log p(y|x,D)\right]\\
&= \E_{x,D}[\text{H}(p(\cdot | x,D), q_\theta(\cdot | x,D))] - \E_{x,D}[\text{H}(p(\cdot|x,D))]]\\
&= \ell_\theta + C, \label{transformation:th1}
\end{align}
where $C = - \E_{x,D}[\text{H}(\cdot|x,D)]$ is a constant that does not depend on $\theta$.
\end{proof}
\section{Proof of Corollary \ref{corollary:exactness}}
\label{sec:proof_exactness}
\begin{proof}
We assume a $\theta$ with $q_\theta = p$ exists and we know that the cross-entropy is minimized for equal distributions; thus the optimum $\theta^*$ fulfills $q_{\theta^*}(\cdot|x,D) = p(\cdot|x,D)$ for all $x,D$ where the density of the prior fulfills $p(x,D) > 0$, which is exactly the subset for which the conditional is defined.
\end{proof} % Todo could move to appendix if space is short.
\section{Proof Theorem \ref{theorem:distribution_approximation}}
\label{sec:proof_distribution_approximation}
\begin{proof}
Given an arbitrary distribution $p$, an arbitrary bound $\epsilon$. For arbitrary minimum (maximum) bucket borders $a$ ($b$), we consider the function $b_\buckets{}$ that maps each position $y$ to its bucket $b\in \buckets{}$, where $\buckets{}$ is an arbitrary set of buckets.
We define the upper bound of $p$ for each bucket
\begin{align}
u(y) = \sup_{y' \in b_s(y)} p(y').
\end{align}
Similarly, we define the lower bound
\begin{align}
l(y) = \inf_{y' \in b_s(y)} p(y').
\end{align}
For each of these we consider their $\log$ form, too. That is $\hat{u}(y) = \log u(y) = \sup_{y' \in b_s(y)} \log p(y')$ and $\hat{l}(y) = \log l(y) = \inf_{y' \in b_s(y)} \log p(y')$. We can move the $\log$ inside the $\sup$ ($\inf$), since $\log$ is steadily increasing.
Integrals over $\hat{u}$ and $\hat{l}$ thus handily represent Darboux\textquotesingle{s} upper and lower sums of $\log p$, if $\log p$ is Riemann integrable and finite. We assumed that $p$ is Riemann integrable, thus $\log p$ is also integrable, since the composition of continuous functions, which $\log x$ is for $x>0$, with an almost everywhere continuous function is almost everywhere continuous and any finite value $x$ has a finite value $\log x$, if $x>0$. The requirement $x>0$ is given due to the assumed full support of $p$.
Thus, we can use $\forall \epsilon' \exists \buckets{}. \int_a^b \hat{u}(y)-\hat{l}(y) dy < \epsilon'$.
We define $t = \sup_{y \in [a,b]} p(y)$, $T = \sup_{y \in \mathbb{R}} p(y)$ and the total probability mass of the interval $P=p(y\in [a,b])$.
We use $\epsilon' = \log 1/P$, which fulfills $\epsilon' > 0$ since $p$ has full support and thus for any borders $a$ and $b$ we have $P<1$, get $\buckets{}$ s.t.\
\begin{align}
&\int_a^b \hat{u}(y)-\hat{l}(y) dy < \log \frac{1}{P}\\
&\Rightarrow \int_a^b t(\hat{u}(y)-\hat{l}(y)) dy <t \log 1/P \\
&\Rightarrow \int_a^b t\log\frac{u(y)}{l(y)} dy < t \log 1/P\\
&\Rightarrow \int_a^b u(y)\log\frac{u(y)}{l(y)} dy <t \log 1/P\\
\intertext{Let $c$ be a constant s.t.\ $c u(y)$ yields a valid probability distribution with $\int c u(y) dy= \int_a^b c u(y) dy = 1$.}
&\Rightarrow \int_a^b c u(y)\log\frac{u(y)}{l(y)} dy < ct \log 1/P\\
&\Rightarrow \int_a^b c u(y)\log\frac{c u(y)}{l(y)} dy < c t \log 1/P + \int_a^b c u(y) \log c\\
&\Rightarrow \int_a^b c u(y)\log\frac{c u(y)}{l(y)} dy < c t\log 1/P + \log c\\
\intertext{We use $c < 1/P$, as $1/P$ would be the factor $c'$ to make $\int_a^b c' p(y) dy = 1$ and $c$ is this factor for $u(y)$, which has $u(y) \ge p(y)$ for any $y \in [a,b]$.}
&\Rightarrow \int_a^b c u(y)\log\frac{c u(y)}{l(y)} dy < c t\log 1/P + \log 1/P\\
&\Rightarrow \int_a^b c u(y)\log\frac{c u(y)}{l(y)} dy < (c t + 1)\log 1/P\\
&\Rightarrow \int_a^b c u(y)\log\frac{c u(y)}{l(y)} dy < - t \frac{\log P}{P} - \log P\\
&\Rightarrow \int_a^b c u(y)\log\frac{c u(y)}{l(y)} dy < - T \frac{\log P}{P} - \log P\\
&\Rightarrow \int_a^b c u(y)\log\frac{c u(y)}{p(y)} dy < - T \frac{\log P}{P} - \log P\\
&\Rightarrow \kl{}(cu(y),p(y)) < - T \frac{\log P}{P} - \log P\\
&\Rightarrow \kl{}(cu(y),p(y)) < - (T + P) \frac{\log P}{P}\\
&\Rightarrow \kl{}(cu(y),p(y)) < - (T + 1) \frac{\log P}{P}.
\end{align}
Here, $cu(y)$ is a valid bar distributions.
Now we want to find $a$ and $b$ such that $- (T + 1) \frac{\log P}{P} < \epsilon$, which is equivalent to
\begin{align}
\frac{1}{P} \log 1/P < \frac{\epsilon}{T+1}
\end{align}
Here, we can substitute the Lambert W function to arrive at the final result, that we should choose $a$ and $b$ such that $p([a,b]) \ge e^{-W_0(\frac{\epsilon}{T+1})}$, which is always possible as $W_0$ maps to $(0,\infty)$ for positive inputs, which we have with $\frac{\epsilon}{T+1}$. And we have that $e^{-x} < 1$ and $e^{-x} > 0$ for any $x \in (0,\infty)$.
Thus, we can conclude the proof. We could upper bound the $\kl{}$-Divergence for any $a$ and $b$. Then, based on that bound we found a setting for values $a$ and $b$ such that this upper bounds falls below $\epsilon$.
\end{proof}
\section{General Transformer Training Details}
\label{sec:general_training_details}
We follow the standard Transformer encoder setup.
In the whole discussion one should note the crucial difference to a standard MLE training: Our optimization problem does not allow for overfitting. An improvement in training loss always means an improvement in the closeness to the exact posterior. More training just improves performance and calibration. The only exception of this is the fine-tuning done for the few-shot learning experiment in Section \ref{sec:fewshotlearning}.
We use Adam \citep{kingma2015adam} and cosine decay \citep{loshchilov2016sgdr} with warmup. For all experiments we used a embedding size of 512, only for few-shot classification we used 1024.
The only hyper-parameters that we did fine-tune for the Transformer training were the batch size and learning rate.
Although, it usually was enough to choose a large enough batch size while the learning rate was more important.
Learning rates generally where chosen based on training performance, if not specified differently. We can do this as we generate new data during training and thus do not need to tune on held-out data.
We saw that search over the set $\{.00001,.0001,.0003,.001\}$ for a learning rate generally yielded good enough results.
Usually performance of the models got better with more training, thus the number of samples used for training generally depends on our compute budget.
For setups models considered we could improve performance with more training. We stopped when we were satisfied with the results.
We did not train longer than a few days on a single standard GPU for any \priorfittednetwork{}.
\newcommand{\subfigwidth}{.45\textwidth}
\pgfplotsset{width=\textwidth, height=.8\textwidth, yticklabel style={/pgf/number format/.cd,print sign},
yticklabel={\ifdim\tick pt=0pt$0$\else\axisdefaultticklabel\fi}}
\begin{figure}[h]
\centering
\begin{subfigure}[b]{\subfigwidth}
\input{comp_w_gp_1}
\end{subfigure}
\begin{subfigure}[b]{\subfigwidth}
\input{comp_w_gp_2}
\end{subfigure}
\centering
\begin{subfigure}[b]{\subfigwidth}
\input{comp_w_gp_3}
\end{subfigure}
\begin{subfigure}[b]{\subfigwidth}
\input{comp_w_gp_4}
\end{subfigure}
\caption{We show the mean and 95\% intervals of the original GP posterior in green and the corresponding \priorfittednetwork{} approximations in blue. We used a length scale of $0.1$.}
\label{fig:comp_w_gp}
\end{figure}
\begin{figure}[h]
\centering
\input{anp_4}
\caption{Like in plot \ref{fig:comp_w_gp}, we show the mean and 95\% confidence intervals for the exact GP prior in green and for our \priorfittednetwork{} in blue. (They are almost indistinguishable, with very small differences visible, e.g., at $x=0.4$.) To further highlight what a big step forward our method is, we additionally show the respective values for the relatively new Attentive Neural Processes (ANP) \citep{kim2018ANP}. We trained both methods in very different environments, but ANP was clearly given the bigger budget.}
\label{fig:comp_w_gp_and_anp}
\end{figure}
\section{Transformer Adaptions}
\subsection{An Efficient Architecture} \label{sec:appendix_efficient_architecture}
The first step of the model is to encode $x$ and $y$; in all our experiments, we use simple linear layers for this, but one could also use other application-specific encoding layers.
Our setup is equivariant to the ordering of the input dataset\textquotesingle{s} examples. To achieve this, we remove positional encodings and feed \xy{} pairs together to the Transformer as a sum of their encodings; see Appendix \ref{sec:riemannablation} for an ablation of the impact of the equivariance to the ordering.
In practice, we feed query points together for efficiency reasons; these are the only inputs for which position matters, and we thus use the output at their positions as prediction for their corresponding PPD over $y$.
%
Since different query points should not depend on each other, we use an attention mask that allows attention from every input example to every other input example and allows attention from every query points to every input example.
An example of the model and its attention can be seen in Figure \ref{fig:transformer_visualization}.
%The mask $M$ we use can be characterized by the number of input examples $m$ and the number of query points $n$, we write $M_{m,n}$.
%An example for such a matrix is
%\begin{align}
%M_{3,2} = \begin{bmatrix}
%1& 1& 1& 0& 0\\
%1& 1& 1& 0& 0\\
%1& 1& 1& 0& 0\\
%1& 1& 1& 1& 0\\
%1& 1& 1& 0& 1\\
%\end{bmatrix}.
%\end{align}
During training, we fix the total number of inputs $N=n+m$, where $n$ is the number of inputs and $m$ is the number of queries, and sample $n$ from a distribution, in order to allow the Transformer to learn to work with different dataset sizes.
Since for smaller $n$ we have more query points to train on, we over-sample larger $n$, s.t.\ the number of query points seen during training for each $n$ is approximately equal. That is, we sample each $n \in \{0,\dots,N-1\}$ with a weight $1/m = 1 / (N-n)$.
As prediction head we use a softmax for multi-class classification tasks, a sigmoid for binary classification tasks and the Riemann distribution for regression.
\subsection{Riemann Distribution Definition}
\label{sec:riemmanndefinition}
Given a set of of buckets $\buckets{}$, where each bucket $b \in \buckets{}$ describes an interval such that
\begin{align}
\bigcup\limits_{b \in \buckets{}} b = [a,b]
\end{align}
and for any $b,b' \in \buckets{}$, we have $b \cap b' = \varnothing$. Additionally, we define the upper-bound (lower-bound) of the lowest (highest) bucket to be $l$ ($f$), and a function $B(\cdot)$ that maps a $y \in \mathbb{R}$ to the unique bucket $b$ with $y \in b$. We assume to have a function $w$ mapping to the width of each bucket as $w(b) = \max_{y \in b} y - \min_{y \in b} y$.
Lastly, we assume the model gives us a probability $p_b$ for each bucket $b$.
In the finite case the Riemann distribution is defined as
\begin{align}
p(y) = p_{b(y)} / w(b(y)).
\end{align}
We normalize with the width of the bucket since $p_b$ describes the probability $p(y' \in b)$, but we are interested in the probability $p(y=y')$.
For the case of infinite support, we have
\begin{align}
f(y_q) =
\begin{cases}
2 \cdot f_{\mathcal{N}}(y_q - l), & \text{if}\ y_q \le l \\
2 \cdot f_{\mathcal{N}}(f - y_q), & \text{if}\ y_q \ge f \\
p_{b(y_q)}/b(y_q), & \text{otherwise}. \\
\end{cases}
\end{align}
Here, the Half-Normal distributions are scaled such that half the probability weight is inside the bounds of the first (last) bucket.
\subsection{Ablation Studies}
\label{sec:riemannablation}
Above, we introduce a novel way of doing regression with neural networks that departs from the tradition of using normal distributions.
Additionally, we remove the default positional embeddings from the Transformer. We show the impact of both approaches on the Transformer in Figure \ref{fig:ablation}. Notice that, while the effect of the Riemann Distribution is very pronounced, the effect of making our architecture permutation invariant only slowly increases with sequence length.
For each setup we searched over a set of the same four learning rates and evaluated the best performing on another sample from the distribution.
\begin{figure}[h]
\centering
\input{ablation_riemann}
\caption{In this figure we show the impact of different architectural decisions on the performance of our method in fitting a Gaussian Process. We use the same Gaussian Process as in Section \ref{sec:gpapprox}.}
\label{fig:ablation}
\end{figure}
\section{Details for Section \ref{sec:posteriorapproximationstudies}}
\label{sec:posteriorapproximationstudies_detail}
The error intervals that can be seen in the \priornll{} plots are 95\% percent confidence intervals over different dataset samples from the prior.
\paragraph{Details for Section \ref{sec:gpapprox}}
\label{sec:detail_gpapprox}
Here, we generally followed default settings for GPs. That is, we used the RBF-Kernel with zero-mean, a length scale of 0.6, 1e-4 additional noise on the diagonal of the covariance matrix. The output scale was set to 1. Only for the plots in the Figure \ref{fig:comp_w_gp} and Figure \ref{fig:comp_w_gp_and_anp} the length scale was decreased to 0.1.
The experiments with more examples in Figure \ref{fig:fastgp_main} did use 5 input dimension, while the qualitative visualizations only used one.
For all methods we sample 1000 datasets for evaluation, each of which has a single validation example, for each dataset size.
\paragraph{Details for Section \ref{gp_hyper_priors}}
\label{sec:detail_gp_hyper_priors}
In this section, we followed the default settings that can be found in BoTorch\textquotesingle{s} \emph{SingleTaskGP} \citep{balandat2020botorch}.
We only changed the distribution over the diagonal noise, as the default is very high, such that it is hard to see any improvements with any of the methods in NLL with an increasing number of training samples.
The GP is setup as a Mat\'{e}rn Kernel with $\nu=2.5$ and we have the following hyper-priors: the noise was sampled from $\Gamma(\alpha=0.0001,\beta=1.0)$, the output scale from $\Gamma(\alpha=2.0,\beta=0.15)$ and the length scale from $\Gamma(\alpha=3.0,\beta=6.0)$.
We used the Pyro \citep{bingham2018pyro} implementation of NUTS. Here, we used every sampled hyper-parameter setting as a sample GP. We averaged the output distribution over all sampled GPs to yield an approximation to the PPD. We scaled both the warmup and the rollout steps together: with ``8 (warmup) steps'' we mean 8 warmup steps and then 8 sampling steps.
Only the MCMC experiments were run on CPU, as this was faster than using the GPU.
For all methods we sample 1000 datasets for evaluation, each of which has a single validation example, for each dataset size.
\paragraph{Details for Section \ref{sec:BNN_ppd}}
\label{sec:detail_BNN_ppd}
For SVI (Bayes-by-Backprop) we followed the setup of \citep{blundell-icml15a} and used Adam \citep{kingma2015adam} as optimizer. We leveraged Pyro \citep{bingham2018pyro} for optimization. We scaled both the training steps and number of samples averaged together to increase approximation quality.
For MCMC we used the Pyro \citep{bingham2018pyro} implementation of NUTS and averaged the output distribution over all sampled BNNs to yield an approximation to the PPD. We scaled both the warmup and the rollout steps together to increase approximation performance. We used one MCMC chain.
\priorfittednetwork{} and SVI are evaluated on a GPU (Nvidia Tesla V100) while MCMC is evaluated on a CPU since it was faster on CPU than on GPU.
\iffalse
% Paragraph on discussion of MCMC
MCMC methods can be
considered among the best available and the most popular
solutions for sampling from exact posterior distributions in
Bayesian statistics \citep{bardenet2015markov}. The most relevant MCMC methods for BNNs are
the Metropolis-Hastings algorithms [71]. The property which
makes Metropolis-Hasting algorithms popular is that they do
not require knowledge about the exact probability distribution
P(x) to sample from.
Most software packages for Bayesian statistics implement
the No-U-Turn sampler (NUTS for short) [73], which is an
improvement over the classic Hamiltonian Monte-Carlo algorithm allowing the
hyperparameters of the algorithm to be tweaked automatically
instead of having to be set manually \citep{}. We use NUTS in all conducted experiments.
% SGLD, HMC
% Paragraph on discussion of SVI
MCMC algorithms are the goto tools to sample from the exact posterior. However, their lack of scalability has made them less popular for BNNs, given the size of the models under
consideration. Variational Inference [74], which scales better
than MCMC algorithms, gained a lot of popularity. Variational
Inference is not an exact method. The most widely used method is
Bayes-by-Backprop, a practical implementation of
SVI combined with a reparametrization trick [82] to ensure
backpropagation works as usual.
% MC-Dropout,
% Deep ensembles, ...
% https://arxiv.org/pdf/2007.06823.pdf
% empirical Bayes
\fi
\iffalse
\begin{figure}[t]
\centering
\begin{subfigure}[b]{.49\textwidth}
\centering
\subcaption{}
\input{bayesian_models_eval_samples_small_3}
\end{subfigure}
\centering
\begin{subfigure}[b]{.49\textwidth}
\centering
\subcaption{}
\input{bayesian_models_eval_model_size}
\end{subfigure}
\caption{(a) We vary the number of revealed datapoints and evaluate the negative log likelihood. PFNs compare best across all evaluated datapoints. 95\% confidence interval is plotted. A BNN architecture with 3 features, 2 layers and a hidden dimensionality of 5 is evaluated. PFNs are 2000x faster than SVI and 12000x faster than MCMC. 95\% confidence interval is plotted. Steps refer to sampling and training steps in MCMC and SVI. (b) We vary the size of the inferred Bayesian Neural Network and compare SVI vs PFNs. The generated data as well as the approximating method are both using this model. Thus the task is less specified with larger models and also more computationally complex. Model N uses $2^N$ layers, $16*2^N$ hidden dimensionality, $2^N$ input features. SVI and PFNs scale to larger prior models in a similar way. 95\% confidence interval is plotted. We did not compare MCMC since the time budgets required for larger model sizes exceed feasibility. The crucial difference here is, that our model\textquotesingle{s} compute requirements do not increase, unlike the requirements to train SVI.}
\label{fig:blabla}
\end{figure}
\fi
\section{Details for Section 6}
\label{sec:detail_tabular}
\textbf{Accomodating a variable number of features} Since the evaluated datasets used a variable number of features, the dimensionality of the input is variable. This can not be accommodated by the fixed size of the linear layer used in our \priorfittednetworks{}. Thus, we fixed a maximum number of features (60) and extended the datasets to this maximum number by appending zeros. We did this during prior training, sampling the number of features uniformly at random alongside the model architecture and during inference. We rescale the features by a factor of $\frac{\text{max features}}{\text{features used}}$ during prior training and bayesian inference to keep the mean input scale constant with a varying feature number.\\
\iffalse
\textbf{Ordering of samples improves predictions} Since we have perfectly balanced datasets, we can order training samples in each dataset such that a sample of class one is always followed by a sample of class zero and vice versa. In order to keep the task as constant as possible we apply this ordering during prior training and bayesian inference.\\
\fi
\textbf{Degrees of freedom for model architectures} The evaluated BNN models consider a wide range of model architectures, that is those architectures vary in terms of: number of layers, number of hidden units per layer, layer sparsity, standard deviation of Gaussian noise applied to each unit and activation function applied to each unit. This list can be easily extended by adapting the sampling procedure.\\
\textbf{Tuning hyperparameter ranges for hyper-priors and cross-validation} While the Transformer-based methods need no parameter tuning on each dataset individually, they still require setting the prior hyperparameters. This is similar to setting the range of hyperparameters in a cross-validation grid search and can be reused for all similar problems, e.g., all small-scale tabular classification. We therefore run a hyperparameter search on a second set of validation datasets, listed in Table \ref{collection_of_datasets_tabular_valid}. The hyperparameters considered in this grid search can be found in \ref{table:hyperparameters_PFN_tabular}
In this search, we find good settings for all hyper-parameters of our priors for the average over all validation datasets. This search results in one final hyper-parameter setting and only one single Transformer model that is reused for all datasets. We use the same setup to find cross-validation ranges for our baselines.\\
\textbf{Standardization of datasets} Each subset is standardized to have zero mean and unit variance where the statistics for the standardization are only calculated on the samples considered by the model, thus without any dependence on the test set. \\
%, which yielded the parameters given in Table \ref{table:tabular_hyperparameters}.
% I don't think this is interesting even in the Appendix
% \textbf{Hyperparameter tuning for priors}
% As described in section \ref{sec:tabular_datasets} run a hyperparameter search to find good settings for all hyper-parameters of our priors for the average over all validation datasets resulting in one final hyper-parameter setting for all datasets. The list of hyperparameters used can be found
\textbf{Runtimes of prior fitting step} Fitting our transformer with GP prior was done over 100 epochs, taking 2:44h, while fitting the BNN prior took 3:14h for the same number of epochs. Crucially, this model is reused for all datasets.\\
\begin{table}[h]
\centering
{\input{tabular_datasets_table}}
\caption{Tabular datasets used for evaluation. Taken from the OpenML AutoML Benchmark (filtered for $N_{features}$ $\leq$ 100, no missing values.)}
\label{collection_of_datasets_tabular}
\end{table}
\begin{table}[h]
\centering
{\input{tabular_datasets_table_valid}}
\caption{Tabular datasets used to find good general hyperparameters for hyper-priors as well as hyperparameters used in baseline cross-validation.}
\label{collection_of_datasets_tabular_valid}
\end{table}
\begin{table}[h]
\centering
{\input{tabular_hyperparameters_bnn}}
\caption{Hyperparameters considered during grid search tuning of the PFN-BNN on validation datasets. The activation functions refer to the activation function used in the data generating BNN and not the ones used in the PFN transformer. scaled\_beta\_sampler\_f(a,b,max,min) refers to sampling from a beta distribution with parameters a, b (numpy.random.beta), scaling the output range from [0, 1] to [max, min] and then rounding the output to the integer range. beta\_sampler\_f(a, b) refers to a function that samples from the beta distribution without scaling and rounding.}
\label{table:hyperparameters_PFN_tabular}
\end{table}
\begin{table}[h]
\centering
\small
{\input{tabular_hparams_basleine}}
\caption{Hyperparameters used in cross-validation for each baseline method.}
\label{table:hyperparameters_tabular_baselines}
\end{table}
\begin{table}[h]
\centering
\input{tabular_auc_full}
\caption{ROC AUC Comparison: Evaluation is made on subsets of datasets with 30 training samples. 20 data-subsets are evaluated for each dataset. The ECE is a calibration measure. The novel PFN-BNN performs strongest overall.}
\label{table:tabular_auc_table_full}
\end{table}
\begin{figure}[t]
%\centering
%\input{riemann_distribution}
\centering
\begin{subfigure}[b]{0.32\textwidth}
\centering
\caption{Bayesian Neural Network}
\input{tabular/tabular_calibration_bnn}
\end{subfigure}
\hfill
\begin{subfigure}[b]{0.32\textwidth}
\centering
\caption{PFN-BNN}
\input{tabular/tabular_calibration_transformer_bayesian}
\end{subfigure}
\hfill
\begin{subfigure}[b]{0.32\textwidth}
\centering
\caption{XGBoost}
\input{tabular/tabular_calibration_xgb}
\end{subfigure}
\caption{Comparison of uncertainty calibration curves over all predictions on the evaluated datasets. The dotted line indicates perfect calibration.}
\label{fig:calibration}
\end{figure}
\section{Details for Section \ref{sec:fewshotlearning}}
\label{sec:section7details}
We searched for the hyper-parameters specified as inputs in Figure \ref{fig:strokepriorcode} with a grid search, where we measure performance on the training tasks as validation set.
We then trained our model as a \priorfittednetwork{} on samples from this prior.
After this stage we fine-tuned the model on the training tasks of Omniglot. Here, we applied translations and used a default batch size of 100.
We used a learning rate of 1e-5 to yield the performance shown in the plot after sampling 500,000 samples from the training tasks.
Unlike above, we sample over different samples of our fine-tuned model to yield the confidence interval, to follow the standard reporting on the Omniglot benchmark.
%Something special about this setup is that the Omniglot data actually lies outside the prior we defined. That means some of the Omniglot tasks have zero-probability under the prior, because they use curves, while our prior only consists of straigt lines. We could see that our \priorfittednetwork{} generalizes to Omniglot examples, even without fine-tuning, but this difference of Omniglot to our prior, explains the importance of fine-tuning for this example.
\begin{figure}
\centering
\tiny
\begin{lstlisting}
def pencil_stroke_prior(num_classes=2, size=28, min_max_strokes=(1,3),
min_max_len=(5/28,20/28), min_max_start=(2/28,25/28),
min_max_width=(1/28,4/28),max_offset=4/28, max_target_offset=2/28):
classes = []
for i in range(num_classes):
num_strokes = random.randint(*min_max_strokes)
len_strokes = [random.randint(int(size * min_max_len[0]),
int(size * min_max_len[1]))
for i in range(num_strokes)]
stroke_start_points = [
(random.randint(int(size * min_max_start[0]), int(size * min_max_start[1])),
random.randint(int(size * min_max_start[0]), int(size * min_max_start[1])))
for i in range(num_strokes)]
stroke_directions = []
# i = Image.fromarray(np.zeros((28,28),dtype=np.uint8))
# draw = ImageDraw.Draw(i)
for i in range(num_strokes):
sp, length = stroke_start_points[i], len_strokes[i]
counter = 0
while True:
if counter % 3 == 0:
length = random.randint(int(size * min_max_len[0]),
int(size * min_max_len[1]))
sp = (
random.randint(int(size * min_max_start[0]),
int(size * min_max_start[1])),
random.randint(int(size * min_max_start[0]),
int(size * min_max_start[1])))
stroke_start_points[i], len_strokes[i] = sp, length
radians = random.random() * 2 * math.pi
x_vel = math.cos(radians) * length
y_vel = math.sin(radians) * length
new_p = (sp[0] + x_vel, sp[1] + y_vel)
if not any(n > size - 1 or n < 0 for n in new_p):
break
counter += 1
stroke_directions.append(radians)
classes.append((len_strokes, stroke_start_points, stroke_directions))
generator_functions = []
for c in classes:
def g(c=c):
len_strokes, stroke_start_points, stroke_directions = c
i = Image.fromarray(np.zeros((size, size), dtype=np.uint8))
draw = ImageDraw.Draw(i)
width = random.randint(int(size * min_max_width[0]),
int(size * min_max_width[1]))
offset = (random.randint(int(-size * max_offset), int(size * max_offset)),
random.randint(int(- size * max_offset), int(size * max_offset)))
for sp, length, radians in \textbackslash
zip(stroke_start_points, len_strokes, stroke_directions):
sp = (sp[0] + offset[0], sp[1] + offset[1])
x_vel = math.cos(radians) * length
x_vel += random.randint(int(-size * max_target_offset),
int(size * max_target_offset))
y_vel = math.sin(radians) * length
y_vel += random.randint(int(-size * max_target_offset),
int(size * max_target_offset))
new_p = (sp[0] + x_vel, sp[1] + y_vel)
stroke_directions.append(radians)
draw.line([round(x) for x in sp + new_p], fill=128, width=width)
a_i = np.array(i)
a_i[a_i == 128] = np.random.randint(200, 255, size=a_i.shape)[a_i == 128]
return Image.fromarray(a_i).filter(ImageFilter.GaussianBlur(.2))
generator_functions.append(g)
return generator_functions
\end{lstlisting}
\caption{A prior definition for a prior over straight strokes in python.}
\label{fig:strokepriorcode}
\end{figure}
\begin{figure}
\centering
\begin{tikzpicture}
\begin{axis}[
width=.8\textwidth, height=.4\textwidth,
tick align=outside,
tick pos=left,
x grid style={white!69.0196078431373!black},
ytick = \empty,
xtick = {3,5.1},
axis line style={draw=none},
tick style={draw=none},
ylabel = {\footnotesize Different Samples},
xticklabels = {{\footnotesize Training Examples},{\footnotesize Query Example}},
xtick style={color=black},
y grid style={white!69.0196078431373!black},
ytick style={color=black},
scale only axis,
enlargelimits=false,
axis on top]
\addplot graphics[xmin=.5,xmax=5.5,ymin=.5,ymax=3.5] {strokesample.png};
\end{axis}
\end{tikzpicture}
\caption{In this figure we show different samples from our prior for Omniglot data.}
\label{fig:strokesamples}
\end{figure}
\iffalse
\begin{figure}[h]
\centering
\includegraphics[width=1\columnwidth]{bo-plots/BO_on_function.png}
\caption{\textbf{BO on a toy function sampled from prior.}}
\label{fig:rank_per_space}
\end{figure}
\fi
\end{document}
%%%%% NEW MATH DEFINITIONS %%%%%
\usepackage{amsmath,amsfonts,bm}
% Mark sections of captions for referring to divisions of figures
\newcommand{\figleft}{{\em (Left)}}
\newcommand{\figcenter}{{\em (Center)}}
\newcommand{\figright}{{\em (Right)}}
\newcommand{\figtop}{{\em (Top)}}
\newcommand{\figbottom}{{\em (Bottom)}}
\newcommand{\captiona}{{\em (a)}}
\newcommand{\captionb}{{\em (b)}}
\newcommand{\captionc}{{\em (c)}}
\newcommand{\captiond}{{\em (d)}}
% Highlight a newly defined term
\newcommand{\newterm}[1]{{\bf #1}}
% Figure reference, lower-case.
\def\figref#1{figure~\ref{#1}}
% Figure reference, capital. For start of sentence
\def\Figref#1{Figure~\ref{#1}}
\def\twofigref#1#2{figures \ref{#1} and \ref{#2}}
\def\quadfigref#1#2#3#4{figures \ref{#1}, \ref{#2}, \ref{#3} and \ref{#4}}
% Section reference, lower-case.
\def\secref#1{section~\ref{#1}}
% Section reference, capital.
\def\Secref#1{Section~\ref{#1}}
% Reference to two sections.
\def\twosecrefs#1#2{sections \ref{#1} and \ref{#2}}
% Reference to three sections.
\def\secrefs#1#2#3{sections \ref{#1}, \ref{#2} and \ref{#3}}
% Reference to an equation, lower-case.
\def\eqref#1{equation~\ref{#1}}
% Reference to an equation, upper case
\def\Eqref#1{Equation~\ref{#1}}
% A raw reference to an equation---avoid using if possible
\def\plaineqref#1{\ref{#1}}
% Reference to a chapter, lower-case.
\def\chapref#1{chapter~\ref{#1}}
% Reference to an equation, upper case.
\def\Chapref#1{Chapter~\ref{#1}}
% Reference to a range of chapters
\def\rangechapref#1#2{chapters\ref{#1}--\ref{#2}}
% Reference to an algorithm, lower-case.
\def\algref#1{algorithm~\ref{#1}}
% Reference to an algorithm, upper case.
\def\Algref#1{Algorithm~\ref{#1}}
\def\twoalgref#1#2{algorithms \ref{#1} and \ref{#2}}
\def\Twoalgref#1#2{Algorithms \ref{#1} and \ref{#2}}
% Reference to a part, lower case
\def\partref#1{part~\ref{#1}}
% Reference to a part, upper case
\def\Partref#1{Part~\ref{#1}}
\def\twopartref#1#2{parts \ref{#1} and \ref{#2}}
\def\ceil#1{\lceil #1 \rceil}
\def\floor#1{\lfloor #1 \rfloor}
\def\1{\bm{1}}
\newcommand{\train}{\mathcal{D}}
\newcommand{\valid}{\mathcal{D_{\mathrm{valid}}}}
\newcommand{\test}{\mathcal{D_{\mathrm{test}}}}
\def\eps{{\epsilon}}
% Random variables
\def\reta{{\textnormal{$\eta$}}}
\def\ra{{\textnormal{a}}}
\def\rb{{\textnormal{b}}}
\def\rc{{\textnormal{c}}}
\def\rd{{\textnormal{d}}}
\def\re{{\textnormal{e}}}
\def\rf{{\textnormal{f}}}
\def\rg{{\textnormal{g}}}
\def\rh{{\textnormal{h}}}
\def\ri{{\textnormal{i}}}
\def\rj{{\textnormal{j}}}
\def\rk{{\textnormal{k}}}
\def\rl{{\textnormal{l}}}
% rm is already a command, just don't name any random variables m
\def\rn{{\textnormal{n}}}
\def\ro{{\textnormal{o}}}
\def\rp{{\textnormal{p}}}
\def\rq{{\textnormal{q}}}
\def\rr{{\textnormal{r}}}
\def\rs{{\textnormal{s}}}
\def\rt{{\textnormal{t}}}
\def\ru{{\textnormal{u}}}
\def\rv{{\textnormal{v}}}
\def\rw{{\textnormal{w}}}
\def\rx{{\textnormal{x}}}
\def\ry{{\textnormal{y}}}
\def\rz{{\textnormal{z}}}
% Random vectors
\def\rvepsilon{{\mathbf{\epsilon}}}
\def\rvtheta{{\mathbf{\theta}}}
\def\rva{{\mathbf{a}}}
\def\rvb{{\mathbf{b}}}
\def\rvc{{\mathbf{c}}}
\def\rvd{{\mathbf{d}}}
\def\rve{{\mathbf{e}}}
\def\rvf{{\mathbf{f}}}
\def\rvg{{\mathbf{g}}}
\def\rvh{{\mathbf{h}}}
\def\rvu{{\mathbf{i}}}
\def\rvj{{\mathbf{j}}}
\def\rvk{{\mathbf{k}}}
\def\rvl{{\mathbf{l}}}
\def\rvm{{\mathbf{m}}}
\def\rvn{{\mathbf{n}}}
\def\rvo{{\mathbf{o}}}
\def\rvp{{\mathbf{p}}}
\def\rvq{{\mathbf{q}}}
\def\rvr{{\mathbf{r}}}
\def\rvs{{\mathbf{s}}}
\def\rvt{{\mathbf{t}}}
\def\rvu{{\mathbf{u}}}
\def\rvv{{\mathbf{v}}}
\def\rvw{{\mathbf{w}}}
\def\rvx{{\mathbf{x}}}
\def\rvy{{\mathbf{y}}}
\def\rvz{{\mathbf{z}}}
% Elements of random vectors
\def\erva{{\textnormal{a}}}
\def\ervb{{\textnormal{b}}}
\def\ervc{{\textnormal{c}}}
\def\ervd{{\textnormal{d}}}
\def\erve{{\textnormal{e}}}
\def\ervf{{\textnormal{f}}}
\def\ervg{{\textnormal{g}}}
\def\ervh{{\textnormal{h}}}
\def\ervi{{\textnormal{i}}}
\def\ervj{{\textnormal{j}}}
\def\ervk{{\textnormal{k}}}
\def\ervl{{\textnormal{l}}}
\def\ervm{{\textnormal{m}}}
\def\ervn{{\textnormal{n}}}
\def\ervo{{\textnormal{o}}}
\def\ervp{{\textnormal{p}}}
\def\ervq{{\textnormal{q}}}
\def\ervr{{\textnormal{r}}}
\def\ervs{{\textnormal{s}}}
\def\ervt{{\textnormal{t}}}
\def\ervu{{\textnormal{u}}}
\def\ervv{{\textnormal{v}}}
\def\ervw{{\textnormal{w}}}
\def\ervx{{\textnormal{x}}}
\def\ervy{{\textnormal{y}}}
\def\ervz{{\textnormal{z}}}
% Random matrices
\def\rmA{{\mathbf{A}}}
\def\rmB{{\mathbf{B}}}
\def\rmC{{\mathbf{C}}}
\def\rmD{{\mathbf{D}}}
\def\rmE{{\mathbf{E}}}
\def\rmF{{\mathbf{F}}}
\def\rmG{{\mathbf{G}}}
\def\rmH{{\mathbf{H}}}
\def\rmI{{\mathbf{I}}}
\def\rmJ{{\mathbf{J}}}
\def\rmK{{\mathbf{K}}}
\def\rmL{{\mathbf{L}}}
\def\rmM{{\mathbf{M}}}
\def\rmN{{\mathbf{N}}}
\def\rmO{{\mathbf{O}}}
\def\rmP{{\mathbf{P}}}
\def\rmQ{{\mathbf{Q}}}
\def\rmR{{\mathbf{R}}}
\def\rmS{{\mathbf{S}}}
\def\rmT{{\mathbf{T}}}
\def\rmU{{\mathbf{U}}}
\def\rmV{{\mathbf{V}}}
\def\rmW{{\mathbf{W}}}
\def\rmX{{\mathbf{X}}}
\def\rmY{{\mathbf{Y}}}
\def\rmZ{{\mathbf{Z}}}
% Elements of random matrices
\def\ermA{{\textnormal{A}}}
\def\ermB{{\textnormal{B}}}
\def\ermC{{\textnormal{C}}}
\def\ermD{{\textnormal{D}}}
\def\ermE{{\textnormal{E}}}
\def\ermF{{\textnormal{F}}}
\def\ermG{{\textnormal{G}}}
\def\ermH{{\textnormal{H}}}
\def\ermI{{\textnormal{I}}}
\def\ermJ{{\textnormal{J}}}
\def\ermK{{\textnormal{K}}}
\def\ermL{{\textnormal{L}}}
\def\ermM{{\textnormal{M}}}
\def\ermN{{\textnormal{N}}}
\def\ermO{{\textnormal{O}}}
\def\ermP{{\textnormal{P}}}
\def\ermQ{{\textnormal{Q}}}
\def\ermR{{\textnormal{R}}}
\def\ermS{{\textnormal{S}}}
\def\ermT{{\textnormal{T}}}
\def\ermU{{\textnormal{U}}}
\def\ermV{{\textnormal{V}}}
\def\ermW{{\textnormal{W}}}
\def\ermX{{\textnormal{X}}}
\def\ermY{{\textnormal{Y}}}
\def\ermZ{{\textnormal{Z}}}
% Vectors
\def\vzero{{\bm{0}}}
\def\vone{{\bm{1}}}
\def\vmu{{\bm{\mu}}}
\def\vtheta{{\bm{\theta}}}
\def\va{{\bm{a}}}
\def\vb{{\bm{b}}}
\def\vc{{\bm{c}}}
\def\vd{{\bm{d}}}
\def\ve{{\bm{e}}}
\def\vf{{\bm{f}}}
\def\vg{{\bm{g}}}
\def\vh{{\bm{h}}}
\def\vi{{\bm{i}}}
\def\vj{{\bm{j}}}
\def\vk{{\bm{k}}}
\def\vl{{\bm{l}}}
\def\vm{{\bm{m}}}
\def\vn{{\bm{n}}}
\def\vo{{\bm{o}}}
\def\vp{{\bm{p}}}
\def\vq{{\bm{q}}}
\def\vr{{\bm{r}}}
\def\vs{{\bm{s}}}
\def\vt{{\bm{t}}}
\def\vu{{\bm{u}}}
\def\vv{{\bm{v}}}
\def\vw{{\bm{w}}}
\def\vx{{\bm{x}}}
\def\vy{{\bm{y}}}
\def\vz{{\bm{z}}}
% Elements of vectors
\def\evalpha{{\alpha}}
\def\evbeta{{\beta}}
\def\evepsilon{{\epsilon}}
\def\evlambda{{\lambda}}
\def\evomega{{\omega}}
\def\evmu{{\mu}}
\def\evpsi{{\psi}}
\def\evsigma{{\sigma}}
\def\evtheta{{\theta}}
\def\eva{{a}}
\def\evb{{b}}
\def\evc{{c}}
\def\evd{{d}}
\def\eve{{e}}
\def\evf{{f}}
\def\evg{{g}}
\def\evh{{h}}
\def\evi{{i}}
\def\evj{{j}}
\def\evk{{k}}
\def\evl{{l}}
\def\evm{{m}}
\def\evn{{n}}
\def\evo{{o}}
\def\evp{{p}}
\def\evq{{q}}
\def\evr{{r}}
\def\evs{{s}}
\def\evt{{t}}
\def\evu{{u}}
\def\evv{{v}}
\def\evw{{w}}
\def\evx{{x}}
\def\evy{{y}}
\def\evz{{z}}
% Matrix
\def\mA{{\bm{A}}}
\def\mB{{\bm{B}}}
\def\mC{{\bm{C}}}
\def\mD{{\bm{D}}}
\def\mE{{\bm{E}}}
\def\mF{{\bm{F}}}
\def\mG{{\bm{G}}}
\def\mH{{\bm{H}}}
\def\mI{{\bm{I}}}
\def\mJ{{\bm{J}}}
\def\mK{{\bm{K}}}
\def\mL{{\bm{L}}}
\def\mM{{\bm{M}}}
\def\mN{{\bm{N}}}
\def\mO{{\bm{O}}}
\def\mP{{\bm{P}}}
\def\mQ{{\bm{Q}}}
\def\mR{{\bm{R}}}
\def\mS{{\bm{S}}}
\def\mT{{\bm{T}}}
\def\mU{{\bm{U}}}
\def\mV{{\bm{V}}}
\def\mW{{\bm{W}}}
\def\mX{{\bm{X}}}
\def\mY{{\bm{Y}}}
\def\mZ{{\bm{Z}}}
\def\mBeta{{\bm{\beta}}}
\def\mPhi{{\bm{\Phi}}}
\def\mLambda{{\bm{\Lambda}}}
\def\mSigma{{\bm{\Sigma}}}
% Tensor
\DeclareMathAlphabet{\mathsfit}{\encodingdefault}{\sfdefault}{m}{sl}
\SetMathAlphabet{\mathsfit}{bold}{\encodingdefault}{\sfdefault}{bx}{n}
\newcommand{\tens}[1]{\bm{\mathsfit{#1}}}
\def\tA{{\tens{A}}}
\def\tB{{\tens{B}}}
\def\tC{{\tens{C}}}
\def\tD{{\tens{D}}}
\def\tE{{\tens{E}}}
\def\tF{{\tens{F}}}
\def\tG{{\tens{G}}}
\def\tH{{\tens{H}}}
\def\tI{{\tens{I}}}
\def\tJ{{\tens{J}}}
\def\tK{{\tens{K}}}
\def\tL{{\tens{L}}}
\def\tM{{\tens{M}}}
\def\tN{{\tens{N}}}
\def\tO{{\tens{O}}}
\def\tP{{\tens{P}}}
\def\tQ{{\tens{Q}}}
\def\tR{{\tens{R}}}
\def\tS{{\tens{S}}}
\def\tT{{\tens{T}}}
\def\tU{{\tens{U}}}
\def\tV{{\tens{V}}}
\def\tW{{\tens{W}}}
\def\tX{{\tens{X}}}
\def\tY{{\tens{Y}}}
\def\tZ{{\tens{Z}}}
% Graph
\def\gA{{\mathcal{A}}}
\def\gB{{\mathcal{B}}}
\def\gC{{\mathcal{C}}}
\def\gD{{\mathcal{D}}}
\def\gE{{\mathcal{E}}}
\def\gF{{\mathcal{F}}}
\def\gG{{\mathcal{G}}}
\def\gH{{\mathcal{H}}}
\def\gI{{\mathcal{I}}}
\def\gJ{{\mathcal{J}}}
\def\gK{{\mathcal{K}}}
\def\gL{{\mathcal{L}}}
\def\gM{{\mathcal{M}}}
\def\gN{{\mathcal{N}}}
\def\gO{{\mathcal{O}}}
\def\gP{{\mathcal{P}}}
\def\gQ{{\mathcal{Q}}}
\def\gR{{\mathcal{R}}}
\def\gS{{\mathcal{S}}}
\def\gT{{\mathcal{T}}}
\def\gU{{\mathcal{U}}}
\def\gV{{\mathcal{V}}}
\def\gW{{\mathcal{W}}}
\def\gX{{\mathcal{X}}}
\def\gY{{\mathcal{Y}}}
\def\gZ{{\mathcal{Z}}}
% Sets
\def\sA{{\mathbb{A}}}
\def\sB{{\mathbb{B}}}
\def\sC{{\mathbb{C}}}
\def\sD{{\mathbb{D}}}
% Don't use a set called E, because this would be the same as our symbol
% for expectation.
\def\sF{{\mathbb{F}}}
\def\sG{{\mathbb{G}}}
\def\sH{{\mathbb{H}}}
\def\sI{{\mathbb{I}}}
\def\sJ{{\mathbb{J}}}
\def\sK{{\mathbb{K}}}
\def\sL{{\mathbb{L}}}
\def\sM{{\mathbb{M}}}
\def\sN{{\mathbb{N}}}
\def\sO{{\mathbb{O}}}
\def\sP{{\mathbb{P}}}
\def\sQ{{\mathbb{Q}}}
\def\sR{{\mathbb{R}}}
\def\sS{{\mathbb{S}}}
\def\sT{{\mathbb{T}}}
\def\sU{{\mathbb{U}}}
\def\sV{{\mathbb{V}}}
\def\sW{{\mathbb{W}}}
\def\sX{{\mathbb{X}}}
\def\sY{{\mathbb{Y}}}
\def\sZ{{\mathbb{Z}}}
% Entries of a matrix
\def\emLambda{{\Lambda}}
\def\emA{{A}}
\def\emB{{B}}
\def\emC{{C}}
\def\emD{{D}}
\def\emE{{E}}
\def\emF{{F}}
\def\emG{{G}}
\def\emH{{H}}
\def\emI{{I}}
\def\emJ{{J}}
\def\emK{{K}}
\def\emL{{L}}
\def\emM{{M}}
\def\emN{{N}}
\def\emO{{O}}
\def\emP{{P}}
\def\emQ{{Q}}
\def\emR{{R}}
\def\emS{{S}}
\def\emT{{T}}
\def\emU{{U}}
\def\emV{{V}}
\def\emW{{W}}
\def\emX{{X}}
\def\emY{{Y}}
\def\emZ{{Z}}
\def\emSigma{{\Sigma}}
% entries of a tensor
% Same font as tensor, without \bm wrapper
\newcommand{\etens}[1]{\mathsfit{#1}}
\def\etLambda{{\etens{\Lambda}}}
\def\etA{{\etens{A}}}
\def\etB{{\etens{B}}}
\def\etC{{\etens{C}}}
\def\etD{{\etens{D}}}
\def\etE{{\etens{E}}}
\def\etF{{\etens{F}}}
\def\etG{{\etens{G}}}
\def\etH{{\etens{H}}}
\def\etI{{\etens{I}}}
\def\etJ{{\etens{J}}}
\def\etK{{\etens{K}}}
\def\etL{{\etens{L}}}
\def\etM{{\etens{M}}}
\def\etN{{\etens{N}}}
\def\etO{{\etens{O}}}
\def\etP{{\etens{P}}}
\def\etQ{{\etens{Q}}}
\def\etR{{\etens{R}}}
\def\etS{{\etens{S}}}
\def\etT{{\etens{T}}}
\def\etU{{\etens{U}}}
\def\etV{{\etens{V}}}
\def\etW{{\etens{W}}}
\def\etX{{\etens{X}}}
\def\etY{{\etens{Y}}}
\def\etZ{{\etens{Z}}}
% The true underlying data generating distribution
\newcommand{\pdata}{p_{\rm{data}}}
% The empirical distribution defined by the training set
\newcommand{\ptrain}{\hat{p}_{\rm{data}}}
\newcommand{\Ptrain}{\hat{P}_{\rm{data}}}
% The model distribution
\newcommand{\pmodel}{p_{\rm{model}}}
\newcommand{\Pmodel}{P_{\rm{model}}}
\newcommand{\ptildemodel}{\tilde{p}_{\rm{model}}}
% Stochastic autoencoder distributions
\newcommand{\pencode}{p_{\rm{encoder}}}
\newcommand{\pdecode}{p_{\rm{decoder}}}
\newcommand{\precons}{p_{\rm{reconstruct}}}
\newcommand{\laplace}{\mathrm{Laplace}} % Laplace distribution
\newcommand{\E}{\mathbb{E}}
\newcommand{\Ls}{\mathcal{L}}
\newcommand{\R}{\mathbb{R}}
\newcommand{\emp}{\tilde{p}}
\newcommand{\lr}{\alpha}
\newcommand{\reg}{\lambda}
\newcommand{\rect}{\mathrm{rectifier}}
\newcommand{\softmax}{\mathrm{softmax}}
\newcommand{\sigmoid}{\sigma}
\newcommand{\softplus}{\zeta}
\newcommand{\KL}{D_{\mathrm{KL}}}
\newcommand{\Var}{\mathrm{Var}}
\newcommand{\standarderror}{\mathrm{SE}}
\newcommand{\Cov}{\mathrm{Cov}}
% Wolfram Mathworld says $L^2$ is for function spaces and $\ell^2$ is for vectors
% But then they seem to use $L^2$ for vectors throughout the site, and so does
% wikipedia.
\newcommand{\normlzero}{L^0}
\newcommand{\normlone}{L^1}
\newcommand{\normltwo}{L^2}
\newcommand{\normlp}{L^p}
\newcommand{\normmax}{L^\infty}
\newcommand{\parents}{Pa} % See usage in notation.tex. Chosen to match Daphne's book.
\DeclareMathOperator*{\argmax}{arg\,max}
\DeclareMathOperator*{\argmin}{arg\,min}
\DeclareMathOperator{\sign}{sign}
\DeclareMathOperator{\Tr}{Tr}
\let\ab\allowbreak
% This file was created by tikzplotlib v0.9.9.
\def\points{(-1,3), (2,7), (3,8)}
\begin{tikzpicture}
\definecolor{color0}{rgb}{0.12156862745098,0.466666666666667,0.705882352941177}
\definecolor{color1}{rgb}{1,0.498039215686275,0.0549019607843137}
\definecolor{color2}{rgb}{0.172549019607843,0.627450980392157,0.172549019607843}
\definecolor{color3}{rgb}{0.83921568627451,0.152941176470588,0.156862745098039}
\definecolor{color4}{rgb}{0.580392156862745,0.403921568627451,0.741176470588235}
\definecolor{color5}{rgb}{0.549019607843137,0.337254901960784,0.294117647058824}
\definecolor{color6}{rgb}{0.890196078431372,0.466666666666667,0.76078431372549}
\definecolor{color7}{rgb}{0.737254901960784,0.741176470588235,0.133333333333333}
\definecolor{color8}{rgb}{0.0901960784313725,0.745098039215686,0.811764705882353}
\begin{axis}[
tick align=outside,
tick pos=left,
x grid style={white!69.0196078431373!black},
xmin=-0, xmax=1,
xtick style={color=black},
y grid style={white!69.0196078431373!black},
ymin=-12, ymax=8,
ytick style={color=black},
height=0.35*\textwidth,
width=1*\textwidth,
smooth
]
\addplot [color0, thick]
table {%
0 -0.128308042883873
0.0101010100916028 0.245983392000198
0.0202020201832056 0.222425639629364
0.030303031206131 0.184084668755531
0.0404040403664112 1.14534425735474
0.0505050495266914 1.76704776287079
0.060606062412262 1.72037672996521
0.0707070678472519 0.933646202087402
0.0808080807328224 -0.325637578964233
0.0909090936183929 -1.37419438362122
0.101010099053383 -2.15597367286682
0.111111111938953 -2.48629117012024
0.121212124824524 -1.78280210494995
0.131313130259514 -0.678521335124969
0.141414135694504 0.257147461175919
0.151515156030655 1.51499426364899
0.161616161465645 3.00852012634277
0.171717166900635 3.92923593521118
0.181818187236786 3.90619087219238
0.191919192671776 3.08789300918579
0.202020198106766 2.43498969078064
0.212121218442917 1.53775215148926
0.222222223877907 0.0152533650398254
0.232323229312897 -0.831866264343262
0.242424249649048 -2.20117807388306
0.252525269985199 -3.22051095962524
0.262626260519028 -3.60607242584229
0.272727280855179 -4.36243295669556
0.28282830119133 -3.87960124015808
0.292929291725159 -2.57185935974121
0.30303031206131 -1.77435493469238
0.313131332397461 -0.545357286930084
0.32323232293129 0.489103436470032
0.333333343267441 1.07867515087128
0.34343433380127 0.627202987670898
0.353535354137421 0.271895408630371
0.363636374473572 0.494738101959229
0.373737365007401 0.639668822288513
0.383838385343552 0.608663558959961
0.39393937587738 0.333570182323456
0.404040396213531 -0.404802054166794
0.414141416549683 -1.0490494966507
0.424242407083511 -1.39884042739868
0.434343427419662 -1.79985642433167
0.444444447755814 -2.24785757064819
0.454545438289642 -2.99189305305481
0.464646458625793 -5.40370893478394
0.474747478961945 -7.91668558120728
0.484848499298096 -9.05742073059082
0.494949519634247 -8.60674571990967
0.505050539970398 -7.36346340179443
0.515151500701904 -5.91135597229004
0.525252521038055 -4.49120616912842
0.535353541374207 -3.0394172668457
0.545454561710358 -1.55300641059875
0.555555582046509 -0.452672868967056
0.565656542778015 0.534026563167572
0.575757563114166 2.54583644866943
0.585858583450317 4.2235746383667
0.595959544181824 4.14144611358643
0.606060564517975 2.15625143051147
0.616161584854126 0.163379073143005
0.626262605190277 -1.39372658729553
0.636363625526428 -1.69715309143066
0.646464645862579 -1.97394108772278
0.65656566619873 -2.3638813495636
0.666666686534882 -2.55302739143372
0.676767706871033 -2.63137912750244
0.686868667602539 -2.81758117675781
0.69696968793869 -2.77993869781494
0.707070708274841 -2.00176215171814
0.717171728610992 -0.724480867385864
0.727272748947144 -0.0427038073539734
0.737373769283295 0.255515217781067
0.747474789619446 -0.16065514087677
0.757575750350952 0.595221161842346
0.767676770687103 1.36654281616211
0.777777791023254 2.52859115600586
0.787878811359406 3.82578372955322
0.797979831695557 4.84706974029541
0.808080792427063 4.88980484008789
0.818181812763214 4.35960006713867
0.828282833099365 3.81451535224915
0.838383793830872 3.85455513000488
0.848484814167023 2.99768543243408
0.858585834503174 1.1380707025528
0.868686854839325 0.237297296524048
0.878787875175476 0.857240200042725
0.888888895511627 1.44591414928436
0.898989915847778 1.55497288703918
0.909090936183929 1.12556338310242
0.919191956520081 -0.0453065037727356
0.929292917251587 -1.68896293640137
0.939393937587738 -3.3630485534668
0.949494957923889 -4.44697570800781
0.95959597826004 -4.36831283569336
0.969696998596191 -3.64294242858887
0.979797959327698 -2.09102964401245
0.989898979663849 -0.693680047988892
1 0.357790112495422
};
\filldraw (axis cs:0.111111111938953,-2.48629117012024) circle (2pt) node[align=left, below, font=\footnotesize] {$(x_1,y_1)$};
\filldraw (axis cs:0.434343427419662,-1.79985642433167) circle (2pt) node[align=left, below, font=\footnotesize] {$(x_2,y_2)$};
\filldraw (axis cs:0.484848499298096,-9.05742073059082) circle (2pt) node[align=left, below, font=\footnotesize] {$(x_3,y_3)$};
\filldraw (axis cs:0.666666686534882,-2.55302739143372) circle (2pt) node[align=left, below, font=\footnotesize] {$\mathbf{(x,?)}$};
\filldraw (axis cs:0.808080792427063,4.88980484008789) circle (2pt) node[align=left, below, font=\footnotesize] {$(x_4,y_4)$};
\filldraw (axis cs:0.939393937587738,-3.3630485534668) circle (2pt) node[align=left, below, font=\footnotesize] {$(x_5,y_5)$};
\addplot [semithick, color1, opacity=0.2]
table {%
0 4.28430891036987
0.0101010100916028 4.30051851272583
0.0202020201832056 4.30913305282593
0.030303031206131 4.29600429534912
0.0404040403664112 4.29854488372803
0.0505050495266914 4.26871061325073
0.060606062412262 4.2602481842041
0.0707070678472519 4.25349617004395
0.0808080807328224 4.22980451583862
0.0909090936183929 4.19286060333252
0.101010099053383 4.17360496520996
0.111111111938953 4.14290761947632
0.121212124824524 4.08796787261963
0.131313130259514 4.0594539642334
0.141414135694504 4.00505876541138
0.151515156030655 3.97285652160645
0.161616161465645 3.90496063232422
0.171717166900635 3.83971953392029
0.181818187236786 3.79873895645142
0.191919192671776 3.73129272460938
0.202020198106766 3.65645909309387
0.212121218442917 3.5960009098053
0.222222223877907 3.52322483062744
0.232323229312897 3.43513751029968
0.242424249649048 3.3616304397583
0.252525269985199 3.29762411117554
0.262626260519028 3.20590972900391
0.272727280855179 3.09360074996948
0.28282830119133 2.99378538131714
0.292929291725159 2.91921520233154
0.30303031206131 2.82170128822327
0.313131332397461 2.70830249786377
0.32323232293129 2.59495806694031
0.333333343267441 2.48730635643005
0.34343433380127 2.37643647193909
0.353535354137421 2.26538705825806
0.363636374473572 2.12807965278625
0.373737365007401 1.99019396305084
0.383838385343552 1.85900282859802
0.39393937587738 1.70321273803711
0.404040396213531 1.56479907035828
0.414141416549683 1.40247809886932
0.424242407083511 1.25359463691711
0.434343427419662 1.07534384727478
0.444444447755814 0.930328845977783
0.454545438289642 0.761442422866821
0.464646458625793 0.573360562324524
0.474747478961945 0.395766615867615
0.484848499298096 0.203279376029968
0.494949519634247 0.0321395397186279
0.505050539970398 -0.137906908988953
0.515151500701904 -0.350167036056519
0.525252521038055 -0.5174560546875
0.535353541374207 -0.701683163642883
0.545454561710358 -0.878420829772949
0.555555582046509 -1.07135534286499
0.565656542778015 -1.2477593421936
0.575757563114166 -1.43019163608551
0.585858583450317 -1.63847303390503
0.595959544181824 -1.82557463645935
0.606060564517975 -1.9855409860611
0.616161584854126 -2.18390655517578
0.626262605190277 -2.37359499931335
0.636363625526428 -2.54729723930359
0.646464645862579 -2.71770572662354
0.65656566619873 -2.90459370613098
0.666666686534882 -3.07928419113159
0.676767706871033 -3.22868132591248
0.686868667602539 -3.42115712165833
0.69696968793869 -3.5619421005249
0.707070708274841 -3.75430226325989
0.717171728610992 -3.89226913452148
0.727272748947144 -4.0384578704834
0.737373769283295 -4.20960998535156
0.747474789619446 -4.35432481765747
0.757575750350952 -4.49405670166016
0.767676770687103 -4.61132335662842
0.777777791023254 -4.77033615112305
0.787878811359406 -4.92688179016113
0.797979831695557 -5.05803680419922
0.808080792427063 -5.19231367111206
0.818181812763214 -5.32923221588135
0.828282833099365 -5.46084308624268
0.838383793830872 -5.5722827911377
0.848484814167023 -5.6985387802124
0.858585834503174 -5.81698989868164
0.868686854839325 -5.91272401809692
0.878787875175476 -6.03909826278687
0.888888895511627 -6.12917518615723
0.898989915847778 -6.20251655578613
0.909090936183929 -6.30838394165039
0.919191956520081 -6.37095642089844
0.929292917251587 -6.47658538818359
0.939393937587738 -6.5447998046875
0.949494957923889 -6.61706161499023
0.95959597826004 -6.67179250717163
0.969696998596191 -6.7358078956604
0.979797959327698 -6.79077291488647
0.989898979663849 -6.83802890777588
1 -6.87432765960693
};
\addplot [semithick, color2, opacity=0.2]
table {%
0 4.14231061935425
0.0101010100916028 3.92172241210938
0.0202020201832056 3.71964049339294
0.030303031206131 3.4999213218689
0.0404040403664112 3.28382301330566
0.0505050495266914 3.08523511886597
0.060606062412262 2.86599493026733
0.0707070678472519 2.65185904502869
0.0808080807328224 2.41668844223022
0.0909090936183929 2.21005034446716
0.101010099053383 2.0389940738678
0.111111111938953 1.84878158569336
0.121212124824524 1.66083455085754
0.131313130259514 1.47408068180084
0.141414135694504 1.33761286735535
0.151515156030655 1.18185973167419
0.161616161465645 1.01285338401794
0.171717166900635 0.870226562023163
0.181818187236786 0.729696035385132
0.191919192671776 0.617734789848328
0.202020198106766 0.505572557449341
0.212121218442917 0.418318718671799
0.222222223877907 0.353389859199524
0.232323229312897 0.284276783466339
0.242424249649048 0.226475298404694
0.252525269985199 0.227721810340881
0.262626260519028 0.196745991706848
0.272727280855179 0.127484798431396
0.28282830119133 0.0827680826187134
0.292929291725159 -0.017727792263031
0.30303031206131 -0.076952338218689
0.313131332397461 -0.176372885704041
0.32323232293129 -0.251087307929993
0.333333343267441 -0.338299036026001
0.34343433380127 -0.433988451957703
0.353535354137421 -0.515852093696594
0.363636374473572 -0.604318737983704
0.373737365007401 -0.681199550628662
0.383838385343552 -0.782364964485168
0.39393937587738 -0.875270128250122
0.404040396213531 -0.941336393356323
0.414141416549683 -1.00581991672516
0.424242407083511 -1.08222579956055
0.434343427419662 -1.17498791217804
0.444444447755814 -1.27898406982422
0.454545438289642 -1.38432359695435
0.464646458625793 -1.54030847549438
0.474747478961945 -1.66976761817932
0.484848499298096 -1.82219076156616
0.494949519634247 -1.93062627315521
0.505050539970398 -2.05011200904846
0.515151500701904 -2.15237331390381
0.525252521038055 -2.22703123092651
0.535353541374207 -2.33321952819824
0.545454561710358 -2.4054388999939
0.555555582046509 -2.48199415206909
0.565656542778015 -2.51512098312378
0.575757563114166 -2.55848479270935
0.585858583450317 -2.55114698410034
0.595959544181824 -2.5389666557312
0.606060564517975 -2.56200289726257
0.616161584854126 -2.57610034942627
0.626262605190277 -2.57621812820435
0.636363625526428 -2.56430602073669
0.646464645862579 -2.51722812652588
0.65656566619873 -2.4813084602356
0.666666686534882 -2.44424891471863
0.676767706871033 -2.44372701644897
0.686868667602539 -2.42748475074768
0.69696968793869 -2.49185180664062
0.707070708274841 -2.5750560760498
0.717171728610992 -2.72130346298218
0.727272748947144 -2.9600887298584
0.737373769283295 -3.21958708763123
0.747474789619446 -3.44769048690796
0.757575750350952 -3.716233253479
0.767676770687103 -3.98657369613647
0.777777791023254 -4.27548742294312
0.787878811359406 -4.56593990325928
0.797979831695557 -4.8224630355835
0.808080792427063 -5.1156120300293
0.818181812763214 -5.34454154968262
0.828282833099365 -5.53120374679565
0.838383793830872 -5.67477798461914
0.848484814167023 -5.78527164459229
0.858585834503174 -5.86214256286621
0.868686854839325 -5.89745616912842
0.878787875175476 -5.93156719207764
0.888888895511627 -5.91250896453857
0.898989915847778 -5.85368871688843
0.909090936183929 -5.74918746948242
0.919191956520081 -5.61399078369141
0.929292917251587 -5.45960426330566
0.939393937587738 -5.25064373016357
0.949494957923889 -5.03104114532471
0.95959597826004 -4.76795864105225
0.969696998596191 -4.47224569320679
0.979797959327698 -4.17514562606812
0.989898979663849 -3.88921737670898
1 -3.52382206916809
};
\addplot [semithick, color3, opacity=0.2]
table {%
0 -3.89315915107727
0.0101010100916028 -3.9140830039978
0.0202020201832056 -3.92380595207214
0.030303031206131 -3.93581676483154
0.0404040403664112 -3.96626734733582
0.0505050495266914 -4.00012254714966
0.060606062412262 -4.06429767608643
0.0707070678472519 -4.06983613967896
0.0808080807328224 -4.10595798492432
0.0909090936183929 -4.14091396331787
0.101010099053383 -4.21698141098022
0.111111111938953 -4.26758003234863
0.121212124824524 -4.35522413253784
0.131313130259514 -4.42382621765137
0.141414135694504 -4.49122381210327
0.151515156030655 -4.57467126846313
0.161616161465645 -4.643883228302
0.171717166900635 -4.74175453186035
0.181818187236786 -4.83301591873169
0.191919192671776 -4.894127368927
0.202020198106766 -4.97080850601196
0.212121218442917 -5.04259872436523
0.222222223877907 -5.11234474182129
0.232323229312897 -5.1732325553894
0.242424249649048 -5.25321674346924
0.252525269985199 -5.32610702514648
0.262626260519028 -5.38926935195923
0.272727280855179 -5.43785285949707
0.28282830119133 -5.50970411300659
0.292929291725159 -5.54183387756348
0.30303031206131 -5.58417081832886
0.313131332397461 -5.60928869247437
0.32323232293129 -5.65616655349731
0.333333343267441 -5.68793773651123
0.34343433380127 -5.72240781784058
0.353535354137421 -5.71297979354858
0.363636374473572 -5.74499273300171
0.373737365007401 -5.77703475952148
0.383838385343552 -5.78221750259399
0.39393937587738 -5.77901411056519
0.404040396213531 -5.80095767974854
0.414141416549683 -5.78725242614746
0.424242407083511 -5.77540588378906
0.434343427419662 -5.76472330093384
0.444444447755814 -5.73452281951904
0.454545438289642 -5.7167592048645
0.464646458625793 -5.66441631317139
0.474747478961945 -5.61245489120483
0.484848499298096 -5.55692911148071
0.494949519634247 -5.49705266952515
0.505050539970398 -5.41606521606445
0.515151500701904 -5.35056638717651
0.525252521038055 -5.27668857574463
0.535353541374207 -5.20048189163208
0.545454561710358 -5.10650777816772
0.555555582046509 -4.9834098815918
0.565656542778015 -4.87844562530518
0.575757563114166 -4.74991750717163
0.585858583450317 -4.6294412612915
0.595959544181824 -4.46934175491333
0.606060564517975 -4.3522310256958
0.616161584854126 -4.17410469055176
0.626262605190277 -4.00872039794922
0.636363625526428 -3.84507083892822
0.646464645862579 -3.65667796134949
0.65656566619873 -3.47281837463379
0.666666686534882 -3.29194331169128
0.676767706871033 -3.09021735191345
0.686868667602539 -2.89965105056763
0.69696968793869 -2.69900441169739
0.707070708274841 -2.50690984725952
0.717171728610992 -2.32743382453918
0.727272748947144 -2.1255214214325
0.737373769283295 -1.90307998657227
0.747474789619446 -1.70446395874023
0.757575750350952 -1.52047729492188
0.767676770687103 -1.31373071670532
0.777777791023254 -1.10342025756836
0.787878811359406 -0.872876882553101
0.797979831695557 -0.680119514465332
0.808080792427063 -0.472295045852661
0.818181812763214 -0.255411624908447
0.828282833099365 -0.0371122360229492
0.838383793830872 0.157435655593872
0.848484814167023 0.372454881668091
0.858585834503174 0.569563865661621
0.868686854839325 0.789226531982422
0.878787875175476 0.993486166000366
0.888888895511627 1.18100929260254
0.898989915847778 1.39689254760742
0.909090936183929 1.60964393615723
0.919191956520081 1.83105421066284
0.929292917251587 2.0354642868042
0.939393937587738 2.25547766685486
0.949494957923889 2.49207401275635
0.95959597826004 2.69941067695618
0.969696998596191 2.90244245529175
0.979797959327698 3.11839818954468
0.989898979663849 3.32969903945923
1 3.52446126937866
};
\addplot [semithick, color4, opacity=0.2]
table {%
0 2.46528601646423
0.0101010100916028 1.54548037052155
0.0202020201832056 0.622193932533264
0.030303031206131 -0.1961659938097
0.0404040403664112 -1.00586295127869
0.0505050495266914 -1.51367437839508
0.060606062412262 -1.68306839466095
0.0707070678472519 -1.63141059875488
0.0808080807328224 -1.31586301326752
0.0909090936183929 -0.926587343215942
0.101010099053383 -0.643606901168823
0.111111111938953 -0.463921427726746
0.121212124824524 -0.313905715942383
0.131313130259514 -0.396990060806274
0.141414135694504 -0.759208858013153
0.151515156030655 -1.20748090744019
0.161616161465645 -1.51499378681183
0.171717166900635 -1.63988614082336
0.181818187236786 -1.67946088314056
0.191919192671776 -1.95840728282928
0.202020198106766 -2.56422400474548
0.212121218442917 -3.27244853973389
0.222222223877907 -4.02103328704834
0.232323229312897 -4.59402179718018
0.242424249649048 -4.68208837509155
0.252525269985199 -4.539719581604
0.262626260519028 -4.41975545883179
0.272727280855179 -4.36539602279663
0.28282830119133 -4.15395259857178
0.292929291725159 -3.67101860046387
0.30303031206131 -3.24383330345154
0.313131332397461 -3.03169369697571
0.32323232293129 -2.93568682670593
0.333333343267441 -2.86720085144043
0.34343433380127 -2.88540506362915
0.353535354137421 -2.87862157821655
0.363636374473572 -2.68589973449707
0.373737365007401 -2.24372887611389
0.383838385343552 -1.68738198280334
0.39393937587738 -1.04805374145508
0.404040396213531 -0.405938297510147
0.414141416549683 0.208922266960144
0.424242407083511 0.896941423416138
0.434343427419662 1.60401153564453
0.444444447755814 2.24998569488525
0.454545438289642 2.82000160217285
0.464646458625793 3.26415944099426
0.474747478961945 3.51514220237732
0.484848499298096 3.56877517700195
0.494949519634247 3.53638315200806
0.505050539970398 3.362135887146
0.515151500701904 3.09048700332642
0.525252521038055 2.81417417526245
0.535353541374207 2.45068073272705
0.545454561710358 2.05925178527832
0.555555582046509 1.91878235340118
0.565656542778015 2.03971886634827
0.575757563114166 2.19124674797058
0.585858583450317 2.11785650253296
0.595959544181824 1.83445334434509
0.606060564517975 1.46386516094208
0.616161584854126 1.16525757312775
0.626262605190277 0.895504832267761
0.636363625526428 0.808026731014252
0.646464645862579 0.810245871543884
0.65656566619873 0.54091739654541
0.666666686534882 -0.048281192779541
0.676767706871033 -0.870970368385315
0.686868667602539 -1.72436261177063
0.69696968793869 -2.41218066215515
0.707070708274841 -2.89082074165344
0.717171728610992 -3.14114379882812
0.727272748947144 -3.17641019821167
0.737373769283295 -3.07627534866333
0.747474789619446 -2.97537541389465
0.757575750350952 -2.93889617919922
0.767676770687103 -3.01350355148315
0.777777791023254 -3.2125198841095
0.787878811359406 -3.20800399780273
0.797979831695557 -3.17361879348755
0.808080792427063 -3.13849759101868
0.818181812763214 -3.07326793670654
0.828282833099365 -2.90618515014648
0.838383793830872 -2.8127908706665
0.848484814167023 -2.68862652778625
0.858585834503174 -2.56095623970032
0.868686854839325 -2.58732581138611
0.878787875175476 -2.72913551330566
0.888888895511627 -2.83773231506348
0.898989915847778 -2.85185432434082
0.909090936183929 -2.89771318435669
0.919191956520081 -3.1075963973999
0.929292917251587 -3.42373704910278
0.939393937587738 -3.8373875617981
0.949494957923889 -4.37008428573608
0.95959597826004 -4.85936975479126
0.969696998596191 -5.25982999801636
0.979797959327698 -5.51366710662842
0.989898979663849 -5.62331867218018
1 -5.60990810394287
};
\addplot [semithick, color5, opacity=0.2]
table {%
0 -2.86187481880188
0.0101010100916028 -2.62218189239502
0.0202020201832056 -2.39012742042542
0.030303031206131 -2.20059466362
0.0404040403664112 -2.00310182571411
0.0505050495266914 -1.83596611022949
0.060606062412262 -1.68469834327698
0.0707070678472519 -1.51345205307007
0.0808080807328224 -1.35335505008698
0.0909090936183929 -1.20136284828186
0.101010099053383 -1.07422947883606
0.111111111938953 -0.975073158740997
0.121212124824524 -0.85306054353714
0.131313130259514 -0.759288668632507
0.141414135694504 -0.674479484558105
0.151515156030655 -0.552119255065918
0.161616161465645 -0.477047681808472
0.171717166900635 -0.384566336870193
0.181818187236786 -0.311709225177765
0.191919192671776 -0.242770314216614
0.202020198106766 -0.17211651802063
0.212121218442917 -0.114948809146881
0.222222223877907 -0.117016106843948
0.232323229312897 -0.071947455406189
0.242424249649048 -0.0353503823280334
0.252525269985199 -0.0364787578582764
0.262626260519028 0.0276833176612854
0.272727280855179 0.125517070293427
0.28282830119133 0.22416353225708
0.292929291725159 0.314024209976196
0.30303031206131 0.441555261611938
0.313131332397461 0.566126525402069
0.32323232293129 0.726855516433716
0.333333343267441 0.912408709526062
0.34343433380127 1.12490940093994
0.353535354137421 1.31817054748535
0.363636374473572 1.56701123714447
0.373737365007401 1.76337480545044
0.383838385343552 1.95535576343536
0.39393937587738 2.08425259590149
0.404040396213531 2.22731065750122
0.414141416549683 2.33449077606201
0.424242407083511 2.41153311729431
0.434343427419662 2.44321846961975
0.444444447755814 2.45274829864502
0.454545438289642 2.43688321113586
0.464646458625793 2.42810726165771
0.474747478961945 2.375159740448
0.484848499298096 2.31538271903992
0.494949519634247 2.24959897994995
0.505050539970398 2.19455003738403
0.515151500701904 2.12445044517517
0.525252521038055 2.05948305130005
0.535353541374207 1.93953227996826
0.545454561710358 1.86074113845825
0.555555582046509 1.76603055000305
0.565656542778015 1.63189673423767
0.575757563114166 1.50387382507324
0.585858583450317 1.38946437835693
0.595959544181824 1.30501794815063
0.606060564517975 1.21184921264648
0.616161584854126 1.15730762481689
0.626262605190277 1.0838828086853
0.636363625526428 0.998543262481689
0.646464645862579 0.906335353851318
0.65656566619873 0.833240032196045
0.666666686534882 0.754873991012573
0.676767706871033 0.689798831939697
0.686868667602539 0.530774116516113
0.69696968793869 0.359215021133423
0.707070708274841 0.15999960899353
0.717171728610992 -0.0388262271881104
0.727272748947144 -0.237374782562256
0.737373769283295 -0.482639074325562
0.747474789619446 -0.714397430419922
0.757575750350952 -0.999650478363037
0.767676770687103 -1.30259728431702
0.777777791023254 -1.58717036247253
0.787878811359406 -1.94076716899872
0.797979831695557 -2.24068546295166
0.808080792427063 -2.55944967269897
0.818181812763214 -2.86688470840454
0.828282833099365 -3.09230327606201
0.838383793830872 -3.35045576095581
0.848484814167023 -3.62021994590759
0.858585834503174 -3.882169008255
0.868686854839325 -4.15410184860229
0.878787875175476 -4.40132665634155
0.888888895511627 -4.64856624603271
0.898989915847778 -4.88014316558838
0.909090936183929 -5.1095986366272
0.919191956520081 -5.30170440673828
0.929292917251587 -5.53704738616943
0.939393937587738 -5.75914096832275
0.949494957923889 -5.96171236038208
0.95959597826004 -6.1424388885498
0.969696998596191 -6.30466032028198
0.979797959327698 -6.49424648284912
0.989898979663849 -6.6247673034668
1 -6.7179012298584
};
\addplot [semithick, color6, opacity=0.2]
table {%
0 5.95280885696411
0.0101010100916028 5.95169639587402
0.0202020201832056 5.9330267906189
0.030303031206131 5.88185977935791
0.0404040403664112 5.82428073883057
0.0505050495266914 5.77828502655029
0.060606062412262 5.73845338821411
0.0707070678472519 5.68677616119385
0.0808080807328224 5.63261842727661
0.0909090936183929 5.54524326324463
0.101010099053383 5.41500473022461
0.111111111938953 5.27171468734741
0.121212124824524 5.13113164901733
0.131313130259514 4.98302555084229
0.141414135694504 4.83715391159058
0.151515156030655 4.65837335586548
0.161616161465645 4.44286870956421
0.171717166900635 4.21702289581299
0.181818187236786 3.95788764953613
0.191919192671776 3.66205143928528
0.202020198106766 3.35986495018005
0.212121218442917 3.03336644172668
0.222222223877907 2.70661735534668
0.232323229312897 2.40367102622986
0.242424249649048 2.13293266296387
0.252525269985199 1.88935720920563
0.262626260519028 1.65596330165863
0.272727280855179 1.46180665493011
0.28282830119133 1.27818071842194
0.292929291725159 1.07687616348267
0.30303031206131 0.882564425468445
0.313131332397461 0.661325812339783
0.32323232293129 0.439331650733948
0.333333343267441 0.209851741790771
0.34343433380127 -0.0276787281036377
0.353535354137421 -0.268971920013428
0.363636374473572 -0.511618137359619
0.373737365007401 -0.753656387329102
0.383838385343552 -0.995855152606964
0.39393937587738 -1.20773363113403
0.404040396213531 -1.40303289890289
0.414141416549683 -1.56743335723877
0.424242407083511 -1.7117030620575
0.434343427419662 -1.82864856719971
0.444444447755814 -1.93554103374481
0.454545438289642 -2.01902008056641
0.464646458625793 -2.10376834869385
0.474747478961945 -2.19225811958313
0.484848499298096 -2.28332924842834
0.494949519634247 -2.35733222961426
0.505050539970398 -2.44428300857544
0.515151500701904 -2.50272464752197
0.525252521038055 -2.54421257972717
0.535353541374207 -2.56124830245972
0.545454561710358 -2.55449151992798
0.555555582046509 -2.54093837738037
0.565656542778015 -2.54586601257324
0.575757563114166 -2.55957007408142
0.585858583450317 -2.5912082195282
0.595959544181824 -2.65414619445801
0.606060564517975 -2.7531476020813
0.616161584854126 -2.87583374977112
0.626262605190277 -2.99206829071045
0.636363625526428 -3.08332753181458
0.646464645862579 -3.14518475532532
0.65656566619873 -3.16532444953918
0.666666686534882 -3.18153047561646
0.676767706871033 -3.20060849189758
0.686868667602539 -3.20878791809082
0.69696968793869 -3.18452572822571
0.707070708274841 -3.14734625816345
0.717171728610992 -3.11039924621582
0.727272748947144 -3.06184196472168
0.737373769283295 -2.99918842315674
0.747474789619446 -2.92262244224548
0.757575750350952 -2.81612777709961
0.767676770687103 -2.68505883216858
0.777777791023254 -2.53911685943604
0.787878811359406 -2.39456534385681
0.797979831695557 -2.2294704914093
0.808080792427063 -2.07440519332886
0.818181812763214 -1.95263671875
0.828282833099365 -1.86927676200867
0.838383793830872 -1.83278703689575
0.848484814167023 -1.84203946590424
0.858585834503174 -1.87581849098206
0.868686854839325 -1.95167756080627
0.878787875175476 -2.02603459358215
0.888888895511627 -2.099609375
0.898989915847778 -2.18276786804199
0.909090936183929 -2.24555540084839
0.919191956520081 -2.29351735115051
0.929292917251587 -2.32762145996094
0.939393937587738 -2.36430597305298
0.949494957923889 -2.40922999382019
0.95959597826004 -2.45865201950073
0.969696998596191 -2.49583315849304
0.979797959327698 -2.52491474151611
0.989898979663849 -2.53641271591187
1 -2.54450869560242
};
\addplot [semithick, white!49.8039215686275!black, opacity=0.2]
table {%
0 -3.59747743606567
0.0101010100916028 -3.29682993888855
0.0202020201832056 -2.94431853294373
0.030303031206131 -2.55584764480591
0.0404040403664112 -2.17785501480103
0.0505050495266914 -1.84661030769348
0.060606062412262 -1.54352796077728
0.0707070678472519 -1.25441980361938
0.0808080807328224 -1.06884098052979
0.0909090936183929 -0.834848582744598
0.101010099053383 -0.614958167076111
0.111111111938953 -0.354739487171173
0.121212124824524 -0.12578971683979
0.131313130259514 0.159320741891861
0.141414135694504 0.393746674060822
0.151515156030655 0.644728541374207
0.161616161465645 0.900069355964661
0.171717166900635 1.17559337615967
0.181818187236786 1.47611808776855
0.191919192671776 1.76929831504822
0.202020198106766 2.05007791519165
0.212121218442917 2.31686067581177
0.222222223877907 2.63215112686157
0.232323229312897 2.83664703369141
0.242424249649048 3.0353045463562
0.252525269985199 3.19684791564941
0.262626260519028 3.36614751815796
0.272727280855179 3.50832438468933
0.28282830119133 3.63970923423767
0.292929291725159 3.76753449440002
0.30303031206131 3.83977794647217
0.313131332397461 3.91308450698853
0.32323232293129 3.97997999191284
0.333333343267441 4.0412769317627
0.34343433380127 4.122727394104
0.353535354137421 4.19057464599609
0.363636374473572 4.3127121925354
0.373737365007401 4.42207670211792
0.383838385343552 4.53272151947021
0.39393937587738 4.68329620361328
0.404040396213531 4.82325029373169
0.414141416549683 4.97995662689209
0.424242407083511 5.1298942565918
0.434343427419662 5.30047798156738
0.444444447755814 5.40030193328857
0.454545438289642 5.47818470001221
0.464646458625793 5.52140140533447
0.474747478961945 5.543776512146
0.484848499298096 5.4680962562561
0.494949519634247 5.38214731216431
0.505050539970398 5.18300104141235
0.515151500701904 4.9354944229126
0.525252521038055 4.65586519241333
0.535353541374207 4.40804576873779
0.545454561710358 4.14279937744141
0.555555582046509 3.88438296318054
0.565656542778015 3.62577295303345
0.575757563114166 3.36249351501465
0.585858583450317 3.10263013839722
0.595959544181824 2.89540147781372
0.606060564517975 2.6480860710144
0.616161584854126 2.39052653312683
0.626262605190277 2.11986970901489
0.636363625526428 1.80144643783569
0.646464645862579 1.45344662666321
0.65656566619873 1.14352655410767
0.666666686534882 0.831650733947754
0.676767706871033 0.559635400772095
0.686868667602539 0.386631965637207
0.69696968793869 0.291438579559326
0.707070708274841 0.267782211303711
0.717171728610992 0.300568580627441
0.727272748947144 0.316277980804443
0.737373769283295 0.368246078491211
0.747474789619446 0.410303115844727
0.757575750350952 0.44727087020874
0.767676770687103 0.464563846588135
0.777777791023254 0.511908054351807
0.787878811359406 0.567136287689209
0.797979831695557 0.550230026245117
0.808080792427063 0.544064521789551
0.818181812763214 0.447702407836914
0.828282833099365 0.304447174072266
0.838383793830872 0.104161739349365
0.848484814167023 -0.129960060119629
0.858585834503174 -0.379889488220215
0.868686854839325 -0.63122034072876
0.878787875175476 -0.913817405700684
0.888888895511627 -1.2254319190979
0.898989915847778 -1.52106165885925
0.909090936183929 -1.79755520820618
0.919191956520081 -2.10931468009949
0.929292917251587 -2.34630846977234
0.939393937587738 -2.57134437561035
0.949494957923889 -2.73612499237061
0.95959597826004 -2.89165997505188
0.969696998596191 -2.98543787002563
0.979797959327698 -3.02404856681824
0.989898979663849 -3.01928424835205
1 -2.98271822929382
};
\addplot [semithick, color7, opacity=0.2]
table {%
0 0.421791017055511
0.0101010100916028 0.577111959457397
0.0202020201832056 0.535125613212585
0.030303031206131 0.636250734329224
0.0404040403664112 1.06264114379883
0.0505050495266914 2.08302545547485
0.060606062412262 3.06881380081177
0.0707070678472519 3.66153788566589
0.0808080807328224 3.93818187713623
0.0909090936183929 3.46718764305115
0.101010099053383 2.49211597442627
0.111111111938953 1.49933099746704
0.121212124824524 0.521971881389618
0.131313130259514 -0.299798369407654
0.141414135694504 -0.573574542999268
0.151515156030655 -0.58588707447052
0.161616161465645 -0.576081871986389
0.171717166900635 -0.219863265752792
0.181818187236786 0.0818308591842651
0.191919192671776 0.113368988037109
0.202020198106766 0.0836582779884338
0.212121218442917 0.174755185842514
0.222222223877907 0.213173091411591
0.232323229312897 -0.166073799133301
0.242424249649048 -0.82199090719223
0.252525269985199 -1.47608280181885
0.262626260519028 -1.75679087638855
0.272727280855179 -1.72552132606506
0.28282830119133 -1.58354926109314
0.292929291725159 -1.51372611522675
0.30303031206131 -1.40375566482544
0.313131332397461 -0.857235789299011
0.32323232293129 0.237660050392151
0.333333343267441 1.52424740791321
0.34343433380127 2.82251739501953
0.353535354137421 3.76652956008911
0.363636374473572 4.35452032089233
0.373737365007401 4.78870820999146
0.383838385343552 5.0697193145752
0.39393937587738 5.02291393280029
0.404040396213531 4.85803174972534
0.414141416549683 4.87737417221069
0.424242407083511 5.54745101928711
0.434343427419662 6.48709106445312
0.444444447755814 6.92037200927734
0.454545438289642 6.39060211181641
0.464646458625793 5.76686859130859
0.474747478961945 5.15970897674561
0.484848499298096 4.57790660858154
0.494949519634247 4.10962009429932
0.505050539970398 3.82723379135132
0.515151500701904 3.65205025672913
0.525252521038055 3.48824977874756
0.535353541374207 3.14233589172363
0.545454561710358 2.84482765197754
0.555555582046509 2.35973763465881
0.565656542778015 1.30864787101746
0.575757563114166 0.00722408294677734
0.585858583450317 -1.08328425884247
0.595959544181824 -1.26426017284393
0.606060564517975 -1.00247156620026
0.616161584854126 -0.318897843360901
0.626262605190277 0.44938051700592
0.636363625526428 1.20082604885101
0.646464645862579 2.00184082984924
0.65656566619873 2.70579981803894
0.666666686534882 3.09378576278687
0.676767706871033 3.35222792625427
0.686868667602539 3.37046790122986
0.69696968793869 3.08511018753052
0.707070708274841 2.39889788627625
0.717171728610992 1.3162237405777
0.727272748947144 0.296457946300507
0.737373769283295 -0.00300717353820801
0.747474789619446 0.0508651733398438
0.757575750350952 0.255613446235657
0.767676770687103 0.787260174751282
0.777777791023254 1.32513785362244
0.787878811359406 1.65337538719177
0.797979831695557 1.62621819972992
0.808080792427063 1.53757619857788
0.818181812763214 1.49968886375427
0.828282833099365 1.30638813972473
0.838383793830872 1.07405781745911
0.848484814167023 0.950636506080627
0.858585834503174 0.850877344608307
0.868686854839325 0.915914416313171
0.878787875175476 0.987419962882996
0.888888895511627 0.948026537895203
0.898989915847778 0.844117879867554
0.909090936183929 1.00458514690399
0.919191956520081 1.22486770153046
0.929292917251587 1.72203266620636
0.939393937587738 1.87751913070679
0.949494957923889 1.53947949409485
0.95959597826004 1.14309239387512
0.969696998596191 1.01519632339478
0.979797959327698 0.873121023178101
0.989898979663849 0.706565499305725
1 0.601293087005615
};
\addplot [semithick, color8, opacity=0.2]
table {%
0 -4.95777034759521
0.0101010100916028 -4.93180227279663
0.0202020201832056 -4.93481254577637
0.030303031206131 -4.94995975494385
0.0404040403664112 -4.99836254119873
0.0505050495266914 -5.05578899383545
0.060606062412262 -5.12817478179932
0.0707070678472519 -5.2290997505188
0.0808080807328224 -5.32506656646729
0.0909090936183929 -5.42766380310059
0.101010099053383 -5.51918077468872
0.111111111938953 -5.56703615188599
0.121212124824524 -5.60059070587158
0.131313130259514 -5.59094572067261
0.141414135694504 -5.56976842880249
0.151515156030655 -5.54290962219238
0.161616161465645 -5.4847993850708
0.171717166900635 -5.40227556228638
0.181818187236786 -5.31867742538452
0.191919192671776 -5.22524929046631
0.202020198106766 -5.11183214187622
0.212121218442917 -5.0323691368103
0.222222223877907 -4.96536445617676
0.232323229312897 -4.90866374969482
0.242424249649048 -4.84531927108765
0.252525269985199 -4.74398469924927
0.262626260519028 -4.63236141204834
0.272727280855179 -4.50383615493774
0.28282830119133 -4.3685245513916
0.292929291725159 -4.21056461334229
0.30303031206131 -4.03595399856567
0.313131332397461 -3.85059332847595
0.32323232293129 -3.59477996826172
0.333333343267441 -3.29541969299316
0.34343433380127 -2.94675135612488
0.353535354137421 -2.57384204864502
0.363636374473572 -2.18923568725586
0.373737365007401 -1.83432304859161
0.383838385343552 -1.52652502059937
0.39393937587738 -1.2524356842041
0.404040396213531 -1.02265477180481
0.414141416549683 -0.822609782218933
0.424242407083511 -0.63211190700531
0.434343427419662 -0.449043214321136
0.444444447755814 -0.268008470535278
0.454545438289642 -0.0736866593360901
0.464646458625793 0.117129027843475
0.474747478961945 0.306960105895996
0.484848499298096 0.528746008872986
0.494949519634247 0.758202075958252
0.505050539970398 0.993643164634705
0.515151500701904 1.2079735994339
0.525252521038055 1.43219327926636
0.535353541374207 1.61249494552612
0.545454561710358 1.75649118423462
0.555555582046509 1.86894869804382
0.565656542778015 1.90865588188171
0.575757563114166 1.93545436859131
0.585858583450317 1.93186450004578
0.595959544181824 1.92697739601135
0.606060564517975 1.9201991558075
0.616161584854126 1.92062902450562
0.626262605190277 1.92802357673645
0.636363625526428 1.97084820270538
0.646464645862579 2.04103326797485
0.65656566619873 2.12601971626282
0.666666686534882 2.22507214546204
0.676767706871033 2.31685209274292
0.686868667602539 2.4227876663208
0.69696968793869 2.51340246200562
0.707070708274841 2.59801864624023
0.717171728610992 2.68294882774353
0.727272748947144 2.74202513694763
0.737373769283295 2.79034471511841
0.747474789619446 2.82125639915466
0.757575750350952 2.87277364730835
0.767676770687103 2.93100476264954
0.777777791023254 2.97197270393372
0.787878811359406 2.99164772033691
0.797979831695557 3.00309419631958
0.808080792427063 3.01275205612183
0.818181812763214 3.02092218399048
0.828282833099365 3.0117654800415
0.838383793830872 2.97434449195862
0.848484814167023 2.92585420608521
0.858585834503174 2.85221576690674
0.868686854839325 2.77306604385376
0.878787875175476 2.67831516265869
0.888888895511627 2.58190488815308
0.898989915847778 2.4694619178772
0.909090936183929 2.34329700469971
0.919191956520081 2.20934629440308
0.929292917251587 2.0544011592865
0.939393937587738 1.8892594575882
0.949494957923889 1.72562980651855
0.95959597826004 1.55829977989197
0.969696998596191 1.37887561321259
0.979797959327698 1.20626258850098
0.989898979663849 1.04822325706482
1 0.920554637908936
};
\addplot [semithick, color0, opacity=0.2]
table {%
0 -0.571947932243347
0.0101010100916028 -0.738592803478241
0.0202020201832056 -0.904007256031036
0.030303031206131 -1.04260611534119
0.0404040403664112 -1.16836607456207
0.0505050495266914 -1.27741384506226
0.060606062412262 -1.37406086921692
0.0707070678472519 -1.48308873176575
0.0808080807328224 -1.61940431594849
0.0909090936183929 -1.82771337032318
0.101010099053383 -2.0908374786377
0.111111111938953 -2.41590619087219
0.121212124824524 -2.80124592781067
0.131313130259514 -3.21526098251343
0.141414135694504 -3.6118597984314
0.151515156030655 -3.96976590156555
0.161616161465645 -4.3033971786499
0.171717166900635 -4.601393699646
0.181818187236786 -4.8320517539978
0.191919192671776 -4.96630764007568
0.202020198106766 -5.03707695007324
0.212121218442917 -5.02712631225586
0.222222223877907 -4.94709777832031
0.232323229312897 -4.78352737426758
0.242424249649048 -4.56254720687866
0.252525269985199 -4.29003429412842
0.262626260519028 -3.97683620452881
0.272727280855179 -3.63165760040283
0.28282830119133 -3.2418966293335
0.292929291725159 -2.8350396156311
0.30303031206131 -2.44271612167358
0.313131332397461 -2.11173009872437
0.32323232293129 -1.83833611011505
0.333333343267441 -1.60403668880463
0.34343433380127 -1.38946926593781
0.353535354137421 -1.16692411899567
0.363636374473572 -0.944236278533936
0.373737365007401 -0.74404901266098
0.383838385343552 -0.554502964019775
0.39393937587738 -0.391778230667114
0.404040396213531 -0.257056474685669
0.414141416549683 -0.186674654483795
0.424242407083511 -0.186238974332809
0.434343427419662 -0.24503518640995
0.444444447755814 -0.354584366083145
0.454545438289642 -0.46845144033432
0.464646458625793 -0.542056679725647
0.474747478961945 -0.592584252357483
0.484848499298096 -0.595092296600342
0.494949519634247 -0.565030217170715
0.505050539970398 -0.492619633674622
0.515151500701904 -0.37942361831665
0.525252521038055 -0.220048010349274
0.535353541374207 -0.0349883437156677
0.545454561710358 0.202317714691162
0.555555582046509 0.47649872303009
0.565656542778015 0.756312310695648
0.575757563114166 1.04069185256958
0.585858583450317 1.35056924819946
0.595959544181824 1.680251121521
0.606060564517975 2.01633715629578
0.616161584854126 2.34476900100708
0.626262605190277 2.65335059165955
0.636363625526428 2.93943548202515
0.646464645862579 3.17552042007446
0.65656566619873 3.30970454216003
0.666666686534882 3.31797552108765
0.676767706871033 3.2625584602356
0.686868667602539 3.1625030040741
0.69696968793869 3.07010364532471
0.707070708274841 2.99110722541809
0.717171728610992 2.93292427062988
0.727272748947144 2.888587474823
0.737373769283295 2.83605599403381
0.747474789619446 2.7781548500061
0.757575750350952 2.70557451248169
0.767676770687103 2.62793588638306
0.777777791023254 2.53652453422546
0.787878811359406 2.43431186676025
0.797979831695557 2.34720635414124
0.808080792427063 2.28218388557434
0.818181812763214 2.19441747665405
0.828282833099365 2.10538268089294
0.838383793830872 1.98117923736572
0.848484814167023 1.82456111907959
0.858585834503174 1.60641884803772
0.868686854839325 1.37415170669556
0.878787875175476 1.15052914619446
0.888888895511627 0.953238725662231
0.898989915847778 0.773709654808044
0.909090936183929 0.607815384864807
0.919191956520081 0.456663489341736
0.929292917251587 0.277816772460938
0.939393937587738 0.115437984466553
0.949494957923889 -0.0296986103057861
0.95959597826004 -0.131018161773682
0.969696998596191 -0.187404155731201
0.979797959327698 -0.196021914482117
0.989898979663849 -0.173290014266968
1 -0.14887011051178
};
\addplot [semithick, color1, opacity=0.2]
table {%
0 -1.07125818729401
0.0101010100916028 -1.20651865005493
0.0202020201832056 -1.35744476318359
0.030303031206131 -1.50552368164062
0.0404040403664112 -1.63422918319702
0.0505050495266914 -1.77012836933136
0.060606062412262 -1.91808223724365
0.0707070678472519 -2.04829692840576
0.0808080807328224 -2.18521738052368
0.0909090936183929 -2.24966812133789
0.101010099053383 -2.31995868682861
0.111111111938953 -2.34159255027771
0.121212124824524 -2.34347605705261
0.131313130259514 -2.32412672042847
0.141414135694504 -2.32966279983521
0.151515156030655 -2.2955379486084
0.161616161465645 -2.2720639705658
0.171717166900635 -2.28017997741699
0.181818187236786 -2.22630500793457
0.191919192671776 -2.18588352203369
0.202020198106766 -2.15864849090576
0.212121218442917 -2.09330153465271
0.222222223877907 -2.03028512001038
0.232323229312897 -1.96130204200745
0.242424249649048 -1.88730335235596
0.252525269985199 -1.7842001914978
0.262626260519028 -1.70646798610687
0.272727280855179 -1.5933084487915
0.28282830119133 -1.4421706199646
0.292929291725159 -1.28122019767761
0.30303031206131 -1.11497020721436
0.313131332397461 -0.895405292510986
0.32323232293129 -0.723376035690308
0.333333343267441 -0.509889841079712
0.34343433380127 -0.308495998382568
0.353535354137421 -0.0706202983856201
0.363636374473572 0.157665729522705
0.373737365007401 0.414125442504883
0.383838385343552 0.67303991317749
0.39393937587738 0.972550868988037
0.404040396213531 1.24752902984619
0.414141416549683 1.51975274085999
0.424242407083511 1.80967116355896
0.434343427419662 2.09967112541199
0.444444447755814 2.36991739273071
0.454545438289642 2.65269231796265
0.464646458625793 2.90242218971252
0.474747478961945 3.18071413040161
0.484848499298096 3.42765474319458
0.494949519634247 3.68067669868469
0.505050539970398 3.90465617179871
0.515151500701904 4.1582236289978
0.525252521038055 4.38878536224365
0.535353541374207 4.59318256378174
0.545454561710358 4.79006767272949
0.555555582046509 4.97662448883057
0.565656542778015 5.16480112075806
0.575757563114166 5.29720973968506
0.585858583450317 5.47407197952271
0.595959544181824 5.59535884857178
0.606060564517975 5.72644233703613
0.616161584854126 5.80872583389282
0.626262605190277 5.8619966506958
0.636363625526428 5.92036771774292
0.646464645862579 5.9495644569397
0.65656566619873 5.95765686035156
0.666666686534882 5.8813304901123
0.676767706871033 5.81364440917969
0.686868667602539 5.72372817993164
0.69696968793869 5.59320449829102
0.707070708274841 5.47699737548828
0.717171728610992 5.31642246246338
0.727272748947144 5.14597940444946
0.737373769283295 4.96121501922607
0.747474789619446 4.80610942840576
0.757575750350952 4.6405611038208
0.767676770687103 4.46778345108032
0.777777791023254 4.31003665924072
0.787878811359406 4.17093229293823
0.797979831695557 3.99192762374878
0.808080792427063 3.83279395103455
0.818181812763214 3.70338749885559
0.828282833099365 3.5218563079834
0.838383793830872 3.32343578338623
0.848484814167023 3.14878511428833
0.858585834503174 2.93203377723694
0.868686854839325 2.72745895385742
0.878787875175476 2.51325750350952
0.888888895511627 2.28960394859314
0.898989915847778 2.05752229690552
0.909090936183929 1.82735300064087
0.919191956520081 1.58472061157227
0.929292917251587 1.33153700828552
0.939393937587738 1.06008052825928
0.949494957923889 0.858110427856445
0.95959597826004 0.587775707244873
0.969696998596191 0.320348501205444
0.979797959327698 0.0571231842041016
0.989898979663849 -0.224284172058105
1 -0.469157218933105
};
\addplot [semithick, color2, opacity=0.2]
table {%
0 -1.41597509384155
0.0101010100916028 -1.99473011493683
0.0202020201832056 -2.53858065605164
0.030303031206131 -2.68976998329163
0.0404040403664112 -2.5623619556427
0.0505050495266914 -2.23090744018555
0.060606062412262 -1.64044594764709
0.0707070678472519 -0.904882788658142
0.0808080807328224 -0.396123647689819
0.0909090936183929 -0.288911104202271
0.101010099053383 -0.516821146011353
0.111111111938953 -0.862318694591522
0.121212124824524 -1.36962711811066
0.131313130259514 -1.69382989406586
0.141414135694504 -1.56359422206879
0.151515156030655 -1.05743420124054
0.161616161465645 -0.354796290397644
0.171717166900635 0.131229519844055
0.181818187236786 0.30054634809494
0.191919192671776 0.344394624233246
0.202020198106766 0.253494471311569
0.212121218442917 0.104639202356339
0.222222223877907 -0.317575991153717
0.232323229312897 -0.737549781799316
0.242424249649048 -1.03241610527039
0.252525269985199 -1.13863289356232
0.262626260519028 -1.12267887592316
0.272727280855179 -1.05534708499908
0.28282830119133 -1.04909801483154
0.292929291725159 -0.90547251701355
0.30303031206131 -0.854245603084564
0.313131332397461 -0.912733137607574
0.32323232293129 -0.84191358089447
0.333333343267441 -0.71999716758728
0.34343433380127 -0.832317054271698
0.353535354137421 -1.10293710231781
0.363636374473572 -1.53071343898773
0.373737365007401 -1.99647092819214
0.383838385343552 -2.54695320129395
0.39393937587738 -3.05651044845581
0.404040396213531 -3.38011026382446
0.414141416549683 -3.77904033660889
0.424242407083511 -4.32626247406006
0.434343427419662 -4.93825912475586
0.444444447755814 -5.30629968643188
0.454545438289642 -5.29307651519775
0.464646458625793 -4.81762886047363
0.474747478961945 -3.92064619064331
0.484848499298096 -2.85588264465332
0.494949519634247 -2.1294162273407
0.505050539970398 -1.72646856307983
0.515151500701904 -1.34104537963867
0.525252521038055 -0.823871493339539
0.535353541374207 -0.346075356006622
0.545454561710358 0.203857779502869
0.555555582046509 0.733048915863037
0.565656542778015 1.10441112518311
0.575757563114166 1.44505858421326
0.585858583450317 1.87796437740326
0.595959544181824 2.14697647094727
0.606060564517975 2.03155922889709
0.616161584854126 1.70090270042419
0.626262605190277 1.22797501087189
0.636363625526428 0.762781202793121
0.646464645862579 0.479481339454651
0.65656566619873 0.154757425189018
0.666666686534882 -0.153501003980637
0.676767706871033 -0.231868237257004
0.686868667602539 -0.0402351915836334
0.69696968793869 0.348876237869263
0.707070708274841 0.574583888053894
0.717171728610992 0.599006533622742
0.727272748947144 0.340336263179779
0.737373769283295 -0.176667183637619
0.747474789619446 -0.812671065330505
0.757575750350952 -1.62606930732727
0.767676770687103 -2.12503385543823
0.777777791023254 -2.23041677474976
0.787878811359406 -2.11082863807678
0.797979831695557 -1.8713800907135
0.808080792427063 -1.71615242958069
0.818181812763214 -1.48895466327667
0.828282833099365 -1.07746911048889
0.838383793830872 -0.489868372678757
0.848484814167023 0.0920155197381973
0.858585834503174 0.498546361923218
0.868686854839325 0.968432664871216
0.878787875175476 1.48415505886078
0.888888895511627 1.81713843345642
0.898989915847778 2.07900238037109
0.909090936183929 2.40730786323547
0.919191956520081 2.53045463562012
0.929292917251587 2.52864956855774
0.939393937587738 2.43041229248047
0.949494957923889 2.06297540664673
0.95959597826004 1.5041811466217
0.969696998596191 1.05711817741394
0.979797959327698 0.615318417549133
0.989898979663849 0.0640649795532227
1 -0.447825580835342
};
\addplot [semithick, color3, opacity=0.2]
table {%
0 2.90011262893677
0.0101010100916028 3.03687763214111
0.0202020201832056 3.18427920341492
0.030303031206131 3.32455730438232
0.0404040403664112 3.45987963676453
0.0505050495266914 3.58613848686218
0.060606062412262 3.70821332931519
0.0707070678472519 3.81804442405701
0.0808080807328224 3.91963481903076
0.0909090936183929 3.99790668487549
0.101010099053383 4.07720708847046
0.111111111938953 4.14429759979248
0.121212124824524 4.21782636642456
0.131313130259514 4.28282690048218
0.141414135694504 4.34779834747314
0.151515156030655 4.4031720161438
0.161616161465645 4.4646201133728
0.171717166900635 4.52310514450073
0.181818187236786 4.57928848266602
0.191919192671776 4.63597679138184
0.202020198106766 4.67889881134033
0.212121218442917 4.70472431182861
0.222222223877907 4.71663331985474
0.232323229312897 4.70709753036499
0.242424249649048 4.68992519378662
0.252525269985199 4.66161966323853
0.262626260519028 4.622314453125
0.272727280855179 4.56910037994385
0.28282830119133 4.49072313308716
0.292929291725159 4.4106764793396
0.30303031206131 4.30852270126343
0.313131332397461 4.19661855697632
0.32323232293129 4.08191680908203
0.333333343267441 3.95339798927307
0.34343433380127 3.82743692398071
0.353535354137421 3.6978554725647
0.363636374473572 3.57883954048157
0.373737365007401 3.46673727035522
0.383838385343552 3.36007332801819
0.39393937587738 3.27371406555176
0.404040396213531 3.19136118888855
0.414141416549683 3.12346458435059
0.424242407083511 3.04407262802124
0.434343427419662 2.97619485855103
0.444444447755814 2.89642667770386
0.454545438289642 2.83861207962036
0.464646458625793 2.77109718322754
0.474747478961945 2.70964169502258
0.484848499298096 2.63884210586548
0.494949519634247 2.56396865844727
0.505050539970398 2.48022532463074
0.515151500701904 2.38640284538269
0.525252521038055 2.27986478805542
0.535353541374207 2.17165374755859
0.545454561710358 2.0510630607605
0.555555582046509 1.92832851409912
0.565656542778015 1.81780850887299
0.575757563114166 1.69365727901459
0.585858583450317 1.57465648651123
0.595959544181824 1.4530553817749
0.606060564517975 1.33926677703857
0.616161584854126 1.21828413009644
0.626262605190277 1.10796689987183
0.636363625526428 0.991786599159241
0.646464645862579 0.861504137516022
0.65656566619873 0.735552132129669
0.666666686534882 0.583832919597626
0.676767706871033 0.445592939853668
0.686868667602539 0.29640007019043
0.69696968793869 0.153591722249985
0.707070708274841 0.0128487944602966
0.717171728610992 -0.123796254396439
0.727272748947144 -0.262515395879745
0.737373769283295 -0.40680980682373
0.747474789619446 -0.551224112510681
0.757575750350952 -0.688287079334259
0.767676770687103 -0.821904122829437
0.777777791023254 -0.945089042186737
0.787878811359406 -1.06519961357117
0.797979831695557 -1.17910599708557
0.808080792427063 -1.30472922325134
0.818181812763214 -1.42693543434143
0.828282833099365 -1.53871476650238
0.838383793830872 -1.66994214057922
0.848484814167023 -1.79529285430908
0.858585834503174 -1.91989421844482
0.868686854839325 -2.05387902259827
0.878787875175476 -2.1777982711792
0.888888895511627 -2.29853272438049
0.898989915847778 -2.39657878875732
0.909090936183929 -2.48925566673279
0.919191956520081 -2.5761559009552
0.929292917251587 -2.64608764648438
0.939393937587738 -2.71730971336365
0.949494957923889 -2.77013444900513
0.95959597826004 -2.82481288909912
0.969696998596191 -2.8751802444458
0.979797959327698 -2.92331123352051
0.989898979663849 -2.9700620174408
1 -3.03093433380127
};
\addplot [semithick, color4, opacity=0.2]
table {%
0 4.53728437423706
0.0101010100916028 4.59860563278198
0.0202020201832056 4.62341356277466
0.030303031206131 4.59741687774658
0.0404040403664112 4.52568006515503
0.0505050495266914 4.46154069900513
0.060606062412262 4.37432909011841
0.0707070678472519 4.27840185165405
0.0808080807328224 4.21013402938843
0.0909090936183929 4.15636444091797
0.101010099053383 4.10617399215698
0.111111111938953 4.06322336196899
0.121212124824524 4.00348949432373
0.131313130259514 3.87709307670593
0.141414135694504 3.67223620414734
0.151515156030655 3.40988445281982
0.161616161465645 3.09457588195801
0.171717166900635 2.74282598495483
0.181818187236786 2.35467934608459
0.191919192671776 1.98825120925903
0.202020198106766 1.63520181179047
0.212121218442917 1.30397498607635
0.222222223877907 0.995622754096985
0.232323229312897 0.697671592235565
0.242424249649048 0.357267022132874
0.252525269985199 -0.0148732662200928
0.262626260519028 -0.380893349647522
0.272727280855179 -0.712016403675079
0.28282830119133 -1.02250552177429
0.292929291725159 -1.21168613433838
0.30303031206131 -1.33234906196594
0.313131332397461 -1.40752649307251
0.32323232293129 -1.45952475070953
0.333333343267441 -1.51082932949066
0.34343433380127 -1.55615425109863
0.353535354137421 -1.62386012077332
0.363636374473572 -1.68691349029541
0.373737365007401 -1.69925165176392
0.383838385343552 -1.70557451248169
0.39393937587738 -1.69317007064819
0.404040396213531 -1.6895694732666
0.414141416549683 -1.67129611968994
0.424242407083511 -1.60815834999084
0.434343427419662 -1.51878595352173
0.444444447755814 -1.36625027656555
0.454545438289642 -1.13568735122681
0.464646458625793 -0.84857702255249
0.474747478961945 -0.527719736099243
0.484848499298096 -0.173161745071411
0.494949519634247 0.181560277938843
0.505050539970398 0.52055287361145
0.515151500701904 0.759220600128174
0.525252521038055 0.91455078125
0.535353541374207 0.984449625015259
0.545454561710358 0.984122037887573
0.555555582046509 0.952024459838867
0.565656542778015 0.874354839324951
0.575757563114166 0.800104260444641
0.585858583450317 0.7730712890625
0.595959544181824 0.840593695640564
0.606060564517975 1.00959122180939
0.616161584854126 1.26332330703735
0.626262605190277 1.54149508476257
0.636363625526428 1.86788487434387
0.646464645862579 2.22876167297363
0.65656566619873 2.54001522064209
0.666666686534882 2.8362455368042
0.676767706871033 3.09804201126099
0.686868667602539 3.34182024002075
0.69696968793869 3.57347726821899
0.707070708274841 3.75619459152222
0.717171728610992 3.88219881057739
0.727272748947144 3.99460458755493
0.737373769283295 4.10030317306519
0.747474789619446 4.19482755661011
0.757575750350952 4.25527667999268
0.767676770687103 4.30463600158691
0.777777791023254 4.38711214065552
0.787878811359406 4.53424215316772
0.797979831695557 4.71334171295166
0.808080792427063 4.89998626708984
0.818181812763214 5.08161973953247
0.828282833099365 5.25718402862549
0.838383793830872 5.4548602104187
0.848484814167023 5.65133476257324
0.858585834503174 5.7822437286377
0.868686854839325 5.8221583366394
0.878787875175476 5.75061511993408
0.888888895511627 5.54752492904663
0.898989915847778 5.3140344619751
0.909090936183929 5.06823587417603
0.919191956520081 4.8929328918457
0.929292917251587 4.80693340301514
0.939393937587738 4.80895900726318
0.949494957923889 4.84252071380615
0.95959597826004 4.87360191345215
0.969696998596191 4.89314651489258
0.979797959327698 4.91093063354492
0.989898979663849 4.9050931930542
1 4.85746622085571
};
\addplot [semithick, color5, opacity=0.2]
table {%
0 -1.04631972312927
0.0101010100916028 -0.949383020401001
0.0202020201832056 -0.830657005310059
0.030303031206131 -0.753649473190308
0.0404040403664112 -0.610736668109894
0.0505050495266914 -0.491872668266296
0.060606062412262 -0.365663290023804
0.0707070678472519 -0.215200394392014
0.0808080807328224 -0.0748999938368797
0.0909090936183929 0.0525348223745823
0.101010099053383 0.226044178009033
0.111111111938953 0.374838054180145
0.121212124824524 0.561473727226257
0.131313130259514 0.742501139640808
0.141414135694504 0.896478533744812
0.151515156030655 1.09724903106689
0.161616161465645 1.28134715557098
0.171717166900635 1.49056434631348
0.181818187236786 1.67613911628723
0.191919192671776 1.87965655326843
0.202020198106766 2.10092401504517
0.212121218442917 2.29077768325806
0.222222223877907 2.5069842338562
0.232323229312897 2.69590759277344
0.242424249649048 2.91653966903687
0.252525269985199 3.11203622817993
0.262626260519028 3.29525780677795
0.272727280855179 3.44791197776794
0.28282830119133 3.63743209838867
0.292929291725159 3.78944873809814
0.30303031206131 3.97464370727539
0.313131332397461 4.117431640625
0.32323232293129 4.25167846679688
0.333333343267441 4.39271974563599
0.34343433380127 4.51337814331055
0.353535354137421 4.62599945068359
0.363636374473572 4.76418256759644
0.373737365007401 4.86494159698486
0.383838385343552 4.96113014221191
0.39393937587738 5.04225254058838
0.404040396213531 5.16114711761475
0.414141416549683 5.23874282836914
0.424242407083511 5.28753185272217
0.434343427419662 5.38054418563843
0.444444447755814 5.44680452346802
0.454545438289642 5.50806474685669
0.464646458625793 5.56092929840088
0.474747478961945 5.60378742218018
0.484848499298096 5.66351509094238
0.494949519634247 5.70379543304443
0.505050539970398 5.75484704971313
0.515151500701904 5.8149209022522
0.525252521038055 5.85479927062988
0.535353541374207 5.90870761871338
0.545454561710358 5.98610210418701
0.555555582046509 6.03032779693604
0.565656542778015 6.06417274475098
0.575757563114166 6.13775491714478
0.585858583450317 6.18766260147095
0.595959544181824 6.22336483001709
0.606060564517975 6.26369142532349
0.616161584854126 6.29533004760742
0.626262605190277 6.30706644058228
0.636363625526428 6.3376669883728
0.646464645862579 6.35718250274658
0.65656566619873 6.36270093917847
0.666666686534882 6.36445331573486
0.676767706871033 6.34895896911621
0.686868667602539 6.31422472000122
0.69696968793869 6.29611253738403
0.707070708274841 6.29061698913574
0.717171728610992 6.24510955810547
0.727272748947144 6.20179843902588
0.737373769283295 6.1452693939209
0.747474789619446 6.07259798049927
0.757575750350952 6.04434108734131
0.767676770687103 5.94684600830078
0.777777791023254 5.88479614257812
0.787878811359406 5.82306814193726
0.797979831695557 5.71962928771973
0.808080792427063 5.65135288238525
0.818181812763214 5.55506992340088
0.828282833099365 5.46424198150635
0.838383793830872 5.40532112121582
0.848484814167023 5.2839789390564
0.858585834503174 5.20161294937134
0.868686854839325 5.07499170303345
0.878787875175476 4.9753098487854
0.888888895511627 4.88469171524048
0.898989915847778 4.78041076660156
0.909090936183929 4.69060468673706
0.919191956520081 4.60182094573975
0.929292917251587 4.49099254608154
0.939393937587738 4.39595460891724
0.949494957923889 4.32589340209961
0.95959597826004 4.21141672134399
0.969696998596191 4.10668277740479
0.979797959327698 4.01978492736816
0.989898979663849 3.92296934127808
1 3.82549667358398
};
\addplot [semithick, color6, opacity=0.2]
table {%
0 -2.85499167442322
0.0101010100916028 -2.94648480415344
0.0202020201832056 -3.04143095016479
0.030303031206131 -3.14543485641479
0.0404040403664112 -3.24359846115112
0.0505050495266914 -3.35092949867249
0.060606062412262 -3.46445941925049
0.0707070678472519 -3.58558130264282
0.0808080807328224 -3.72168970108032
0.0909090936183929 -3.8671715259552
0.101010099053383 -4.01843452453613
0.111111111938953 -4.17528867721558
0.121212124824524 -4.32670783996582
0.131313130259514 -4.48854684829712
0.141414135694504 -4.62784719467163
0.151515156030655 -4.77119016647339
0.161616161465645 -4.89512920379639
0.171717166900635 -5.0172700881958
0.181818187236786 -5.12356233596802
0.191919192671776 -5.2249174118042
0.202020198106766 -5.3045072555542
0.212121218442917 -5.37464427947998
0.222222223877907 -5.43828058242798
0.232323229312897 -5.47788286209106
0.242424249649048 -5.53148078918457
0.252525269985199 -5.5509991645813
0.262626260519028 -5.55774641036987
0.272727280855179 -5.54863786697388
0.28282830119133 -5.54200458526611
0.292929291725159 -5.50639009475708
0.30303031206131 -5.46097040176392
0.313131332397461 -5.40846347808838
0.32323232293129 -5.33580636978149
0.333333343267441 -5.25540018081665
0.34343433380127 -5.16555023193359
0.353535354137421 -5.0587043762207
0.363636374473572 -4.94284582138062
0.373737365007401 -4.83011245727539
0.383838385343552 -4.70008039474487
0.39393937587738 -4.56570482254028
0.404040396213531 -4.43624258041382
0.414141416549683 -4.29303216934204
0.424242407083511 -4.16455507278442
0.434343427419662 -4.02478551864624
0.444444447755814 -3.90038824081421
0.454545438289642 -3.77710795402527
0.464646458625793 -3.67243981361389
0.474747478961945 -3.56473064422607
0.484848499298096 -3.45006585121155
0.494949519634247 -3.34804320335388
0.505050539970398 -3.25616645812988
0.515151500701904 -3.15467309951782
0.525252521038055 -3.06312227249146
0.535353541374207 -2.97076892852783
0.545454561710358 -2.87954449653625
0.555555582046509 -2.80078363418579
0.565656542778015 -2.72139358520508
0.575757563114166 -2.649165391922
0.585858583450317 -2.56876945495605
0.595959544181824 -2.49055433273315
0.606060564517975 -2.40904998779297
0.616161584854126 -2.31949377059937
0.626262605190277 -2.23162698745728
0.636363625526428 -2.1365761756897
0.646464645862579 -2.04481101036072
0.65656566619873 -1.96288514137268
0.666666686534882 -1.87503004074097
0.676767706871033 -1.80468702316284
0.686868667602539 -1.72990393638611
0.69696968793869 -1.65618789196014
0.707070708274841 -1.59323501586914
0.717171728610992 -1.51458334922791
0.727272748947144 -1.42903780937195
0.737373769283295 -1.34500896930695
0.747474789619446 -1.2594358921051
0.757575750350952 -1.17745959758759
0.767676770687103 -1.07633519172668
0.777777791023254 -0.977434933185577
0.787878811359406 -0.881283640861511
0.797979831695557 -0.76569926738739
0.808080792427063 -0.664943277835846
0.818181812763214 -0.547435402870178
0.828282833099365 -0.444242298603058
0.838383793830872 -0.327868223190308
0.848484814167023 -0.209941327571869
0.858585834503174 -0.0992040038108826
0.868686854839325 0.0162366628646851
0.878787875175476 0.15179306268692
0.888888895511627 0.274383306503296
0.898989915847778 0.40609335899353
0.909090936183929 0.540075421333313
0.919191956520081 0.680586159229279
0.929292917251587 0.812853395938873
0.939393937587738 0.963186025619507
0.949494957923889 1.09713172912598
0.95959597826004 1.22349095344543
0.969696998596191 1.33857882022858
0.979797959327698 1.45320427417755
0.989898979663849 1.56988215446472
1 1.67705059051514
};
\addplot [semithick, white!49.8039215686275!black, opacity=0.2]
table {%
0 1.35104405879974
0.0101010100916028 1.21539282798767
0.0202020201832056 1.05275893211365
0.030303031206131 0.845260441303253
0.0404040403664112 0.587349593639374
0.0505050495266914 0.337263524532318
0.060606062412262 0.0984273254871368
0.0707070678472519 -0.1112991720438
0.0808080807328224 -0.315586060285568
0.0909090936183929 -0.512107849121094
0.101010099053383 -0.668996930122375
0.111111111938953 -0.790766417980194
0.121212124824524 -0.825496017932892
0.131313130259514 -0.838222742080688
0.141414135694504 -0.833304166793823
0.151515156030655 -0.802125453948975
0.161616161465645 -0.785033762454987
0.171717166900635 -0.787507712841034
0.181818187236786 -0.76988273859024
0.191919192671776 -0.740726232528687
0.202020198106766 -0.688096761703491
0.212121218442917 -0.625508666038513
0.222222223877907 -0.56617134809494
0.232323229312897 -0.527588248252869
0.242424249649048 -0.501333355903625
0.252525269985199 -0.476342439651489
0.262626260519028 -0.454761862754822
0.272727280855179 -0.435355544090271
0.28282830119133 -0.450817823410034
0.292929291725159 -0.482116043567657
0.30303031206131 -0.543460249900818
0.313131332397461 -0.610179305076599
0.32323232293129 -0.724994659423828
0.333333343267441 -0.888031244277954
0.34343433380127 -1.08760666847229
0.353535354137421 -1.3196108341217
0.363636374473572 -1.59593224525452
0.373737365007401 -1.89323019981384
0.383838385343552 -2.17987465858459
0.39393937587738 -2.45591187477112
0.404040396213531 -2.74982213973999
0.414141416549683 -3.04846215248108
0.424242407083511 -3.38920593261719
0.434343427419662 -3.73768162727356
0.444444447755814 -4.07441520690918
0.454545438289642 -4.38291263580322
0.464646458625793 -4.63119506835938
0.474747478961945 -4.8270378112793
0.484848499298096 -4.97166156768799
0.494949519634247 -5.06199550628662
0.505050539970398 -5.12060070037842
0.515151500701904 -5.12368106842041
0.525252521038055 -5.05576229095459
0.535353541374207 -4.94504022598267
0.545454561710358 -4.80658912658691
0.555555582046509 -4.63091421127319
0.565656542778015 -4.44632720947266
0.575757563114166 -4.28469133377075
0.585858583450317 -4.11631679534912
0.595959544181824 -3.92223477363586
0.606060564517975 -3.71471500396729
0.616161584854126 -3.45397877693176
0.626262605190277 -3.1499719619751
0.636363625526428 -2.84679841995239
0.646464645862579 -2.55780458450317
0.65656566619873 -2.26444673538208
0.666666686534882 -1.98461627960205
0.676767706871033 -1.70987415313721
0.686868667602539 -1.40734934806824
0.69696968793869 -1.09948801994324
0.707070708274841 -0.803181231021881
0.717171728610992 -0.519817173480988
0.727272748947144 -0.253726840019226
0.737373769283295 0.00119662284851074
0.747474789619446 0.250497221946716
0.757575750350952 0.507363319396973
0.767676770687103 0.816270232200623
0.777777791023254 1.1226749420166
0.787878811359406 1.35685014724731
0.797979831695557 1.48968911170959
0.808080792427063 1.59751451015472
0.818181812763214 1.64660608768463
0.828282833099365 1.61181044578552
0.838383793830872 1.50673651695251
0.848484814167023 1.37093651294708
0.858585834503174 1.22919487953186
0.868686854839325 1.10125815868378
0.878787875175476 0.967206656932831
0.888888895511627 0.864370346069336
0.898989915847778 0.807151257991791
0.909090936183929 0.800308167934418
0.919191956520081 0.813098669052124
0.929292917251587 0.824967503547668
0.939393937587738 0.83734667301178
0.949494957923889 0.848156809806824
0.95959597826004 0.826453447341919
0.969696998596191 0.75333309173584
0.979797959327698 0.690777540206909
0.989898979663849 0.595033168792725
1 0.48309326171875
};
\addplot [semithick, color7, opacity=0.2]
table {%
0 -1.60989451408386
0.0101010100916028 -1.51542150974274
0.0202020201832056 -1.40991246700287
0.030303031206131 -1.30580127239227
0.0404040403664112 -1.17768526077271
0.0505050495266914 -1.08806324005127
0.060606062412262 -0.93954199552536
0.0707070678472519 -0.807285308837891
0.0808080807328224 -0.656267285346985
0.0909090936183929 -0.525127649307251
0.101010099053383 -0.396870046854019
0.111111111938953 -0.282980561256409
0.121212124824524 -0.158603221178055
0.131313130259514 -0.0824716687202454
0.141414135694504 0.0243114829063416
0.151515156030655 0.130483895540237
0.161616161465645 0.198617219924927
0.171717166900635 0.25265371799469
0.181818187236786 0.318867683410645
0.191919192671776 0.368633031845093
0.202020198106766 0.40425580739975
0.212121218442917 0.48121166229248
0.222222223877907 0.543250024318695
0.232323229312897 0.612955212593079
0.242424249649048 0.692381262779236
0.252525269985199 0.76584780216217
0.262626260519028 0.843113899230957
0.272727280855179 0.905805826187134
0.28282830119133 1.02371919155121
0.292929291725159 1.12751889228821
0.30303031206131 1.22075486183167
0.313131332397461 1.32865250110626
0.32323232293129 1.4065648317337
0.333333343267441 1.48401379585266
0.34343433380127 1.57999205589294
0.353535354137421 1.62660217285156
0.363636374473572 1.7029721736908
0.373737365007401 1.74622011184692
0.383838385343552 1.78225481510162
0.39393937587738 1.81250631809235
0.404040396213531 1.80551910400391
0.414141416549683 1.81247735023499
0.424242407083511 1.77532303333282
0.434343427419662 1.78273725509644
0.444444447755814 1.77546942234039
0.454545438289642 1.74428462982178
0.464646458625793 1.72541308403015
0.474747478961945 1.68495535850525
0.484848499298096 1.67687940597534
0.494949519634247 1.65420985221863
0.505050539970398 1.65649056434631
0.515151500701904 1.65864670276642
0.525252521038055 1.64165091514587
0.535353541374207 1.63704812526703
0.545454561710358 1.63117372989655
0.555555582046509 1.61493682861328
0.565656542778015 1.61841642856598
0.575757563114166 1.63306522369385
0.585858583450317 1.61587107181549
0.595959544181824 1.61895203590393
0.606060564517975 1.65060722827911
0.616161584854126 1.67486238479614
0.626262605190277 1.72279453277588
0.636363625526428 1.75927770137787
0.646464645862579 1.81955897808075
0.65656566619873 1.89747047424316
0.666666686534882 1.9976681470871
0.676767706871033 2.11934757232666
0.686868667602539 2.21988105773926
0.69696968793869 2.35192251205444
0.707070708274841 2.49123859405518
0.717171728610992 2.64172506332397
0.727272748947144 2.74934554100037
0.737373769283295 2.91775894165039
0.747474789619446 3.01979207992554
0.757575750350952 3.15104246139526
0.767676770687103 3.24878072738647
0.777777791023254 3.33579301834106
0.787878811359406 3.41999340057373
0.797979831695557 3.50799465179443
0.808080792427063 3.55657982826233
0.818181812763214 3.61788773536682
0.828282833099365 3.72170114517212
0.838383793830872 3.78883695602417
0.848484814167023 3.85903644561768
0.858585834503174 3.97314596176147
0.868686854839325 4.01713371276855
0.878787875175476 4.12168598175049
0.888888895511627 4.22345781326294
0.898989915847778 4.28727865219116
0.909090936183929 4.38500261306763
0.919191956520081 4.49235486984253
0.929292917251587 4.54872608184814
0.939393937587738 4.66223239898682
0.949494957923889 4.7370285987854
0.95959597826004 4.80878973007202
0.969696998596191 4.85925197601318
0.979797959327698 4.91715288162231
0.989898979663849 4.98196935653687
1 4.98609924316406
};
\end{axis}
\end{tikzpicture}
%\documentclass{minimal}
\iffalse
\usepackage[dvipsnames]{xcolor}
\usepackage{tikz}
\usetikzlibrary{positioning,calc,backgrounds,fit,arrows.meta}
\fi
%\begin{document}
\newcommand{\lastidx}{n+1}
\newcommand{\evalidx}{n+1}
\newcommand{\dataset}{\mathcal{D}}
\newcommand{\setofpairs}[2]{\{(x_1^{#1}, y_1^{#1}),\dots,(x_{#2}^{#1}, y_{#2}^{#1})\}}
\newcommand{\seq}[1]{$\setofpairs{#1}{\lastidx{}}$}
\begin{tikzpicture}[remember picture,
inner/.style={circle,draw=blue!50,fill=blue!20,thick,inner sep=3pt},
]
\tikzstyle{every node} = [font=\footnotesize,]
\tikzstyle{bigbox} = [thick, fill=white, rounded corners, rectangle,draw=black!30]
\tikzstyle{sample} = [thick, minimum size=0.6cm, rounded corners,rectangle, fill=Salmon!10,draw=black!30]
\tikzstyle{loss} = [thick, minimum size=0.6cm, rounded corners,rectangle, fill=PineGreen!10,draw=black!30]
\tikzstyle{pfmbox} = [thick,minimum size=0.6cm, rounded corners,rectangle, fill=MidnightBlue!10,draw=black!30]
\node[] (top) {Sample prior datasets $D^{(i)} \sim p(\dataset{})$};
\node[sample,below=1.8em of top.west, anchor=west] (1) {$D^{(1)} = D^{(1)}_{train} \cup \{(x^{(1)}_{test},y^{(1)}_{test})\}$};
%\node[sample,below=1em of 1] (2) {\seq{2}};
%\node[sample,below=1em of 2] (3) {\seq{3}};
\node[below=-5pt of 1] (dots) {$\vdots$};
\node[sample,below=1pt of dots] (2) {$D^{(K)} = D^{(K)}_{train} \cup \{(x^{(K)}_{test},y^{(K)}_{test})\}$};
\begin{pgfonlayer}{background}
\node[bigbox] [fit = (top) (2) (dots)] (metadataset) {
};
\end{pgfonlayer}
\node[loss,right=of metadataset.north east,anchor=north west, align=center, text width=4.5cm] (l) {Train the PFN by minimizing \\ $-\sum_{i=1}^K \log q_\theta(y_{test}^{(i)}|x^{(i)}_{test}, D^{(i)}_{train})$};
\draw [-{Triangle[length=3mm, width=2mm]}] (metadataset.east|-l) -- (l);
\node[below=7.7em of top.west, anchor=west] (evaltop) {Actual training dataset and test input};
\node[sample,below=1.5em of evaltop.west, anchor=west] (evaldata) {$(D_{train}, x_{test})$};
\node[pfmbox,align=center,text width=4.5cm, below=2.5em of l] (pfm) {Bayesian inference via the trained PFN, with the actual training data and a test point as input:\\$q_{\theta^*}(y_{test}^{}|x^{}_{test}, D_{train}) \approx p(y_{test}^{}|x^{}_{test}, D_{train})$};
\draw [-{Triangle[length=3mm, width=2mm]}] (evaldata) -- (pfm.west|-evaldata);
\draw [-{Triangle[length=3mm, width=2mm]}] (l) -- (pfm) node[midway,right] {PFN with parameters $\theta^*$};
\end{tikzpicture}
%\end{document}
\iffalse
%\documentclass{article}
\usepackage[dvipsnames]{xcolor}
\usepackage{tikz}
\usetikzlibrary{positioning,calc,backgrounds,fit,arrows.meta}
\usetikzlibrary{arrows}
\usetikzlibrary{decorations.pathreplacing,calligraphy}
\fi
%\begin{document}
\newcommand{\lastidx}{n+1}
\newcommand{\evalidx}{n+1}
\newcommand{\setofpairs}[2]{\{(x_1^{#1}, y_1^{#1}),\dots,(x_{#2}^{#1}, y_{#2}^{#1})\}}
\newcommand{\seq}[1]{$\setofpairs{#1}{\lastidx{}}$}
\newcommand{\xypair}[1]{$(x_{#1},y_{#1})$}
\newcommand{\qattbend}{40}
\def\innerdistance{2.2em}
\def\outdistane{1.2em}
\def\indistane{1.em}
\begin{tikzpicture}[remember picture,
inner/.style={circle,draw=blue!50,fill=blue!20,thick,inner sep=3pt},
]
\tikzstyle{bigbox} = [thick, fill=white, rounded corners, rectangle,draw=black!30]
\tikzstyle{sample} = [thick, minimum size=0.6cm, rounded corners,rectangle, fill=Salmon!10,draw=black!30]
\tikzstyle{loss} = [thick, minimum size=0.6cm, rounded corners,rectangle, fill=PineGreen!10,draw=black!30]
\tikzstyle{timestep} = [thick,minimum width=0.2cm,minimum height=.6cm, rounded corners,rectangle, fill=MidnightBlue!10,draw=black!30]
\tikzstyle{querytimestep} = [thick,minimum width=0.2cm,minimum height=.6cm, rounded corners,rectangle, fill=MidnightBlue!10,draw=black!30]
\tikzstyle{innerattention} = [draw=blue!30,-{Triangle[length=2mm, width=1mm]},]
\tikzstyle{queryattention} = [draw=red!30,-{Triangle[length=2mm, width=1mm]},]
\tikzstyle{output} = [draw=black!70,-{Triangle[length=2mm, width=1mm]},]
\tikzstyle{input} = [draw=black!70,-{Triangle[length=2mm, width=1mm]},]
\tikzstyle{every node} = [font=\footnotesize,]
\node[timestep] (t1) {};
\node[timestep, right=\innerdistance of t1] (t2) {};
\node[timestep, right=\innerdistance of t2] (t3) {};
\draw [ innerattention] (t1) to [bend left=15] (t2);
\draw [ innerattention] (t1) to [bend left=30] (t3);
\draw [ innerattention] (t2) to [bend right=15] (t1);
\draw [ innerattention] (t2) to [bend left=15] (t3);
\draw [ innerattention] (t3) to [bend right=30] (t1);
\draw [ innerattention] (t3) to [bend right=15] (t2);
\node[below=\indistane of t1] (i1) {\xypair{1}};
\draw [ input] (i1) -- (t1);
\node[below=\indistane of t2] (i2) {\xypair{2}};
\draw [ input] (i2) -- (t2);
\node[below=\indistane of t3] (i3) {\xypair{3}};
\draw [ input] (i3) -- (t3);
\draw [pen colour={black!70},
decorate,
decoration = {calligraphic brace, mirror,
raise=.7em,
amplitude=5pt}] (i1.west) -- (i3.east)
node[pos=0.5,below=1.2em,black]{$D$};
\node[querytimestep, right=2em of t3] (q1) {};
\node[querytimestep, right=3.2em of q1] (q2) {};
\draw [ queryattention] (q1) to [bend right=40] (t1);
\draw [ queryattention] (q1) to [bend right=40] (t2);
\draw [ queryattention] (q1) to [bend right=40] (t3);
\draw [ queryattention] (q2) to [bend right=40] (t1);
\draw [ queryattention] (q2) to [bend right=40] (t2);
\draw [ queryattention] (q2) to [bend right=40] (t3);
\node[above=\outdistane of q2] (d2) {$q(\cdot|x_5,D)$};
\node[above=\outdistane of q1] (d1) {$q(\cdot|x_4,D)$};
\draw [ output] (q1) -- (d1);
\node[] at (q1|-i1) (i4) {$x_4$};
\draw [ input] (i4) -- (q1);
\draw [ output] (q2) -- (d2);
\node[] at (q2|-i1) (i5) {$x_5$};
\draw [ input] (i5) -- (q2);
\end{tikzpicture}
%\end{document}
\begin{tikzpicture}
\begin{axis}[width=\textwidth,
height=.98\textwidth,
axis lines=center,
axis y line*=left,
xmin=-4,xmax=4,
ymin=0,ymax=.5,
grid=major,
yminorgrids,
ylabel near ticks,
xlabel near ticks,
major grid style={thick},
tick style={thick},
axis line style={thick},
xlabel=$y$,
ylabel=Density,
xtick={-3,-1,0,1,3},
ytick={0,.2,.4},
ticklabel style={font=\tiny},
label style={font=\tiny},
enlarge x limits={upper,value=0.02},
enlarge y limits={upper,value=0.1},
]
\addplot[BarStyle]coordinates{(-3,.1)(-1.5,.1)};
\addplot[BarStyle]coordinates{(-1.5,.15)(-.5,.15)};
\addplot[BarStyle]coordinates{(-.5,.25)(0,.25)};
\addplot[BarStyle]coordinates{(.0,.35)(.5,.25)};
\addplot[BarStyle]coordinates{(.5,.15)(2,.15)};
\addplot[BarStyle]coordinates{(2,.05)(3.,.1)};
\end{axis}
\end{tikzpicture}
\pgfmathdeclarefunction{gauss}{3}{%
\pgfmathparse{1/(#2*sqrt(2*pi))*#3*exp(-((x-#1)^2)/(2*#2^2))}%
}
\usepgfplotslibrary{fillbetween}
\begin{tikzpicture}
\begin{axis}[width=\textwidth,
height=.98\textwidth,
axis lines=center,
axis y line*=left,
xmin=-4,xmax=4,
ymin=0,ymax=.5,
grid=major,
yminorgrids,
ylabel near ticks,
xlabel near ticks,
major grid style={thick},
tick style={thick},
axis line style={thick},
xlabel=$y_q$,
ylabel=Density,
xtick={-3,-1,0,1,3},
ytick={0,.2,.4},
ticklabel style={font=\tiny},
label style={font=\tiny},
enlarge x limits={upper,value=0.02},
enlarge y limits={upper,value=0.1},
]
\addplot [name path=A,domain=-10:-1.5] {gauss(-1.5, .75, .1*2)};
\addplot[name path=line, domain=-10:-1.5, gray, no markers, line width=1pt] {0};
\addplot[BarStyle] fill between [of=A and line];
\addplot[BarStyle]coordinates{(-1.5,.15)(-.5,.15)};
\addplot[BarStyle]coordinates{(-.5,.25)(0,.25)};
\addplot[BarStyle]coordinates{(.0,.35)(.5,.25)};
\addplot[BarStyle]coordinates{(.5,.20)(2,.15)};
\addplot [name path=B,domain=2:10] {gauss(2, .5,.05*2)};
\addplot[name path=line, domain=2:10, gray, no markers, line width=1pt] {0};
\addplot[BarStyle] fill between [of=B and line];
\end{axis}
\end{tikzpicture}
\begin{tikzpicture}
\begin{axis}[width=\textwidth,
height=1.\textwidth,
axis lines=center,
axis y line*=left,
xmin=-4,xmax=4,
ymin=0,ymax=.5,
grid=major,
yminorgrids,
ylabel near ticks,
xlabel near ticks,
major grid style={thick},
tick style={thick},
axis line style={thick},
xlabel=$y$,
ylabel=Density,
xtick={-3,-2,-1,0,1,2,3},
ytick={0,.2,.4},
ticklabel style={font=\tiny},
label style={font=\tiny},
enlarge x limits={upper,value=0.02},
enlarge y limits={upper,value=0.1},
]
\addplot[BarStyle]coordinates{(-3,.1)(-2,.1)};
\addplot[BarStyle]coordinates{(-2,.15)(-1,.15)};
\addplot[BarStyle]coordinates{(-1,.25)(0,.25)};
\addplot[BarStyle]coordinates{(.0,.35)(1.,.25)};
\addplot[BarStyle]coordinates{(1,.20)(2.,.15)};
\addplot[BarStyle]coordinates{(2,.05)(3.,.1)};
\end{axis}
\end{tikzpicture}
% This file was created with tikzplotlib v0.9.12.
\iffalse
%\documentclass{minimal}
\usepackage[dvipsnames]{xcolor}
\usepackage{tikz}
\usepackage{pgfplots}
\usetikzlibrary{positioning,calc,backgrounds,fit,arrows.meta}
\fi
%\begin{document}
\begin{tikzpicture}
\definecolor{color0}{rgb}{0.12156862745098,0.466666666666667,0.705882352941177}
\definecolor{color1}{rgb}{1,0.498039215686275,0.0549019607843137}
\definecolor{color2}{rgb}{0.172549019607843,0.627450980392157,0.172549019607843}
\begin{axis}[
hide axis,
width=1.\textwidth,
height=.2\textwidth,
legend columns=3,legend style={draw=none,column sep=5ex},
legend cell align={left},
legend style={fill opacity=0.8, draw opacity=1, text opacity=1, draw=white!80!black},
log basis x={10},
tick align=outside,
tick pos=left,
x grid style={white!69.0196078431373!black},
xlabel={Training time per task},
xmin=0.005, xmax=300,
xmode=log,
xtick style={color=black},
y grid style={white!69.0196078431373!black},
ylabel={Negative Log Likelihood},
ymin=-1, ymax=15,
ytick style={color=black}
]
\addlegendimage{semithick, black, mark=*, mark size=1.5, mark options={solid}, only marks}
\addlegendentry{Given examples}
\addlegendimage{semithick, green!50.1960784313725!black}
\addlegendentry{GP mean}
\addlegendimage{draw=green!50.1960784313725!black, fill=green!50.1960784313725!black, opacity=0.2,area legend}
\addlegendentry{GP 95\% confidence}
\addlegendimage{draw=none, left color=blue!0!white, right color=blue!55!white, opacity=1.,area legend,}
\addlegendentry{\priorfittednetwork{} density}
\addlegendimage{semithick, blue}
\addlegendentry{\priorfittednetwork{} mean}
\addlegendimage{draw=blue, fill=blue, opacity=0.2,area legend,}
\addlegendentry{\priorfittednetwork{} 95\% confidence}
\end{axis}
\end{tikzpicture}
%\end{document}
% This file was created by tikzplotlib v0.9.9.
\begin{tikzpicture}
\definecolor{color0}{rgb}{0.12156862745098,0.466666666666667,0.705882352941177}
\definecolor{color1}{rgb}{1,0.498039215686275,0.0549019607843137}
\definecolor{color2}{rgb}{0.172549019607843,0.627450980392157,0.172549019607843}
\definecolor{color3}{rgb}{0.83921568627451,0.152941176470588,0.156862745098039}
\definecolor{color4}{rgb}{0.580392156862745,0.403921568627451,0.741176470588235}
\begin{axis}[
legend cell align={left},
legend style={fill opacity=0.8, draw opacity=1, text opacity=1, draw=white!80!black},
tick align=outside,
tick pos=left,
x grid style={white!69.0196078431373!black},
xlabel={Number of revealed data points (n)},
%xmin=0, xmax=1901,
xtick style={color=black},
y grid style={white!69.0196078431373!black},
ylabel={Negative Log-Likelihood},
ymin=-3.26597546491281, ymax=1.87508452316904,
ytick style={color=black}
]
\path [fill=color0, fill opacity=0.1]
(axis cs:1,1.64139997825623)
--(axis cs:1,1.50919997825623)
--(axis cs:101,-0.214500003623962)
--(axis cs:201,-0.994199993610382)
--(axis cs:301,-1.37150002880096)
--(axis cs:401,-1.52599999103546)
--(axis cs:501,-1.70860005130768)
--(axis cs:601,-1.80539994239807)
--(axis cs:701,-1.901799949646)
--(axis cs:801,-1.99730005340576)
--(axis cs:901,-2.05490009880066)
--(axis cs:1001,-2.10530000534058)
--(axis cs:1101,-2.08919996109009)
--(axis cs:1201,-2.1077999874115)
--(axis cs:1301,-2.15710003013611)
--(axis cs:1401,-2.17160008354187)
--(axis cs:1501,-2.16509994812012)
--(axis cs:1601,-2.18009991416931)
--(axis cs:1701,-2.20169988594055)
--(axis cs:1801,-2.20999988250732)
--(axis cs:1901,-2.21329998512268)
--(axis cs:1901,-2.15289998512268)
--(axis cs:1901,-2.15289998512268)
--(axis cs:1801,-2.15639988250732)
--(axis cs:1701,-2.14749988594055)
--(axis cs:1601,-2.12489991416931)
--(axis cs:1501,-2.10869994812012)
--(axis cs:1401,-2.11820008354187)
--(axis cs:1301,-2.09470003013611)
--(axis cs:1201,-2.0385999874115)
--(axis cs:1101,-2.02239996109009)
--(axis cs:1001,-2.04850000534058)
--(axis cs:901,-1.97790009880066)
--(axis cs:801,-1.91570005340576)
--(axis cs:701,-1.808799949646)
--(axis cs:601,-1.70539994239807)
--(axis cs:501,-1.60380005130768)
--(axis cs:401,-1.40779999103546)
--(axis cs:301,-1.24810002880096)
--(axis cs:201,-0.864199993610382)
--(axis cs:101,-0.0729000036239624)
--(axis cs:1,1.64139997825623)
--cycle;
\path [fill=color1, fill opacity=0.1]
(axis cs:1,1.57399995288849)
--(axis cs:1,1.43979995288849)
--(axis cs:101,-0.161200002610683)
--(axis cs:201,-1.00589999456406)
--(axis cs:301,-1.42989998817444)
--(axis cs:401,-1.68630002098083)
--(axis cs:501,-1.95980000457764)
--(axis cs:601,-2.03300005397797)
--(axis cs:701,-2.20119991722107)
--(axis cs:801,-2.24460011138916)
--(axis cs:901,-2.35440009078979)
--(axis cs:1001,-2.40680003395081)
--(axis cs:1101,-2.45300009803772)
--(axis cs:1201,-2.48029992752075)
--(axis cs:1301,-2.50299999885559)
--(axis cs:1401,-2.55880003814697)
--(axis cs:1501,-2.56000010757446)
--(axis cs:1601,-2.57419996795654)
--(axis cs:1701,-2.60130009346008)
--(axis cs:1801,-2.63660007362366)
--(axis cs:1901,-2.65770010566711)
--(axis cs:1901,-2.59070010566711)
--(axis cs:1901,-2.59070010566711)
--(axis cs:1801,-2.56400007362366)
--(axis cs:1701,-2.52770009346008)
--(axis cs:1601,-2.50299996795654)
--(axis cs:1501,-2.48440010757446)
--(axis cs:1401,-2.48620003814697)
--(axis cs:1301,-2.41939999885559)
--(axis cs:1201,-2.39669992752075)
--(axis cs:1101,-2.36140009803772)
--(axis cs:1001,-2.31200003395081)
--(axis cs:901,-2.2602000907898)
--(axis cs:801,-2.14180011138916)
--(axis cs:701,-2.10239991722107)
--(axis cs:601,-1.91880005397797)
--(axis cs:501,-1.85060000457764)
--(axis cs:401,-1.56970002098083)
--(axis cs:301,-1.30989998817444)
--(axis cs:201,-0.875499994564056)
--(axis cs:101,-0.000200002610683439)
--(axis cs:1,1.57399995288849)
--cycle;
\path [fill=color2, fill opacity=0.1]
(axis cs:1,1.53610004787445)
--(axis cs:1,1.41850004787445)
--(axis cs:101,-0.423900004482269)
--(axis cs:201,-1.1468999578476)
--(axis cs:301,-1.59239999980927)
--(axis cs:401,-1.82359996299744)
--(axis cs:501,-2.05369994888306)
--(axis cs:601,-2.16679989814758)
--(axis cs:701,-2.33459992599487)
--(axis cs:801,-2.43039991836548)
--(axis cs:901,-2.47009997673035)
--(axis cs:1001,-2.56189996910095)
--(axis cs:1101,-2.57800011024475)
--(axis cs:1201,-2.66499992485046)
--(axis cs:1301,-2.66360004844666)
--(axis cs:1401,-2.7143001033783)
--(axis cs:1501,-2.74930004119873)
--(axis cs:1601,-2.72959992752075)
--(axis cs:1701,-2.73749999084473)
--(axis cs:1801,-2.78500010299683)
--(axis cs:1901,-2.80920006256104)
--(axis cs:1901,-2.72460006256104)
--(axis cs:1901,-2.72460006256104)
--(axis cs:1801,-2.69900010299683)
--(axis cs:1701,-2.64169999084473)
--(axis cs:1601,-2.64739992752075)
--(axis cs:1501,-2.66930004119873)
--(axis cs:1401,-2.6289001033783)
--(axis cs:1301,-2.57980004844666)
--(axis cs:1201,-2.57259992485046)
--(axis cs:1101,-2.48080011024475)
--(axis cs:1001,-2.46789996910095)
--(axis cs:901,-2.36869997673035)
--(axis cs:801,-2.32579991836548)
--(axis cs:701,-2.23059992599487)
--(axis cs:601,-2.05179989814758)
--(axis cs:501,-1.93849994888306)
--(axis cs:401,-1.70899996299744)
--(axis cs:301,-1.47799999980927)
--(axis cs:201,-1.0364999578476)
--(axis cs:101,-0.296700004482269)
--(axis cs:1,1.53610004787445)
--cycle;
\path [fill=color3, fill opacity=0.1]
(axis cs:1,1.4560000377655)
--(axis cs:1,1.3548000377655)
--(axis cs:101,-0.563699977302551)
--(axis cs:201,-1.24819998035431)
--(axis cs:301,-1.76560000190735)
--(axis cs:401,-2.03379995269775)
--(axis cs:501,-2.21080011329651)
--(axis cs:601,-2.3103000541687)
--(axis cs:701,-2.45200000495911)
--(axis cs:801,-2.53810003089905)
--(axis cs:901,-2.62220009307861)
--(axis cs:1001,-2.69090005340576)
--(axis cs:1101,-2.74170004196167)
--(axis cs:1201,-2.75059997215271)
--(axis cs:1301,-2.80069998168945)
--(axis cs:1401,-2.83270008277893)
--(axis cs:1501,-2.80979994506836)
--(axis cs:1601,-2.87620011062622)
--(axis cs:1701,-2.87940008049011)
--(axis cs:1801,-2.86849996337891)
--(axis cs:1901,-2.86959999351501)
--(axis cs:1901,-2.77399999351501)
--(axis cs:1901,-2.77399999351501)
--(axis cs:1801,-2.77329996337891)
--(axis cs:1701,-2.78680008049011)
--(axis cs:1601,-2.79180011062622)
--(axis cs:1501,-2.71539994506836)
--(axis cs:1401,-2.73870008277893)
--(axis cs:1301,-2.70769998168945)
--(axis cs:1201,-2.65279997215271)
--(axis cs:1101,-2.64530004196167)
--(axis cs:1001,-2.59710005340576)
--(axis cs:901,-2.52260009307861)
--(axis cs:801,-2.43210003089905)
--(axis cs:701,-2.34260000495911)
--(axis cs:601,-2.1961000541687)
--(axis cs:501,-2.09660011329651)
--(axis cs:401,-1.92539995269775)
--(axis cs:301,-1.65540000190735)
--(axis cs:201,-1.12799998035431)
--(axis cs:101,-0.438899977302551)
--(axis cs:1,1.4560000377655)
--cycle;
\path [fill=color4, fill opacity=0.1]
(axis cs:1,1.33120845)
--(axis cs:1,1.24612155)
--(axis cs:101,-0.92246298)
--(axis cs:201,-1.64867954)
--(axis cs:301,-2.0809544)
--(axis cs:401,-2.30874495)
--(axis cs:501,-2.49409526)
--(axis cs:601,-2.5663452)
--(axis cs:701,-2.70197438)
--(axis cs:801,-2.70035709)
--(axis cs:901,-2.79028498)
--(axis cs:1001,-2.86375295)
--(axis cs:1101,-2.8604834)
--(axis cs:1201,-2.87069433)
--(axis cs:1301,-2.92779786)
--(axis cs:1401,-2.95633085)
--(axis cs:1501,-2.93521017)
--(axis cs:1601,-2.98606405)
--(axis cs:1701,-2.99001125)
--(axis cs:1801,-2.98982687)
--(axis cs:1901,-3.03229092)
--(axis cs:1901,-2.93948748)
--(axis cs:1901,-2.93948748)
--(axis cs:1801,-2.88905113)
--(axis cs:1701,-2.89319175)
--(axis cs:1601,-2.88503075)
--(axis cs:1501,-2.83525043)
--(axis cs:1401,-2.85179375)
--(axis cs:1301,-2.83008094)
--(axis cs:1201,-2.77036507)
--(axis cs:1101,-2.758808)
--(axis cs:1001,-2.77149905)
--(axis cs:901,-2.68344742)
--(axis cs:801,-2.58958811)
--(axis cs:701,-2.59693362)
--(axis cs:601,-2.4529404)
--(axis cs:501,-2.37603154)
--(axis cs:401,-2.18558705)
--(axis cs:301,-1.954828)
--(axis cs:201,-1.52498786)
--(axis cs:101,-0.80469562)
--(axis cs:1,1.33120845)
--cycle;
\addplot [semithick, color0]
table {%
1 1.57529997825623
101 -0.143700003623962
201 -0.929199993610382
301 -1.30980002880096
401 -1.46689999103546
501 -1.65620005130768
601 -1.75539994239807
701 -1.855299949646
801 -1.95650005340576
901 -2.01640009880066
1001 -2.07690000534058
1101 -2.05579996109009
1201 -2.0731999874115
1301 -2.12590003013611
1401 -2.14490008354187
1501 -2.13689994812012
1601 -2.15249991416931
1701 -2.17459988594055
1801 -2.18319988250732
1901 -2.18309998512268
};
\addlegendentry{\priorfittednetwork{} (500K datasets)}
\addplot [semithick, color1]
table {%
1 1.50689995288849
101 -0.0807000026106834
201 -0.940699994564056
301 -1.36989998817444
401 -1.62800002098083
501 -1.90520000457764
601 -1.97590005397797
701 -2.15179991722107
801 -2.19320011138916
901 -2.30730009078979
1001 -2.35940003395081
1101 -2.40720009803772
1201 -2.43849992752075
1301 -2.46119999885559
1401 -2.52250003814697
1501 -2.52220010757446
1601 -2.53859996795654
1701 -2.56450009346008
1801 -2.60030007362366
1901 -2.62420010566711
};
\addlegendentry{\priorfittednetwork{} (1M datasets)}
\addplot [semithick, color2]
table {%
1 1.47730004787445
101 -0.360300004482269
201 -1.0916999578476
301 -1.53519999980927
401 -1.76629996299744
501 -1.99609994888306
601 -2.10929989814758
701 -2.28259992599487
801 -2.37809991836548
901 -2.41939997673035
1001 -2.51489996910095
1101 -2.52940011024475
1201 -2.61879992485046
1301 -2.62170004844666
1401 -2.6716001033783
1501 -2.70930004119873
1601 -2.68849992752075
1701 -2.68959999084473
1801 -2.74200010299683
1901 -2.76690006256104
};
\addlegendentry{\priorfittednetwork{} (2M datasets)}
\addplot [semithick, color3]
table {%
1 1.4054000377655
101 -0.501299977302551
201 -1.18809998035431
301 -1.71050000190735
401 -1.97959995269775
501 -2.15370011329651
601 -2.2532000541687
701 -2.39730000495911
801 -2.48510003089905
901 -2.57240009307861
1001 -2.64400005340576
1101 -2.69350004196167
1201 -2.70169997215271
1301 -2.75419998168945
1401 -2.78570008277893
1501 -2.76259994506836
1601 -2.83400011062622
1701 -2.83310008049011
1801 -2.82089996337891
1901 -2.82179999351501
};
\addlegendentry{\priorfittednetwork{} (4M datasets)}
\addplot [semithick, color4]
table {%
1 1.288665
101 -0.8635793
201 -1.5868337
301 -2.0178912
401 -2.247166
501 -2.4350634
601 -2.5096428
701 -2.649454
801 -2.6449726
901 -2.7368662
1001 -2.817626
1101 -2.8096457
1201 -2.8205297
1301 -2.8789394
1401 -2.9040623
1501 -2.8852303
1601 -2.9355474
1701 -2.9416015
1801 -2.939439
1901 -2.9858892
};
\addlegendentry{GP}
\end{axis}
\end{tikzpicture}
% This file was created by tikzplotlib v0.9.9.
\begin{tikzpicture}
\definecolor{color0}{rgb}{0.12156862745098,0.466666666666667,0.705882352941177}
\definecolor{color1}{rgb}{1,0.498039215686275,0.0549019607843137}
\definecolor{color2}{rgb}{0.172549019607843,0.627450980392157,0.172549019607843}
\definecolor{color3}{rgb}{0.83921568627451,0.152941176470588,0.156862745098039}
\definecolor{color4}{rgb}{0.580392156862745,0.403921568627451,0.741176470588235}
\definecolor{color5}{rgb}{0.549019607843137,0.337254901960784,0.294117647058824}
\definecolor{color6}{rgb}{0.890196078431372,0.466666666666667,0.76078431372549}
\begin{axis}[
legend cell align={left},
tick align=outside,
tick pos=left,
x grid style={white!69.0196078431373!black},
xlabel={Number of revealed data points (n)},
ylabel={},
xtick style={color=black},
y grid style={white!69.0196078431373!black},
ymin=-3.5, ymax=5,
ytick style={color=black},
]
\path [fill=color0, fill opacity=0.1]
(axis cs:5,-0.54249999704361)
--(axis cs:5,-0.79869999704361)
--(axis cs:10,-2.08589995040894)
--(axis cs:15,-2.69029995651245)
--(axis cs:20,-3.00829996185303)
--(axis cs:25,-3.20990006637573)
--(axis cs:30,-3.3432999874115)
--(axis cs:30,-3.1780999874115)
--(axis cs:30,-3.1780999874115)
--(axis cs:25,-3.04090006637573)
--(axis cs:20,-2.82169996185303)
--(axis cs:15,-2.48589995651245)
--(axis cs:10,-1.84309995040894)
--(axis cs:5,-0.54249999704361)
--cycle;
\path [fill=color1, fill opacity=0.1]
(axis cs:5,84.2237)
--(axis cs:5,33.1961)
--(axis cs:10,4.7666)
--(axis cs:15,-2.4755)
--(axis cs:20,-2.3216)
--(axis cs:25,-2.918)
--(axis cs:30,-2.5018)
--(axis cs:30,-1.0842)
--(axis cs:30,-1.0842)
--(axis cs:25,-0.1318)
--(axis cs:20,-1.8504)
--(axis cs:15,3.5405)
--(axis cs:10,27.253)
--(axis cs:5,84.2237)
--cycle;
\iffalse
\path [fill=color2, fill opacity=0.1]
(axis cs:5,1.5472)
--(axis cs:5,0.6062)
--(axis cs:10,-0.565)
--(axis cs:15,-2.7004)
--(axis cs:20,-1.6706)
--(axis cs:25,-1.8334)
--(axis cs:30,-2.0293)
--(axis cs:30,-1.6907)
--(axis cs:30,-1.6907)
--(axis cs:25,-0.973)
--(axis cs:20,3.8746)
--(axis cs:15,4.0772)
--(axis cs:10,2.4722)
--(axis cs:5,1.5472)
--cycle;
\path [fill=color3, fill opacity=0.1]
(axis cs:5,0.7434)
--(axis cs:5,0.4748)
--(axis cs:10,-1.0065)
--(axis cs:15,-1.7445)
--(axis cs:20,-2.0838)
--(axis cs:25,-2.4326)
--(axis cs:30,-2.5718)
--(axis cs:30,-1.906)
--(axis cs:30,-1.906)
--(axis cs:25,-1.3524)
--(axis cs:20,-1.6528)
--(axis cs:15,-1.2881)
--(axis cs:10,-0.6889)
--(axis cs:5,0.7434)
--cycle;
\fi
\path [fill=color4, fill opacity=0.1]
(axis cs:5,0.8434)
--(axis cs:5,0.5396)
--(axis cs:10,-0.9257)
--(axis cs:15,-1.9269)
--(axis cs:20,-2.1159)
--(axis cs:25,-2.6006)
--(axis cs:30,-2.9769)
--(axis cs:30,-1.7411)
--(axis cs:30,-1.7411)
--(axis cs:25,-2.3474)
--(axis cs:20,-1.3431)
--(axis cs:15,-0.2307)
--(axis cs:10,-0.4689)
--(axis cs:5,0.8434)
--cycle;
\iffalse
\path [fill=color5, fill opacity=0.1]
(axis cs:5,0.5576)
--(axis cs:5,0.3086)
--(axis cs:10,-1.1073)
--(axis cs:15,-1.9394)
--(axis cs:20,-2.401)
--(axis cs:25,-2.6625)
--(axis cs:30,-2.8433)
--(axis cs:30,-2.6255)
--(axis cs:30,-2.6255)
--(axis cs:25,-2.4293)
--(axis cs:20,-1.8026)
--(axis cs:15,-1.6668)
--(axis cs:10,-0.7715)
--(axis cs:5,0.5576)
--cycle;
\path [fill=color6, fill opacity=0.1]
(axis cs:5,0.62)
--(axis cs:5,0.358)
--(axis cs:10,-1.115)
--(axis cs:15,-2.0284)
--(axis cs:20,-2.3769)
--(axis cs:25,-2.7635)
--(axis cs:30,-2.95)
--(axis cs:30,-2.7546)
--(axis cs:30,-2.7546)
--(axis cs:25,-2.5461)
--(axis cs:20,-2.1365)
--(axis cs:15,-1.7688)
--(axis cs:10,-0.8212)
--(axis cs:5,0.62)
--cycle;
\fi
\path [fill=white!49.8039215686275!black, fill opacity=0.1]
(axis cs:5,0.6305)
--(axis cs:5,0.3861)
--(axis cs:10,-1.1322)
--(axis cs:15,-1.9057)
--(axis cs:20,-2.4349)
--(axis cs:25,-2.5839)
--(axis cs:30,-2.8762)
--(axis cs:30,-2.6672)
--(axis cs:30,-2.6672)
--(axis cs:25,-2.3391)
--(axis cs:20,-2.2041)
--(axis cs:15,-1.6387)
--(axis cs:10,-0.8558)
--(axis cs:5,0.6305)
--cycle;
\addplot [semithick, color1]
table {%
5 58.7099
10 16.0098
15 0.5325
20 -2.086
25 -1.5249
30 -1.793
};
\addlegendentry{MLE-II}
\iffalse
\addplot [semithick, color2]
table {%
5 1.0767
10 0.9536
15 0.6884
20 1.102
25 -1.4032
30 -1.86
};
\addlegendentry{NUTS w/ 8 (warmup) steps}
\addplot [semithick, color3]
table {%
5 0.6091
10 -0.8477
15 -1.5163
20 -1.8683
25 -1.8925
30 -2.2389
};
\addlegendentry{NUTS w/ 16 (warmup) steps}
\fi
\addplot [semithick, color4]
table {%
5 0.6915
10 -0.6973
15 -1.0788
20 -1.7295
25 -2.474
30 -2.359
};
\addlegendentry{NUTS w/ 32 (warmup) steps}
\iffalse
\addplot [semithick, color5]
table {%
5 0.4331
10 -0.9394
15 -1.8031
20 -2.1018
25 -2.5459
30 -2.7344
};
\addlegendentry{NUTS w/ 64 (warmup) steps}
\addplot [semithick, color6]
table {%
5 0.489
10 -0.9681
15 -1.8986
20 -2.2567
25 -2.6548
30 -2.8523
};
\addlegendentry{NUTS w/ 128 (warmup) steps}
\fi
\addplot [semithick, white!49.8039215686275!black]
table {%
5 0.5083
10 -0.994
15 -1.7722
20 -2.3195
25 -2.4615
30 -2.7717
};
\addlegendentry{NUTS w/ 256 (warmup) steps}
\addplot [semithick, color0]
table {%
5 -0.67059999704361
10 -1.96449995040894
15 -2.58809995651245
20 -2.91499996185303
25 -3.12540006637573
30 -3.2606999874115
};
\addlegendentry{\priorfittednetwork{} (4M datasets)}
\end{axis}
\end{tikzpicture}
% This file was created by tikzplotlib v0.9.9.
\begin{tikzpicture}
\definecolor{color0}{rgb}{0.12156862745098,0.466666666666667,0.705882352941177}
\definecolor{color1}{rgb}{1,0.498039215686275,0.0549019607843137}
\begin{axis}[
legend cell align={left},
legend style={
fill opacity=0.8,
draw opacity=1,
text opacity=1,
at={(0.03,0.97)},
anchor=north west,
draw=white!80!black,
},
tick align=outside,
tick pos=left,
x grid style={white!69.0196078431373!black},
xlabel={Dataset Size},
xmin=-215, xmax=6715,
xtick style={color=black},
y grid style={white!69.0196078431373!black},
ylabel={Time (s)},
ymin=-0.0354870457435027, ymax=0.82241697490681,
ytick style={color=black}
]
\addplot [semithick, color0]
table {%
100 0.00350859155878425
200 0.00393559923395514
400 0.00490951538085938
800 0.00864667911082506
1600 0.0256963483989239
3200 0.0979035645723343
6400 0.334685176610947
};
\addlegendentry{Transformer}
\addplot [semithick, color1]
table {%
100 0.00438466062769294
200 0.00424602022394538
400 0.00515887746587396
800 0.00612738123163581
1600 0.0480283498764038
3200 0.358485877513885
6400 0.783421337604523
};
\addlegendentry{GP}
\end{axis}
\end{tikzpicture}
% This file was created with tikzplotlib v0.9.12.
\begin{tikzpicture}
\definecolor{color2}{rgb}{0.12156862745098,0.466666666666667,0.705882352941177}
\definecolor{color0}{rgb}{1,0.498039215686275,0.0549019607843137}
\definecolor{color1}{rgb}{0.172549019607843,0.627450980392157,0.172549019607843}
\begin{axis}[
width=\textwidth,
height=.6\textwidth,
legend cell align={left},
legend style={fill opacity=0.8, draw opacity=1, text opacity=1, draw=white!80!black},
log basis x={10},
log basis y={10},
tick align=outside,
tick pos=left,
x grid style={white!69.0196078431373!black},
xlabel={Training time per task (s)},
ymin=0.175,
xmax=1600,
xmode=log,
xtick style={color=black},
y grid style={white!69.0196078431373!black},
ylabel={Negative Log-Likelihood},
ymax=15,
ymode=log,
ytick style={color=black}
]
\path [fill=color0, fill opacity=0.1]
(axis cs:0.0418464517593384,28.6230624102461)
--(axis cs:0.0418464517593384,25.3493954754961)
--(axis cs:0.0720097708702087,9.57554276554838)
--(axis cs:0.131692745685577,2.36477256723705)
--(axis cs:0.247699303627014,0.895739120962369)
--(axis cs:0.485526397228241,0.700326532898776)
--(axis cs:0.962719347476959,0.603478311881157)
--(axis cs:1.87292000293732,0.520048095198353)
--(axis cs:3.83550430297852,0.398479560589327)
--(axis cs:7.7248609495163,0.286872201027544)
--(axis cs:15.2928001856804,0.241997082941626)
--(axis cs:57.2935777544975,0.23660640582682)
--(axis cs:104.077784936428,0.232635993615833)
--(axis cs:108.094298963547,0.23454723918296)
--(axis cs:220.728208065033,0.229639362985707)
--(axis cs:401.523287701607,0.228987175914848)
--(axis cs:401.523287701607,0.275727372672951)
--(axis cs:401.523287701607,0.275727372672951)
--(axis cs:220.728208065033,0.276086318796062)
--(axis cs:108.094298963547,0.279817718626488)
--(axis cs:104.077784936428,0.278612654550824)
--(axis cs:57.2935777544975,0.281539811042534)
--(axis cs:15.2928001856804,0.284922699220087)
--(axis cs:7.7248609495163,0.326217837272017)
--(axis cs:3.83550430297852,0.437510153079496)
--(axis cs:1.87292000293732,0.567643092660228)
--(axis cs:0.962719347476959,0.668091416970161)
--(axis cs:0.485526397228241,0.782786994399198)
--(axis cs:0.247699303627014,1.19978662061478)
--(axis cs:0.131692745685577,3.26846574834522)
--(axis cs:0.0720097708702087,12.5917298022006)
--(axis cs:0.0418464517593384,28.6230624102461)
--cycle;
\path [fill=color1, fill opacity=0.1]
(axis cs:0.0883133125305176,20.7927346661883)
--(axis cs:0.0883133125305176,16.1772245928449)
--(axis cs:0.715808699131012,4.31497475329588)
--(axis cs:3.15951771736145,1.59803524051658)
--(axis cs:6.84223338842392,0.796135094346223)
--(axis cs:10.9540012979507,0.463260159658297)
--(axis cs:25.1598756217957,0.315318169053697)
--(axis cs:51.3041855764389,0.28769263616981)
--(axis cs:103.125261063576,0.239040406632114)
--(axis cs:207.198462553024,0.234454886689715)
--(axis cs:254.926835448742,0.234194416111427)
--(axis cs:442.418867218494,0.230273543079467)
--(axis cs:442.418867218494,0.27689361743155)
--(axis cs:442.418867218494,0.27689361743155)
--(axis cs:254.926835448742,0.282957356864495)
--(axis cs:207.198462553024,0.287605149969526)
--(axis cs:103.125261063576,0.303675917697262)
--(axis cs:51.3041855764389,0.389533788047415)
--(axis cs:25.1598756217957,0.438775239530898)
--(axis cs:10.9540012979507,0.657710089517728)
--(axis cs:6.84223338842392,1.11624738913614)
--(axis cs:3.15951771736145,2.17642316783913)
--(axis cs:0.715808699131012,6.10618689831545)
--(axis cs:0.0883133125305176,20.7927346661883)
--cycle;
\path [fill=color2, fill opacity=0.1, dash pattern=on 3.7pt off 1.6pt]
(axis cs:0.0107475638389587,0.274339572707999)
--(axis cs:0.0107475638389587,0.226971133430612)
--(axis cs:107475.638389587,0.226971133430612)
--(axis cs:107475.638389587,0.274339572707999)
--(axis cs:107475.638389587,0.274339572707999)
--(axis cs:0.0107475638389587,0.274339572707999)
--cycle;
\addplot [semithick, color0]
table {%
0.0418464517593384 26.9862289428711
0.0720097708702087 11.0836362838745
0.131692745685577 2.81661915779114
0.247699303627014 1.04776287078857
0.485526397228241 0.741556763648987
0.962719347476959 0.635784864425659
1.87292000293732 0.543845593929291
3.83550430297852 0.417994856834412
7.7248609495163 0.30654501914978
15.2928001856804 0.263459891080856
57.2935777544975 0.259073108434677
104.077784936428 0.255624324083328
108.094298963547 0.257182478904724
220.728208065033 0.252862840890884
401.523287701607 0.2523572742939
};
%\addlegendentry{SVI (Adam /w N train + sampling steps)}
\addplot [semithick, color1]
table {%
0.0883133125305176 18.4849796295166
0.715808699131012 5.21058082580566
3.15951771736145 1.88722920417786
6.84223338842392 0.95619124174118
10.9540012979507 0.560485124588013
25.1598756217957 0.377046704292297
51.3041855764389 0.338613212108612
103.125261063576 0.271358162164688
207.198462553024 0.26103001832962
254.926835448742 0.258575886487961
442.418867218494 0.253583580255508
};
%\addlegendentry{MCMC (NUTS /w N warmup + sampling steps)}
\addplot [semithick, color2, dashed, mark=*, mark size=3, mark options={solid}]
table {%
0.0107475638389587 0.250655353069305
107475.638389587 0.250655353069305
};
%\addlegendentry{Transformer (4M Prior Samples)}
\end{axis}
\end{tikzpicture}
% This file was created with tikzplotlib v0.9.12.
\begin{tikzpicture}
\definecolor{color2}{rgb}{0.12156862745098,0.466666666666667,0.705882352941177}
\definecolor{color0}{rgb}{1,0.498039215686275,0.0549019607843137}
\definecolor{color1}{rgb}{0.172549019607843,0.627450980392157,0.172549019607843}
\begin{axis}[
width=\textwidth,
height=.6\textwidth,
legend cell align={left},
legend style={fill opacity=0.8, draw opacity=1, text opacity=1, draw=white!80!black},
log basis x={10},
log basis y={10},
tick align=outside,
tick pos=left,
x grid style={white!69.0196078431373!black},
xlabel={Training time per task (s)},
xmax=6000,
xmode=log,
xtick style={color=black},
y grid style={white!69.0196078431373!black},
ylabel={},
ymax=15,
ymin=0.07,
ymode=log,
ytick style={color=black}
]
\path [fill=color0, fill opacity=0.1]
(axis cs:0.048541316986084,39.3192959880337)
--(axis cs:0.048541316986084,35.0526217365756)
--(axis cs:0.0798813033103943,20.5155863747431)
--(axis cs:0.148253695964813,12.6495487202318)
--(axis cs:0.276684482097626,5.04581297752074)
--(axis cs:1.01501720905304,1.27658528344386)
--(axis cs:2.23568467617035,0.321478489183742)
--(axis cs:4.50809059143066,0.220259158880709)
--(axis cs:9.05898730278015,0.17067608142147)
--(axis cs:17.7422209644318,0.140818446859858)
--(axis cs:33.9822976613045,0.136259373049486)
--(axis cs:56.7953882718086,0.155521874948685)
--(axis cs:99.6093530392647,0.140373215826301)
--(axis cs:177.1265239048,0.148466273441112)
--(axis cs:360.465931067467,0.121898997415161)
--(axis cs:1600.08951223135,0.11947893352626)
--(axis cs:1600.08951223135,0.128935314701813)
--(axis cs:1600.08951223135,0.128935314701813)
--(axis cs:360.465931067467,0.13095481337204)
--(axis cs:177.1265239048,0.156839773760045)
--(axis cs:99.6093530392647,0.149126245824547)
--(axis cs:56.7953882718086,0.164087588266189)
--(axis cs:33.9822976613045,0.144711766619932)
--(axis cs:17.7422209644318,0.149753749385336)
--(axis cs:9.05898730278015,0.18161085343113)
--(axis cs:4.50809059143066,0.235575229852201)
--(axis cs:2.23568467617035,0.44302206919257)
--(axis cs:1.01501720905304,2.13870650274999)
--(axis cs:0.276684482097626,7.1265989411194)
--(axis cs:0.148253695964813,16.2224929820387)
--(axis cs:0.0798813033103943,24.290264607923)
--(axis cs:0.048541316986084,39.3192959880337)
--cycle;
\path [fill=color1, fill opacity=0.1]
(axis cs:0.606662583351135,27.2806178422731)
--(axis cs:0.606662583351135,21.7565945295531)
--(axis cs:5.22408721923828,2.16055210098613)
--(axis cs:43.905959854126,0.79514638265564)
--(axis cs:67.348067741394,0.556576663043823)
--(axis cs:83.5251543235779,0.283285046997827)
--(axis cs:124.119333703518,0.109526378292852)
--(axis cs:252.408518188,0.101282463541341)
--(axis cs:339.109047954082,0.0894096422006328)
--(axis cs:656.239313220978,0.0876084541932425)
--(axis cs:1602.78304531813,0.0890523056510918)
--(axis cs:1602.78304531813,0.113934366869546)
--(axis cs:1602.78304531813,0.113934366869546)
--(axis cs:656.239313220978,0.115068125162951)
--(axis cs:339.109047954082,0.12323450269206)
--(axis cs:252.408518188,0.158341153392482)
--(axis cs:124.119333703518,0.210706728320307)
--(axis cs:83.5251543235779,0.447266255435187)
--(axis cs:67.348067741394,0.700851983043823)
--(axis cs:43.905959854126,1.06067284265564)
--(axis cs:5.22408721923828,2.99760095611226)
--(axis cs:0.606662583351135,27.2806178422731)
--cycle;
\path [fill=color2, fill opacity=0.1, dash pattern=on 3.7pt off 1.6pt]
(axis cs:0.0104961490631104,0.113167430664947)
--(axis cs:0.0104961490631104,0.100950632784912)
--(axis cs:104961.490631104,0.100950632784912)
--(axis cs:104961.490631104,0.113167430664947)
--(axis cs:104961.490631104,0.113167430664947)
--(axis cs:0.0104961490631104,0.113167430664947)
--cycle;
\addplot [semithick, color0]
table {%
0.048541316986084 37.1859588623047
0.0798813033103943 22.402925491333
0.148253695964813 14.4360208511353
0.276684482097626 6.08620595932007
1.01501720905304 1.70764589309692
2.23568467617035 0.382250279188156
4.50809059143066 0.227917194366455
9.05898730278015 0.1761434674263
17.7422209644318 0.145286098122597
33.9822976613045 0.140485569834709
56.7953882718086 0.159804731607437
99.6093530392647 0.144749730825424
177.1265239048 0.152653023600578
360.465931067467 0.1264269053936
1600.08951223135 0.124207124114037
};
\addplot [semithick, color1]
table {%
0.606662583351135 24.5186061859131
5.22408721923828 2.57907652854919
43.905959854126 0.92790961265564
67.348067741394 0.628714323043823
83.5251543235779 0.365275651216507
124.119333703518 0.16011655330658
252.408518188 0.129811808466911
339.109047954082 0.106322072446346
656.239313220978 0.101338289678097
1602.78304531813 0.101493336260319
};
\addplot [semithick, color2, dashed, mark=*, mark size=3, mark options={solid}]
table {%
0.0104961490631104 0.10705903172493
104961.490631104 0.10705903172493
};
\end{axis}
\end{tikzpicture}
% This file was created with tikzplotlib v0.9.12.
\begin{tikzpicture}
\definecolor{color0}{rgb}{0.12156862745098,0.466666666666667,0.705882352941177}
\definecolor{color1}{rgb}{1,0.498039215686275,0.0549019607843137}
\definecolor{color2}{rgb}{0.172549019607843,0.627450980392157,0.172549019607843}
\begin{axis}[
hide axis,
width=1.\textwidth,
height=.2\textwidth,
legend columns=-1,legend style={draw=none,column sep=5ex},
legend cell align={left},
legend style={fill opacity=0.8, draw opacity=1, text opacity=1, draw=white!80!black},
log basis x={10},
tick align=outside,
tick pos=left,
x grid style={white!69.0196078431373!black},
xlabel={Training time per task},
xmin=0.005, xmax=300,
xmode=log,
xtick style={color=black},
y grid style={white!69.0196078431373!black},
ylabel={Negative Log Likelihood},
ymin=-1, ymax=15,
ytick style={color=black}
]
\addlegendimage{semithick, color2}
\addlegendentry{NUTS}
\addlegendimage{semithick, color1}
\addlegendentry{SVI}
\addlegendimage{semithick, color0, dashed, mark=*, mark size=3, mark options={solid}}
\addlegendentry{\priorfittednetwork{} (4M Datasets)}
\end{axis}
\end{tikzpicture}
\begin{tikzpicture}[-Latex,auto,node distance =0.65 cm and 0.5 cm,semithick,
state/.style ={ellipse, draw, minimum width = 1.0 cm, minimum height = 0.7 cm},every node/.style={scale=0.52},
point/.style = {circle, draw, inner sep=0.04cm,fill,node contents={}},
bidirected/.style={Latex-Latex,dashed},
el/.style = {inner sep=2pt, align=left, sloped}]
\node[state,fill=gray!40!white] (1) {$x^{(1)}$};
\node[state,fill=gray!40!white] (2) [right =of 1] {$x^{(2)}$};
\node[state] (3) [above left =of 1,yshift=0.2cm] {$H_1$};
\node[state] (4) [above =of 1] {$H_2$};
\node[state] (5) [above =of 2] {$H_3$};
\node[state] (6) [above right =of 2,yshift=0.2cm] {$H_4$};
\node[state] (7) [above =of 4,xshift=0.8cm,fill=gray!40!white] {$y$};
\path (1) edge (3);
\path (1) edge (4);
\path (1) edge (5);
\path (1) edge(6);
\path (2) edge (3);
\path (2) edge (4);
\path (2) edge (5);
\path (2) edge (6);
\path (3) edge (7);
\path (4) edge (7);
\path (5) edge (7);
\path (6) edge (7);
\end{tikzpicture}
% \documentclass{article}
% \usepackage{pgfplots}
% \usepackage{tikz}
% \usetikzlibrary{shapes,decorations,arrows,calc,arrows.meta,fit,positioning}
% \begin{document}
\begin{tikzpicture}[-Latex,auto,node distance =0.2 cm and 0.3 cm,semithick,
state/.style ={ellipse, draw, minimum width = .7 cm, minimum height = 0.7 cm},every node/.style={scale=0.3},
point/.style = {circle, draw, inner sep=0.04cm,fill,node contents={}},
bidirected/.style={Latex-Latex,dashed},
el/.style = {inner sep=2pt, align=left, sloped}]
\node[state,fill=gray!40!white] (1) {$x^{(1)}$};
\node[state,fill=gray!40!white] (2) [right =of 1] {$x^{(2)}$};
\node[state] (3) [above left =of 1,yshift=0.2cm, xshift=1cm] {$H_1$};
\node[state] (4) [above =of 1, xshift=1cm] {$H_2$};
\node[state] (5) [above =of 2, xshift=1cm] {$H_3$};
\node[state] (8) [above =of 3] {$H_1$};
\node[state] (9) [above =of 4] {$H_2$};
\node[state] (10) [above =of 5] {$H_3$};
\node[state] (7) [above =of 9,fill=gray!40!white] {$y$};
\path (1) edge (3);
\path (1) edge (4);
\path (1) edge (5);
\path (2) edge (3);
\path (2) edge (4);
\path (2) edge (5);
\path (3) edge (8);
\path (4) edge (8);
\path (5) edge (8);
\path (3) edge (9);
\path (4) edge (9);
\path (5) edge (9);
\path (3) edge (10);
\path (4) edge (10);
\path (5) edge (10);
\path (8) edge (7);
\path (9) edge (7);
\path (10) edge (7);
\end{tikzpicture}
\begin{tikzpicture}[-Latex,auto,node distance =0.3 cm and 0.3 cm,semithick,
state/.style ={ellipse, draw, minimum width = .7 cm, minimum height = 0.7 cm},every node/.style={scale=0.3},
point/.style = {circle, draw, inner sep=0.04cm,fill,node contents={}},
bidirected/.style={Latex-Latex,dashed},
el/.style = {inner sep=2pt, align=left, sloped}]
\node[state,fill=gray!40!white] (1) {$x^{(1)}$};
\node[state,fill=gray!40!white] (2) [right =of 1] {$x^{(2)}$};
\node[state] (3) [above left =of 1,yshift=0.2cm] {$H_1$};
\node[state] (4) [above =of 1] {$H_2$};
\node[state] (5) [above =of 2] {$H_3$};
\node[state] (6) [above right =of 2,yshift=0.2cm] {$H_4$};
\node[state] (7) [above =of 4,xshift=0.8cm,fill=gray!40!white] {$y$};
\path (1) edge (3);
\path (1) edge (4);
\path (1) edge (5);
\path (1) edge(6);
\path (2) edge (3);
\path (2) edge (4);
\path (2) edge (5);
\path (2) edge (6);
\path (3) edge (7);
\path (4) edge (7);
\path (5) edge (7);
\path (6) edge (7);
\end{tikzpicture}
\begin{tikzpicture}[-Latex,auto,node distance =0.3 cm and 0.15 cm,semithick,
state/.style ={ellipse, draw, minimum width = .7 cm, minimum height = 0.7 cm},every node/.style={scale=0.3},
point/.style = {circle, draw, inner sep=0.04cm,fill,node contents={}},
bidirected/.style={Latex-Latex,dashed},
el/.style = {inner sep=2pt, align=left, sloped}]
\node[state,fill=gray!40!white] (1) {$x^{(1)}$};
\node[state,fill=gray!40!white] (2) [right =of 1] {$x^{(2)}$};
\node[state] (3) [above left =of 1,yshift=0.2cm] {$H_1$};
\node[state] (4) [above =of 1] {$H_2$};
\node[state] (5) [above =of 2] {$H_3$};
\node[state] (8) [left =of 3] {$H_2$};
\node[state] (6) [above right =of 2,yshift=0.2cm] {$H_4$};
\node[state] (9) [right =of 6] {$H_3$};
\node[state] (7) [above =of 4,xshift=0.8cm,fill=gray!40!white] {$y$};
\path (1) edge (3);
\path (1) edge (4);
\path (1) edge (5);
\path (1) edge(6);
\path (1) edge (8);
\path (1) edge (9);
\path (2) edge (3);
\path (2) edge (4);
\path (2) edge (5);
\path (2) edge (6);
\path (2) edge (8);
\path (2) edge (9);
\path (3) edge (7);
\path (4) edge (7);
\path (5) edge (7);
\path (6) edge (7);
\path (8) edge (7);
\path (9) edge (7);
\end{tikzpicture}
% \end{document}
\footnotesize
\setlength{\tabcolsep}{0.2em}
\begin{tabular}{lcc|cccccc}
\toprule
Metric & PFN-BNN & PFN-GP & Log. Reg. & GP & KNN & BNN & Catboost & XGB \\
\midrule
Mean rank ROC AUC & \textbf{2.786} & 3.833 & 4.690 & 5.286 & 6.214 & 5.000 & 4.833 & 3.357 \\
Loss/Tie/Win vs PFN-BNN & -- & 14/1/6 & 16/1/4 & 17/0/4 & 17/1/3 & 17/1/3 & 13/1/7 & 12/2/7 \\
Wilcoxon p. vs PFN-BNN & -- & 3.6e-13 & 3.6e-13 & 3.9e-13 & 3.6e-13 & 3.6e-13 & 1.9e-12 & 1.6e-06 \\
Expected Calibration Error & \textbf{0.025} & 0.067 & 0.157 & 0.095 & 0.093 & 0.089 & 0.157 & 0.066 \\
\multirow{2}{*}{Benchmark Time} & \multicolumn{2}{c|}{GPU: \textbf{0:00:13}} & \multirow{2}{*}{0:09:56} & \multirow{2}{*}{0:24:30} & \multirow{2}{*}{0:00:34} & \multirow{2}{*}{12:04:41} & \multirow{2}{*}{2:05:20} & \multirow{2}{*}{20:59:46} \\
& \multicolumn{2}{c|}{CPU: 0:04:23}&&&&&&\\
\bottomrule
\end{tabular}
% This file was created by tikzplotlib v0.9.9.
\begin{tikzpicture}
\begin{axis}[
ylabel=y,
tick align=outside,
tick pos=left,
x grid style={white!69.0196078431373!black},
xmin=-0.05, xmax=1.05,
xtick style={color=black},
y grid style={white!69.0196078431373!black},
ymin=-2.36598739624023, ymax=2.75804862976074,
ytick style={color=black}
]
\path [draw=blue, fill=blue, opacity=0.2]
(axis cs:0,0.48836225271225)
--(axis cs:0,0.415137112140656)
--(axis cs:0.0101010100916028,0.234063774347305)
--(axis cs:0.0202020201832056,0.00553166028112173)
--(axis cs:0.030303031206131,-0.219634175300598)
--(axis cs:0.0404040403664112,-0.441207051277161)
--(axis cs:0.0505050495266914,-0.660769522190094)
--(axis cs:0.060606062412262,-0.866878986358643)
--(axis cs:0.0707070678472519,-1.05090737342834)
--(axis cs:0.0808080807328224,-1.24810099601746)
--(axis cs:0.0909090936183929,-1.44007933139801)
--(axis cs:0.101010099053383,-1.60583162307739)
--(axis cs:0.111111111938953,-1.73368620872498)
--(axis cs:0.121212124824524,-1.83448457717896)
--(axis cs:0.131313130259514,-1.9359358549118)
--(axis cs:0.141414135694504,-2.02791571617126)
--(axis cs:0.151515156030655,-2.08280920982361)
--(axis cs:0.161616161465645,-2.10896468162537)
--(axis cs:0.171717166900635,-2.11702990531921)
--(axis cs:0.181818187236786,-2.10278820991516)
--(axis cs:0.191919192671776,-2.0577974319458)
--(axis cs:0.202020198106766,-1.97708141803741)
--(axis cs:0.212121218442917,-1.86951541900635)
--(axis cs:0.222222223877907,-1.7485511302948)
--(axis cs:0.232323229312897,-1.60003483295441)
--(axis cs:0.242424249649048,-1.4387458562851)
--(axis cs:0.252525269985199,-1.28506243228912)
--(axis cs:0.262626260519028,-1.40837919712067)
--(axis cs:0.272727280855179,-1.56119740009308)
--(axis cs:0.28282830119133,-1.68955564498901)
--(axis cs:0.292929291725159,-1.78406238555908)
--(axis cs:0.30303031206131,-1.86246454715729)
--(axis cs:0.313131332397461,-1.91156911849976)
--(axis cs:0.32323232293129,-1.91824328899384)
--(axis cs:0.333333343267441,-1.87828612327576)
--(axis cs:0.34343433380127,-1.81035017967224)
--(axis cs:0.353535354137421,-1.72607338428497)
--(axis cs:0.363636374473572,-1.60809350013733)
--(axis cs:0.373737365007401,-1.45037758350372)
--(axis cs:0.383838385343552,-1.24851429462433)
--(axis cs:0.39393937587738,-0.998290956020355)
--(axis cs:0.404040396213531,-0.750182688236237)
--(axis cs:0.414141416549683,-0.525704860687256)
--(axis cs:0.424242407083511,-0.357743710279465)
--(axis cs:0.434343427419662,-0.135300785303116)
--(axis cs:0.444444447755814,0.126165747642517)
--(axis cs:0.454545438289642,0.403996556997299)
--(axis cs:0.464646458625793,0.657515227794647)
--(axis cs:0.474747478961945,0.89471846818924)
--(axis cs:0.484848499298096,1.12940919399261)
--(axis cs:0.494949519634247,1.33079588413239)
--(axis cs:0.505050539970398,1.50211012363434)
--(axis cs:0.515151500701904,1.35694479942322)
--(axis cs:0.525252521038055,1.16336917877197)
--(axis cs:0.535353541374207,0.950490415096283)
--(axis cs:0.545454561710358,0.723466515541077)
--(axis cs:0.555555582046509,0.504927337169647)
--(axis cs:0.565656542778015,0.281702965497971)
--(axis cs:0.575757563114166,0.0685240551829338)
--(axis cs:0.585858583450317,-0.13225944340229)
--(axis cs:0.595959544181824,-0.312537550926208)
--(axis cs:0.606060564517975,-0.467220991849899)
--(axis cs:0.616161584854126,-0.595295667648315)
--(axis cs:0.626262605190277,-0.699680984020233)
--(axis cs:0.636363625526428,-0.767002463340759)
--(axis cs:0.646464645862579,-0.776046991348267)
--(axis cs:0.65656566619873,-0.774105429649353)
--(axis cs:0.666666686534882,-0.733565211296082)
--(axis cs:0.676767706871033,-0.691394984722137)
--(axis cs:0.686868667602539,-0.637468218803406)
--(axis cs:0.69696968793869,-0.539371430873871)
--(axis cs:0.707070708274841,-0.411705821752548)
--(axis cs:0.717171728610992,-0.268257588148117)
--(axis cs:0.727272748947144,-0.116832487285137)
--(axis cs:0.737373769283295,0.031189676374197)
--(axis cs:0.747474789619446,0.188553214073181)
--(axis cs:0.757575750350952,0.312197506427765)
--(axis cs:0.767676770687103,0.13393634557724)
--(axis cs:0.777777791023254,-0.0825264304876328)
--(axis cs:0.787878811359406,-0.301064342260361)
--(axis cs:0.797979831695557,-0.516804218292236)
--(axis cs:0.808080792427063,-0.720233798027039)
--(axis cs:0.818181812763214,-0.904540479183197)
--(axis cs:0.828282833099365,-1.06612980365753)
--(axis cs:0.838383793830872,-1.22843110561371)
--(axis cs:0.848484814167023,-1.38574481010437)
--(axis cs:0.858585834503174,-1.50933611392975)
--(axis cs:0.868686854839325,-1.60499787330627)
--(axis cs:0.878787875175476,-1.68155801296234)
--(axis cs:0.888888895511627,-1.72222781181335)
--(axis cs:0.898989915847778,-1.72970712184906)
--(axis cs:0.909090936183929,-1.73046946525574)
--(axis cs:0.919191956520081,-1.71048104763031)
--(axis cs:0.929292917251587,-1.6622371673584)
--(axis cs:0.939393937587738,-1.58365535736084)
--(axis cs:0.949494957923889,-1.47647070884705)
--(axis cs:0.95959597826004,-1.35282254219055)
--(axis cs:0.969696998596191,-1.21870219707489)
--(axis cs:0.979797959327698,-1.07435023784637)
--(axis cs:0.989898979663849,-0.900119543075562)
--(axis cs:1,-0.746024668216705)
--(axis cs:1,-0.673951804637909)
--(axis cs:1,-0.673951804637909)
--(axis cs:0.989898979663849,-0.507326245307922)
--(axis cs:0.979797959327698,-0.306597948074341)
--(axis cs:0.969696998596191,-0.114666670560837)
--(axis cs:0.95959597826004,0.0890414640307426)
--(axis cs:0.949494957923889,0.290912091732025)
--(axis cs:0.939393937587738,0.482827752828598)
--(axis cs:0.929292917251587,0.65446412563324)
--(axis cs:0.919191956520081,0.804228365421295)
--(axis cs:0.909090936183929,0.933906435966492)
--(axis cs:0.898989915847778,1.05554437637329)
--(axis cs:0.888888895511627,1.16974520683289)
--(axis cs:0.878787875175476,1.24376940727234)
--(axis cs:0.868686854839325,1.28814744949341)
--(axis cs:0.858585834503174,1.31901741027832)
--(axis cs:0.848484814167023,1.32284557819366)
--(axis cs:0.838383793830872,1.29121851921082)
--(axis cs:0.828282833099365,1.24036478996277)
--(axis cs:0.818181812763214,1.16481220722198)
--(axis cs:0.808080792427063,1.06135177612305)
--(axis cs:0.797979831695557,0.940897345542908)
--(axis cs:0.787878811359406,0.803660213947296)
--(axis cs:0.777777791023254,0.660230994224548)
--(axis cs:0.767676770687103,0.507468283176422)
--(axis cs:0.757575750350952,0.382923036813736)
--(axis cs:0.747474789619446,0.568678319454193)
--(axis cs:0.737373769283295,0.784480810165405)
--(axis cs:0.727272748947144,1.00219941139221)
--(axis cs:0.717171728610992,1.1989334821701)
--(axis cs:0.707070708274841,1.39756238460541)
--(axis cs:0.69696968793869,1.57316732406616)
--(axis cs:0.686868667602539,1.73068594932556)
--(axis cs:0.676767706871033,1.90267431735992)
--(axis cs:0.666666686534882,2.0696439743042)
--(axis cs:0.65656566619873,2.21041941642761)
--(axis cs:0.646464645862579,2.31671452522278)
--(axis cs:0.636363625526428,2.39649868011475)
--(axis cs:0.626262605190277,2.45100498199463)
--(axis cs:0.616161584854126,2.48830580711365)
--(axis cs:0.606060564517975,2.50864386558533)
--(axis cs:0.595959544181824,2.50856685638428)
--(axis cs:0.585858583450317,2.48201560974121)
--(axis cs:0.575757563114166,2.43362832069397)
--(axis cs:0.565656542778015,2.37192559242249)
--(axis cs:0.555555582046509,2.29341602325439)
--(axis cs:0.545454561710358,2.18715262413025)
--(axis cs:0.535353541374207,2.06191420555115)
--(axis cs:0.525252521038055,1.91972386837006)
--(axis cs:0.515151500701904,1.74595034122467)
--(axis cs:0.505050539970398,1.59932494163513)
--(axis cs:0.494949519634247,1.72287714481354)
--(axis cs:0.484848499298096,1.86946165561676)
--(axis cs:0.474747478961945,1.98051631450653)
--(axis cs:0.464646458625793,2.07214260101318)
--(axis cs:0.454545438289642,2.14373254776001)
--(axis cs:0.444444447755814,2.18675327301025)
--(axis cs:0.434343427419662,2.20286750793457)
--(axis cs:0.424242407083511,2.18358945846558)
--(axis cs:0.414141416549683,2.12909626960754)
--(axis cs:0.404040396213531,2.04254627227783)
--(axis cs:0.39393937587738,1.90732991695404)
--(axis cs:0.383838385343552,1.74706768989563)
--(axis cs:0.373737365007401,1.58211135864258)
--(axis cs:0.363636374473572,1.39323019981384)
--(axis cs:0.353535354137421,1.17267620563507)
--(axis cs:0.34343433380127,0.920457720756531)
--(axis cs:0.333333343267441,0.668398439884186)
--(axis cs:0.32323232293129,0.420982718467712)
--(axis cs:0.313131332397461,0.177429169416428)
--(axis cs:0.30303031206131,-0.0690865516662598)
--(axis cs:0.292929291725159,-0.318397551774979)
--(axis cs:0.28282830119133,-0.552216172218323)
--(axis cs:0.272727280855179,-0.79360032081604)
--(axis cs:0.262626260519028,-1.02164494991302)
--(axis cs:0.252525269985199,-1.19548094272614)
--(axis cs:0.242424249649048,-1.0438551902771)
--(axis cs:0.232323229312897,-0.839464962482452)
--(axis cs:0.222222223877907,-0.621898174285889)
--(axis cs:0.212121218442917,-0.406501770019531)
--(axis cs:0.202020198106766,-0.190933123230934)
--(axis cs:0.191919192671776,0.0385102108120918)
--(axis cs:0.181818187236786,0.268366992473602)
--(axis cs:0.171717166900635,0.486332714557648)
--(axis cs:0.161616161465645,0.66456401348114)
--(axis cs:0.151515156030655,0.790132761001587)
--(axis cs:0.141414135694504,0.922284424304962)
--(axis cs:0.131313130259514,1.05679023265839)
--(axis cs:0.121212124824524,1.18532228469849)
--(axis cs:0.111111111938953,1.2858852148056)
--(axis cs:0.101010099053383,1.33695435523987)
--(axis cs:0.0909090936183929,1.34189283847809)
--(axis cs:0.0808080807328224,1.3314266204834)
--(axis cs:0.0707070678472519,1.31959629058838)
--(axis cs:0.060606062412262,1.27801203727722)
--(axis cs:0.0505050495266914,1.18698132038116)
--(axis cs:0.0404040403664112,1.06947243213654)
--(axis cs:0.030303031206131,0.937995493412018)
--(axis cs:0.0202020201832056,0.784277558326721)
--(axis cs:0.0101010100916028,0.623385548591614)
--(axis cs:0,0.48836225271225)
--cycle;
\path [draw=green!50.1960784313725!black, fill=green!50.1960784313725!black, opacity=0.2]
(axis cs:0,0.42320853471756)
--(axis cs:0,0.478630483150482)
--(axis cs:0.0101010100916028,0.63086324930191)
--(axis cs:0.0202020201832056,0.79552835226059)
--(axis cs:0.030303031206131,0.94506299495697)
--(axis cs:0.0404040403664112,1.07579600811005)
--(axis cs:0.0505050495266914,1.18488883972168)
--(axis cs:0.060606062412262,1.26998770236969)
--(axis cs:0.0707070678472519,1.32928156852722)
--(axis cs:0.0808080807328224,1.36147749423981)
--(axis cs:0.0909090936183929,1.36582171916962)
--(axis cs:0.101010099053383,1.34206390380859)
--(axis cs:0.111111111938953,1.29045009613037)
--(axis cs:0.121212124824524,1.21168637275696)
--(axis cs:0.131313130259514,1.10691666603088)
--(axis cs:0.141414135694504,0.977699398994446)
--(axis cs:0.151515156030655,0.825999021530151)
--(axis cs:0.161616161465645,0.654172658920288)
--(axis cs:0.171717166900635,0.464965999126434)
--(axis cs:0.181818187236786,0.261504948139191)
--(axis cs:0.191919192671776,0.0472662448883057)
--(axis cs:0.202020198106766,-0.173944175243378)
--(axis cs:0.212121218442917,-0.398045301437378)
--(axis cs:0.222222223877907,-0.620745956897736)
--(axis cs:0.232323229312897,-0.837598621845245)
--(axis cs:0.242424249649048,-1.0437605381012)
--(axis cs:0.252525269985199,-1.21087288856506)
--(axis cs:0.262626260519028,-1.02080011367798)
--(axis cs:0.272727280855179,-0.790893852710724)
--(axis cs:0.28282830119133,-0.548761963844299)
--(axis cs:0.292929291725159,-0.298509538173676)
--(axis cs:0.30303031206131,-0.0439335107803345)
--(axis cs:0.313131332397461,0.211251735687256)
--(axis cs:0.32323232293129,0.463441848754883)
--(axis cs:0.333333343267441,0.709164977073669)
--(axis cs:0.34343433380127,0.945122003555298)
--(axis cs:0.353535354137421,1.16818904876709)
--(axis cs:0.363636374473572,1.37542080879211)
--(axis cs:0.373737365007401,1.56404685974121)
--(axis cs:0.383838385343552,1.73147392272949)
--(axis cs:0.39393937587738,1.87529420852661)
--(axis cs:0.404040396213531,1.99331879615784)
--(axis cs:0.414141416549683,2.08361768722534)
--(axis cs:0.424242407083511,2.14458775520325)
--(axis cs:0.434343427419662,2.17502784729004)
--(axis cs:0.444444447755814,2.1742205619812)
--(axis cs:0.454545438289642,2.142014503479)
--(axis cs:0.464646458625793,2.07889604568481)
--(axis cs:0.474747478961945,1.98604822158813)
--(axis cs:0.484848499298096,1.86541771888733)
--(axis cs:0.494949519634247,1.72002959251404)
--(axis cs:0.505050539970398,1.57721316814423)
--(axis cs:0.515151500701904,1.75488746166229)
--(axis cs:0.525252521038055,1.93632102012634)
--(axis cs:0.535353541374207,2.09533667564392)
--(axis cs:0.545454561710358,2.23000621795654)
--(axis cs:0.555555582046509,2.33938312530518)
--(axis cs:0.565656542778015,2.42315769195557)
--(axis cs:0.575757563114166,2.48155117034912)
--(axis cs:0.585858583450317,2.51521921157837)
--(axis cs:0.595959544181824,2.52513790130615)
--(axis cs:0.606060564517975,2.51249122619629)
--(axis cs:0.616161584854126,2.47855472564697)
--(axis cs:0.626262605190277,2.42459559440613)
--(axis cs:0.636363625526428,2.35179829597473)
--(axis cs:0.646464645862579,2.26121759414673)
--(axis cs:0.65656566619873,2.15377068519592)
--(axis cs:0.666666686534882,2.0302562713623)
--(axis cs:0.676767706871033,1.8914076089859)
--(axis cs:0.686868667602539,1.73795938491821)
--(axis cs:0.69696968793869,1.5707323551178)
--(axis cs:0.707070708274841,1.3907128572464)
--(axis cs:0.717171728610992,1.19912457466125)
--(axis cs:0.727272748947144,0.997484028339386)
--(axis cs:0.737373769283295,0.787675678730011)
--(axis cs:0.747474789619446,0.572265446186066)
--(axis cs:0.757575750350952,0.377738237380981)
--(axis cs:0.767676770687103,0.519419372081757)
--(axis cs:0.777777791023254,0.679764151573181)
--(axis cs:0.787878811359406,0.830590486526489)
--(axis cs:0.797979831695557,0.967712759971619)
--(axis cs:0.808080792427063,1.08774089813232)
--(axis cs:0.818181812763214,1.18783736228943)
--(axis cs:0.828282833099365,1.26569366455078)
--(axis cs:0.838383793830872,1.31954658031464)
--(axis cs:0.848484814167023,1.3481719493866)
--(axis cs:0.858585834503174,1.35086405277252)
--(axis cs:0.868686854839325,1.32741022109985)
--(axis cs:0.878787875175476,1.27805984020233)
--(axis cs:0.888888895511627,1.20350527763367)
--(axis cs:0.898989915847778,1.10485076904297)
--(axis cs:0.909090936183929,0.983589291572571)
--(axis cs:0.919191956520081,0.841610193252563)
--(axis cs:0.929292917251587,0.681154727935791)
--(axis cs:0.939393937587738,0.504822611808777)
--(axis cs:0.949494957923889,0.315531313419342)
--(axis cs:0.95959597826004,0.116456389427185)
--(axis cs:0.969696998596191,-0.0889670848846436)
--(axis cs:0.979797959327698,-0.297123461961746)
--(axis cs:0.989898979663849,-0.503996789455414)
--(axis cs:1,-0.681941628456116)
--(axis cs:1,-0.737388372421265)
--(axis cs:1,-0.737388372421265)
--(axis cs:0.989898979663849,-0.898661077022552)
--(axis cs:0.979797959327698,-1.07233858108521)
--(axis cs:0.969696998596191,-1.23119270801544)
--(axis cs:0.95959597826004,-1.37203598022461)
--(axis cs:0.949494957923889,-1.49246001243591)
--(axis cs:0.939393937587738,-1.59056580066681)
--(axis cs:0.929292917251587,-1.66496157646179)
--(axis cs:0.919191956520081,-1.71470522880554)
--(axis cs:0.909090936183929,-1.73928844928741)
--(axis cs:0.898989915847778,-1.73859596252441)
--(axis cs:0.888888895511627,-1.71284866333008)
--(axis cs:0.878787875175476,-1.6625679731369)
--(axis cs:0.868686854839325,-1.58853077888489)
--(axis cs:0.858585834503174,-1.4917494058609)
--(axis cs:0.848484814167023,-1.37345361709595)
--(axis cs:0.838383793830872,-1.23509514331818)
--(axis cs:0.828282833099365,-1.0783417224884)
--(axis cs:0.818181812763214,-0.905088424682617)
--(axis cs:0.808080792427063,-0.717463672161102)
--(axis cs:0.797979831695557,-0.517817258834839)
--(axis cs:0.787878811359406,-0.308697313070297)
--(axis cs:0.777777791023254,-0.0928693413734436)
--(axis cs:0.767676770687103,0.126435399055481)
--(axis cs:0.757575750350952,0.32230806350708)
--(axis cs:0.747474789619446,0.178770244121552)
--(axis cs:0.737373769283295,0.0129687786102295)
--(axis cs:0.727272748947144,-0.146591484546661)
--(axis cs:0.717171728610992,-0.29519647359848)
--(axis cs:0.707070708274841,-0.428772538900375)
--(axis cs:0.69696968793869,-0.543674409389496)
--(axis cs:0.686868667602539,-0.6367227435112)
--(axis cs:0.676767706871033,-0.705268025398254)
--(axis cs:0.666666686534882,-0.747236609458923)
--(axis cs:0.65656566619873,-0.761165380477905)
--(axis cs:0.646464645862579,-0.74621593952179)
--(axis cs:0.636363625526428,-0.702185213565826)
--(axis cs:0.626262605190277,-0.629502475261688)
--(axis cs:0.616161584854126,-0.52922523021698)
--(axis cs:0.606060564517975,-0.403024315834045)
--(axis cs:0.595959544181824,-0.253169655799866)
--(axis cs:0.585858583450317,-0.0825041532516479)
--(axis cs:0.575757563114166,0.105599284172058)
--(axis cs:0.565656542778015,0.307281136512756)
--(axis cs:0.555555582046509,0.518273711204529)
--(axis cs:0.545454561710358,0.733989238739014)
--(axis cs:0.535353541374207,0.949618995189667)
--(axis cs:0.525252521038055,1.16021490097046)
--(axis cs:0.515151500701904,1.36050999164581)
--(axis cs:0.505050539970398,1.52178299427032)
--(axis cs:0.494949519634247,1.32564640045166)
--(axis cs:0.484848499298096,1.08930349349976)
--(axis cs:0.474747478961945,0.840315163135529)
--(axis cs:0.464646458625793,0.582855105400085)
--(axis cs:0.454545438289642,0.320865750312805)
--(axis cs:0.444444447755814,0.058283805847168)
--(axis cs:0.434343427419662,-0.20101273059845)
--(axis cs:0.424242407083511,-0.45326042175293)
--(axis cs:0.414141416549683,-0.694863140583038)
--(axis cs:0.404040396213531,-0.922431945800781)
--(axis cs:0.39393937587738,-1.1327977180481)
--(axis cs:0.383838385343552,-1.32303166389465)
--(axis cs:0.373737365007401,-1.49045872688293)
--(axis cs:0.363636374473572,-1.63267183303833)
--(axis cs:0.353535354137421,-1.74756288528442)
--(axis cs:0.34343433380127,-1.83336162567139)
--(axis cs:0.333333343267441,-1.88868582248688)
--(axis cs:0.32323232293129,-1.9126033782959)
--(axis cs:0.313131332397461,-1.90468955039978)
--(axis cs:0.30303031206131,-1.86508977413177)
--(axis cs:0.292929291725159,-1.79455709457397)
--(axis cs:0.28282830119133,-1.69449770450592)
--(axis cs:0.272727280855179,-1.56701231002808)
--(axis cs:0.262626260519028,-1.41518235206604)
--(axis cs:0.252525269985199,-1.26630306243896)
--(axis cs:0.242424249649048,-1.43814611434937)
--(axis cs:0.232323229312897,-1.61372065544128)
--(axis cs:0.222222223877907,-1.76650309562683)
--(axis cs:0.212121218442917,-1.89412474632263)
--(axis cs:0.202020198106766,-1.99515414237976)
--(axis cs:0.191919192671776,-2.06876611709595)
--(axis cs:0.181818187236786,-2.11467456817627)
--(axis cs:0.171717166900635,-2.13307666778564)
--(axis cs:0.161616161465645,-2.12457966804504)
--(axis cs:0.151515156030655,-2.09012055397034)
--(axis cs:0.141414135694504,-2.03088426589966)
--(axis cs:0.131313130259514,-1.94823455810547)
--(axis cs:0.121212124824524,-1.84365153312683)
--(axis cs:0.111111111938953,-1.71869897842407)
--(axis cs:0.101010099053383,-1.57500529289246)
--(axis cs:0.0909090936183929,-1.41427481174469)
--(axis cs:0.0808080807328224,-1.23830473423004)
--(axis cs:0.0707070678472519,-1.04903149604797)
--(axis cs:0.060606062412262,-0.848540902137756)
--(axis cs:0.0505050495266914,-0.639123797416687)
--(axis cs:0.0404040403664112,-0.423254698514938)
--(axis cs:0.030303031206131,-0.203639417886734)
--(axis cs:0.0202020201832056,0.016850471496582)
--(axis cs:0.0101010100916028,0.234802171587944)
--(axis cs:0,0.42320853471756)
--cycle;
\addplot [semithick, black, mark=*, mark size=3, mark options={solid}, only marks]
table {%
0 0.450969964265823
0.252525269985199 -1.23872041702271
0.505050539970398 1.54965722560883
0.757575750350952 0.350055396556854
1 -0.709737658500671
};
\addplot [semithick, blue]
table {%
0 0.451404750347137
0.0101010100916028 0.429508864879608
0.0202020201832056 0.395687580108643
0.030303031206131 0.359092146158218
0.0404040403664112 0.312889993190765
0.0505050495266914 0.261233478784561
0.060606062412262 0.203510195016861
0.0707070678472519 0.129626274108887
0.0808080807328224 0.0392259284853935
0.0909090936183929 -0.0495683066546917
0.101010099053383 -0.136123538017273
0.111111111938953 -0.226568162441254
0.121212124824524 -0.328051686286926
0.131313130259514 -0.441125929355621
0.141414135694504 -0.553359568119049
0.151515156030655 -0.646482348442078
0.161616161465645 -0.721714496612549
0.171717166900635 -0.810608148574829
0.181818187236786 -0.912096858024597
0.191919192671776 -1.00756561756134
0.202020198106766 -1.08372116088867
0.212121218442917 -1.14134109020233
0.222222223877907 -1.18776392936707
0.232323229312897 -1.22145795822144
0.242424249649048 -1.24008917808533
0.252525269985199 -1.24087452888489
0.262626260519028 -1.21322524547577
0.272727280855179 -1.18115615844727
0.28282830119133 -1.12498712539673
0.292929291725159 -1.05412817001343
0.30303031206131 -0.966374933719635
0.313131332397461 -0.863375544548035
0.32323232293129 -0.745270907878876
0.333333343267441 -0.605263948440552
0.34343433380127 -0.447585672140121
0.353535354137421 -0.279474765062332
0.363636374473572 -0.106364138424397
0.373737365007401 0.0700160041451454
0.383838385343552 0.253800392150879
0.39393937587738 0.456855267286301
0.404040396213531 0.645587027072906
0.414141416549683 0.801948666572571
0.424242407083511 0.914830088615417
0.434343427419662 1.03487551212311
0.444444447755814 1.16097414493561
0.454545438289642 1.27622497081757
0.464646458625793 1.36799550056458
0.474747478961945 1.44127821922302
0.484848499298096 1.49789786338806
0.494949519634247 1.52496671676636
0.505050539970398 1.5487414598465
0.515151500701904 1.55028939247131
0.525252521038055 1.53875482082367
0.535353541374207 1.50630915164948
0.545454561710358 1.459308385849
0.555555582046509 1.40213072299957
0.565656542778015 1.33318710327148
0.575757563114166 1.25758647918701
0.585858583450317 1.18111801147461
0.595959544181824 1.10618090629578
0.606060564517975 1.02639639377594
0.616161584854126 0.948617815971375
0.626262605190277 0.876832485198975
0.636363625526428 0.815645337104797
0.646464645862579 0.770972907543182
0.65656566619873 0.719010055065155
0.666666686534882 0.669880330562592
0.676767706871033 0.60925030708313
0.686868667602539 0.551652431488037
0.69696968793869 0.518530607223511
0.707070708274841 0.49223804473877
0.717171728610992 0.467531263828278
0.727272748947144 0.440430611371994
0.737373769283295 0.409383833408356
0.747474789619446 0.378106713294983
0.757575750350952 0.348360985517502
0.767676770687103 0.320502936840057
0.777777791023254 0.288082242012024
0.787878811359406 0.249972492456436
0.797979831695557 0.210989579558372
0.808080792427063 0.171478629112244
0.818181812763214 0.129068806767464
0.828282833099365 0.082429438829422
0.838383793830872 0.028945567086339
0.848484814167023 -0.0323014184832573
0.858585834503174 -0.0969058275222778
0.868686854839325 -0.161361858248711
0.878787875175476 -0.221974015235901
0.888888895511627 -0.279919773340225
0.898989915847778 -0.340129196643829
0.909090936183929 -0.400968372821808
0.919191956520081 -0.454226464033127
0.929292917251587 -0.502343833446503
0.939393937587738 -0.548811614513397
0.949494957923889 -0.593340992927551
0.95959597826004 -0.635101020336151
0.969696998596191 -0.669027984142303
0.979797959327698 -0.690099358558655
0.989898979663849 -0.705365180969238
1 -0.710029423236847
};
\addplot [semithick, green!50.1960784313725!black]
table {%
0 0.450919508934021
0.0101010100916028 0.432832717895508
0.0202020201832056 0.406189411878586
0.030303031206131 0.370711773633957
0.0404040403664112 0.326270669698715
0.0505050495266914 0.272882550954819
0.060606062412262 0.210723400115967
0.0707070678472519 0.140125021338463
0.0808080807328224 0.0615863353013992
0.0909090936183929 -0.0242265295237303
0.101010099053383 -0.116470664739609
0.111111111938953 -0.214124441146851
0.121212124824524 -0.315982550382614
0.131313130259514 -0.42065891623497
0.141414135694504 -0.526592493057251
0.151515156030655 -0.632060766220093
0.161616161465645 -0.735203504562378
0.171717166900635 -0.834055364131927
0.181818187236786 -0.926584780216217
0.191919192671776 -1.01074993610382
0.202020198106766 -1.08454918861389
0.212121218442917 -1.14608502388
0.222222223877907 -1.19362449645996
0.232323229312897 -1.22565960884094
0.242424249649048 -1.24095332622528
0.252525269985199 -1.23858797550201
0.262626260519028 -1.21799123287201
0.272727280855179 -1.17895305156708
0.28282830119133 -1.12162983417511
0.292929291725159 -1.04653334617615
0.30303031206131 -0.954511642456055
0.313131332397461 -0.846718907356262
0.32323232293129 -0.724580764770508
0.333333343267441 -0.589760422706604
0.34343433380127 -0.444119840860367
0.353535354137421 -0.289686918258667
0.363636374473572 -0.128625512123108
0.373737365007401 0.036794126033783
0.383838385343552 0.204221129417419
0.39393937587738 0.371248245239258
0.404040396213531 0.535443425178528
0.414141416549683 0.694377243518829
0.424242407083511 0.845663666725159
0.434343427419662 0.98700749874115
0.444444447755814 1.11625218391418
0.454545438289642 1.23144018650055
0.464646458625793 1.33087551593781
0.474747478961945 1.41318166255951
0.484848499298096 1.47736060619354
0.494949519634247 1.52283799648285
0.505050539970398 1.54949808120728
0.515151500701904 1.55769872665405
0.525252521038055 1.5482679605484
0.535353541374207 1.52247786521912
0.545454561710358 1.48199772834778
0.555555582046509 1.4288284778595
0.565656542778015 1.36521935462952
0.575757563114166 1.29357528686523
0.585858583450317 1.21635758876801
0.595959544181824 1.13598418235779
0.606060564517975 1.05473351478577
0.616161584854126 0.974664688110352
0.626262605190277 0.897546589374542
0.636363625526428 0.82480651140213
0.646464645862579 0.757500886917114
0.65656566619873 0.696302652359009
0.666666686534882 0.641509771347046
0.676767706871033 0.593069791793823
0.686868667602539 0.550618350505829
0.69696968793869 0.513529002666473
0.707070708274841 0.480970174074173
0.717171728610992 0.45196408033371
0.727272748947144 0.425446271896362
0.737373769283295 0.40032222867012
0.747474789619446 0.375517845153809
0.757575750350952 0.350023150444031
0.767676770687103 0.322927385568619
0.777777791023254 0.293447405099869
0.787878811359406 0.260946601629257
0.797979831695557 0.22494775056839
0.808080792427063 0.185138627886772
0.818181812763214 0.141374453902245
0.828282833099365 0.0936759263277054
0.838383793830872 0.0422257483005524
0.848484814167023 -0.0126408934593201
0.858585834503174 -0.0704426318407059
0.868686854839325 -0.130560308694839
0.878787875175476 -0.192254021763802
0.888888895511627 -0.254671663045883
0.898989915847778 -0.316872596740723
0.909090936183929 -0.377849578857422
0.919191956520081 -0.436547487974167
0.929292917251587 -0.491903454065323
0.939393937587738 -0.542871594429016
0.949494957923889 -0.588464379310608
0.95959597826004 -0.627789795398712
0.969696998596191 -0.660079896450043
0.979797959327698 -0.684731006622314
0.989898979663849 -0.701328933238983
1 -0.70966500043869
};
\end{axis}
\end{tikzpicture}
% This file was created by tikzplotlib v0.9.9.
\begin{tikzpicture}
\begin{axis}[
tick align=outside,
tick pos=left,
x grid style={white!69.0196078431373!black},
xmin=-0.05, xmax=1.05,
xtick style={color=black},
y grid style={white!69.0196078431373!black},
ymin=-2.79759725928307, ymax=2.13650438189507,
ytick style={color=black}
]
\path [draw=blue, fill=blue, opacity=0.2]
(axis cs:0,-1.77572655677795)
--(axis cs:0,-1.87970924377441)
--(axis cs:0.0101010100916028,-2.00187110900879)
--(axis cs:0.0202020201832056,-2.15863394737244)
--(axis cs:0.030303031206131,-2.28728747367859)
--(axis cs:0.0404040403664112,-2.38705277442932)
--(axis cs:0.0505050495266914,-2.45314168930054)
--(axis cs:0.060606062412262,-2.49665284156799)
--(axis cs:0.0707070678472519,-2.5063214302063)
--(axis cs:0.0808080807328224,-2.4753429889679)
--(axis cs:0.0909090936183929,-2.4353232383728)
--(axis cs:0.101010099053383,-2.3624529838562)
--(axis cs:0.111111111938953,-2.26170635223389)
--(axis cs:0.121212124824524,-2.1325991153717)
--(axis cs:0.131313130259514,-1.97519659996033)
--(axis cs:0.141414135694504,-1.78515672683716)
--(axis cs:0.151515156030655,-1.60323643684387)
--(axis cs:0.161616161465645,-1.43953013420105)
--(axis cs:0.171717166900635,-1.25933599472046)
--(axis cs:0.181818187236786,-1.03491818904877)
--(axis cs:0.191919192671776,-0.793911457061768)
--(axis cs:0.202020198106766,-0.547502636909485)
--(axis cs:0.212121218442917,-0.3149234354496)
--(axis cs:0.222222223877907,-0.0801777467131615)
--(axis cs:0.232323229312897,0.138718947768211)
--(axis cs:0.242424249649048,0.358084589242935)
--(axis cs:0.252525269985199,0.542679131031036)
--(axis cs:0.262626260519028,0.397037357091904)
--(axis cs:0.272727280855179,0.215411454439163)
--(axis cs:0.28282830119133,0.0303839799016714)
--(axis cs:0.292929291725159,-0.157315015792847)
--(axis cs:0.30303031206131,-0.334902703762054)
--(axis cs:0.313131332397461,-0.493901938199997)
--(axis cs:0.32323232293129,-0.6366286277771)
--(axis cs:0.333333343267441,-0.764340698719025)
--(axis cs:0.34343433380127,-0.891273558139801)
--(axis cs:0.353535354137421,-1.02008903026581)
--(axis cs:0.363636374473572,-1.12022066116333)
--(axis cs:0.373737365007401,-1.17197895050049)
--(axis cs:0.383838385343552,-1.18416094779968)
--(axis cs:0.39393937587738,-1.17172348499298)
--(axis cs:0.404040396213531,-1.14077091217041)
--(axis cs:0.414141416549683,-1.12282228469849)
--(axis cs:0.424242407083511,-1.09629058837891)
--(axis cs:0.434343427419662,-1.0104466676712)
--(axis cs:0.444444447755814,-0.879726827144623)
--(axis cs:0.454545438289642,-0.71797639131546)
--(axis cs:0.464646458625793,-0.555372953414917)
--(axis cs:0.474747478961945,-0.391676753759384)
--(axis cs:0.484848499298096,-0.220648884773254)
--(axis cs:0.494949519634247,-0.0452536791563034)
--(axis cs:0.505050539970398,0.108546540141106)
--(axis cs:0.515151500701904,-0.0519117377698421)
--(axis cs:0.525252521038055,-0.246114492416382)
--(axis cs:0.535353541374207,-0.433059334754944)
--(axis cs:0.545454561710358,-0.610726177692413)
--(axis cs:0.555555582046509,-0.781551241874695)
--(axis cs:0.565656542778015,-0.942324638366699)
--(axis cs:0.575757563114166,-1.08462738990784)
--(axis cs:0.585858583450317,-1.22177624702454)
--(axis cs:0.595959544181824,-1.34274470806122)
--(axis cs:0.606060564517975,-1.44603896141052)
--(axis cs:0.616161584854126,-1.52484440803528)
--(axis cs:0.626262605190277,-1.57947039604187)
--(axis cs:0.636363625526428,-1.60667824745178)
--(axis cs:0.646464645862579,-1.59282433986664)
--(axis cs:0.65656566619873,-1.55604517459869)
--(axis cs:0.666666686534882,-1.50695443153381)
--(axis cs:0.676767706871033,-1.43778586387634)
--(axis cs:0.686868667602539,-1.36080408096313)
--(axis cs:0.69696968793869,-1.25526130199432)
--(axis cs:0.707070708274841,-1.12573778629303)
--(axis cs:0.717171728610992,-0.979223430156708)
--(axis cs:0.727272748947144,-0.828780889511108)
--(axis cs:0.737373769283295,-0.66575425863266)
--(axis cs:0.747474789619446,-0.499670147895813)
--(axis cs:0.757575750350952,-0.366113781929016)
--(axis cs:0.767676770687103,-0.547487080097198)
--(axis cs:0.777777791023254,-0.765745460987091)
--(axis cs:0.787878811359406,-0.985893607139587)
--(axis cs:0.797979831695557,-1.19172096252441)
--(axis cs:0.808080792427063,-1.39268219470978)
--(axis cs:0.818181812763214,-1.57867097854614)
--(axis cs:0.828282833099365,-1.74434018135071)
--(axis cs:0.838383793830872,-1.90369021892548)
--(axis cs:0.848484814167023,-2.05849671363831)
--(axis cs:0.858585834503174,-2.19442415237427)
--(axis cs:0.868686854839325,-2.31568145751953)
--(axis cs:0.878787875175476,-2.42093086242676)
--(axis cs:0.888888895511627,-2.49319911003113)
--(axis cs:0.898989915847778,-2.5311963558197)
--(axis cs:0.909090936183929,-2.55570244789124)
--(axis cs:0.919191956520081,-2.55562853813171)
--(axis cs:0.929292917251587,-2.53653478622437)
--(axis cs:0.939393937587738,-2.49681448936462)
--(axis cs:0.949494957923889,-2.43126797676086)
--(axis cs:0.95959597826004,-2.33654594421387)
--(axis cs:0.969696998596191,-2.21987628936768)
--(axis cs:0.979797959327698,-2.06782817840576)
--(axis cs:0.989898979663849,-1.91583180427551)
--(axis cs:1,-1.76515650749207)
--(axis cs:1,-1.66605830192566)
--(axis cs:1,-1.66605830192566)
--(axis cs:0.989898979663849,-1.52185416221619)
--(axis cs:0.979797959327698,-1.30338525772095)
--(axis cs:0.969696998596191,-1.10261225700378)
--(axis cs:0.95959597826004,-0.882315099239349)
--(axis cs:0.949494957923889,-0.64478075504303)
--(axis cs:0.939393937587738,-0.434544771909714)
--(axis cs:0.929292917251587,-0.234077304601669)
--(axis cs:0.919191956520081,-0.0475783832371235)
--(axis cs:0.909090936183929,0.115869373083115)
--(axis cs:0.898989915847778,0.263330489397049)
--(axis cs:0.888888895511627,0.403941661119461)
--(axis cs:0.878787875175476,0.507823646068573)
--(axis cs:0.868686854839325,0.581855833530426)
--(axis cs:0.858585834503174,0.633602917194366)
--(axis cs:0.848484814167023,0.639456391334534)
--(axis cs:0.838383793830872,0.60640150308609)
--(axis cs:0.828282833099365,0.552226603031158)
--(axis cs:0.818181812763214,0.477211534976959)
--(axis cs:0.808080792427063,0.37996906042099)
--(axis cs:0.797979831695557,0.261111557483673)
--(axis cs:0.787878811359406,0.126361683011055)
--(axis cs:0.777777791023254,-0.0193902719765902)
--(axis cs:0.767676770687103,-0.168602973222733)
--(axis cs:0.757575750350952,-0.295247882604599)
--(axis cs:0.747474789619446,-0.118694685399532)
--(axis cs:0.737373769283295,0.093021847307682)
--(axis cs:0.727272748947144,0.29423925280571)
--(axis cs:0.717171728610992,0.485918313264847)
--(axis cs:0.707070708274841,0.670939743518829)
--(axis cs:0.69696968793869,0.83789587020874)
--(axis cs:0.686868667602539,0.987226068973541)
--(axis cs:0.676767706871033,1.12052929401398)
--(axis cs:0.666666686534882,1.25316417217255)
--(axis cs:0.65656566619873,1.36738109588623)
--(axis cs:0.646464645862579,1.46610462665558)
--(axis cs:0.636363625526428,1.53365981578827)
--(axis cs:0.626262605190277,1.54835689067841)
--(axis cs:0.616161584854126,1.53176665306091)
--(axis cs:0.606060564517975,1.49397838115692)
--(axis cs:0.595959544181824,1.4367538690567)
--(axis cs:0.585858583450317,1.36165976524353)
--(axis cs:0.575757563114166,1.26514422893524)
--(axis cs:0.565656542778015,1.1414270401001)
--(axis cs:0.555555582046509,0.996169924736023)
--(axis cs:0.545454561710358,0.840539634227753)
--(axis cs:0.535353541374207,0.670153915882111)
--(axis cs:0.525252521038055,0.502714931964874)
--(axis cs:0.515151500701904,0.322328299283981)
--(axis cs:0.505050539970398,0.179380863904953)
--(axis cs:0.494949519634247,0.345961809158325)
--(axis cs:0.484848499298096,0.53890860080719)
--(axis cs:0.474747478961945,0.716320991516113)
--(axis cs:0.464646458625793,0.897793412208557)
--(axis cs:0.454545438289642,1.07271635532379)
--(axis cs:0.444444447755814,1.24993705749512)
--(axis cs:0.434343427419662,1.39227640628815)
--(axis cs:0.424242407083511,1.49866318702698)
--(axis cs:0.414141416549683,1.58034634590149)
--(axis cs:0.404040396213531,1.68093109130859)
--(axis cs:0.39393937587738,1.77172708511353)
--(axis cs:0.383838385343552,1.8503440618515)
--(axis cs:0.373737365007401,1.8945746421814)
--(axis cs:0.363636374473572,1.91222703456879)
--(axis cs:0.353535354137421,1.89711332321167)
--(axis cs:0.34343433380127,1.84722018241882)
--(axis cs:0.333333343267441,1.7752081155777)
--(axis cs:0.32323232293129,1.68998897075653)
--(axis cs:0.313131332397461,1.58956789970398)
--(axis cs:0.30303031206131,1.46627533435822)
--(axis cs:0.292929291725159,1.31597030162811)
--(axis cs:0.28282830119133,1.16358602046967)
--(axis cs:0.272727280855179,0.979778409004211)
--(axis cs:0.262626260519028,0.783105134963989)
--(axis cs:0.252525269985199,0.613874852657318)
--(axis cs:0.242424249649048,0.741821348667145)
--(axis cs:0.232323229312897,0.898408591747284)
--(axis cs:0.222222223877907,1.04929006099701)
--(axis cs:0.212121218442917,1.16176664829254)
--(axis cs:0.202020198106766,1.25690054893494)
--(axis cs:0.191919192671776,1.32071375846863)
--(axis cs:0.181818187236786,1.34844136238098)
--(axis cs:0.171717166900635,1.35224223136902)
--(axis cs:0.161616161465645,1.32400548458099)
--(axis cs:0.151515156030655,1.25749289989471)
--(axis cs:0.141414135694504,1.14788377285004)
--(axis cs:0.131313130259514,1.0135190486908)
--(axis cs:0.121212124824524,0.881045937538147)
--(axis cs:0.111111111938953,0.757622241973877)
--(axis cs:0.101010099053383,0.591476440429688)
--(axis cs:0.0909090936183929,0.359183371067047)
--(axis cs:0.0808080807328224,0.135236889123917)
--(axis cs:0.0707070678472519,-0.110840737819672)
--(axis cs:0.060606062412262,-0.357632696628571)
--(axis cs:0.0505050495266914,-0.595200181007385)
--(axis cs:0.0404040403664112,-0.872254192829132)
--(axis cs:0.030303031206131,-1.13448119163513)
--(axis cs:0.0202020201832056,-1.37505924701691)
--(axis cs:0.0101010100916028,-1.60263919830322)
--(axis cs:0,-1.77572655677795)
--cycle;
\path [draw=green!50.1960784313725!black, fill=green!50.1960784313725!black, opacity=0.2]
(axis cs:0,-1.85805284976959)
--(axis cs:0,-1.80263102054596)
--(axis cs:0.0101010100916028,-1.61525213718414)
--(axis cs:0.0202020201832056,-1.38656258583069)
--(axis cs:0.030303031206131,-1.14443910121918)
--(axis cs:0.0404040403664112,-0.89355081319809)
--(axis cs:0.0505050495266914,-0.638325870037079)
--(axis cs:0.060606062412262,-0.383272409439087)
--(axis cs:0.0707070678472519,-0.132801651954651)
--(axis cs:0.0808080807328224,0.108807802200317)
--(axis cs:0.0909090936183929,0.337543725967407)
--(axis cs:0.101010099053383,0.549685418605804)
--(axis cs:0.111111111938953,0.741891503334045)
--(axis cs:0.121212124824524,0.911214709281921)
--(axis cs:0.131313130259514,1.05514252185822)
--(axis cs:0.141414135694504,1.17161178588867)
--(axis cs:0.151515156030655,1.25904428958893)
--(axis cs:0.161616161465645,1.31636393070221)
--(axis cs:0.171717166900635,1.34303748607635)
--(axis cs:0.181818187236786,1.33909153938293)
--(axis cs:0.191919192671776,1.30514168739319)
--(axis cs:0.202020198106766,1.24240612983704)
--(axis cs:0.212121218442917,1.15270841121674)
--(axis cs:0.222222223877907,1.03846395015717)
--(axis cs:0.232323229312897,0.902679145336151)
--(axis cs:0.242424249649048,0.749224781990051)
--(axis cs:0.252525269985199,0.605998039245605)
--(axis cs:0.262626260519028,0.791195750236511)
--(axis cs:0.272727280855179,0.98805171251297)
--(axis cs:0.28282830119133,1.17005181312561)
--(axis cs:0.292929291725159,1.33465731143951)
--(axis cs:0.30303031206131,1.48006272315979)
--(axis cs:0.313131332397461,1.60490596294403)
--(axis cs:0.32323232293129,1.70822560787201)
--(axis cs:0.333333343267441,1.78941380977631)
--(axis cs:0.34343433380127,1.84817981719971)
--(axis cs:0.353535354137421,1.8844883441925)
--(axis cs:0.363636374473572,1.89850223064423)
--(axis cs:0.373737365007401,1.89052665233612)
--(axis cs:0.383838385343552,1.86096894741058)
--(axis cs:0.39393937587738,1.81031441688538)
--(axis cs:0.404040396213531,1.73912596702576)
--(axis cs:0.414141416549683,1.64806354045868)
--(axis cs:0.424242407083511,1.53792321681976)
--(axis cs:0.434343427419662,1.40968179702759)
--(axis cs:0.444444447755814,1.26455414295197)
--(axis cs:0.454545438289642,1.10403668880463)
--(axis cs:0.464646458625793,0.929942429065704)
--(axis cs:0.474747478961945,0.744427561759949)
--(axis cs:0.484848499298096,0.55001974105835)
--(axis cs:0.494949519634247,0.349909543991089)
--(axis cs:0.505050539970398,0.171157568693161)
--(axis cs:0.515151500701904,0.33097892999649)
--(axis cs:0.525252521038055,0.511515200138092)
--(axis cs:0.535353541374207,0.685072958469391)
--(axis cs:0.545454561710358,0.847866773605347)
--(axis cs:0.555555582046509,0.996819972991943)
--(axis cs:0.565656542778015,1.12929940223694)
--(axis cs:0.575757563114166,1.24310171604156)
--(axis cs:0.585858583450317,1.3364497423172)
--(axis cs:0.595959544181824,1.40797531604767)
--(axis cs:0.606060564517975,1.45669651031494)
--(axis cs:0.616161584854126,1.48197627067566)
--(axis cs:0.626262605190277,1.48348760604858)
--(axis cs:0.636363625526428,1.46118128299713)
--(axis cs:0.646464645862579,1.41526257991791)
--(axis cs:0.65656566619873,1.34618389606476)
--(axis cs:0.666666686534882,1.25464558601379)
--(axis cs:0.676767706871033,1.14160990715027)
--(axis cs:0.686868667602539,1.0083132982254)
--(axis cs:0.69696968793869,0.856286764144897)
--(axis cs:0.707070708274841,0.687360048294067)
--(axis cs:0.717171728610992,0.503658950328827)
--(axis cs:0.727272748947144,0.307585388422012)
--(axis cs:0.737373769283295,0.101826220750809)
--(axis cs:0.747474789619446,-0.11039075255394)
--(axis cs:0.757575750350952,-0.302101731300354)
--(axis cs:0.767676770687103,-0.157713502645493)
--(axis cs:0.777777791023254,0.00527751445770264)
--(axis cs:0.787878811359406,0.158524096012115)
--(axis cs:0.797979831695557,0.297487705945969)
--(axis cs:0.808080792427063,0.418275147676468)
--(axis cs:0.818181812763214,0.517448246479034)
--(axis cs:0.828282833099365,0.59205687046051)
--(axis cs:0.838383793830872,0.639720439910889)
--(axis cs:0.848484814167023,0.658683717250824)
--(axis cs:0.858585834503174,0.647855877876282)
--(axis cs:0.868686854839325,0.606834530830383)
--(axis cs:0.878787875175476,0.535911738872528)
--(axis cs:0.888888895511627,0.436081290245056)
--(axis cs:0.898989915847778,0.309010624885559)
--(axis cs:0.909090936183929,0.157005190849304)
--(axis cs:0.919191956520081,-0.0170046091079712)
--(axis cs:0.929292917251587,-0.209562420845032)
--(axis cs:0.939393937587738,-0.41672670841217)
--(axis cs:0.949494957923889,-0.634182870388031)
--(axis cs:0.95959597826004,-0.857377707958221)
--(axis cs:0.969696998596191,-1.08157849311829)
--(axis cs:0.979797959327698,-1.30203056335449)
--(axis cs:0.989898979663849,-1.51379466056824)
--(axis cs:1,-1.68855941295624)
--(axis cs:1,-1.7440060377121)
--(axis cs:1,-1.7440060377121)
--(axis cs:0.989898979663849,-1.90845894813538)
--(axis cs:0.979797959327698,-2.07724571228027)
--(axis cs:0.969696998596191,-2.22380399703979)
--(axis cs:0.95959597826004,-2.34587001800537)
--(axis cs:0.949494957923889,-2.44217419624329)
--(axis cs:0.939393937587738,-2.51211500167847)
--(axis cs:0.929292917251587,-2.5556788444519)
--(axis cs:0.919191956520081,-2.57331991195679)
--(axis cs:0.909090936183929,-2.56587266921997)
--(axis cs:0.898989915847778,-2.53443622589111)
--(axis cs:0.888888895511627,-2.48027276992798)
--(axis cs:0.878787875175476,-2.40471601486206)
--(axis cs:0.868686854839325,-2.30910634994507)
--(axis cs:0.858585834503174,-2.19475746154785)
--(axis cs:0.848484814167023,-2.06294178962708)
--(axis cs:0.838383793830872,-1.91492128372192)
--(axis cs:0.828282833099365,-1.75197851657867)
--(axis cs:0.818181812763214,-1.57547760009766)
--(axis cs:0.808080792427063,-1.38692951202393)
--(axis cs:0.797979831695557,-1.18804228305817)
--(axis cs:0.787878811359406,-0.980763733386993)
--(axis cs:0.777777791023254,-0.767355978488922)
--(axis cs:0.767676770687103,-0.550697445869446)
--(axis cs:0.757575750350952,-0.357531905174255)
--(axis cs:0.747474789619446,-0.503885984420776)
--(axis cs:0.737373769283295,-0.67288064956665)
--(axis cs:0.727272748947144,-0.836490154266357)
--(axis cs:0.717171728610992,-0.990662157535553)
--(axis cs:0.707070708274841,-1.13212537765503)
--(axis cs:0.69696968793869,-1.25812005996704)
--(axis cs:0.686868667602539,-1.36636888980865)
--(axis cs:0.676767706871033,-1.45506572723389)
--(axis cs:0.666666686534882,-1.52284717559814)
--(axis cs:0.65656566619873,-1.56875216960907)
--(axis cs:0.646464645862579,-1.5921710729599)
--(axis cs:0.636363625526428,-1.59280216693878)
--(axis cs:0.626262605190277,-1.57061052322388)
--(axis cs:0.616161584854126,-1.525803565979)
--(axis cs:0.606060564517975,-1.45881915092468)
--(axis cs:0.595959544181824,-1.37033236026764)
--(axis cs:0.585858583450317,-1.26127374172211)
--(axis cs:0.575757563114166,-1.13285028934479)
--(axis cs:0.565656542778015,-0.986577093601227)
--(axis cs:0.555555582046509,-0.824289560317993)
--(axis cs:0.545454561710358,-0.648150205612183)
--(axis cs:0.535353541374207,-0.46064481139183)
--(axis cs:0.525252521038055,-0.264590799808502)
--(axis cs:0.515151500701904,-0.0633985847234726)
--(axis cs:0.505050539970398,0.115727387368679)
--(axis cs:0.494949519634247,-0.0444737672805786)
--(axis cs:0.484848499298096,-0.226094469428062)
--(axis cs:0.474747478961945,-0.401305437088013)
--(axis cs:0.464646458625793,-0.566098392009735)
--(axis cs:0.454545438289642,-0.717112243175507)
--(axis cs:0.444444447755814,-0.851382672786713)
--(axis cs:0.434343427419662,-0.966358661651611)
--(axis cs:0.424242407083511,-1.05992496013641)
--(axis cs:0.414141416549683,-1.13041722774506)
--(axis cs:0.404040396213531,-1.17662477493286)
--(axis cs:0.39393937587738,-1.19777750968933)
--(axis cs:0.383838385343552,-1.19353663921356)
--(axis cs:0.373737365007401,-1.16397893428802)
--(axis cs:0.363636374473572,-1.10959041118622)
--(axis cs:0.353535354137421,-1.03126358985901)
--(axis cs:0.34343433380127,-0.930303812026978)
--(axis cs:0.333333343267441,-0.808436989784241)
--(axis cs:0.32323232293129,-0.667819619178772)
--(axis cs:0.313131332397461,-0.511035323143005)
--(axis cs:0.30303031206131,-0.341093599796295)
--(axis cs:0.292929291725159,-0.16139030456543)
--(axis cs:0.28282830119133,0.0243160128593445)
--(axis cs:0.272727280855179,0.211933314800262)
--(axis cs:0.262626260519028,0.396813541650772)
--(axis cs:0.252525269985199,0.550567865371704)
--(axis cs:0.242424249649048,0.354839116334915)
--(axis cs:0.232323229312897,0.126557171344757)
--(axis cs:0.222222223877907,-0.107293099164963)
--(axis cs:0.212121218442917,-0.343371033668518)
--(axis cs:0.202020198106766,-0.578803896903992)
--(axis cs:0.191919192671776,-0.810890674591064)
--(axis cs:0.181818187236786,-1.03708791732788)
--(axis cs:0.171717166900635,-1.25500524044037)
--(axis cs:0.161616161465645,-1.46238839626312)
--(axis cs:0.151515156030655,-1.65707528591156)
--(axis cs:0.141414135694504,-1.83697199821472)
--(axis cs:0.131313130259514,-2.00000858306885)
--(axis cs:0.121212124824524,-2.14412307739258)
--(axis cs:0.111111111938953,-2.26725769042969)
--(axis cs:0.101010099053383,-2.3673837184906)
--(axis cs:0.0909090936183929,-2.4425528049469)
--(axis cs:0.0808080807328224,-2.49097442626953)
--(axis cs:0.0707070678472519,-2.51111459732056)
--(axis cs:0.060606062412262,-2.50180101394653)
--(axis cs:0.0505050495266914,-2.4623384475708)
--(axis cs:0.0404040403664112,-2.39260149002075)
--(axis cs:0.030303031206131,-2.29314136505127)
--(axis cs:0.0202020201832056,-2.16524052619934)
--(axis cs:0.0101010100916028,-2.01131319999695)
--(axis cs:0,-1.85805284976959)
--cycle;
\addplot [semithick, black, mark=*, mark size=3, mark options={solid}, only marks]
table {%
0 -1.83052742481232
0.252525269985199 0.578347861766815
0.505050539970398 0.143455162644386
0.757575750350952 -0.329841434955597
1 -1.71645307540894
};
\addplot [semithick, blue]
table {%
0 -1.82794237136841
0.0101010100916028 -1.80248129367828
0.0202020201832056 -1.76602864265442
0.030303031206131 -1.70956516265869
0.0404040403664112 -1.62978672981262
0.0505050495266914 -1.52824091911316
0.060606062412262 -1.42358565330505
0.0707070678472519 -1.30808448791504
0.0808080807328224 -1.17485213279724
0.0909090936183929 -1.0412905216217
0.101010099053383 -0.890201032161713
0.111111111938953 -0.753471970558167
0.121212124824524 -0.625776410102844
0.131313130259514 -0.481359660625458
0.141414135694504 -0.32117035984993
0.151515156030655 -0.176426291465759
0.161616161465645 -0.060685858130455
0.171717166900635 0.0448338836431503
0.181818187236786 0.155033096671104
0.191919192671776 0.262386471033096
0.202020198106766 0.352682650089264
0.212121218442917 0.425931394100189
0.222222223877907 0.481675624847412
0.232323229312897 0.519096493721008
0.242424249649048 0.550521075725555
0.252525269985199 0.57817006111145
0.262626260519028 0.589876890182495
0.272727280855179 0.598390579223633
0.28282830119133 0.595390200614929
0.292929291725159 0.581771910190582
0.30303031206131 0.566249370574951
0.313131332397461 0.548824667930603
0.32323232293129 0.529731571674347
0.333333343267441 0.508555829524994
0.34343433380127 0.479685932397842
0.353535354137421 0.439207851886749
0.363636374473572 0.397185504436493
0.373737365007401 0.363430440425873
0.383838385343552 0.335638046264648
0.39393937587738 0.302829444408417
0.404040396213531 0.270835518836975
0.414141416549683 0.229634076356888
0.424242407083511 0.20330935716629
0.434343427419662 0.192685633897781
0.444444447755814 0.185996800661087
0.454545438289642 0.177534490823746
0.464646458625793 0.167088866233826
0.474747478961945 0.159218907356262
0.484848499298096 0.15802001953125
0.494949519634247 0.150111839175224
0.505050539970398 0.143772453069687
0.515151500701904 0.134222000837326
0.525252521038055 0.128199756145477
0.535353541374207 0.116369612514973
0.545454561710358 0.109461665153503
0.555555582046509 0.104083627462387
0.565656542778015 0.0965288132429123
0.575757563114166 0.0854566991329193
0.585858583450317 0.0691086053848267
0.595959544181824 0.047811821103096
0.606060564517975 0.0244831629097462
0.616161584854126 0.00292238127440214
0.626262605190277 -0.0167179498821497
0.636363625526428 -0.0378357581794262
0.646464645862579 -0.0640752613544464
0.65656566619873 -0.0947883874177933
0.666666686534882 -0.127549380064011
0.676767706871033 -0.160839423537254
0.686868667602539 -0.188652247190475
0.69696968793869 -0.210562288761139
0.707070708274841 -0.229276537895203
0.717171728610992 -0.247611045837402
0.727272748947144 -0.266524791717529
0.737373769283295 -0.287339419126511
0.747474789619446 -0.311479836702347
0.757575750350952 -0.330682456493378
0.767676770687103 -0.358491271734238
0.777777791023254 -0.390957772731781
0.787878811359406 -0.428439497947693
0.797979831695557 -0.46640282869339
0.808080792427063 -0.504439830780029
0.818181812763214 -0.545271098613739
0.828282833099365 -0.590579509735107
0.838383793830872 -0.645362079143524
0.848484814167023 -0.709442734718323
0.858585834503174 -0.782322883605957
0.868686854839325 -0.86964076757431
0.878787875175476 -0.957900643348694
0.888888895511627 -1.047159075737
0.898989915847778 -1.1404824256897
0.909090936183929 -1.22821366786957
0.919191956520081 -1.3053172826767
0.929292917251587 -1.38584876060486
0.939393937587738 -1.46851992607117
0.949494957923889 -1.54457664489746
0.95959597826004 -1.61110246181488
0.969696998596191 -1.66065716743469
0.979797959327698 -1.68942046165466
0.989898979663849 -1.71514797210693
1 -1.71599495410919
};
\addplot [semithick, green!50.1960784313725!black]
table {%
0 -1.83034193515778
0.0101010100916028 -1.81328272819519
0.0202020201832056 -1.77590155601501
0.030303031206131 -1.71879029273987
0.0404040403664112 -1.64307618141174
0.0505050495266914 -1.55033218860626
0.060606062412262 -1.44253671169281
0.0707070678472519 -1.32195818424225
0.0808080807328224 -1.19108331203461
0.0909090936183929 -1.05250453948975
0.101010099053383 -0.90884917974472
0.111111111938953 -0.762683033943176
0.121212124824524 -0.616454243659973
0.131313130259514 -0.472433090209961
0.141414135694504 -0.332680106163025
0.151515156030655 -0.199015542864799
0.161616161465645 -0.0730122104287148
0.171717166900635 0.044016107916832
0.181818187236786 0.151001811027527
0.191919192671776 0.24712547659874
0.202020198106766 0.3318010866642
0.212121218442917 0.404668688774109
0.222222223877907 0.465585440397263
0.232323229312897 0.514618158340454
0.242424249649048 0.552031934261322
0.252525269985199 0.578282952308655
0.262626260519028 0.59400463104248
0.272727280855179 0.599992513656616
0.28282830119133 0.597183883190155
0.292929291725159 0.586633503437042
0.30303031206131 0.569484531879425
0.313131332397461 0.546935319900513
0.32323232293129 0.520202994346619
0.333333343267441 0.49048838019371
0.34343433380127 0.458938002586365
0.353535354137421 0.426612436771393
0.363636374473572 0.394455879926682
0.373737365007401 0.36327388882637
0.383838385343552 0.333716124296188
0.39393937587738 0.306268513202667
0.404040396213531 0.281250655651093
0.414141416549683 0.258823156356812
0.424242407083511 0.238999098539352
0.434343427419662 0.221661582589149
0.444444447755814 0.206585720181465
0.454545438289642 0.193462207913399
0.464646458625793 0.181922003626823
0.474747478961945 0.171561047434807
0.484848499298096 0.161962643265724
0.494949519634247 0.152717888355255
0.505050539970398 0.14344248175621
0.515151500701904 0.13379018008709
0.525252521038055 0.123462207615376
0.535353541374207 0.1122140660882
0.545454561710358 0.0998582988977432
0.555555582046509 0.0862651765346527
0.565656542778015 0.0713611245155334
0.575757563114166 0.0551257468760014
0.585858583450317 0.0375879779458046
0.595959544181824 0.0188214853405952
0.606060564517975 -0.00106131285429001
0.616161584854126 -0.0219136625528336
0.626262605190277 -0.043561477214098
0.636363625526428 -0.0658104941248894
0.646464645862579 -0.0884542316198349
0.65656566619873 -0.111284092068672
0.666666686534882 -0.134100794792175
0.676767706871033 -0.156727954745293
0.686868667602539 -0.179027780890465
0.69696968793869 -0.200916662812233
0.707070708274841 -0.22238264977932
0.717171728610992 -0.243501603603363
0.727272748947144 -0.264452368021011
0.737373769283295 -0.285527229309082
0.747474789619446 -0.307138353586197
0.757575750350952 -0.329816818237305
0.767676770687103 -0.35420548915863
0.777777791023254 -0.38103923201561
0.787878811359406 -0.411119818687439
0.797979831695557 -0.44527730345726
0.808080792427063 -0.484327167272568
0.818181812763214 -0.529014647006989
0.828282833099365 -0.579960823059082
0.838383793830872 -0.637600421905518
0.848484814167023 -0.702129065990448
0.858585834503174 -0.77345085144043
0.868686854839325 -0.851135969161987
0.878787875175476 -0.934402167797089
0.888888895511627 -1.02209568023682
0.898989915847778 -1.11271274089813
0.909090936183929 -1.20443367958069
0.919191956520081 -1.29516232013702
0.929292917251587 -1.38262057304382
0.939393937587738 -1.46442091464996
0.949494957923889 -1.53817856311798
0.95959597826004 -1.60162389278412
0.969696998596191 -1.65269124507904
0.979797959327698 -1.68963813781738
0.989898979663849 -1.71112680435181
1 -1.71628272533417
};
\end{axis}
\end{tikzpicture}
% This file was created by tikzplotlib v0.9.9.
\begin{tikzpicture}
\begin{axis}[
xlabel=x, ylabel=y,
tick align=outside,
tick pos=left,
x grid style={white!69.0196078431373!black},
xmin=-0.05, xmax=1.05,
xtick style={color=black},
y grid style={white!69.0196078431373!black},
ymin=-2.36563382148743, ymax=2.85721144676209,
ytick style={color=black}
]
\path [draw=blue, fill=blue, opacity=0.2]
(axis cs:0,0.464962095022202)
--(axis cs:0,0.392874389886856)
--(axis cs:0.0101010100916028,0.250869572162628)
--(axis cs:0.0202020201832056,0.0732533037662506)
--(axis cs:0.030303031206131,-0.0798227787017822)
--(axis cs:0.0404040403664112,-0.238363087177277)
--(axis cs:0.0505050495266914,-0.389174342155457)
--(axis cs:0.060606062412262,-0.512137651443481)
--(axis cs:0.0707070678472519,-0.584345400333405)
--(axis cs:0.0808080807328224,-0.61897486448288)
--(axis cs:0.0909090936183929,-0.653846323490143)
--(axis cs:0.101010099053383,-0.689997375011444)
--(axis cs:0.111111111938953,-0.700381457805634)
--(axis cs:0.121212124824524,-0.652755796909332)
--(axis cs:0.131313130259514,-0.546591758728027)
--(axis cs:0.141414135694504,-0.437991589307785)
--(axis cs:0.151515156030655,-0.313125461339951)
--(axis cs:0.161616161465645,-0.187646955251694)
--(axis cs:0.171717166900635,-0.0167126394808292)
--(axis cs:0.181818187236786,0.208796858787537)
--(axis cs:0.191919192671776,0.441302984952927)
--(axis cs:0.202020198106766,0.651185154914856)
--(axis cs:0.212121218442917,0.86050271987915)
--(axis cs:0.222222223877907,1.09129989147186)
--(axis cs:0.232323229312897,1.30849456787109)
--(axis cs:0.242424249649048,1.51908099651337)
--(axis cs:0.252525269985199,1.67173433303833)
--(axis cs:0.262626260519028,1.51373958587646)
--(axis cs:0.272727280855179,1.29504072666168)
--(axis cs:0.28282830119133,1.0685703754425)
--(axis cs:0.292929291725159,0.816317439079285)
--(axis cs:0.30303031206131,0.571208655834198)
--(axis cs:0.313131332397461,0.331234306097031)
--(axis cs:0.32323232293129,0.10523396730423)
--(axis cs:0.333333343267441,-0.109325297176838)
--(axis cs:0.34343433380127,-0.312302857637405)
--(axis cs:0.353535354137421,-0.493835002183914)
--(axis cs:0.363636374473572,-0.67211925983429)
--(axis cs:0.373737365007401,-0.821274697780609)
--(axis cs:0.383838385343552,-0.932058274745941)
--(axis cs:0.39393937587738,-1.03159308433533)
--(axis cs:0.404040396213531,-1.09010243415833)
--(axis cs:0.414141416549683,-1.11366665363312)
--(axis cs:0.424242407083511,-1.10496437549591)
--(axis cs:0.434343427419662,-1.06017708778381)
--(axis cs:0.444444447755814,-0.980633735656738)
--(axis cs:0.454545438289642,-0.870244383811951)
--(axis cs:0.464646458625793,-0.748034179210663)
--(axis cs:0.474747478961945,-0.616547524929047)
--(axis cs:0.484848499298096,-0.472347736358643)
--(axis cs:0.494949519634247,-0.312524110078812)
--(axis cs:0.505050539970398,-0.179713264107704)
--(axis cs:0.515151500701904,-0.367924302816391)
--(axis cs:0.525252521038055,-0.577349603176117)
--(axis cs:0.535353541374207,-0.787624418735504)
--(axis cs:0.545454561710358,-0.982816934585571)
--(axis cs:0.555555582046509,-1.16539990901947)
--(axis cs:0.565656542778015,-1.33365571498871)
--(axis cs:0.575757563114166,-1.48352062702179)
--(axis cs:0.585858583450317,-1.61894869804382)
--(axis cs:0.595959544181824,-1.73657429218292)
--(axis cs:0.606060564517975,-1.82720935344696)
--(axis cs:0.616161584854126,-1.90044450759888)
--(axis cs:0.626262605190277,-1.95948076248169)
--(axis cs:0.636363625526428,-1.99543976783752)
--(axis cs:0.646464645862579,-2.00260257720947)
--(axis cs:0.65656566619873,-1.99804413318634)
--(axis cs:0.666666686534882,-1.97670364379883)
--(axis cs:0.676767706871033,-1.92967855930328)
--(axis cs:0.686868667602539,-1.86906218528748)
--(axis cs:0.69696968793869,-1.7828973531723)
--(axis cs:0.707070708274841,-1.66263103485107)
--(axis cs:0.717171728610992,-1.52109289169312)
--(axis cs:0.727272748947144,-1.37280917167664)
--(axis cs:0.737373769283295,-1.21131896972656)
--(axis cs:0.747474789619446,-1.03360176086426)
--(axis cs:0.757575750350952,-0.876796543598175)
--(axis cs:0.767676770687103,-1.02847981452942)
--(axis cs:0.777777791023254,-1.20529973506927)
--(axis cs:0.787878811359406,-1.3728084564209)
--(axis cs:0.797979831695557,-1.52649760246277)
--(axis cs:0.808080792427063,-1.66553819179535)
--(axis cs:0.818181812763214,-1.78472077846527)
--(axis cs:0.828282833099365,-1.88052749633789)
--(axis cs:0.838383793830872,-1.95784795284271)
--(axis cs:0.848484814167023,-2.01791906356812)
--(axis cs:0.858585834503174,-2.06451559066772)
--(axis cs:0.868686854839325,-2.09642744064331)
--(axis cs:0.878787875175476,-2.11618375778198)
--(axis cs:0.888888895511627,-2.12604594230652)
--(axis cs:0.898989915847778,-2.12823176383972)
--(axis cs:0.909090936183929,-2.12404990196228)
--(axis cs:0.919191956520081,-2.11521315574646)
--(axis cs:0.929292917251587,-2.10285210609436)
--(axis cs:0.939393937587738,-2.08835196495056)
--(axis cs:0.949494957923889,-2.07121419906616)
--(axis cs:0.95959597826004,-2.05223488807678)
--(axis cs:0.969696998596191,-2.03345131874084)
--(axis cs:0.979797959327698,-2.01660680770874)
--(axis cs:0.989898979663849,-2.00282025337219)
--(axis cs:1,-1.99191737174988)
--(axis cs:1,1.91023433208466)
--(axis cs:1,1.91023433208466)
--(axis cs:0.989898979663849,1.89902853965759)
--(axis cs:0.979797959327698,1.8847815990448)
--(axis cs:0.969696998596191,1.86517632007599)
--(axis cs:0.95959597826004,1.83823275566101)
--(axis cs:0.949494957923889,1.80206739902496)
--(axis cs:0.939393937587738,1.75725138187408)
--(axis cs:0.929292917251587,1.70507323741913)
--(axis cs:0.919191956520081,1.64439260959625)
--(axis cs:0.909090936183929,1.57527410984039)
--(axis cs:0.898989915847778,1.49697411060333)
--(axis cs:0.888888895511627,1.40691816806793)
--(axis cs:0.878787875175476,1.30233263969421)
--(axis cs:0.868686854839325,1.18148338794708)
--(axis cs:0.858585834503174,1.04776549339294)
--(axis cs:0.848484814167023,0.899827718734741)
--(axis cs:0.838383793830872,0.739595293998718)
--(axis cs:0.828282833099365,0.563165307044983)
--(axis cs:0.818181812763214,0.376980721950531)
--(axis cs:0.808080792427063,0.18477275967598)
--(axis cs:0.797979831695557,-0.018585417419672)
--(axis cs:0.787878811359406,-0.225396454334259)
--(axis cs:0.777777791023254,-0.430193483829498)
--(axis cs:0.767676770687103,-0.635524332523346)
--(axis cs:0.757575750350952,-0.801201283931732)
--(axis cs:0.747474789619446,-0.634715914726257)
--(axis cs:0.737373769283295,-0.43555822968483)
--(axis cs:0.727272748947144,-0.240364700555801)
--(axis cs:0.717171728610992,-0.0404745638370514)
--(axis cs:0.707070708274841,0.156209975481033)
--(axis cs:0.69696968793869,0.340395987033844)
--(axis cs:0.686868667602539,0.505993247032166)
--(axis cs:0.676767706871033,0.650733530521393)
--(axis cs:0.666666686534882,0.774729490280151)
--(axis cs:0.65656566619873,0.872848153114319)
--(axis cs:0.646464645862579,0.960156440734863)
--(axis cs:0.636363625526428,1.02971410751343)
--(axis cs:0.626262605190277,1.05283772945404)
--(axis cs:0.616161584854126,1.05243539810181)
--(axis cs:0.606060564517975,1.04862987995148)
--(axis cs:0.595959544181824,1.03306555747986)
--(axis cs:0.585858583450317,0.981883108615875)
--(axis cs:0.575757563114166,0.898656666278839)
--(axis cs:0.565656542778015,0.791954576969147)
--(axis cs:0.555555582046509,0.660297572612762)
--(axis cs:0.545454561710358,0.509933710098267)
--(axis cs:0.535353541374207,0.357499241828918)
--(axis cs:0.525252521038055,0.192929863929749)
--(axis cs:0.515151500701904,0.026222575455904)
--(axis cs:0.505050539970398,-0.111870504915714)
--(axis cs:0.494949519634247,0.0747952833771706)
--(axis cs:0.484848499298096,0.297604501247406)
--(axis cs:0.474747478961945,0.528253197669983)
--(axis cs:0.464646458625793,0.749299228191376)
--(axis cs:0.454545438289642,0.972333192825317)
--(axis cs:0.444444447755814,1.18762814998627)
--(axis cs:0.434343427419662,1.36597692966461)
--(axis cs:0.424242407083511,1.52292144298553)
--(axis cs:0.414141416549683,1.69837033748627)
--(axis cs:0.404040396213531,1.87235939502716)
--(axis cs:0.39393937587738,2.0366063117981)
--(axis cs:0.383838385343552,2.17166018486023)
--(axis cs:0.373737365007401,2.28205156326294)
--(axis cs:0.363636374473572,2.37658166885376)
--(axis cs:0.353535354137421,2.44508361816406)
--(axis cs:0.34343433380127,2.48610639572144)
--(axis cs:0.333333343267441,2.50590538978577)
--(axis cs:0.32323232293129,2.49697709083557)
--(axis cs:0.313131332397461,2.45892453193665)
--(axis cs:0.30303031206131,2.4023060798645)
--(axis cs:0.292929291725159,2.32116603851318)
--(axis cs:0.28282830119133,2.20727109909058)
--(axis cs:0.272727280855179,2.07056021690369)
--(axis cs:0.262626260519028,1.90620589256287)
--(axis cs:0.252525269985199,1.77389574050903)
--(axis cs:0.242424249649048,1.91233241558075)
--(axis cs:0.232323229312897,2.08235335350037)
--(axis cs:0.222222223877907,2.22931957244873)
--(axis cs:0.212121218442917,2.35422706604004)
--(axis cs:0.202020198106766,2.44974732398987)
--(axis cs:0.191919192671776,2.53086566925049)
--(axis cs:0.181818187236786,2.59006690979004)
--(axis cs:0.171717166900635,2.61875009536743)
--(axis cs:0.161616161465645,2.61980938911438)
--(axis cs:0.151515156030655,2.61716294288635)
--(axis cs:0.141414135694504,2.58792161941528)
--(axis cs:0.131313130259514,2.51811718940735)
--(axis cs:0.121212124824524,2.43652963638306)
--(axis cs:0.111111111938953,2.34858751296997)
--(axis cs:0.101010099053383,2.23931097984314)
--(axis cs:0.0909090936183929,2.10158348083496)
--(axis cs:0.0808080807328224,1.9399881362915)
--(axis cs:0.0707070678472519,1.77408516407013)
--(axis cs:0.060606062412262,1.61782586574554)
--(axis cs:0.0505050495266914,1.45003628730774)
--(axis cs:0.0404040403664112,1.25799596309662)
--(axis cs:0.030303031206131,1.07006287574768)
--(axis cs:0.0202020201832056,0.854456007480621)
--(axis cs:0.0101010100916028,0.645568668842316)
--(axis cs:0,0.464962095022202)
--cycle;
\path [draw=green!50.1960784313725!black, fill=green!50.1960784313725!black, opacity=0.2]
(axis cs:0,0.401168584823608)
--(axis cs:0,0.456590533256531)
--(axis cs:0.0101010100916028,0.645147025585175)
--(axis cs:0.0202020201832056,0.855617165565491)
--(axis cs:0.030303031206131,1.0615348815918)
--(axis cs:0.0404040403664112,1.26027595996857)
--(axis cs:0.0505050495266914,1.44995176792145)
--(axis cs:0.060606062412262,1.62900018692017)
--(axis cs:0.0707070678472519,1.79618144035339)
--(axis cs:0.0808080807328224,1.95048046112061)
--(axis cs:0.0909090936183929,2.09105777740479)
--(axis cs:0.101010099053383,2.2171459197998)
--(axis cs:0.111111111938953,2.32798004150391)
--(axis cs:0.121212124824524,2.42271685600281)
--(axis cs:0.131313130259514,2.50038456916809)
--(axis cs:0.141414135694504,2.55985999107361)
--(axis cs:0.151515156030655,2.59988355636597)
--(axis cs:0.161616161465645,2.61910653114319)
--(axis cs:0.171717166900635,2.61617517471313)
--(axis cs:0.181818187236786,2.58983945846558)
--(axis cs:0.191919192671776,2.5390739440918)
--(axis cs:0.202020198106766,2.46321082115173)
--(axis cs:0.212121218442917,2.36205983161926)
--(axis cs:0.222222223877907,2.23601651191711)
--(axis cs:0.232323229312897,2.08617162704468)
--(axis cs:0.242424249649048,1.91468131542206)
--(axis cs:0.252525269985199,1.74801003932953)
--(axis cs:0.262626260519028,1.90325927734375)
--(axis cs:0.272727280855179,2.06293821334839)
--(axis cs:0.28282830119133,2.2001953125)
--(axis cs:0.292929291725159,2.31252384185791)
--(axis cs:0.30303031206131,2.39849519729614)
--(axis cs:0.313131332397461,2.45742321014404)
--(axis cs:0.32323232293129,2.48926401138306)
--(axis cs:0.333333343267441,2.49451112747192)
--(axis cs:0.34343433380127,2.47409296035767)
--(axis cs:0.353535354137421,2.42925024032593)
--(axis cs:0.363636374473572,2.36142444610596)
--(axis cs:0.373737365007401,2.27215266227722)
--(axis cs:0.383838385343552,2.16298913955688)
--(axis cs:0.39393937587738,2.0354528427124)
--(axis cs:0.404040396213531,1.89100551605225)
--(axis cs:0.414141416549683,1.73106026649475)
--(axis cs:0.424242407083511,1.55701577663422)
--(axis cs:0.434343427419662,1.37029981613159)
--(axis cs:0.444444447755814,1.17242991924286)
--(axis cs:0.454545438289642,0.965063095092773)
--(axis cs:0.464646458625793,0.750034630298615)
--(axis cs:0.474747478961945,0.52939361333847)
--(axis cs:0.484848499298096,0.305437088012695)
--(axis cs:0.494949519634247,0.0810083523392677)
--(axis cs:0.505050539970398,-0.117286570370197)
--(axis cs:0.515151500701904,0.0272237062454224)
--(axis cs:0.525252521038055,0.196040466427803)
--(axis cs:0.535353541374207,0.360769838094711)
--(axis cs:0.545454561710358,0.516877353191376)
--(axis cs:0.555555582046509,0.660513699054718)
--(axis cs:0.565656542778015,0.788279414176941)
--(axis cs:0.575757563114166,0.897237300872803)
--(axis cs:0.585858583450317,0.984941482543945)
--(axis cs:0.595959544181824,1.04945433139801)
--(axis cs:0.606060564517975,1.08934926986694)
--(axis cs:0.616161584854126,1.10369765758514)
--(axis cs:0.626262605190277,1.09205663204193)
--(axis cs:0.636363625526428,1.05444967746735)
--(axis cs:0.646464645862579,0.991351962089539)
--(axis cs:0.65656566619873,0.903680503368378)
--(axis cs:0.666666686534882,0.792787134647369)
--(axis cs:0.676767706871033,0.660451531410217)
--(axis cs:0.686868667602539,0.50886744260788)
--(axis cs:0.69696968793869,0.340630888938904)
--(axis cs:0.707070708274841,0.158703923225403)
--(axis cs:0.717171728610992,-0.0336306095123291)
--(axis cs:0.727272748947144,-0.232834577560425)
--(axis cs:0.737373769283295,-0.435153961181641)
--(axis cs:0.747474789619446,-0.636407613754272)
--(axis cs:0.757575750350952,-0.808807253837585)
--(axis cs:0.767676770687103,-0.632020771503448)
--(axis cs:0.777777791023254,-0.425070762634277)
--(axis cs:0.787878811359406,-0.215451896190643)
--(axis cs:0.797979831695557,-0.00691777467727661)
--(axis cs:0.808080792427063,0.197320461273193)
--(axis cs:0.818181812763214,0.394403755664825)
--(axis cs:0.828282833099365,0.581859648227692)
--(axis cs:0.838383793830872,0.757668972015381)
--(axis cs:0.848484814167023,0.920315086841583)
--(axis cs:0.858585834503174,1.06879091262817)
--(axis cs:0.868686854839325,1.20258140563965)
--(axis cs:0.878787875175476,1.32163786888123)
--(axis cs:0.888888895511627,1.42631030082703)
--(axis cs:0.898989915847778,1.51728224754333)
--(axis cs:0.909090936183929,1.59548902511597)
--(axis cs:0.919191956520081,1.66203367710114)
--(axis cs:0.929292917251587,1.71811616420746)
--(axis cs:0.939393937587738,1.76496148109436)
--(axis cs:0.949494957923889,1.80377101898193)
--(axis cs:0.95959597826004,1.83567714691162)
--(axis cs:0.969696998596191,1.86171853542328)
--(axis cs:0.979797959327698,1.8828272819519)
--(axis cs:0.989898979663849,1.89982235431671)
--(axis cs:1,1.91341269016266)
--(axis cs:1,-2.00120401382446)
--(axis cs:1,-2.00120401382446)
--(axis cs:0.989898979663849,-2.01140236854553)
--(axis cs:0.979797959327698,-2.02320241928101)
--(axis cs:0.969696998596191,-2.03652715682983)
--(axis cs:0.95959597826004,-2.05115675926208)
--(axis cs:0.949494957923889,-2.06668925285339)
--(axis cs:0.939393937587738,-2.08250665664673)
--(axis cs:0.929292917251587,-2.09774661064148)
--(axis cs:0.919191956520081,-2.11128568649292)
--(axis cs:0.909090936183929,-2.12173843383789)
--(axis cs:0.898989915847778,-2.12747621536255)
--(axis cs:0.888888895511627,-2.12666773796082)
--(axis cs:0.878787875175476,-2.11733913421631)
--(axis cs:0.868686854839325,-2.09745502471924)
--(axis cs:0.858585834503174,-2.0650155544281)
--(axis cs:0.848484814167023,-2.01816272735596)
--(axis cs:0.838383793830872,-1.95529413223267)
--(axis cs:0.828282833099365,-1.87516760826111)
--(axis cs:0.818181812763214,-1.7769992351532)
--(axis cs:0.808080792427063,-1.66054940223694)
--(axis cs:0.797979831695557,-1.52617788314819)
--(axis cs:0.787878811359406,-1.37487721443176)
--(axis cs:0.777777791023254,-1.20832645893097)
--(axis cs:0.767676770687103,-1.02916264533997)
--(axis cs:0.757575750350952,-0.864237427711487)
--(axis cs:0.747474789619446,-1.0324432849884)
--(axis cs:0.737373769283295,-1.21383774280548)
--(axis cs:0.727272748947144,-1.38152670860291)
--(axis cs:0.717171728610992,-1.53268384933472)
--(axis cs:0.707070708274841,-1.66530239582062)
--(axis cs:0.69696968793869,-1.77789676189423)
--(axis cs:0.686868667602539,-1.86944270133972)
--(axis cs:0.676767706871033,-1.93933141231537)
--(axis cs:0.666666686534882,-1.98730540275574)
--(axis cs:0.65656566619873,-2.01338601112366)
--(axis cs:0.646464645862579,-2.01779556274414)
--(axis cs:0.636363625526428,-2.00088787078857)
--(axis cs:0.626262605190277,-1.96309292316437)
--(axis cs:0.616161584854126,-1.9048844575882)
--(axis cs:0.606060564517975,-1.82676815986633)
--(axis cs:0.595959544181824,-1.72929584980011)
--(axis cs:0.585858583450317,-1.6131010055542)
--(axis cs:0.575757563114166,-1.47894024848938)
--(axis cs:0.565656542778015,-1.32775151729584)
--(axis cs:0.555555582046509,-1.16069793701172)
--(axis cs:0.545454561710358,-0.979204714298248)
--(axis cs:0.535353541374207,-0.784985899925232)
--(axis cs:0.525252521038055,-0.580085039138794)
--(axis cs:0.515151500701904,-0.367161929607391)
--(axis cs:0.505050539970398,-0.172716751694679)
--(axis cs:0.494949519634247,-0.313379615545273)
--(axis cs:0.484848499298096,-0.470684230327606)
--(axis cs:0.474747478961945,-0.616347372531891)
--(axis cs:0.464646458625793,-0.746014773845673)
--(axis cs:0.454545438289642,-0.856094002723694)
--(axis cs:0.444444447755814,-0.943513989448547)
--(axis cs:0.434343427419662,-1.00574684143066)
--(axis cs:0.424242407083511,-1.0408376455307)
--(axis cs:0.414141416549683,-1.04742503166199)
--(axis cs:0.404040396213531,-1.02474904060364)
--(axis cs:0.39393937587738,-0.972641825675964)
--(axis cs:0.383838385343552,-0.891518771648407)
--(axis cs:0.373737365007401,-0.782354891300201)
--(axis cs:0.363636374473572,-0.646669745445251)
--(axis cs:0.353535354137421,-0.486502826213837)
--(axis cs:0.34343433380127,-0.304391622543335)
--(axis cs:0.333333343267441,-0.103340148925781)
--(axis cs:0.32323232293129,0.113218426704407)
--(axis cs:0.313131332397461,0.341481447219849)
--(axis cs:0.30303031206131,0.577338576316833)
--(axis cs:0.292929291725159,0.816476047039032)
--(axis cs:0.28282830119133,1.05445957183838)
--(axis cs:0.272727280855179,1.28681969642639)
--(axis cs:0.262626260519028,1.50887608528137)
--(axis cs:0.252525269985199,1.69257986545563)
--(axis cs:0.242424249649048,1.52029573917389)
--(axis cs:0.232323229312897,1.31004977226257)
--(axis cs:0.222222223877907,1.09025931358337)
--(axis cs:0.212121218442917,0.865980207920074)
--(axis cs:0.202020198106766,0.642000734806061)
--(axis cs:0.191919192671776,0.423041462898254)
--(axis cs:0.181818187236786,0.213659644126892)
--(axis cs:0.171717166900635,0.0181325674057007)
--(axis cs:0.161616161465645,-0.159645795822144)
--(axis cs:0.151515156030655,-0.316236138343811)
--(axis cs:0.141414135694504,-0.448723554611206)
--(axis cs:0.131313130259514,-0.554766714572906)
--(axis cs:0.121212124824524,-0.632621049880981)
--(axis cs:0.111111111938953,-0.681168913841248)
--(axis cs:0.101010099053383,-0.699923396110535)
--(axis cs:0.0909090936183929,-0.689038872718811)
--(axis cs:0.0808080807328224,-0.649301707744598)
--(axis cs:0.0707070678472519,-0.582131624221802)
--(axis cs:0.060606062412262,-0.489528477191925)
--(axis cs:0.0505050495266914,-0.374060869216919)
--(axis cs:0.0404040403664112,-0.23877477645874)
--(axis cs:0.030303031206131,-0.087167501449585)
--(axis cs:0.0202020201832056,0.0769392848014832)
--(axis cs:0.0101010100916028,0.249085947871208)
--(axis cs:0,0.401168584823608)
--cycle;
\addplot [semithick, black, mark=*, mark size=3, mark options={solid}, only marks]
table {%
0 0.42891538143158
0.252525269985199 1.72046613693237
0.505050539970398 -0.145019799470901
0.757575750350952 -0.836605310440063
};
\addplot [semithick, blue]
table {%
0 0.429002434015274
0.0101010100916028 0.449568629264832
0.0202020201832056 0.466691017150879
0.030303031206131 0.492810070514679
0.0404040403664112 0.51272714138031
0.0505050495266914 0.531257390975952
0.060606062412262 0.556295394897461
0.0707070678472519 0.600713908672333
0.0808080807328224 0.665284812450409
0.0909090936183929 0.726065814495087
0.101010099053383 0.774799704551697
0.111111111938953 0.823861718177795
0.121212124824524 0.891816556453705
0.131313130259514 0.987799167633057
0.141414135694504 1.08036923408508
0.151515156030655 1.16028547286987
0.161616161465645 1.22470355033875
0.171717166900635 1.31197154521942
0.181818187236786 1.41286015510559
0.191919192671776 1.49680757522583
0.202020198106766 1.55821013450623
0.212121218442917 1.60935091972351
0.222222223877907 1.65796196460724
0.232323229312897 1.69381618499756
0.242424249649048 1.7143886089325
0.252525269985199 1.72138953208923
0.262626260519028 1.70866513252258
0.272727280855179 1.68004441261292
0.28282830119133 1.63407242298126
0.292929291725159 1.56972742080688
0.30303031206131 1.49287414550781
0.313131332397461 1.40419781208038
0.32323232293129 1.30894672870636
0.333333343267441 1.20494246482849
0.34343433380127 1.09350109100342
0.353535354137421 0.978851199150085
0.363636374473572 0.852492690086365
0.373737365007401 0.73059868812561
0.383838385343552 0.621208071708679
0.39393937587738 0.504603922367096
0.404040396213531 0.393550843000412
0.414141416549683 0.293620675802231
0.424242407083511 0.211314737796783
0.434343427419662 0.154186218976974
0.444444447755814 0.103725463151932
0.454545438289642 0.0511239841580391
0.464646458625793 -0.00171189161483198
0.474747478961945 -0.0473881289362907
0.484848499298096 -0.0878624394536018
0.494949519634247 -0.118413135409355
0.505050539970398 -0.14593343436718
0.515151500701904 -0.170164078474045
0.525252521038055 -0.194224715232849
0.535353541374207 -0.213615581393242
0.545454561710358 -0.236107140779495
0.555555582046509 -0.255205810070038
0.565656542778015 -0.272782981395721
0.575757563114166 -0.292775005102158
0.585858583450317 -0.319390565156937
0.595959544181824 -0.354456424713135
0.606060564517975 -0.392676711082458
0.616161584854126 -0.425936698913574
0.626262605190277 -0.454382687807083
0.636363625526428 -0.483855485916138
0.646464645862579 -0.523193001747131
0.65656566619873 -0.565433025360107
0.666666686534882 -0.603338956832886
0.676767706871033 -0.6394904255867
0.686868667602539 -0.678069472312927
0.69696968793869 -0.716697037220001
0.707070708274841 -0.752980828285217
0.717171728610992 -0.785297572612762
0.727272748947144 -0.806873083114624
0.737373769283295 -0.825129270553589
0.747474789619446 -0.833795845508575
0.757575750350952 -0.838339805603027
0.767676770687103 -0.831737875938416
0.777777791023254 -0.818080961704254
0.787878811359406 -0.796720087528229
0.797979831695557 -0.771999478340149
0.808080792427063 -0.735652446746826
0.818181812763214 -0.695408821105957
0.828282833099365 -0.652559041976929
0.838383793830872 -0.606214344501495
0.848484814167023 -0.556795120239258
0.858585834503174 -0.505458772182465
0.868686854839325 -0.45381098985672
0.878787875175476 -0.403565585613251
0.888888895511627 -0.355746448040009
0.898989915847778 -0.310498654842377
0.909090936183929 -0.267804741859436
0.919191956520081 -0.228074938058853
0.929292917251587 -0.19177633523941
0.939393937587738 -0.158764868974686
0.949494957923889 -0.128859519958496
0.95959597826004 -0.102910496294498
0.969696998596191 -0.0816257894039154
0.979797959327698 -0.0647270381450653
0.989898979663849 -0.0516116879880428
1 -0.0410993248224258
};
\addplot [semithick, green!50.1960784313725!black]
table {%
0 0.42887955904007
0.0101010100916028 0.447116494178772
0.0202020201832056 0.466278225183487
0.030303031206131 0.487183690071106
0.0404040403664112 0.510750591754913
0.0505050495266914 0.537945449352264
0.060606062412262 0.569735825061798
0.0707070678472519 0.607024908065796
0.0808080807328224 0.650589406490326
0.0909090936183929 0.701009392738342
0.101010099053383 0.75861120223999
0.111111111938953 0.823405623435974
0.121212124824524 0.895047903060913
0.131313130259514 0.97280889749527
0.141414135694504 1.0555682182312
0.151515156030655 1.14182376861572
0.161616161465645 1.22973036766052
0.171717166900635 1.31715393066406
0.181818187236786 1.40174949169159
0.191919192671776 1.48105764389038
0.202020198106766 1.55260574817657
0.212121218442917 1.61401998996735
0.222222223877907 1.66313791275024
0.232323229312897 1.69811069965363
0.242424249649048 1.71748852729797
0.252525269985199 1.72029495239258
0.262626260519028 1.70606768131256
0.272727280855179 1.67487895488739
0.28282830119133 1.62732744216919
0.292929291725159 1.56449997425079
0.30303031206131 1.48791682720184
0.313131332397461 1.39945232868195
0.32323232293129 1.30124115943909
0.333333343267441 1.19558548927307
0.34343433380127 1.08485066890717
0.353535354137421 0.971373736858368
0.363636374473572 0.857377290725708
0.373737365007401 0.744898855686188
0.383838385343552 0.635735213756561
0.39393937587738 0.531405568122864
0.404040396213531 0.433128178119659
0.414141416549683 0.341817587614059
0.424242407083511 0.25808909535408
0.434343427419662 0.182276472449303
0.444444447755814 0.114457994699478
0.454545438289642 0.0544845759868622
0.464646458625793 0.0020099300891161
0.474747478961945 -0.0434768833220005
0.484848499298096 -0.0826235860586166
0.494949519634247 -0.116185627877712
0.505050539970398 -0.145001664757729
0.515151500701904 -0.169969111680984
0.525252521038055 -0.192022278904915
0.535353541374207 -0.212108045816422
0.545454561710358 -0.231163665652275
0.555555582046509 -0.250092089176178
0.565656542778015 -0.269736051559448
0.575757563114166 -0.290851473808289
0.585858583450317 -0.314079761505127
0.595959544181824 -0.339920729398727
0.606060564517975 -0.368709444999695
0.616161584854126 -0.400593400001526
0.626262605190277 -0.435518115758896
0.636363625526428 -0.473219037055969
0.646464645862579 -0.513221859931946
0.65656566619873 -0.554852783679962
0.666666686534882 -0.597259104251862
0.676767706871033 -0.639439940452576
0.686868667602539 -0.680287659168243
0.69696968793869 -0.718632936477661
0.707070708274841 -0.753299236297607
0.717171728610992 -0.783157229423523
0.727272748947144 -0.807180643081665
0.737373769283295 -0.824495851993561
0.747474789619446 -0.834425449371338
0.757575750350952 -0.836522340774536
0.767676770687103 -0.830591678619385
0.777777791023254 -0.816698610782623
0.787878811359406 -0.795164585113525
0.797979831695557 -0.766547799110413
0.808080792427063 -0.731614470481873
0.818181812763214 -0.691297709941864
0.828282833099365 -0.646653950214386
0.838383793830872 -0.598812580108643
0.848484814167023 -0.548923790454865
0.858585834503174 -0.498112380504608
0.868686854839325 -0.447436809539795
0.878787875175476 -0.397850662469864
0.888888895511627 -0.350178778171539
0.898989915847778 -0.305097043514252
0.909090936183929 -0.26312467455864
0.919191956520081 -0.224625989794731
0.929292917251587 -0.189815253019333
0.939393937587738 -0.158772617578506
0.949494957923889 -0.131459057331085
0.95959597826004 -0.107739813625813
0.969696998596191 -0.0874043554067612
0.979797959327698 -0.070187583565712
0.989898979663849 -0.0557899922132492
1 -0.0438957214355469
};
\end{axis}
\end{tikzpicture}
% This file was created by tikzplotlib v0.9.9.
\begin{tikzpicture}
\begin{axis}[
xlabel=x,
tick align=outside,
tick pos=left,
x grid style={white!69.0196078431373!black},
xmin=-0.05, xmax=1.05,
xtick style={color=black},
y grid style={white!69.0196078431373!black},
ymin=-2.16745010018349, ymax=3.12008813023567,
ytick style={color=black}
]
\path [draw=blue, fill=blue, opacity=0.2]
(axis cs:0,1.98500108718872)
--(axis cs:0,-1.92710745334625)
--(axis cs:0.0101010100916028,-1.91843044757843)
--(axis cs:0.0202020201832056,-1.90787065029144)
--(axis cs:0.030303031206131,-1.89372849464417)
--(axis cs:0.0404040403664112,-1.87335824966431)
--(axis cs:0.0505050495266914,-1.84506177902222)
--(axis cs:0.060606062412262,-1.80905723571777)
--(axis cs:0.0707070678472519,-1.7683961391449)
--(axis cs:0.0808080807328224,-1.72120869159698)
--(axis cs:0.0909090936183929,-1.66169774532318)
--(axis cs:0.101010099053383,-1.58740091323853)
--(axis cs:0.111111111938953,-1.50508248806)
--(axis cs:0.121212124824524,-1.41461455821991)
--(axis cs:0.131313130259514,-1.31361186504364)
--(axis cs:0.141414135694504,-1.19883632659912)
--(axis cs:0.151515156030655,-1.06410658359528)
--(axis cs:0.161616161465645,-0.913222432136536)
--(axis cs:0.171717166900635,-0.749062836170197)
--(axis cs:0.181818187236786,-0.569547057151794)
--(axis cs:0.191919192671776,-0.382551908493042)
--(axis cs:0.202020198106766,-0.189664423465729)
--(axis cs:0.212121218442917,0.011099249124527)
--(axis cs:0.222222223877907,0.226816698908806)
--(axis cs:0.232323229312897,0.432996988296509)
--(axis cs:0.242424249649048,0.634175658226013)
--(axis cs:0.252525269985199,0.800479888916016)
--(axis cs:0.262626260519028,0.636476039886475)
--(axis cs:0.272727280855179,0.431088596582413)
--(axis cs:0.28282830119133,0.227119952440262)
--(axis cs:0.292929291725159,0.0218948982656002)
--(axis cs:0.30303031206131,-0.178923323750496)
--(axis cs:0.313131332397461,-0.376851558685303)
--(axis cs:0.32323232293129,-0.567261397838593)
--(axis cs:0.333333343267441,-0.745040237903595)
--(axis cs:0.34343433380127,-0.904806196689606)
--(axis cs:0.353535354137421,-1.05080592632294)
--(axis cs:0.363636374473572,-1.18501901626587)
--(axis cs:0.373737365007401,-1.30529713630676)
--(axis cs:0.383838385343552,-1.41746151447296)
--(axis cs:0.39393937587738,-1.51297807693481)
--(axis cs:0.404040396213531,-1.5883070230484)
--(axis cs:0.414141416549683,-1.64914977550507)
--(axis cs:0.424242407083511,-1.69944798946381)
--(axis cs:0.434343427419662,-1.74102914333344)
--(axis cs:0.444444447755814,-1.77392423152924)
--(axis cs:0.454545438289642,-1.79622673988342)
--(axis cs:0.464646458625793,-1.8147212266922)
--(axis cs:0.474747478961945,-1.83138263225555)
--(axis cs:0.484848499298096,-1.84374535083771)
--(axis cs:0.494949519634247,-1.84802341461182)
--(axis cs:0.505050539970398,-1.84121942520142)
--(axis cs:0.515151500701904,-1.82212865352631)
--(axis cs:0.525252521038055,-1.79447841644287)
--(axis cs:0.535353541374207,-1.76881587505341)
--(axis cs:0.545454561710358,-1.74074363708496)
--(axis cs:0.555555582046509,-1.69804894924164)
--(axis cs:0.565656542778015,-1.62600779533386)
--(axis cs:0.575757563114166,-1.53296530246735)
--(axis cs:0.585858583450317,-1.44539356231689)
--(axis cs:0.595959544181824,-1.35083496570587)
--(axis cs:0.606060564517975,-1.22498393058777)
--(axis cs:0.616161584854126,-1.0719473361969)
--(axis cs:0.626262605190277,-0.896340548992157)
--(axis cs:0.636363625526428,-0.710041105747223)
--(axis cs:0.646464645862579,-0.514549314975739)
--(axis cs:0.65656566619873,-0.309899508953094)
--(axis cs:0.666666686534882,-0.0740100294351578)
--(axis cs:0.676767706871033,0.169833391904831)
--(axis cs:0.686868667602539,0.419698536396027)
--(axis cs:0.69696968793869,0.672138273715973)
--(axis cs:0.707070708274841,0.928317964076996)
--(axis cs:0.717171728610992,1.17512774467468)
--(axis cs:0.727272748947144,1.43138539791107)
--(axis cs:0.737373769283295,1.6711094379425)
--(axis cs:0.747474789619446,1.89680397510529)
--(axis cs:0.757575750350952,2.05261588096619)
--(axis cs:0.767676770687103,1.89358699321747)
--(axis cs:0.777777791023254,1.67562162876129)
--(axis cs:0.787878811359406,1.42896902561188)
--(axis cs:0.797979831695557,1.17226147651672)
--(axis cs:0.808080792427063,0.925967335700989)
--(axis cs:0.818181812763214,0.676955342292786)
--(axis cs:0.828282833099365,0.43437597155571)
--(axis cs:0.838383793830872,0.18850964307785)
--(axis cs:0.848484814167023,-0.0529684722423553)
--(axis cs:0.858585834503174,-0.290052622556686)
--(axis cs:0.868686854839325,-0.496278613805771)
--(axis cs:0.878787875175476,-0.689031422138214)
--(axis cs:0.888888895511627,-0.866735637187958)
--(axis cs:0.898989915847778,-1.02908301353455)
--(axis cs:0.909090936183929,-1.16996788978577)
--(axis cs:0.919191956520081,-1.29108393192291)
--(axis cs:0.929292917251587,-1.40031158924103)
--(axis cs:0.939393937587738,-1.49750220775604)
--(axis cs:0.949494957923889,-1.58441519737244)
--(axis cs:0.95959597826004,-1.65891087055206)
--(axis cs:0.969696998596191,-1.71931850910187)
--(axis cs:0.979797959327698,-1.76862835884094)
--(axis cs:0.989898979663849,-1.81006503105164)
--(axis cs:1,-1.84511435031891)
--(axis cs:1,2.04970598220825)
--(axis cs:1,2.04970598220825)
--(axis cs:0.989898979663849,2.07993626594543)
--(axis cs:0.979797959327698,2.1182873249054)
--(axis cs:0.969696998596191,2.16444873809814)
--(axis cs:0.95959597826004,2.21735954284668)
--(axis cs:0.949494957923889,2.27454042434692)
--(axis cs:0.939393937587738,2.33408498764038)
--(axis cs:0.929292917251587,2.39461159706116)
--(axis cs:0.919191956520081,2.45581865310669)
--(axis cs:0.909090936183929,2.52138304710388)
--(axis cs:0.898989915847778,2.58818221092224)
--(axis cs:0.888888895511627,2.65474724769592)
--(axis cs:0.878787875175476,2.71962451934814)
--(axis cs:0.868686854839325,2.77756881713867)
--(axis cs:0.858585834503174,2.82620120048523)
--(axis cs:0.848484814167023,2.85846781730652)
--(axis cs:0.838383793830872,2.87061309814453)
--(axis cs:0.828282833099365,2.85940146446228)
--(axis cs:0.818181812763214,2.82613706588745)
--(axis cs:0.808080792427063,2.77104115486145)
--(axis cs:0.797979831695557,2.70123243331909)
--(axis cs:0.787878811359406,2.59972071647644)
--(axis cs:0.777777791023254,2.46223568916321)
--(axis cs:0.767676770687103,2.30354070663452)
--(axis cs:0.757575750350952,2.165287733078)
--(axis cs:0.747474789619446,2.30963683128357)
--(axis cs:0.737373769283295,2.47782850265503)
--(axis cs:0.727272748947144,2.61131882667542)
--(axis cs:0.717171728610992,2.70228600502014)
--(axis cs:0.707070708274841,2.76338958740234)
--(axis cs:0.69696968793869,2.82027339935303)
--(axis cs:0.686868667602539,2.85828471183777)
--(axis cs:0.676767706871033,2.87668704986572)
--(axis cs:0.666666686534882,2.86823868751526)
--(axis cs:0.65656566619873,2.83794021606445)
--(axis cs:0.646464645862579,2.7862536907196)
--(axis cs:0.636363625526428,2.71716022491455)
--(axis cs:0.626262605190277,2.63588547706604)
--(axis cs:0.616161584854126,2.54988050460815)
--(axis cs:0.606060564517975,2.46271204948425)
--(axis cs:0.595959544181824,2.39025187492371)
--(axis cs:0.585858583450317,2.33558678627014)
--(axis cs:0.575757563114166,2.2866747379303)
--(axis cs:0.565656542778015,2.22834968566895)
--(axis cs:0.555555582046509,2.17148470878601)
--(axis cs:0.545454561710358,2.13398599624634)
--(axis cs:0.535353541374207,2.11118912696838)
--(axis cs:0.525252521038055,2.09175038337708)
--(axis cs:0.515151500701904,2.07117462158203)
--(axis cs:0.505050539970398,2.05747771263123)
--(axis cs:0.494949519634247,2.05258679389954)
--(axis cs:0.484848499298096,2.05643033981323)
--(axis cs:0.474747478961945,2.0670440196991)
--(axis cs:0.464646458625793,2.08023309707642)
--(axis cs:0.454545438289642,2.09176301956177)
--(axis cs:0.444444447755814,2.10444378852844)
--(axis cs:0.434343427419662,2.11742615699768)
--(axis cs:0.424242407083511,2.11942577362061)
--(axis cs:0.414141416549683,2.11741590499878)
--(axis cs:0.404040396213531,2.12085914611816)
--(axis cs:0.39393937587738,2.12505912780762)
--(axis cs:0.383838385343552,2.12435984611511)
--(axis cs:0.373737365007401,2.11491298675537)
--(axis cs:0.363636374473572,2.09559512138367)
--(axis cs:0.353535354137421,2.06394100189209)
--(axis cs:0.34343433380127,2.01600337028503)
--(axis cs:0.333333343267441,1.95046496391296)
--(axis cs:0.32323232293129,1.86820018291473)
--(axis cs:0.313131332397461,1.77208387851715)
--(axis cs:0.30303031206131,1.65686798095703)
--(axis cs:0.292929291725159,1.51379323005676)
--(axis cs:0.28282830119133,1.36814641952515)
--(axis cs:0.272727280855179,1.2164386510849)
--(axis cs:0.262626260519028,1.03931415081024)
--(axis cs:0.252525269985199,0.878695130348206)
--(axis cs:0.242424249649048,1.03784227371216)
--(axis cs:0.232323229312897,1.21392667293549)
--(axis cs:0.222222223877907,1.37174332141876)
--(axis cs:0.212121218442917,1.52050149440765)
--(axis cs:0.202020198106766,1.6555609703064)
--(axis cs:0.191919192671776,1.76137208938599)
--(axis cs:0.181818187236786,1.85245549678802)
--(axis cs:0.171717166900635,1.93596935272217)
--(axis cs:0.161616161465645,2.00641560554504)
--(axis cs:0.151515156030655,2.05797600746155)
--(axis cs:0.141414135694504,2.09089970588684)
--(axis cs:0.131313130259514,2.11018896102905)
--(axis cs:0.121212124824524,2.11881136894226)
--(axis cs:0.111111111938953,2.11970162391663)
--(axis cs:0.101010099053383,2.11482262611389)
--(axis cs:0.0909090936183929,2.10598278045654)
--(axis cs:0.0808080807328224,2.09379649162292)
--(axis cs:0.0707070678472519,2.08066082000732)
--(axis cs:0.060606062412262,2.06659936904907)
--(axis cs:0.0505050495266914,2.05064582824707)
--(axis cs:0.0404040403664112,2.03330206871033)
--(axis cs:0.030303031206131,2.01718163490295)
--(axis cs:0.0202020201832056,2.00390291213989)
--(axis cs:0.0101010100916028,1.99329304695129)
--(axis cs:0,1.98500108718872)
--cycle;
\path [draw=green!50.1960784313725!black, fill=green!50.1960784313725!black, opacity=0.2]
(axis cs:0,-1.92374169826508)
--(axis cs:0,1.99304831027985)
--(axis cs:0.0101010100916028,2.00180768966675)
--(axis cs:0.0202020201832056,2.0121705532074)
--(axis cs:0.030303031206131,2.02417087554932)
--(axis cs:0.0404040403664112,2.03773617744446)
--(axis cs:0.0505050495266914,2.05265116691589)
--(axis cs:0.060606062412262,2.0685179233551)
--(axis cs:0.0707070678472519,2.08472275733948)
--(axis cs:0.0808080807328224,2.10040521621704)
--(axis cs:0.0909090936183929,2.11444330215454)
--(axis cs:0.101010099053383,2.12545108795166)
--(axis cs:0.111111111938953,2.13179636001587)
--(axis cs:0.121212124824524,2.13164162635803)
--(axis cs:0.131313130259514,2.12300324440002)
--(axis cs:0.141414135694504,2.10383105278015)
--(axis cs:0.151515156030655,2.07210683822632)
--(axis cs:0.161616161465645,2.02594709396362)
--(axis cs:0.171717166900635,1.96371579170227)
--(axis cs:0.181818187236786,1.88413190841675)
--(axis cs:0.191919192671776,1.78636121749878)
--(axis cs:0.202020198106766,1.67009937763214)
--(axis cs:0.212121218442917,1.53562712669373)
--(axis cs:0.222222223877907,1.38384485244751)
--(axis cs:0.232323229312897,1.21630311012268)
--(axis cs:0.242424249649048,1.03549635410309)
--(axis cs:0.252525269985199,0.868089973926544)
--(axis cs:0.262626260519028,1.03550279140472)
--(axis cs:0.272727280855179,1.21631956100464)
--(axis cs:0.28282830119133,1.38386929035187)
--(axis cs:0.292929291725159,1.53567051887512)
--(axis cs:0.30303031206131,1.67016887664795)
--(axis cs:0.313131332397461,1.78646898269653)
--(axis cs:0.32323232293129,1.88430035114288)
--(axis cs:0.333333343267441,1.96397638320923)
--(axis cs:0.34343433380127,2.02634477615356)
--(axis cs:0.353535354137421,2.07270812988281)
--(axis cs:0.363636374473572,2.10473108291626)
--(axis cs:0.373737365007401,2.1243360042572)
--(axis cs:0.383838385343552,2.13359594345093)
--(axis cs:0.39393937587738,2.13463187217712)
--(axis cs:0.404040396213531,2.12952280044556)
--(axis cs:0.414141416549683,2.12023115158081)
--(axis cs:0.424242407083511,2.10854721069336)
--(axis cs:0.434343427419662,2.0960578918457)
--(axis cs:0.444444447755814,2.08413481712341)
--(axis cs:0.454545438289642,2.07394218444824)
--(axis cs:0.464646458625793,2.06645703315735)
--(axis cs:0.474747478961945,2.06250143051147)
--(axis cs:0.484848499298096,2.06277561187744)
--(axis cs:0.494949519634247,2.06788921356201)
--(axis cs:0.505050539970398,2.0783851146698)
--(axis cs:0.515151500701904,2.09474968910217)
--(axis cs:0.525252521038055,2.1174111366272)
--(axis cs:0.535353541374207,2.1467182636261)
--(axis cs:0.545454561710358,2.18290662765503)
--(axis cs:0.555555582046509,2.22605013847351)
--(axis cs:0.565656542778015,2.27600240707397)
--(axis cs:0.575757563114166,2.33233642578125)
--(axis cs:0.585858583450317,2.3942813873291)
--(axis cs:0.595959544181824,2.46067547798157)
--(axis cs:0.606060564517975,2.52993059158325)
--(axis cs:0.616161584854126,2.60002040863037)
--(axis cs:0.626262605190277,2.66849517822266)
--(axis cs:0.636363625526428,2.73253059387207)
--(axis cs:0.646464645862579,2.7890043258667)
--(axis cs:0.65656566619873,2.83460092544556)
--(axis cs:0.666666686534882,2.86594390869141)
--(axis cs:0.676767706871033,2.87974548339844)
--(axis cs:0.686868667602539,2.87296533584595)
--(axis cs:0.69696968793869,2.84296846389771)
--(axis cs:0.707070708274841,2.787677526474)
--(axis cs:0.717171728610992,2.70570254325867)
--(axis cs:0.727272748947144,2.59645080566406)
--(axis cs:0.737373769283295,2.46023440361023)
--(axis cs:0.747474789619446,2.2986056804657)
--(axis cs:0.757575750350952,2.13765525817871)
--(axis cs:0.767676770687103,2.29860305786133)
--(axis cs:0.777777791023254,2.46023035049438)
--(axis cs:0.787878811359406,2.59644150733948)
--(axis cs:0.797979831695557,2.70568633079529)
--(axis cs:0.808080792427063,2.78765058517456)
--(axis cs:0.818181812763214,2.84292602539062)
--(axis cs:0.828282833099365,2.87289810180664)
--(axis cs:0.838383793830872,2.87964177131653)
--(axis cs:0.848484814167023,2.86578559875488)
--(axis cs:0.858585834503174,2.83436107635498)
--(axis cs:0.868686854839325,2.78864598274231)
--(axis cs:0.878787875175476,2.73199963569641)
--(axis cs:0.888888895511627,2.66771697998047)
--(axis cs:0.898989915847778,2.59889221191406)
--(axis cs:0.909090936183929,2.52831125259399)
--(axis cs:0.919191956520081,2.45837497711182)
--(axis cs:0.929292917251587,2.39104700088501)
--(axis cs:0.939393937587738,2.32783913612366)
--(axis cs:0.949494957923889,2.26981520652771)
--(axis cs:0.95959597826004,2.2176308631897)
--(axis cs:0.969696998596191,2.1715784072876)
--(axis cs:0.979797959327698,2.13165020942688)
--(axis cs:0.989898979663849,2.09760308265686)
--(axis cs:1,2.06902670860291)
--(axis cs:1,-1.84559941291809)
--(axis cs:1,-1.84559941291809)
--(axis cs:0.989898979663849,-1.813636302948)
--(axis cs:0.979797959327698,-1.77440333366394)
--(axis cs:0.969696998596191,-1.72670435905457)
--(axis cs:0.95959597826004,-1.66925930976868)
--(axis cs:0.949494957923889,-1.60072898864746)
--(axis cs:0.939393937587738,-1.51975178718567)
--(axis cs:0.929292917251587,-1.42499113082886)
--(axis cs:0.919191956520081,-1.31519055366516)
--(axis cs:0.909090936183929,-1.1892557144165)
--(axis cs:0.898989915847778,-1.04632580280304)
--(axis cs:0.888888895511627,-0.885871648788452)
--(axis cs:0.878787875175476,-0.707773923873901)
--(axis cs:0.868686854839325,-0.512410163879395)
--(axis cs:0.858585834503174,-0.300726056098938)
--(axis cs:0.848484814167023,-0.0742698907852173)
--(axis cs:0.838383793830872,0.164776563644409)
--(axis cs:0.828282833099365,0.413631319999695)
--(axis cs:0.818181812763214,0.668958306312561)
--(axis cs:0.808080792427063,0.926944732666016)
--(axis cs:0.797979831695557,1.183434009552)
--(axis cs:0.787878811359406,1.43407225608826)
--(axis cs:0.777777791023254,1.67441618442535)
--(axis cs:0.767676770687103,1.89981114864349)
--(axis cs:0.757575750350952,2.08223342895508)
--(axis cs:0.747474789619446,1.89981377124786)
--(axis cs:0.737373769283295,1.67442440986633)
--(axis cs:0.727272748947144,1.43408489227295)
--(axis cs:0.717171728610992,1.18345379829407)
--(axis cs:0.707070708274841,0.926973342895508)
--(axis cs:0.69696968793869,0.669003129005432)
--(axis cs:0.686868667602539,0.413701891899109)
--(axis cs:0.676767706871033,0.164881825447083)
--(axis cs:0.666666686534882,-0.0741101503372192)
--(axis cs:0.65656566619873,-0.300484180450439)
--(axis cs:0.646464645862579,-0.51205050945282)
--(axis cs:0.636363625526428,-0.70724093914032)
--(axis cs:0.626262605190277,-0.885091423988342)
--(axis cs:0.616161584854126,-1.04519402980804)
--(axis cs:0.606060564517975,-1.18762826919556)
--(axis cs:0.595959544181824,-1.31287503242493)
--(axis cs:0.585858583450317,-1.42172610759735)
--(axis cs:0.575757563114166,-1.51519668102264)
--(axis cs:0.565656542778015,-1.59443211555481)
--(axis cs:0.555555582046509,-1.66063690185547)
--(axis cs:0.545454561710358,-1.71500623226166)
--(axis cs:0.535353541374207,-1.75867486000061)
--(axis cs:0.525252521038055,-1.79267251491547)
--(axis cs:0.515151500701904,-1.8178927898407)
--(axis cs:0.505050539970398,-1.83506846427917)
--(axis cs:0.494949519634247,-1.84475314617157)
--(axis cs:0.484848499298096,-1.84730792045593)
--(axis cs:0.474747478961945,-1.84289169311523)
--(axis cs:0.464646458625793,-1.83145594596863)
--(axis cs:0.454545438289642,-1.81274473667145)
--(axis cs:0.444444447755814,-1.78629970550537)
--(axis cs:0.434343427419662,-1.7514750957489)
--(axis cs:0.424242407083511,-1.70746040344238)
--(axis cs:0.414141416549683,-1.65331900119781)
--(axis cs:0.404040396213531,-1.58803582191467)
--(axis cs:0.39393937587738,-1.51058197021484)
--(axis cs:0.383838385343552,-1.41999018192291)
--(axis cs:0.373737365007401,-1.31543469429016)
--(axis cs:0.363636374473572,-1.19632351398468)
--(axis cs:0.353535354137421,-1.06237745285034)
--(axis cs:0.34343433380127,-0.913710415363312)
--(axis cs:0.333333343267441,-0.750886917114258)
--(axis cs:0.32323232293129,-0.574962973594666)
--(axis cs:0.313131332397461,-0.387496471405029)
--(axis cs:0.30303031206131,-0.190538167953491)
--(axis cs:0.292929291725159,0.0134218335151672)
--(axis cs:0.28282830119133,0.221506357192993)
--(axis cs:0.272727280855179,0.430509626865387)
--(axis cs:0.262626260519028,0.63671088218689)
--(axis cs:0.252525269985199,0.812668025493622)
--(axis cs:0.242424249649048,0.636704385280609)
--(axis cs:0.232323229312897,0.430497705936432)
--(axis cs:0.222222223877907,0.221478879451752)
--(axis cs:0.212121218442917,0.0133807063102722)
--(axis cs:0.202020198106766,-0.19060480594635)
--(axis cs:0.191919192671776,-0.387605249881744)
--(axis cs:0.181818187236786,-0.575132429599762)
--(axis cs:0.171717166900635,-0.751147508621216)
--(axis cs:0.161616161465645,-0.91410905122757)
--(axis cs:0.151515156030655,-1.06298005580902)
--(axis cs:0.141414135694504,-1.1972244977951)
--(axis cs:0.131313130259514,-1.31676971912384)
--(axis cs:0.121212124824524,-1.42194652557373)
--(axis cs:0.111111111938953,-1.51342177391052)
--(axis cs:0.101010099053383,-1.59211587905884)
--(axis cs:0.0909090936183929,-1.65912234783173)
--(axis cs:0.0808080807328224,-1.71563231945038)
--(axis cs:0.0707070678472519,-1.76286816596985)
--(axis cs:0.060606062412262,-1.80202627182007)
--(axis cs:0.0505050495266914,-1.83423900604248)
--(axis cs:0.0404040403664112,-1.86054658889771)
--(axis cs:0.030303031206131,-1.88188254833221)
--(axis cs:0.0202020201832056,-1.89906895160675)
--(axis cs:0.0101010100916028,-1.91281843185425)
--(axis cs:0,-1.92374169826508)
--cycle;
\addplot [semithick, black, mark=*, mark size=3, mark options={solid}, only marks]
table {%
0.252525269985199 0.840462923049927
0.757575750350952 2.11015510559082
};
\addplot [semithick, blue]
table {%
0 0.0308567881584167
0.0101010100916028 0.0393742509186268
0.0202020201832056 0.0498128533363342
0.030303031206131 0.0631895810365677
0.0404040403664112 0.080877885222435
0.0505050495266914 0.103159934282303
0.060606062412262 0.128578275442123
0.0707070678472519 0.155851751565933
0.0808080807328224 0.185887217521667
0.0909090936183929 0.221555009484291
0.101010099053383 0.262855350971222
0.111111111938953 0.306741744279861
0.121212124824524 0.352196574211121
0.131313130259514 0.399467825889587
0.141414135694504 0.448443531990051
0.151515156030655 0.498202502727509
0.161616161465645 0.547618985176086
0.171717166900635 0.596477270126343
0.181818187236786 0.64507269859314
0.191919192671776 0.692316830158234
0.202020198106766 0.734460711479187
0.212121218442917 0.769253015518188
0.222222223877907 0.800988078117371
0.232323229312897 0.822312235832214
0.242424249649048 0.836183667182922
0.252525269985199 0.839445233345032
0.262626260519028 0.837767004966736
0.272727280855179 0.823115706443787
0.28282830119133 0.801139891147614
0.292929291725159 0.772008717060089
0.30303031206131 0.742075562477112
0.313131332397461 0.70261937379837
0.32323232293129 0.655998826026917
0.333333343267441 0.607333898544312
0.34343433380127 0.558225512504578
0.353535354137421 0.508806586265564
0.363636374473572 0.45853316783905
0.373737365007401 0.406327277421951
0.383838385343552 0.353450775146484
0.39393937587738 0.305453151464462
0.404040396213531 0.265832602977753
0.414141416549683 0.234343737363815
0.424242407083511 0.210796415805817
0.434343427419662 0.189303502440453
0.444444447755814 0.167154580354691
0.454545438289642 0.150646448135376
0.464646458625793 0.136024415493011
0.474747478961945 0.121325835585594
0.484848499298096 0.110021524131298
0.494949519634247 0.106431640684605
0.505050539970398 0.112946219742298
0.515151500701904 0.130984932184219
0.525252521038055 0.157817229628563
0.535353541374207 0.182506293058395
0.545454561710358 0.209796115756035
0.555555582046509 0.251445412635803
0.565656542778015 0.317189902067184
0.575757563114166 0.393440157175064
0.585858583450317 0.462025463581085
0.595959544181824 0.536317706108093
0.606060564517975 0.634371697902679
0.616161584854126 0.752949178218842
0.626262605190277 0.880801975727081
0.636363625526428 1.01347148418427
0.646464645862579 1.14765751361847
0.65656566619873 1.28319263458252
0.666666686534882 1.41874361038208
0.676767706871033 1.54730010032654
0.686868667602539 1.66267800331116
0.69696968793869 1.76654243469238
0.707070708274841 1.86280965805054
0.717171728610992 1.95162105560303
0.727272748947144 2.023921251297
0.737373769283295 2.07430911064148
0.747474789619446 2.10313844680786
0.757575750350952 2.10999274253845
0.767676770687103 2.09903764724731
0.777777791023254 2.06746387481689
0.787878811359406 2.01587343215942
0.797979831695557 1.94715082645416
0.808080792427063 1.8631899356842
0.818181812763214 1.76966428756714
0.828282833099365 1.6663510799408
0.838383793830872 1.54981851577759
0.848484814167023 1.42020428180695
0.858585834503174 1.28500366210938
0.868686854839325 1.15176200866699
0.878787875175476 1.02355313301086
0.888888895511627 0.90151047706604
0.898989915847778 0.786918163299561
0.909090936183929 0.681069135665894
0.919191956520081 0.584406018257141
0.929292917251587 0.49615490436554
0.939393937587738 0.414726346731186
0.949494957923889 0.339198648929596
0.95959597826004 0.271315425634384
0.969696998596191 0.213868230581284
0.979797959327698 0.167203456163406
0.989898979663849 0.129614263772964
1 0.0993368923664093
};
\addplot [semithick, green!50.1960784313725!black]
table {%
0 0.034653302282095
0.0101010100916028 0.0444946177303791
0.0202020201832056 0.0565508417785168
0.030303031206131 0.0711441785097122
0.0404040403664112 0.0885948464274406
0.0505050495266914 0.109206035733223
0.060606062412262 0.133245825767517
0.0707070678472519 0.160927295684814
0.0808080807328224 0.192386478185654
0.0909090936183929 0.227660521864891
0.101010099053383 0.266667574644089
0.111111111938953 0.309187322854996
0.121212124824524 0.354847580194473
0.131313130259514 0.40311673283577
0.141414135694504 0.453303307294846
0.151515156030655 0.504563331604004
0.161616161465645 0.555918991565704
0.171717166900635 0.606284141540527
0.181818187236786 0.654499709606171
0.191919192671776 0.699377954006195
0.202020198106766 0.739747285842896
0.212121218442917 0.774503886699677
0.222222223877907 0.802661895751953
0.232323229312897 0.823400378227234
0.242424249649048 0.836100339889526
0.252525269985199 0.840378999710083
0.262626260519028 0.836106836795807
0.272727280855179 0.823414623737335
0.28282830119133 0.80268782377243
0.292929291725159 0.774546205997467
0.30303031206131 0.739815354347229
0.313131332397461 0.699486255645752
0.32323232293129 0.654668688774109
0.333333343267441 0.606544733047485
0.34343433380127 0.556317150592804
0.353535354137421 0.505165278911591
0.363636374473572 0.454203814268112
0.373737365007401 0.404450595378876
0.383838385343552 0.356802850961685
0.39393937587738 0.312024891376495
0.404040396213531 0.270743548870087
0.414141416549683 0.233456045389175
0.424242407083511 0.200543373823166
0.434343427419662 0.172291427850723
0.444444447755814 0.14891754090786
0.454545438289642 0.130598768591881
0.464646458625793 0.117500565946102
0.474747478961945 0.109804928302765
0.484848499298096 0.107733845710754
0.494949519634247 0.111568078398705
0.505050539970398 0.121658369898796
0.515151500701904 0.138428434729576
0.525252521038055 0.162369251251221
0.535353541374207 0.194021642208099
0.545454561710358 0.233950227499008
0.555555582046509 0.282706677913666
0.565656542778015 0.340785145759583
0.575757563114166 0.408569812774658
0.585858583450317 0.48627769947052
0.595959544181824 0.573900163173676
0.606060564517975 0.671151220798492
0.616161584854126 0.777413129806519
0.626262605190277 0.891701817512512
0.636363625526428 1.01264488697052
0.646464645862579 1.13847696781158
0.65656566619873 1.26705837249756
0.666666686534882 1.39591693878174
0.676767706871033 1.5223137140274
0.686868667602539 1.64333355426788
0.69696968793869 1.75598585605621
0.707070708274841 1.85732543468475
0.717171728610992 1.94457817077637
0.727272748947144 2.01526784896851
0.737373769283295 2.06732940673828
0.747474789619446 2.09920978546143
0.757575750350952 2.10994434356689
0.767676770687103 2.09920716285706
0.777777791023254 2.06732320785522
0.787878811359406 2.01525688171387
0.797979831695557 1.94456017017365
0.808080792427063 1.85729765892029
0.818181812763214 1.75594210624695
0.828282833099365 1.64326477050781
0.838383793830872 1.52220916748047
0.848484814167023 1.39575779438019
0.858585834503174 1.26681756973267
0.868686854839325 1.13811790943146
0.878787875175476 1.01211285591125
0.888888895511627 0.890922665596008
0.898989915847778 0.776283264160156
0.909090936183929 0.669527769088745
0.919191956520081 0.571592152118683
0.929292917251587 0.483027964830399
0.939393937587738 0.404043644666672
0.949494957923889 0.334543168544769
0.95959597826004 0.274185717105865
0.969696998596191 0.222436994314194
0.979797959327698 0.178623408079147
0.989898979663849 0.141983419656754
1 0.111713595688343
};
\end{axis}
\end{tikzpicture}
% This file was created by tikzplotlib v0.9.9.
%\documentclass{minimal}
\iffalse
\usepackage[dvipsnames]{xcolor}
\usepackage{tikz}
\usetikzlibrary{positioning,calc,backgrounds,fit,arrows.meta}
\usepackage{pgfplots}
\fi
%\begin{document}
% This file was created by tikzplotlib v0.9.9.
\begin{tikzpicture}
\begin{axis}[
width=.6\textwidth,
height=.5\textwidth,
ylabel=y,
xlabel=x,
tick align=outside,
tick pos=left,
x grid style={white!69.0196078431373!black},
xmin=-0.05, xmax=1.05,
xtick style={color=black},
y grid style={white!69.0196078431373!black},
ymin=-4, ymax=2.75804862976074,
ytick style={color=black}
]
\path [draw=blue, fill=blue, opacity=0.2]
(axis cs:0,0.48836225271225)
--(axis cs:0,0.415137112140656)
--(axis cs:0.0101010100916028,0.234063774347305)
--(axis cs:0.0202020201832056,0.00553166028112173)
--(axis cs:0.030303031206131,-0.219634175300598)
--(axis cs:0.0404040403664112,-0.441207051277161)
--(axis cs:0.0505050495266914,-0.660769522190094)
--(axis cs:0.060606062412262,-0.866878986358643)
--(axis cs:0.0707070678472519,-1.05090737342834)
--(axis cs:0.0808080807328224,-1.24810099601746)
--(axis cs:0.0909090936183929,-1.44007933139801)
--(axis cs:0.101010099053383,-1.60583162307739)
--(axis cs:0.111111111938953,-1.73368620872498)
--(axis cs:0.121212124824524,-1.83448457717896)
--(axis cs:0.131313130259514,-1.9359358549118)
--(axis cs:0.141414135694504,-2.02791571617126)
--(axis cs:0.151515156030655,-2.08280920982361)
--(axis cs:0.161616161465645,-2.10896468162537)
--(axis cs:0.171717166900635,-2.11702990531921)
--(axis cs:0.181818187236786,-2.10278820991516)
--(axis cs:0.191919192671776,-2.0577974319458)
--(axis cs:0.202020198106766,-1.97708141803741)
--(axis cs:0.212121218442917,-1.86951541900635)
--(axis cs:0.222222223877907,-1.7485511302948)
--(axis cs:0.232323229312897,-1.60003483295441)
--(axis cs:0.242424249649048,-1.4387458562851)
--(axis cs:0.252525269985199,-1.28506243228912)
--(axis cs:0.262626260519028,-1.40837919712067)
--(axis cs:0.272727280855179,-1.56119740009308)
--(axis cs:0.28282830119133,-1.68955564498901)
--(axis cs:0.292929291725159,-1.78406238555908)
--(axis cs:0.30303031206131,-1.86246454715729)
--(axis cs:0.313131332397461,-1.91156911849976)
--(axis cs:0.32323232293129,-1.91824328899384)
--(axis cs:0.333333343267441,-1.87828612327576)
--(axis cs:0.34343433380127,-1.81035017967224)
--(axis cs:0.353535354137421,-1.72607338428497)
--(axis cs:0.363636374473572,-1.60809350013733)
--(axis cs:0.373737365007401,-1.45037758350372)
--(axis cs:0.383838385343552,-1.24851429462433)
--(axis cs:0.39393937587738,-0.998290956020355)
--(axis cs:0.404040396213531,-0.750182688236237)
--(axis cs:0.414141416549683,-0.525704860687256)
--(axis cs:0.424242407083511,-0.357743710279465)
--(axis cs:0.434343427419662,-0.135300785303116)
--(axis cs:0.444444447755814,0.126165747642517)
--(axis cs:0.454545438289642,0.403996556997299)
--(axis cs:0.464646458625793,0.657515227794647)
--(axis cs:0.474747478961945,0.89471846818924)
--(axis cs:0.484848499298096,1.12940919399261)
--(axis cs:0.494949519634247,1.33079588413239)
--(axis cs:0.505050539970398,1.50211012363434)
--(axis cs:0.515151500701904,1.35694479942322)
--(axis cs:0.525252521038055,1.16336917877197)
--(axis cs:0.535353541374207,0.950490415096283)
--(axis cs:0.545454561710358,0.723466515541077)
--(axis cs:0.555555582046509,0.504927337169647)
--(axis cs:0.565656542778015,0.281702965497971)
--(axis cs:0.575757563114166,0.0685240551829338)
--(axis cs:0.585858583450317,-0.13225944340229)
--(axis cs:0.595959544181824,-0.312537550926208)
--(axis cs:0.606060564517975,-0.467220991849899)
--(axis cs:0.616161584854126,-0.595295667648315)
--(axis cs:0.626262605190277,-0.699680984020233)
--(axis cs:0.636363625526428,-0.767002463340759)
--(axis cs:0.646464645862579,-0.776046991348267)
--(axis cs:0.65656566619873,-0.774105429649353)
--(axis cs:0.666666686534882,-0.733565211296082)
--(axis cs:0.676767706871033,-0.691394984722137)
--(axis cs:0.686868667602539,-0.637468218803406)
--(axis cs:0.69696968793869,-0.539371430873871)
--(axis cs:0.707070708274841,-0.411705821752548)
--(axis cs:0.717171728610992,-0.268257588148117)
--(axis cs:0.727272748947144,-0.116832487285137)
--(axis cs:0.737373769283295,0.031189676374197)
--(axis cs:0.747474789619446,0.188553214073181)
--(axis cs:0.757575750350952,0.312197506427765)
--(axis cs:0.767676770687103,0.13393634557724)
--(axis cs:0.777777791023254,-0.0825264304876328)
--(axis cs:0.787878811359406,-0.301064342260361)
--(axis cs:0.797979831695557,-0.516804218292236)
--(axis cs:0.808080792427063,-0.720233798027039)
--(axis cs:0.818181812763214,-0.904540479183197)
--(axis cs:0.828282833099365,-1.06612980365753)
--(axis cs:0.838383793830872,-1.22843110561371)
--(axis cs:0.848484814167023,-1.38574481010437)
--(axis cs:0.858585834503174,-1.50933611392975)
--(axis cs:0.868686854839325,-1.60499787330627)
--(axis cs:0.878787875175476,-1.68155801296234)
--(axis cs:0.888888895511627,-1.72222781181335)
--(axis cs:0.898989915847778,-1.72970712184906)
--(axis cs:0.909090936183929,-1.73046946525574)
--(axis cs:0.919191956520081,-1.71048104763031)
--(axis cs:0.929292917251587,-1.6622371673584)
--(axis cs:0.939393937587738,-1.58365535736084)
--(axis cs:0.949494957923889,-1.47647070884705)
--(axis cs:0.95959597826004,-1.35282254219055)
--(axis cs:0.969696998596191,-1.21870219707489)
--(axis cs:0.979797959327698,-1.07435023784637)
--(axis cs:0.989898979663849,-0.900119543075562)
--(axis cs:1,-0.746024668216705)
--(axis cs:1,-0.673951804637909)
--(axis cs:1,-0.673951804637909)
--(axis cs:0.989898979663849,-0.507326245307922)
--(axis cs:0.979797959327698,-0.306597948074341)
--(axis cs:0.969696998596191,-0.114666670560837)
--(axis cs:0.95959597826004,0.0890414640307426)
--(axis cs:0.949494957923889,0.290912091732025)
--(axis cs:0.939393937587738,0.482827752828598)
--(axis cs:0.929292917251587,0.65446412563324)
--(axis cs:0.919191956520081,0.804228365421295)
--(axis cs:0.909090936183929,0.933906435966492)
--(axis cs:0.898989915847778,1.05554437637329)
--(axis cs:0.888888895511627,1.16974520683289)
--(axis cs:0.878787875175476,1.24376940727234)
--(axis cs:0.868686854839325,1.28814744949341)
--(axis cs:0.858585834503174,1.31901741027832)
--(axis cs:0.848484814167023,1.32284557819366)
--(axis cs:0.838383793830872,1.29121851921082)
--(axis cs:0.828282833099365,1.24036478996277)
--(axis cs:0.818181812763214,1.16481220722198)
--(axis cs:0.808080792427063,1.06135177612305)
--(axis cs:0.797979831695557,0.940897345542908)
--(axis cs:0.787878811359406,0.803660213947296)
--(axis cs:0.777777791023254,0.660230994224548)
--(axis cs:0.767676770687103,0.507468283176422)
--(axis cs:0.757575750350952,0.382923036813736)
--(axis cs:0.747474789619446,0.568678319454193)
--(axis cs:0.737373769283295,0.784480810165405)
--(axis cs:0.727272748947144,1.00219941139221)
--(axis cs:0.717171728610992,1.1989334821701)
--(axis cs:0.707070708274841,1.39756238460541)
--(axis cs:0.69696968793869,1.57316732406616)
--(axis cs:0.686868667602539,1.73068594932556)
--(axis cs:0.676767706871033,1.90267431735992)
--(axis cs:0.666666686534882,2.0696439743042)
--(axis cs:0.65656566619873,2.21041941642761)
--(axis cs:0.646464645862579,2.31671452522278)
--(axis cs:0.636363625526428,2.39649868011475)
--(axis cs:0.626262605190277,2.45100498199463)
--(axis cs:0.616161584854126,2.48830580711365)
--(axis cs:0.606060564517975,2.50864386558533)
--(axis cs:0.595959544181824,2.50856685638428)
--(axis cs:0.585858583450317,2.48201560974121)
--(axis cs:0.575757563114166,2.43362832069397)
--(axis cs:0.565656542778015,2.37192559242249)
--(axis cs:0.555555582046509,2.29341602325439)
--(axis cs:0.545454561710358,2.18715262413025)
--(axis cs:0.535353541374207,2.06191420555115)
--(axis cs:0.525252521038055,1.91972386837006)
--(axis cs:0.515151500701904,1.74595034122467)
--(axis cs:0.505050539970398,1.59932494163513)
--(axis cs:0.494949519634247,1.72287714481354)
--(axis cs:0.484848499298096,1.86946165561676)
--(axis cs:0.474747478961945,1.98051631450653)
--(axis cs:0.464646458625793,2.07214260101318)
--(axis cs:0.454545438289642,2.14373254776001)
--(axis cs:0.444444447755814,2.18675327301025)
--(axis cs:0.434343427419662,2.20286750793457)
--(axis cs:0.424242407083511,2.18358945846558)
--(axis cs:0.414141416549683,2.12909626960754)
--(axis cs:0.404040396213531,2.04254627227783)
--(axis cs:0.39393937587738,1.90732991695404)
--(axis cs:0.383838385343552,1.74706768989563)
--(axis cs:0.373737365007401,1.58211135864258)
--(axis cs:0.363636374473572,1.39323019981384)
--(axis cs:0.353535354137421,1.17267620563507)
--(axis cs:0.34343433380127,0.920457720756531)
--(axis cs:0.333333343267441,0.668398439884186)
--(axis cs:0.32323232293129,0.420982718467712)
--(axis cs:0.313131332397461,0.177429169416428)
--(axis cs:0.30303031206131,-0.0690865516662598)
--(axis cs:0.292929291725159,-0.318397551774979)
--(axis cs:0.28282830119133,-0.552216172218323)
--(axis cs:0.272727280855179,-0.79360032081604)
--(axis cs:0.262626260519028,-1.02164494991302)
--(axis cs:0.252525269985199,-1.19548094272614)
--(axis cs:0.242424249649048,-1.0438551902771)
--(axis cs:0.232323229312897,-0.839464962482452)
--(axis cs:0.222222223877907,-0.621898174285889)
--(axis cs:0.212121218442917,-0.406501770019531)
--(axis cs:0.202020198106766,-0.190933123230934)
--(axis cs:0.191919192671776,0.0385102108120918)
--(axis cs:0.181818187236786,0.268366992473602)
--(axis cs:0.171717166900635,0.486332714557648)
--(axis cs:0.161616161465645,0.66456401348114)
--(axis cs:0.151515156030655,0.790132761001587)
--(axis cs:0.141414135694504,0.922284424304962)
--(axis cs:0.131313130259514,1.05679023265839)
--(axis cs:0.121212124824524,1.18532228469849)
--(axis cs:0.111111111938953,1.2858852148056)
--(axis cs:0.101010099053383,1.33695435523987)
--(axis cs:0.0909090936183929,1.34189283847809)
--(axis cs:0.0808080807328224,1.3314266204834)
--(axis cs:0.0707070678472519,1.31959629058838)
--(axis cs:0.060606062412262,1.27801203727722)
--(axis cs:0.0505050495266914,1.18698132038116)
--(axis cs:0.0404040403664112,1.06947243213654)
--(axis cs:0.030303031206131,0.937995493412018)
--(axis cs:0.0202020201832056,0.784277558326721)
--(axis cs:0.0101010100916028,0.623385548591614)
--(axis cs:0,0.48836225271225)
--cycle;
\path [draw=green!50.1960784313725!black, fill=green!50.1960784313725!black, opacity=0.2]
(axis cs:0,0.42320853471756)
--(axis cs:0,0.478630483150482)
--(axis cs:0.0101010100916028,0.63086324930191)
--(axis cs:0.0202020201832056,0.79552835226059)
--(axis cs:0.030303031206131,0.94506299495697)
--(axis cs:0.0404040403664112,1.07579600811005)
--(axis cs:0.0505050495266914,1.18488883972168)
--(axis cs:0.060606062412262,1.26998770236969)
--(axis cs:0.0707070678472519,1.32928156852722)
--(axis cs:0.0808080807328224,1.36147749423981)
--(axis cs:0.0909090936183929,1.36582171916962)
--(axis cs:0.101010099053383,1.34206390380859)
--(axis cs:0.111111111938953,1.29045009613037)
--(axis cs:0.121212124824524,1.21168637275696)
--(axis cs:0.131313130259514,1.10691666603088)
--(axis cs:0.141414135694504,0.977699398994446)
--(axis cs:0.151515156030655,0.825999021530151)
--(axis cs:0.161616161465645,0.654172658920288)
--(axis cs:0.171717166900635,0.464965999126434)
--(axis cs:0.181818187236786,0.261504948139191)
--(axis cs:0.191919192671776,0.0472662448883057)
--(axis cs:0.202020198106766,-0.173944175243378)
--(axis cs:0.212121218442917,-0.398045301437378)
--(axis cs:0.222222223877907,-0.620745956897736)
--(axis cs:0.232323229312897,-0.837598621845245)
--(axis cs:0.242424249649048,-1.0437605381012)
--(axis cs:0.252525269985199,-1.21087288856506)
--(axis cs:0.262626260519028,-1.02080011367798)
--(axis cs:0.272727280855179,-0.790893852710724)
--(axis cs:0.28282830119133,-0.548761963844299)
--(axis cs:0.292929291725159,-0.298509538173676)
--(axis cs:0.30303031206131,-0.0439335107803345)
--(axis cs:0.313131332397461,0.211251735687256)
--(axis cs:0.32323232293129,0.463441848754883)
--(axis cs:0.333333343267441,0.709164977073669)
--(axis cs:0.34343433380127,0.945122003555298)
--(axis cs:0.353535354137421,1.16818904876709)
--(axis cs:0.363636374473572,1.37542080879211)
--(axis cs:0.373737365007401,1.56404685974121)
--(axis cs:0.383838385343552,1.73147392272949)
--(axis cs:0.39393937587738,1.87529420852661)
--(axis cs:0.404040396213531,1.99331879615784)
--(axis cs:0.414141416549683,2.08361768722534)
--(axis cs:0.424242407083511,2.14458775520325)
--(axis cs:0.434343427419662,2.17502784729004)
--(axis cs:0.444444447755814,2.1742205619812)
--(axis cs:0.454545438289642,2.142014503479)
--(axis cs:0.464646458625793,2.07889604568481)
--(axis cs:0.474747478961945,1.98604822158813)
--(axis cs:0.484848499298096,1.86541771888733)
--(axis cs:0.494949519634247,1.72002959251404)
--(axis cs:0.505050539970398,1.57721316814423)
--(axis cs:0.515151500701904,1.75488746166229)
--(axis cs:0.525252521038055,1.93632102012634)
--(axis cs:0.535353541374207,2.09533667564392)
--(axis cs:0.545454561710358,2.23000621795654)
--(axis cs:0.555555582046509,2.33938312530518)
--(axis cs:0.565656542778015,2.42315769195557)
--(axis cs:0.575757563114166,2.48155117034912)
--(axis cs:0.585858583450317,2.51521921157837)
--(axis cs:0.595959544181824,2.52513790130615)
--(axis cs:0.606060564517975,2.51249122619629)
--(axis cs:0.616161584854126,2.47855472564697)
--(axis cs:0.626262605190277,2.42459559440613)
--(axis cs:0.636363625526428,2.35179829597473)
--(axis cs:0.646464645862579,2.26121759414673)
--(axis cs:0.65656566619873,2.15377068519592)
--(axis cs:0.666666686534882,2.0302562713623)
--(axis cs:0.676767706871033,1.8914076089859)
--(axis cs:0.686868667602539,1.73795938491821)
--(axis cs:0.69696968793869,1.5707323551178)
--(axis cs:0.707070708274841,1.3907128572464)
--(axis cs:0.717171728610992,1.19912457466125)
--(axis cs:0.727272748947144,0.997484028339386)
--(axis cs:0.737373769283295,0.787675678730011)
--(axis cs:0.747474789619446,0.572265446186066)
--(axis cs:0.757575750350952,0.377738237380981)
--(axis cs:0.767676770687103,0.519419372081757)
--(axis cs:0.777777791023254,0.679764151573181)
--(axis cs:0.787878811359406,0.830590486526489)
--(axis cs:0.797979831695557,0.967712759971619)
--(axis cs:0.808080792427063,1.08774089813232)
--(axis cs:0.818181812763214,1.18783736228943)
--(axis cs:0.828282833099365,1.26569366455078)
--(axis cs:0.838383793830872,1.31954658031464)
--(axis cs:0.848484814167023,1.3481719493866)
--(axis cs:0.858585834503174,1.35086405277252)
--(axis cs:0.868686854839325,1.32741022109985)
--(axis cs:0.878787875175476,1.27805984020233)
--(axis cs:0.888888895511627,1.20350527763367)
--(axis cs:0.898989915847778,1.10485076904297)
--(axis cs:0.909090936183929,0.983589291572571)
--(axis cs:0.919191956520081,0.841610193252563)
--(axis cs:0.929292917251587,0.681154727935791)
--(axis cs:0.939393937587738,0.504822611808777)
--(axis cs:0.949494957923889,0.315531313419342)
--(axis cs:0.95959597826004,0.116456389427185)
--(axis cs:0.969696998596191,-0.0889670848846436)
--(axis cs:0.979797959327698,-0.297123461961746)
--(axis cs:0.989898979663849,-0.503996789455414)
--(axis cs:1,-0.681941628456116)
--(axis cs:1,-0.737388372421265)
--(axis cs:1,-0.737388372421265)
--(axis cs:0.989898979663849,-0.898661077022552)
--(axis cs:0.979797959327698,-1.07233858108521)
--(axis cs:0.969696998596191,-1.23119270801544)
--(axis cs:0.95959597826004,-1.37203598022461)
--(axis cs:0.949494957923889,-1.49246001243591)
--(axis cs:0.939393937587738,-1.59056580066681)
--(axis cs:0.929292917251587,-1.66496157646179)
--(axis cs:0.919191956520081,-1.71470522880554)
--(axis cs:0.909090936183929,-1.73928844928741)
--(axis cs:0.898989915847778,-1.73859596252441)
--(axis cs:0.888888895511627,-1.71284866333008)
--(axis cs:0.878787875175476,-1.6625679731369)
--(axis cs:0.868686854839325,-1.58853077888489)
--(axis cs:0.858585834503174,-1.4917494058609)
--(axis cs:0.848484814167023,-1.37345361709595)
--(axis cs:0.838383793830872,-1.23509514331818)
--(axis cs:0.828282833099365,-1.0783417224884)
--(axis cs:0.818181812763214,-0.905088424682617)
--(axis cs:0.808080792427063,-0.717463672161102)
--(axis cs:0.797979831695557,-0.517817258834839)
--(axis cs:0.787878811359406,-0.308697313070297)
--(axis cs:0.777777791023254,-0.0928693413734436)
--(axis cs:0.767676770687103,0.126435399055481)
--(axis cs:0.757575750350952,0.32230806350708)
--(axis cs:0.747474789619446,0.178770244121552)
--(axis cs:0.737373769283295,0.0129687786102295)
--(axis cs:0.727272748947144,-0.146591484546661)
--(axis cs:0.717171728610992,-0.29519647359848)
--(axis cs:0.707070708274841,-0.428772538900375)
--(axis cs:0.69696968793869,-0.543674409389496)
--(axis cs:0.686868667602539,-0.6367227435112)
--(axis cs:0.676767706871033,-0.705268025398254)
--(axis cs:0.666666686534882,-0.747236609458923)
--(axis cs:0.65656566619873,-0.761165380477905)
--(axis cs:0.646464645862579,-0.74621593952179)
--(axis cs:0.636363625526428,-0.702185213565826)
--(axis cs:0.626262605190277,-0.629502475261688)
--(axis cs:0.616161584854126,-0.52922523021698)
--(axis cs:0.606060564517975,-0.403024315834045)
--(axis cs:0.595959544181824,-0.253169655799866)
--(axis cs:0.585858583450317,-0.0825041532516479)
--(axis cs:0.575757563114166,0.105599284172058)
--(axis cs:0.565656542778015,0.307281136512756)
--(axis cs:0.555555582046509,0.518273711204529)
--(axis cs:0.545454561710358,0.733989238739014)
--(axis cs:0.535353541374207,0.949618995189667)
--(axis cs:0.525252521038055,1.16021490097046)
--(axis cs:0.515151500701904,1.36050999164581)
--(axis cs:0.505050539970398,1.52178299427032)
--(axis cs:0.494949519634247,1.32564640045166)
--(axis cs:0.484848499298096,1.08930349349976)
--(axis cs:0.474747478961945,0.840315163135529)
--(axis cs:0.464646458625793,0.582855105400085)
--(axis cs:0.454545438289642,0.320865750312805)
--(axis cs:0.444444447755814,0.058283805847168)
--(axis cs:0.434343427419662,-0.20101273059845)
--(axis cs:0.424242407083511,-0.45326042175293)
--(axis cs:0.414141416549683,-0.694863140583038)
--(axis cs:0.404040396213531,-0.922431945800781)
--(axis cs:0.39393937587738,-1.1327977180481)
--(axis cs:0.383838385343552,-1.32303166389465)
--(axis cs:0.373737365007401,-1.49045872688293)
--(axis cs:0.363636374473572,-1.63267183303833)
--(axis cs:0.353535354137421,-1.74756288528442)
--(axis cs:0.34343433380127,-1.83336162567139)
--(axis cs:0.333333343267441,-1.88868582248688)
--(axis cs:0.32323232293129,-1.9126033782959)
--(axis cs:0.313131332397461,-1.90468955039978)
--(axis cs:0.30303031206131,-1.86508977413177)
--(axis cs:0.292929291725159,-1.79455709457397)
--(axis cs:0.28282830119133,-1.69449770450592)
--(axis cs:0.272727280855179,-1.56701231002808)
--(axis cs:0.262626260519028,-1.41518235206604)
--(axis cs:0.252525269985199,-1.26630306243896)
--(axis cs:0.242424249649048,-1.43814611434937)
--(axis cs:0.232323229312897,-1.61372065544128)
--(axis cs:0.222222223877907,-1.76650309562683)
--(axis cs:0.212121218442917,-1.89412474632263)
--(axis cs:0.202020198106766,-1.99515414237976)
--(axis cs:0.191919192671776,-2.06876611709595)
--(axis cs:0.181818187236786,-2.11467456817627)
--(axis cs:0.171717166900635,-2.13307666778564)
--(axis cs:0.161616161465645,-2.12457966804504)
--(axis cs:0.151515156030655,-2.09012055397034)
--(axis cs:0.141414135694504,-2.03088426589966)
--(axis cs:0.131313130259514,-1.94823455810547)
--(axis cs:0.121212124824524,-1.84365153312683)
--(axis cs:0.111111111938953,-1.71869897842407)
--(axis cs:0.101010099053383,-1.57500529289246)
--(axis cs:0.0909090936183929,-1.41427481174469)
--(axis cs:0.0808080807328224,-1.23830473423004)
--(axis cs:0.0707070678472519,-1.04903149604797)
--(axis cs:0.060606062412262,-0.848540902137756)
--(axis cs:0.0505050495266914,-0.639123797416687)
--(axis cs:0.0404040403664112,-0.423254698514938)
--(axis cs:0.030303031206131,-0.203639417886734)
--(axis cs:0.0202020201832056,0.016850471496582)
--(axis cs:0.0101010100916028,0.234802171587944)
--(axis cs:0,0.42320853471756)
--cycle;
\addplot [semithick, black, mark=*, mark size=3, mark options={solid}, only marks]
table {%
0 0.450969964265823
0.252525269985199 -1.23872041702271
0.505050539970398 1.54965722560883
0.757575750350952 0.350055396556854
1 -0.709737658500671
};
\addplot [semithick, blue]
table {%
0 0.451404750347137
0.0101010100916028 0.429508864879608
0.0202020201832056 0.395687580108643
0.030303031206131 0.359092146158218
0.0404040403664112 0.312889993190765
0.0505050495266914 0.261233478784561
0.060606062412262 0.203510195016861
0.0707070678472519 0.129626274108887
0.0808080807328224 0.0392259284853935
0.0909090936183929 -0.0495683066546917
0.101010099053383 -0.136123538017273
0.111111111938953 -0.226568162441254
0.121212124824524 -0.328051686286926
0.131313130259514 -0.441125929355621
0.141414135694504 -0.553359568119049
0.151515156030655 -0.646482348442078
0.161616161465645 -0.721714496612549
0.171717166900635 -0.810608148574829
0.181818187236786 -0.912096858024597
0.191919192671776 -1.00756561756134
0.202020198106766 -1.08372116088867
0.212121218442917 -1.14134109020233
0.222222223877907 -1.18776392936707
0.232323229312897 -1.22145795822144
0.242424249649048 -1.24008917808533
0.252525269985199 -1.24087452888489
0.262626260519028 -1.21322524547577
0.272727280855179 -1.18115615844727
0.28282830119133 -1.12498712539673
0.292929291725159 -1.05412817001343
0.30303031206131 -0.966374933719635
0.313131332397461 -0.863375544548035
0.32323232293129 -0.745270907878876
0.333333343267441 -0.605263948440552
0.34343433380127 -0.447585672140121
0.353535354137421 -0.279474765062332
0.363636374473572 -0.106364138424397
0.373737365007401 0.0700160041451454
0.383838385343552 0.253800392150879
0.39393937587738 0.456855267286301
0.404040396213531 0.645587027072906
0.414141416549683 0.801948666572571
0.424242407083511 0.914830088615417
0.434343427419662 1.03487551212311
0.444444447755814 1.16097414493561
0.454545438289642 1.27622497081757
0.464646458625793 1.36799550056458
0.474747478961945 1.44127821922302
0.484848499298096 1.49789786338806
0.494949519634247 1.52496671676636
0.505050539970398 1.5487414598465
0.515151500701904 1.55028939247131
0.525252521038055 1.53875482082367
0.535353541374207 1.50630915164948
0.545454561710358 1.459308385849
0.555555582046509 1.40213072299957
0.565656542778015 1.33318710327148
0.575757563114166 1.25758647918701
0.585858583450317 1.18111801147461
0.595959544181824 1.10618090629578
0.606060564517975 1.02639639377594
0.616161584854126 0.948617815971375
0.626262605190277 0.876832485198975
0.636363625526428 0.815645337104797
0.646464645862579 0.770972907543182
0.65656566619873 0.719010055065155
0.666666686534882 0.669880330562592
0.676767706871033 0.60925030708313
0.686868667602539 0.551652431488037
0.69696968793869 0.518530607223511
0.707070708274841 0.49223804473877
0.717171728610992 0.467531263828278
0.727272748947144 0.440430611371994
0.737373769283295 0.409383833408356
0.747474789619446 0.378106713294983
0.757575750350952 0.348360985517502
0.767676770687103 0.320502936840057
0.777777791023254 0.288082242012024
0.787878811359406 0.249972492456436
0.797979831695557 0.210989579558372
0.808080792427063 0.171478629112244
0.818181812763214 0.129068806767464
0.828282833099365 0.082429438829422
0.838383793830872 0.028945567086339
0.848484814167023 -0.0323014184832573
0.858585834503174 -0.0969058275222778
0.868686854839325 -0.161361858248711
0.878787875175476 -0.221974015235901
0.888888895511627 -0.279919773340225
0.898989915847778 -0.340129196643829
0.909090936183929 -0.400968372821808
0.919191956520081 -0.454226464033127
0.929292917251587 -0.502343833446503
0.939393937587738 -0.548811614513397
0.949494957923889 -0.593340992927551
0.95959597826004 -0.635101020336151
0.969696998596191 -0.669027984142303
0.979797959327698 -0.690099358558655
0.989898979663849 -0.705365180969238
1 -0.710029423236847
};
\addplot [semithick, green!50.1960784313725!black]
table {%
0 0.450919508934021
0.0101010100916028 0.432832717895508
0.0202020201832056 0.406189411878586
0.030303031206131 0.370711773633957
0.0404040403664112 0.326270669698715
0.0505050495266914 0.272882550954819
0.060606062412262 0.210723400115967
0.0707070678472519 0.140125021338463
0.0808080807328224 0.0615863353013992
0.0909090936183929 -0.0242265295237303
0.101010099053383 -0.116470664739609
0.111111111938953 -0.214124441146851
0.121212124824524 -0.315982550382614
0.131313130259514 -0.42065891623497
0.141414135694504 -0.526592493057251
0.151515156030655 -0.632060766220093
0.161616161465645 -0.735203504562378
0.171717166900635 -0.834055364131927
0.181818187236786 -0.926584780216217
0.191919192671776 -1.01074993610382
0.202020198106766 -1.08454918861389
0.212121218442917 -1.14608502388
0.222222223877907 -1.19362449645996
0.232323229312897 -1.22565960884094
0.242424249649048 -1.24095332622528
0.252525269985199 -1.23858797550201
0.262626260519028 -1.21799123287201
0.272727280855179 -1.17895305156708
0.28282830119133 -1.12162983417511
0.292929291725159 -1.04653334617615
0.30303031206131 -0.954511642456055
0.313131332397461 -0.846718907356262
0.32323232293129 -0.724580764770508
0.333333343267441 -0.589760422706604
0.34343433380127 -0.444119840860367
0.353535354137421 -0.289686918258667
0.363636374473572 -0.128625512123108
0.373737365007401 0.036794126033783
0.383838385343552 0.204221129417419
0.39393937587738 0.371248245239258
0.404040396213531 0.535443425178528
0.414141416549683 0.694377243518829
0.424242407083511 0.845663666725159
0.434343427419662 0.98700749874115
0.444444447755814 1.11625218391418
0.454545438289642 1.23144018650055
0.464646458625793 1.33087551593781
0.474747478961945 1.41318166255951
0.484848499298096 1.47736060619354
0.494949519634247 1.52283799648285
0.505050539970398 1.54949808120728
0.515151500701904 1.55769872665405
0.525252521038055 1.5482679605484
0.535353541374207 1.52247786521912
0.545454561710358 1.48199772834778
0.555555582046509 1.4288284778595
0.565656542778015 1.36521935462952
0.575757563114166 1.29357528686523
0.585858583450317 1.21635758876801
0.595959544181824 1.13598418235779
0.606060564517975 1.05473351478577
0.616161584854126 0.974664688110352
0.626262605190277 0.897546589374542
0.636363625526428 0.82480651140213
0.646464645862579 0.757500886917114
0.65656566619873 0.696302652359009
0.666666686534882 0.641509771347046
0.676767706871033 0.593069791793823
0.686868667602539 0.550618350505829
0.69696968793869 0.513529002666473
0.707070708274841 0.480970174074173
0.717171728610992 0.45196408033371
0.727272748947144 0.425446271896362
0.737373769283295 0.40032222867012
0.747474789619446 0.375517845153809
0.757575750350952 0.350023150444031
0.767676770687103 0.322927385568619
0.777777791023254 0.293447405099869
0.787878811359406 0.260946601629257
0.797979831695557 0.22494775056839
0.808080792427063 0.185138627886772
0.818181812763214 0.141374453902245
0.828282833099365 0.0936759263277054
0.838383793830872 0.0422257483005524
0.848484814167023 -0.0126408934593201
0.858585834503174 -0.0704426318407059
0.868686854839325 -0.130560308694839
0.878787875175476 -0.192254021763802
0.888888895511627 -0.254671663045883
0.898989915847778 -0.316872596740723
0.909090936183929 -0.377849578857422
0.919191956520081 -0.436547487974167
0.929292917251587 -0.491903454065323
0.939393937587738 -0.542871594429016
0.949494957923889 -0.588464379310608
0.95959597826004 -0.627789795398712
0.969696998596191 -0.660079896450043
0.979797959327698 -0.684731006622314
0.989898979663849 -0.701328933238983
1 -0.70966500043869
};
\path [draw=red, fill=red, opacity=0.2]
(axis cs:0,0.24650934)
--(axis cs:0,0.63853204)
--(axis cs:0.01,0.6373885)
--(axis cs:0.02,0.6352033)
--(axis cs:0.03,0.6357253)
--(axis cs:0.04,0.64640945)
--(axis cs:0.05,1.0347253)
--(axis cs:0.05999999,1.4347534)
--(axis cs:0.06999999,1.601848)
--(axis cs:0.07999999,1.6880443)
--(axis cs:0.08999999,1.7022696)
--(axis cs:0.09999999,1.6959207)
--(axis cs:0.10999998,1.675191)
--(axis cs:0.11999998,1.6457186)
--(axis cs:0.12999998,1.568301)
--(axis cs:0.13999999,1.0679722)
--(axis cs:0.14999999,1.0826819)
--(axis cs:0.16,1.0256364)
--(axis cs:0.17,0.9292991)
--(axis cs:0.18,0.71161246)
--(axis cs:0.19000001,0.21484125)
--(axis cs:0.20000002,-0.6569113)
--(axis cs:0.21000002,-1.0159453)
--(axis cs:0.22000003,-1.0364289)
--(axis cs:0.23000003,-1.039857)
--(axis cs:0.24000004,-1.0389485)
--(axis cs:0.25000003,-1.0388527)
--(axis cs:0.26000002,-1.040562)
--(axis cs:0.27,-1.0459695)
--(axis cs:0.28,-1.0217912)
--(axis cs:0.29,0.5209143)
--(axis cs:0.29999998,1.2401094)
--(axis cs:0.30999997,1.4497621)
--(axis cs:0.31999996,1.5292492)
--(axis cs:0.32999995,1.5774112)
--(axis cs:0.33999994,1.6108305)
--(axis cs:0.34999993,1.6353836)
--(axis cs:0.35999992,1.6529938)
--(axis cs:0.36999992,1.66755)
--(axis cs:0.3799999,1.6762488)
--(axis cs:0.3899999,1.8801243)
--(axis cs:0.3999999,2.3253407)
--(axis cs:0.40999988,2.3001733)
--(axis cs:0.41999987,2.2977586)
--(axis cs:0.42999986,2.2929645)
--(axis cs:0.43999985,2.269072)
--(axis cs:0.44999984,1.8053424)
--(axis cs:0.45999983,1.7157716)
--(axis cs:0.46999982,1.7295359)
--(axis cs:0.4799998,1.7307036)
--(axis cs:0.4899998,1.730691)
--(axis cs:0.4999998,1.7311727)
--(axis cs:0.5099998,1.7317371)
--(axis cs:0.5199998,1.7357196)
--(axis cs:0.5299998,1.7379292)
--(axis cs:0.5399998,1.7359166)
--(axis cs:0.5499998,1.780058)
--(axis cs:0.55999976,2.1485832)
--(axis cs:0.56999975,2.2730331)
--(axis cs:0.57999974,2.351571)
--(axis cs:0.58999974,2.4310305)
--(axis cs:0.5999997,2.507622)
--(axis cs:0.6099997,2.5787601)
--(axis cs:0.6199997,2.6344266)
--(axis cs:0.6299997,2.6915865)
--(axis cs:0.6399997,1.4888961)
--(axis cs:0.6499997,1.6969283)
--(axis cs:0.65999967,1.6613034)
--(axis cs:0.66999966,1.6207557)
--(axis cs:0.67999965,1.5262084)
--(axis cs:0.68999964,1.3514929)
--(axis cs:0.69999963,0.94487894)
--(axis cs:0.7099996,0.5657289)
--(axis cs:0.7199996,0.56389296)
--(axis cs:0.7299996,0.56301475)
--(axis cs:0.7399996,0.56435865)
--(axis cs:0.7499996,0.56493723)
--(axis cs:0.7599996,0.56423175)
--(axis cs:0.76999956,0.56350285)
--(axis cs:0.77999955,0.5672554)
--(axis cs:0.78999954,0.57323056)
--(axis cs:0.79999954,0.57641983)
--(axis cs:0.8099995,1.4222553)
--(axis cs:0.8199995,1.9352151)
--(axis cs:0.8299995,1.9602075)
--(axis cs:0.8399995,1.970506)
--(axis cs:0.8499995,1.9758072)
--(axis cs:0.8599995,1.9794533)
--(axis cs:0.86999947,1.991367)
--(axis cs:0.87999946,2.0127635)
--(axis cs:0.88999945,1.2291249)
--(axis cs:0.89999944,1.3802515)
--(axis cs:0.90999943,1.3600812)
--(axis cs:0.9199994,1.3411072)
--(axis cs:0.9299994,1.094925)
--(axis cs:0.9399994,0.5706672)
--(axis cs:0.9499994,0.5534767)
--(axis cs:0.9599994,0.52046376)
--(axis cs:0.9699994,0.38842493)
--(axis cs:0.97999936,-0.33085993)
--(axis cs:0.98999935,-0.44522506)
--(axis cs:0.98999935,-0.8373706)
--(axis cs:0.98999935,-0.8373706)
--(axis cs:0.97999936,-0.9249939)
--(axis cs:0.9699994,-1.5828846)
--(axis cs:0.9599994,-1.7059684)
--(axis cs:0.9499994,-1.7353638)
--(axis cs:0.9399994,-1.7595539)
--(axis cs:0.9299994,-1.9655696)
--(axis cs:0.9199994,-2.0235744)
--(axis cs:0.90999943,-2.0312684)
--(axis cs:0.89999944,-2.0401044)
--(axis cs:0.88999945,-1.9010373)
--(axis cs:0.87999946,-1.5685246)
--(axis cs:0.86999947,-1.5552996)
--(axis cs:0.8599995,-1.5546393)
--(axis cs:0.8499995,-1.5633612)
--(axis cs:0.8399995,-1.5679314)
--(axis cs:0.8299995,-1.5602045)
--(axis cs:0.8199995,-1.4941355)
--(axis cs:0.8099995,-0.76820666)
--(axis cs:0.79999954,0.18440983)
--(axis cs:0.78999954,0.18121965)
--(axis cs:0.77999955,0.17525838)
--(axis cs:0.76999956,0.1715083)
--(axis cs:0.7599996,0.17222254)
--(axis cs:0.7499996,0.17285906)
--(axis cs:0.7399996,0.1720521)
--(axis cs:0.7299996,0.16971003)
--(axis cs:0.7199996,0.16723835)
--(axis cs:0.7099996,0.16587994)
--(axis cs:0.69999963,-0.17217973)
--(axis cs:0.68999964,-0.59302485)
--(axis cs:0.67999965,-0.77202153)
--(axis cs:0.66999966,-0.87129164)
--(axis cs:0.65999967,-0.91670024)
--(axis cs:0.6499997,-0.95686203)
--(axis cs:0.6399997,1.0081999)
--(axis cs:0.6299997,0.10102069)
--(axis cs:0.6199997,0.17730832)
--(axis cs:0.6099997,0.24765384)
--(axis cs:0.5999997,0.33079553)
--(axis cs:0.58999974,0.41899848)
--(axis cs:0.57999974,0.50884163)
--(axis cs:0.56999975,0.60104924)
--(axis cs:0.55999976,0.77383393)
--(axis cs:0.5499998,1.2556278)
--(axis cs:0.5399998,1.3439088)
--(axis cs:0.5299998,1.3459319)
--(axis cs:0.5199998,1.3437217)
--(axis cs:0.5099998,1.339735)
--(axis cs:0.4999998,1.3391083)
--(axis cs:0.4899998,1.3384323)
--(axis cs:0.4799998,1.3380291)
--(axis cs:0.46999982,1.3361462)
--(axis cs:0.45999983,1.3208302)
--(axis cs:0.44999984,0.9501514)
--(axis cs:0.43999985,0.36082757)
--(axis cs:0.42999986,0.31386125)
--(axis cs:0.41999987,0.29957706)
--(axis cs:0.40999988,0.28904796)
--(axis cs:0.3999999,0.2517029)
--(axis cs:0.3899999,0.0806967)
--(axis cs:0.3799999,-3.0292566)
--(axis cs:0.36999992,-3.0366745)
--(axis cs:0.35999992,-3.0414772)
--(axis cs:0.34999993,-3.048777)
--(axis cs:0.33999994,-3.0555055)
--(axis cs:0.32999995,-3.0616465)
--(axis cs:0.31999996,-3.0660372)
--(axis cs:0.30999997,-3.065358)
--(axis cs:0.29999998,-3.0290833)
--(axis cs:0.29,-2.6839592)
--(axis cs:0.28,-1.4144695)
--(axis cs:0.27,-1.4380968)
--(axis cs:0.26000002,-1.4331192)
--(axis cs:0.25000003,-1.4316378)
--(axis cs:0.24000004,-1.4317133)
--(axis cs:0.23000003,-1.4327224)
--(axis cs:0.22000003,-1.4323354)
--(axis cs:0.21000002,-1.4420522)
--(axis cs:0.20000002,-1.7178795)
--(axis cs:0.19000001,-2.527276)
--(axis cs:0.18,-2.8391154)
--(axis cs:0.17,-2.9714592)
--(axis cs:0.16,-3.028898)
--(axis cs:0.14999999,-3.0580747)
--(axis cs:0.13999999,-2.7556512)
--(axis cs:0.12999998,-1.4397676)
--(axis cs:0.11999998,-1.5228312)
--(axis cs:0.10999998,-1.5039216)
--(axis cs:0.09999999,-1.4576817)
--(axis cs:0.08999999,-1.3750691)
--(axis cs:0.07999999,-1.2576098)
--(axis cs:0.06999999,-1.0535241)
--(axis cs:0.05999999,-0.73750234)
--(axis cs:0.05,-0.19283411)
--(axis cs:0.04,0.23220946)
--(axis cs:0.03,0.24243951)
--(axis cs:0.02,0.24292074)
--(axis cs:0.01,0.24532546)
--(axis cs:0,0.24650934)
--cycle;
\addplot [semithick, red]
table {%
0 0.44252068
0.01 0.441357
0.02 0.43906203
0.03 0.4390824
0.04 0.43930945
0.05 0.4209456
0.05999999 0.34862554
0.06999999 0.27416193
0.07999999 0.21521726
0.08999999 0.16360027
0.09999999 0.11911953
0.10999998 0.08563468
0.11999998 0.06144373
0.12999998 0.06426667
0.13999999 -0.84383947
0.14999999 -0.9876964
0.16 -1.0016308
0.17 -1.02108
0.18 -1.0637515
0.19000001 -1.1562173
0.20000002 -1.1873955
0.21000002 -1.2289988
0.22000003 -1.2343822
0.23000003 -1.2362897
0.24000004 -1.2353309
0.25000003 -1.2352452
0.26000002 -1.2368406
0.27 -1.2420331
0.28 -1.2181304
0.29 -1.0815225
0.29999998 -0.8944869
0.30999997 -0.8077979
0.31999996 -0.768394
0.32999995 -0.7421176
0.33999994 -0.7223374
0.34999993 -0.7066967
0.35999992 -0.69424164
0.36999992 -0.6845623
0.3799999 -0.67650396
0.3899999 0.9804105
0.3999999 1.2885219
0.40999988 1.2946106
0.41999987 1.2986678
0.42999986 1.3034129
0.43999985 1.3149499
0.44999984 1.3777469
0.45999983 1.5183009
0.46999982 1.5328411
0.4799998 1.5343664
0.4899998 1.5345616
0.4999998 1.5351405
0.5099998 1.5357361
0.5199998 1.5397207
0.5299998 1.5419306
0.5399998 1.5399127
0.5499998 1.5178429
0.55999976 1.4612086
0.56999975 1.4370412
0.57999974 1.4302064
0.58999974 1.4250145
0.5999997 1.4192088
0.6099997 1.413207
0.6199997 1.4058675
0.6299997 1.3963035
0.6399997 1.248548
0.6499997 0.3700331
0.65999967 0.37230155
0.66999966 0.37473202
0.67999965 0.37709346
0.68999964 0.37923402
0.69999963 0.3863496
0.7099996 0.36580443
0.7199996 0.36556566
0.7299996 0.3663624
0.7399996 0.36820537
0.7499996 0.36889815
0.7599996 0.36822715
0.76999956 0.36750558
0.77999955 0.3712569
0.78999954 0.3772251
0.79999954 0.38041484
0.8099995 0.32702428
0.8199995 0.2205398
0.8299995 0.20000142
0.8399995 0.20128724
0.8499995 0.20622297
0.8599995 0.21240704
0.86999947 0.21803372
0.87999946 0.22211948
0.88999945 -0.33595625
0.89999944 -0.3299265
0.90999943 -0.33559352
0.9199994 -0.34123352
0.9299994 -0.43532228
0.9399994 -0.5944434
0.9499994 -0.5909436
0.9599994 -0.5927523
0.9699994 -0.5972298
0.97999936 -0.6279269
0.98999935 -0.6412978
};
\end{axis}
\end{tikzpicture}
%\end{document}
% This file was created by tikzplotlib v0.9.9.
\begin{tikzpicture}
\definecolor{color0}{rgb}{0.12156862745098,0.466666666666667,0.705882352941177}
\definecolor{color1}{rgb}{1,0.498039215686275,0.0549019607843137}
\begin{axis}[
legend cell align={left},
legend style={
fill opacity=0.8,
draw opacity=1,
text opacity=1,
at={(0.98,0.97)},
anchor=north east,
draw=white!80!black
},
tick align=outside,
tick pos=left,
x grid style={white!69.0196078431373!black},
xlabel={Dataset Size},
xmin=-8.5, xmax=200.5,
xtick style={color=black},
y grid style={white!69.0196078431373!black},
ylabel={Negative Log-Likelihood},
ymin=-1.6, ymax=5,
ytick style={color=black},
height=.4\textwidth,
width=.7\textwidth,
]
\path [fill=color0, fill opacity=0.1]
(axis cs:1,44.6985638971572)
--(axis cs:1,12.2816263799424)
--(axis cs:11,7.49041550300113)
--(axis cs:21,0.182076923001606)
--(axis cs:31,2.07572374880865)
--(axis cs:41,-6.32221257288298)
--(axis cs:51,1.89761279879315)
--(axis cs:61,1.15548817206679)
--(axis cs:71,0.00836860709860376)
--(axis cs:81,0.694454064454148)
--(axis cs:91,0.525893495827115)
--(axis cs:101,-0.823471832878438)
--(axis cs:111,-6.18210003900266)
--(axis cs:121,-1.72154396405576)
--(axis cs:131,0.412787645640149)
--(axis cs:141,0.0610359890022446)
--(axis cs:151,-0.192633856502971)
--(axis cs:161,-0.191392781477042)
--(axis cs:171,-0.155701293196258)
--(axis cs:181,-0.0719032445506264)
--(axis cs:191,-0.187852754603636)
--(axis cs:191,0.818538918505919)
--(axis cs:191,0.818538918505919)
--(axis cs:181,0.692179278428953)
--(axis cs:171,0.656124307360229)
--(axis cs:161,0.917497398595877)
--(axis cs:151,0.590210844246349)
--(axis cs:141,1.39362144641493)
--(axis cs:131,1.50711801689647)
--(axis cs:121,9.50609033933042)
--(axis cs:111,22.8371445946667)
--(axis cs:101,7.14933853209719)
--(axis cs:91,2.04645533725699)
--(axis cs:81,1.91721642962544)
--(axis cs:71,5.07341993278787)
--(axis cs:61,4.87824758003892)
--(axis cs:51,4.51377655209796)
--(axis cs:41,39.0368083675607)
--(axis cs:31,9.23333253323481)
--(axis cs:21,48.1006150325648)
--(axis cs:11,22.9178868041278)
--(axis cs:1,44.6985638971572)
--cycle;
\path [fill=color1, fill opacity=0.1]
(axis cs:1,1.43582652553544)
--(axis cs:1,1.31970359340682)
--(axis cs:11,0.84415761408121)
--(axis cs:21,0.482164694707946)
--(axis cs:31,0.254431246138276)
--(axis cs:41,0.0669480416847419)
--(axis cs:51,-0.0907979931957162)
--(axis cs:61,-0.251689058609849)
--(axis cs:71,-0.351863560128454)
--(axis cs:81,-0.474340895922134)
--(axis cs:91,-0.610353006784202)
--(axis cs:101,-0.706140798083434)
--(axis cs:111,-0.795300029176958)
--(axis cs:121,-0.882441181098347)
--(axis cs:131,-0.929339680375218)
--(axis cs:141,-1.03369877685619)
--(axis cs:151,-1.01877744553965)
--(axis cs:161,-1.10048350988321)
--(axis cs:171,-1.16114655382316)
--(axis cs:181,-1.12751058796905)
--(axis cs:191,-1.21170001692356)
--(axis cs:191,-1.11416310601651)
--(axis cs:191,-1.11416310601651)
--(axis cs:181,-1.02004524012543)
--(axis cs:171,-1.06622609250862)
--(axis cs:161,-0.996421483174039)
--(axis cs:151,-0.912500187173667)
--(axis cs:141,-0.933537145957229)
--(axis cs:131,-0.824919548331141)
--(axis cs:121,-0.777622324074382)
--(axis cs:111,-0.692376114469282)
--(axis cs:101,-0.599217731007447)
--(axis cs:91,-0.498916254575967)
--(axis cs:81,-0.363829215257218)
--(axis cs:71,-0.244383040022608)
--(axis cs:61,-0.131061303547972)
--(axis cs:51,0.0177680131322778)
--(axis cs:41,0.180757999055271)
--(axis cs:31,0.3559759274953)
--(axis cs:21,0.589960740167543)
--(axis cs:11,0.955743632943326)
--(axis cs:1,1.43582652553544)
--cycle;
\addplot [semithick, color0]
table {%
1 28.4900951385498
11 15.2041511535645
21 24.1413459777832
31 5.65452814102173
41 16.3572978973389
51 3.20569467544556
61 3.01686787605286
71 2.54089426994324
81 1.30583524703979
91 1.28617441654205
101 3.16293334960938
111 8.32752227783203
121 3.89227318763733
131 0.959952831268311
141 0.727328717708588
151 0.198788493871689
161 0.363052308559418
171 0.250211507081985
181 0.310138016939163
191 0.315343081951141
};
\addlegendentry{Normal Distribution}
\addplot [semithick, color1]
table {%
1 1.37776505947113
11 0.899950623512268
21 0.536062717437744
31 0.305203586816788
41 0.123853020370007
51 -0.0365149900317192
61 -0.191375181078911
71 -0.298123300075531
81 -0.419085055589676
91 -0.554634630680084
101 -0.652679264545441
111 -0.74383807182312
121 -0.830031752586365
131 -0.87712961435318
141 -0.983617961406708
151 -0.965638816356659
161 -1.04845249652863
171 -1.11368632316589
181 -1.07377791404724
191 -1.16293156147003
};
\addlegendentry{Riemann Distribution}
\path [draw=green!50.1960784313725!black, fill=green!50.1960784313725!black, opacity=0.1]
(axis cs:1,1.42143539155969)
--(axis cs:1,1.31697126661292)
--(axis cs:11,0.798985137418475)
--(axis cs:21,0.487283752558255)
--(axis cs:31,0.189091307691002)
--(axis cs:41,0.0810241985796534)
--(axis cs:51,-0.116909311758604)
--(axis cs:61,-0.270458512406716)
--(axis cs:71,-0.416831171391445)
--(axis cs:81,-0.466799130544771)
--(axis cs:91,-0.546259072836886)
--(axis cs:101,-0.660627420105177)
--(axis cs:111,-0.718855668658776)
--(axis cs:121,-0.786583647666859)
--(axis cs:131,-0.846716104089563)
--(axis cs:141,-0.935805380288497)
--(axis cs:151,-0.947121472465848)
--(axis cs:161,-1.07660102785065)
--(axis cs:171,-1.06775732623217)
--(axis cs:181,-1.08529982597516)
--(axis cs:191,-1.14087934874005)
--(axis cs:191,-1.03748015977435)
--(axis cs:191,-1.03748015977435)
--(axis cs:181,-0.975703859022401)
--(axis cs:171,-0.964129251374026)
--(axis cs:161,-0.981477261181336)
--(axis cs:151,-0.84098353852048)
--(axis cs:141,-0.828792870100602)
--(axis cs:131,-0.734654130399878)
--(axis cs:121,-0.682208433212353)
--(axis cs:111,-0.602135847454506)
--(axis cs:101,-0.548244898162645)
--(axis cs:91,-0.428669306221952)
--(axis cs:81,-0.352961728467833)
--(axis cs:71,-0.315691554667515)
--(axis cs:61,-0.165867305893531)
--(axis cs:51,-0.00291715408650425)
--(axis cs:41,0.197213382673398)
--(axis cs:31,0.307216512390709)
--(axis cs:21,0.602783157231784)
--(axis cs:11,0.90507017187718)
--(axis cs:1,1.42143539155969)
--cycle;
\addplot [semithick, green!50.1960784313725!black]
table {%
1 1.3692033290863
11 0.852027654647827
21 0.54503345489502
31 0.248153910040855
41 0.139118790626526
51 -0.059913232922554
61 -0.218162909150124
71 -0.36626136302948
81 -0.409880429506302
91 -0.487464189529419
101 -0.604436159133911
111 -0.660495758056641
121 -0.734396040439606
131 -0.79068511724472
141 -0.88229912519455
151 -0.894052505493164
161 -1.02903914451599
171 -1.0159432888031
181 -1.03050184249878
191 -1.0891797542572
};
\addlegendentry{Riemann Distribution + Positional Embeddings}
\end{axis}
\end{tikzpicture}
% This file was created with tikzplotlib v0.9.12.
\begin{tikzpicture}
\definecolor{color0}{rgb}{0.12156862745098,0.466666666666667,0.705882352941177}
\definecolor{color1}{rgb}{1,0.498039215686275,0.0549019607843137}
\definecolor{color2}{rgb}{0.172549019607843,0.627450980392157,0.172549019607843}
\begin{axis}[
legend cell align={left},
legend style={fill opacity=0.8, draw opacity=1, text opacity=1, draw=white!80!black},
tick align=outside,
tick pos=left,
x grid style={white!69.0196078431373!black},
xlabel={Number of samples in train-set},
xmin=2, xmax=100,
xtick style={color=black},
y grid style={white!69.0196078431373!black},
ylabel={Negative Log Likelihood},
ymin=0.209750335469239, ymax=0.671255502645545,
ytick style={color=black}
]
\path [fill=color0, fill opacity=0.1]
(axis cs:2,0.650277995046622)
--(axis cs:2,0.59728822809791)
--(axis cs:7,0.437730351704756)
--(axis cs:12,0.381681856735771)
--(axis cs:17,0.35506878675104)
--(axis cs:22,0.337688519528868)
--(axis cs:27,0.325793155416786)
--(axis cs:32,0.314489403406044)
--(axis cs:37,0.314497432525718)
--(axis cs:42,0.305076487076607)
--(axis cs:47,0.301322298083913)
--(axis cs:52,0.301208908812668)
--(axis cs:57,0.292238859860051)
--(axis cs:62,0.295763684315272)
--(axis cs:67,0.295551749163062)
--(axis cs:72,0.28833762524391)
--(axis cs:77,0.281520163124291)
--(axis cs:82,0.290513686842997)
--(axis cs:87,0.291626794577276)
--(axis cs:92,0.285850764841269)
--(axis cs:97,0.285268226093111)
--(axis cs:97,0.324342093044462)
--(axis cs:97,0.324342093044462)
--(axis cs:92,0.325116513639261)
--(axis cs:87,0.329768137692774)
--(axis cs:82,0.329449720673483)
--(axis cs:77,0.32113787024668)
--(axis cs:72,0.327603660843126)
--(axis cs:67,0.334101645059197)
--(axis cs:62,0.334704088168554)
--(axis cs:57,0.332799465926539)
--(axis cs:52,0.341176395161484)
--(axis cs:47,0.340999884571421)
--(axis cs:42,0.343529872882042)
--(axis cs:37,0.353507378284372)
--(axis cs:32,0.354023417791466)
--(axis cs:27,0.365421048417747)
--(axis cs:22,0.381067500540254)
--(axis cs:17,0.398921684612028)
--(axis cs:12,0.428087356463845)
--(axis cs:7,0.489980419855913)
--(axis cs:2,0.650277995046622)
--cycle;
\path [fill=color1, fill opacity=0.1]
(axis cs:2,0.611448274395756)
--(axis cs:2,0.564200534083553)
--(axis cs:7,0.414753425181715)
--(axis cs:12,0.355384345881497)
--(axis cs:17,0.328649560432181)
--(axis cs:22,0.304993590900218)
--(axis cs:27,0.280763467085906)
--(axis cs:32,0.270379804897404)
--(axis cs:37,0.266505835286276)
--(axis cs:42,0.262876658813928)
--(axis cs:47,0.261788753920772)
--(axis cs:52,0.266614909972323)
--(axis cs:57,0.261273557325564)
--(axis cs:62,0.253936559170555)
--(axis cs:67,0.256765229249506)
--(axis cs:72,0.240174391614068)
--(axis cs:77,0.247086359184182)
--(axis cs:82,0.250885188846748)
--(axis cs:87,0.260496722832023)
--(axis cs:92,0.255774128986692)
--(axis cs:97,0.241484021122981)
--(axis cs:97,0.307912672583531)
--(axis cs:97,0.307912672583531)
--(axis cs:92,0.323438834594394)
--(axis cs:87,0.338404548988046)
--(axis cs:82,0.328751325340111)
--(axis cs:77,0.302166806537712)
--(axis cs:72,0.312165579451453)
--(axis cs:67,0.317850785707922)
--(axis cs:62,0.317897945433784)
--(axis cs:57,0.321264987806119)
--(axis cs:52,0.343139413987028)
--(axis cs:47,0.326790483540318)
--(axis cs:42,0.332955391032721)
--(axis cs:37,0.330555798777445)
--(axis cs:32,0.334741867256069)
--(axis cs:27,0.339536110627106)
--(axis cs:22,0.384342649391378)
--(axis cs:17,0.409117539901987)
--(axis cs:12,0.417316977150882)
--(axis cs:7,0.468735825955065)
--(axis cs:2,0.611448274395756)
--cycle;
\path [fill=color2, fill opacity=0.1]
(axis cs:2,0.646168853285116)
--(axis cs:2,0.592935417649943)
--(axis cs:7,0.406003156766676)
--(axis cs:12,0.345008947225597)
--(axis cs:17,0.311446008591257)
--(axis cs:22,0.291826198025125)
--(axis cs:27,0.273880940574623)
--(axis cs:32,0.269565731277367)
--(axis cs:37,0.262754342945373)
--(axis cs:42,0.254920101668477)
--(axis cs:47,0.249876303762764)
--(axis cs:52,0.245681171751597)
--(axis cs:57,0.242542518656337)
--(axis cs:62,0.238663664239137)
--(axis cs:67,0.236665292180219)
--(axis cs:72,0.234862975397861)
--(axis cs:77,0.232717772975753)
--(axis cs:82,0.233305469972395)
--(axis cs:87,0.232684103557753)
--(axis cs:92,0.23198973330383)
--(axis cs:97,0.230727843068162)
--(axis cs:97,0.279204615332565)
--(axis cs:97,0.279204615332565)
--(axis cs:92,0.280476697360232)
--(axis cs:87,0.281190546443773)
--(axis cs:82,0.282182439344621)
--(axis cs:77,0.281625548347642)
--(axis cs:72,0.284236081322872)
--(axis cs:67,0.286847428881488)
--(axis cs:62,0.288218149763854)
--(axis cs:57,0.293381796796239)
--(axis cs:52,0.296985203885458)
--(axis cs:47,0.300343112855583)
--(axis cs:42,0.305548572037578)
--(axis cs:37,0.312876381484711)
--(axis cs:32,0.320098906765083)
--(axis cs:27,0.323850590091729)
--(axis cs:22,0.347278168277366)
--(axis cs:17,0.365880372615255)
--(axis cs:12,0.402300856737111)
--(axis cs:7,0.466533383264757)
--(axis cs:2,0.646168853285116)
--cycle;
\addplot [semithick, color0]
table {%
2 0.623783111572266
7 0.463855385780334
12 0.404884606599808
17 0.376995235681534
22 0.359378010034561
27 0.345607101917267
32 0.334256410598755
37 0.334002405405045
42 0.324303179979324
47 0.321161091327667
52 0.321192651987076
57 0.312519162893295
62 0.315233886241913
67 0.31482669711113
72 0.307970643043518
77 0.301329016685486
82 0.30998170375824
87 0.310697466135025
92 0.305483639240265
97 0.304805159568787
};
\addlegendentry{SVI (Adam /w 512 steps)}
\addplot [semithick, color1]
table {%
2 0.587824404239655
7 0.44174462556839
12 0.38635066151619
17 0.368883550167084
22 0.344668120145798
27 0.310149788856506
32 0.302560836076736
37 0.29853081703186
42 0.297916024923325
47 0.294289618730545
52 0.304877161979675
57 0.291269272565842
62 0.28591725230217
67 0.287308007478714
72 0.276169985532761
77 0.274626582860947
82 0.28981825709343
87 0.299450635910034
92 0.289606481790543
97 0.274698346853256
};
\addlegendentry{MCMC (NUTS /w 256 steps)}
\addplot [semithick, color2]
table {%
2 0.619552135467529
7 0.436268270015717
12 0.373654901981354
17 0.338663190603256
22 0.319552183151245
27 0.298865765333176
32 0.294832319021225
37 0.287815362215042
42 0.280234336853027
47 0.275109708309174
52 0.271333187818527
57 0.267962157726288
62 0.263440907001495
67 0.261756360530853
72 0.259549528360367
77 0.257171660661697
82 0.257743954658508
87 0.256937325000763
92 0.256233215332031
97 0.254966229200363
};
\addlegendentry{Transformer (4M Prior Samples)}
\end{axis}
\end{tikzpicture}
% This file was created with tikzplotlib v0.9.12.
\begin{tikzpicture}
\definecolor{color0}{rgb}{0.12156862745098,0.466666666666667,0.705882352941177}
\definecolor{color1}{rgb}{1,0.498039215686275,0.0549019607843137}
\definecolor{color2}{rgb}{0.172549019607843,0.627450980392157,0.172549019607843}
\begin{axis}[
legend cell align={left},
legend style={fill opacity=0.8, draw opacity=1, text opacity=1, draw=white!80!black},
tick align=outside,
tick pos=left,
x grid style={white!69.0196078431373!black},
xmin=-0.5, xmax=3.5,
xtick style={color=black},
xtick={0,1,2,3},
xticklabels={Model Size 2,Model Size 4,Model Size 8,Model Size 16},
y grid style={white!69.0196078431373!black},
ylabel={Negative Log Likelihood},
ymin=0.0700141092660041, ymax=0.353299669641593,
ytick style={color=black}
]
\path [draw=color0, semithick]
(axis cs:0,0.12442780991962)
--(axis cs:0,0.15308918098042);
\path [draw=color0, semithick]
(axis cs:1,0.138530185215282)
--(axis cs:1,0.147769103057576);
\path [draw=color0, semithick]
(axis cs:2,0.151115703362495)
--(axis cs:2,0.161927205544441);
\path [draw=color0, semithick]
(axis cs:3,0.182709622411117)
--(axis cs:3,0.196806383104935);
\path [draw=color0, semithick]
(axis cs:4,0.277493029430426)
--(axis cs:4,0.307597250148737);
\path [draw=color1, semithick]
(axis cs:0,0.121219145797156)
--(axis cs:0,0.149926585413552);
\path [draw=color1, semithick]
(axis cs:1,0.131473558313108)
--(axis cs:1,0.140817327135348);
\path [draw=color1, semithick]
(axis cs:2,0.149520469648445)
--(axis cs:2,0.160459088342582);
\path [draw=color1, semithick]
(axis cs:3,0.203648500305146)
--(axis cs:3,0.217610038656264);
\path [draw=color1, semithick]
(axis cs:4,0.30288794037185)
--(axis cs:4,0.3198258042685);
\path [draw=color2, semithick]
(axis cs:0,0.0945984106664224)
--(axis cs:0,0.125585364162488);
\path [draw=color2, semithick]
(axis cs:1,0.0828907256467127)
--(axis cs:1,0.0968321199076391);
\path [draw=color2, semithick]
(axis cs:2,0.110152102369697)
--(axis cs:2,0.126214586835473);
\path [draw=color2, semithick]
(axis cs:3,0.189223940211407)
--(axis cs:3,0.211525653761753);
\path [draw=color2, semithick]
(axis cs:4,0.318669492248454)
--(axis cs:4,0.340423053260885);
\addplot [semithick, color1, mark=*, mark size=2, mark options={solid}, only marks]
table {%
0 0.135572865605354
1 0.136145442724228
2 0.154989778995514
3 0.210629269480705
4 0.311356872320175
};
\addlegendentry{SVI (Adam /w 4096 steps)}
\addplot [semithick, color2, mark=*, mark size=2, mark options={solid}, only marks]
table {%
0 0.110091887414455
1 0.0898614227771759
2 0.118183344602585
3 0.20037479698658
4 0.329546272754669
};
\addlegendentry{Transformer (1.6M Prior Samples)}
\end{axis}
\end{tikzpicture}
\begin{tabular}{lrr}
\toprule
Name & Number of Features & Number of Symbolic Features \\
\midrule
kr-vs-kp & 37 & 37 \\
credit-g & 21 & 14 \\
vehicle & 19 & 1 \\
wine & 14 & 1 \\
kc1 & 22 & 1 \\
airlines & 8 & 5 \\
bank-marketing & 17 & 10 \\
blood-transfusion-service-center & 5 & 1 \\
phoneme & 6 & 1 \\
covertype & 55 & 45 \\
numerai28.6 & 22 & 1 \\
connect-4 & 43 & 43 \\
car & 7 & 7 \\
Australian & 15 & 9 \\
segment & 20 & 1 \\
jungle\_chess\_2pcs\_raw\_endgame\_complete & 7 & 1 \\
sylvine & 21 & 1 \\
MiniBooNE & 51 & 1 \\
jannis & 55 & 1 \\
helena & 28 & 1 \\
\bottomrule
\end{tabular}
\begin{tabular}{lrr}
\toprule
Name & Number of Features & Number of Symbolic Features \\
\midrule
haberman & 4 & 2 \\
ionosphere & 35 & 1 \\
sa-heart & 10 & 2 \\
cleve & 14 & 9 \\
four & 2 & 0 \\
german & 24 & 0 \\
\bottomrule
\end{tabular}
\begin{tabular}{lll}
\toprule
& & param \\
index & n & \\
\midrule
prior\_activations & 1 & torch.nn.Tanh \\
& 2 & torch.nn.ReLU \\
prior\_dropout\_sampler & 1 & lambda: 0.0 \\
& 2 & lambda: 0.5 \\
& 3 & beta\_sampler\_f(1.1, 1.4) \\
& 4 & beta\_sampler\_f(1.1, 2.0) \\
& 5 & beta\_sampler\_f(1.1, 4.0) \\
& 6 & beta\_sampler\_f(2.0, 1.4) \\
& 7 & beta\_sampler\_f(2.0, 4.0) \\
prior\_emsize\_sampler & 1 & scaled\_beta\_sampler\_f(1.1, 1.3, 256, max\_features) \\
& 2 & scaled\_beta\_sampler\_f(1.1, 2.0, 256, max\_features) \\
& 3 & scaled\_beta\_sampler\_f(1.1, 4.0, 256, max\_features) \\
& 4 & scaled\_beta\_sampler\_f(2.0, 4.0, 100, 1+max\_features) \\
prior\_nlayers\_sampler & 1 & lambda: 3 \\
& 2 & scaled\_beta\_sampler\_f(1.1, 1.3, 4, 3) \\
& 3 & scaled\_beta\_sampler\_f(1.1, 2.0, 5, 2) \\
& 4 & scaled\_beta\_sampler\_f(1.1, 4.0, 5, 2) \\
& 5 & scaled\_beta\_sampler\_f(2.0, 4.0, 4, 3) \\
prior\_noise\_std\_gamma\_k & 1 & range: [1.5, 5.0] \\
prior\_noise\_std\_gamma\_theta & 1 & range: [0.01, 1.0] \\
prior\_sigma\_gamma\_k & 1 & range: [1.5, 5.0] \\
prior\_sigma\_gamma\_theta & 1 & range: [0.01, 1.0] \\
\bottomrule
\end{tabular}
\small
\begin{tabular}{ll}
\toprule
Log. Regr. & \{'solver': ['saga'], 'penalty': ['l1', 'l2', 'none'], 'tol': [0.01, 0.0001, 1e-10]\\ &, 'max\_iter': [500], 'fit\_intercept': [True, False], 'C': [1e-05, 0.001, 0.01, 0.1, 1.0, 2.0]\} \\
KNN & \{'n\_neighbors (max number of samples)': array([1, 2, 3, 4, 5])\} \\
BNN & \{'embed': [5, 10, 30, 64], 'lr': [0.001, 0.0001], 'num\_training\_steps': [400]\\ &, 'num\_samples\_for\_prediction': [400]\} \\
GP & \{'params\_y\_scale': [0.05, 0.1, 0.5, 1.0, 5.0, 10.0], 'params\_length\_scale': [0.1, 0.5, 1.0, 2.0]\} \\
Catboost & \{'learning\_rate': [0.1, 0.5, 1.0], 'depth': [2, 4, 7], 'l2\_leaf\_reg': [0.0, 0.5, 1]\\ &, 'iterations': [10, 40, 70], 'loss\_function': ['Logloss']\} \\
XGB & \{'min\_child\_weight': [0.5, 1.0], 'learning\_rate': [0.02, 0.2], 'subsample': [0.5, 0.8]\\ &, 'max\_depth': [1, 2], 'colsample\_bytree': [0.8], 'eval\_metric': ['logloss'], 'n\_estimators': [100]\} \\
\bottomrule
\end{tabular}
\footnotesize
\setlength{\tabcolsep}{0.3em}
\begin{tabular}{lcc|cccccc}
\toprule
{} & PFN-BNN & PFN-GP & logistic & GP & KNN & BNN & Catboost & XGB \\
\midrule
kr-vs-kp & 0.928 & 0.897 & 0.920 & 0.886 & 0.754 & 0.843 & 0.653 & \textbf{1.000} \\
credit-g & 0.671 & \textbf{0.681} & 0.621 & 0.562 & 0.593 & 0.607 & 0.602 & 0.637 \\
vehicle & 0.562 & 0.501 & \textbf{0.601} & 0.506 & 0.527 & 0.562 & 0.525 & 0.522 \\
wine & 0.959 & \textbf{0.975} & 0.944 & 0.785 & 0.900 & 0.919 & 0.908 & 0.930 \\
kc1 & 0.614 & 0.695 & 0.730 & 0.695 & 0.666 & 0.724 & 0.708 & \textbf{0.747} \\
airlines & 0.916 & 0.896 & 0.913 & 0.873 & 0.834 & 0.907 & 0.982 & \textbf{0.985} \\
bank-market.. & 0.878 & 0.877 & 0.811 & 0.857 & 0.810 & 0.849 & \textbf{0.908} & 0.885 \\
blood-transfus.. & \textbf{0.588} & 0.576 & 0.558 & 0.577 & \textbf{0.588} & 0.555 & 0.565 & 0.531 \\
phoneme & 0.757 & 0.782 & 0.735 & \textbf{0.785} & 0.772 & 0.733 & 0.766 & \textbf{0.785} \\
covertype & \textbf{0.962} & 0.904 & 0.904 & 0.929 & 0.793 & 0.870 & 0.810 & 0.798 \\
numerai28.6 & 0.449 & 0.454 & 0.460 & \textbf{0.502} & \textbf{0.502} & 0.454 & \textbf{0.502} & 0.464 \\
connect-4 & \textbf{0.981} & 0.961 & 0.951 & 0.933 & 0.871 & 0.934 & 0.945 & 0.976 \\
car & \textbf{0.959} & 0.926 & 0.925 & 0.771 & 0.874 & 0.935 & 0.849 & 0.898 \\
Australian & \textbf{0.906} & 0.903 & 0.862 & 0.866 & 0.865 & 0.873 & 0.848 & 0.871 \\
segment & \textbf{1.000} & \textbf{1.000} & \textbf{1.000} & 0.999 & 0.985 & 0.999 & \textbf{1.000} & \textbf{1.000} \\
jungle\_chess.. & 0.774 & 0.780 & 0.693 & \textbf{0.852} & 0.747 & 0.708 & 0.841 & 0.708 \\
sylvine & 0.833 & 0.791 & 0.915 & 0.708 & 0.664 & 0.785 & 0.885 & \textbf{0.929} \\
MiniBooNE & 0.855 & 0.852 & 0.821 & 0.843 & 0.850 & \textbf{0.861} & 0.818 & 0.855 \\
dionis & \textbf{0.996} & 0.977 & 0.981 & 0.982 & 0.962 & 0.987 & 0.975 & 0.991 \\
jannis & \textbf{0.654} & 0.652 & 0.570 & 0.551 & 0.569 & 0.549 & 0.603 & 0.600 \\
helena & \textbf{0.911} & 0.892 & 0.839 & 0.870 & 0.842 & 0.882 & 0.870 & 0.905 \\
\midrule
Mean rank AUC & \textbf{2.786} & 3.833 & 4.690 & 5.286 & 6.214 & 5.000 & 4.833 & 3.357 \\
Win/Tie/Loss AUC vs T-BNN & 0/21/0 & 14/1/6 & 16/1/4 & 17/0/4 & 17/1/3 & 17/1/3 & 13/1/7 & 12/2/7 \\
Wilcoxon p. AUC vs T-BNN & 0.0e+00 & 3.6e-13 & 3.6e-13 & 3.9e-13 & 3.6e-13 & 3.6e-13 & 1.9e-12 & 1.6e-06 \\
\multirow{2}{*}{Benchmark Time} & \multicolumn{2}{c|}{GPU: 0:00:13} & \multirow{2}{*}{0:09:56} & \multirow{2}{*}{0:24:30} & \multirow{2}{*}{0:00:34} & \multirow{2}{*}{12:04:41} & \multirow{2}{*}{2:05:20} & \multirow{2}{*}{20:59:46} \\
& \multicolumn{2}{c|}{CPU: 0:04:23}&&&&&&\\
ECE & \textbf{0.025} & 0.067 & 0.157 & 0.095 & 0.093 & 0.089 & 0.157 & 0.066 \\
\bottomrule
\end{tabular}
\usepgfplotslibrary{groupplots}
% This file was created with tikzplotlib v0.9.12.
\begin{tikzpicture}
\definecolor{color0}{rgb}{0.12156862745098,0.466666666666667,0.705882352941177}
\begin{axis}[
width=\textwidth,
height=.7\textwidth,
legend cell align={left},
legend style={
fill opacity=0.8,
draw opacity=1,
text opacity=1,
at={(0.03,0.97)},
anchor=north west,
draw=white!80!black
},
tick align=outside,
tick pos=left,
x grid style={white!69.0196078431373!black},
xlabel={Confidence},
xmin=0, xmax=1,
xtick style={color=black},
xtick={0,0.2,0.4,0.6,0.8,1},
xticklabels={0.0,0.2,0.4,0.6,0.8,1.0},
y grid style={white!69.0196078431373!black},
ylabel={Accuracy},
ymin=0, ymax=1,
ytick style={color=black},
ytick={0,0.2,0.4,0.6,0.8,1},
yticklabels={0.0,0.2,0.4,0.6,0.8,1.0}
]
\draw[draw=black,fill=color0] (axis cs:-3.46944695195361e-18,0) rectangle (axis cs:0.05,0.12936200211491);
\draw[draw=black,fill=color0] (axis cs:0.05,0) rectangle (axis cs:0.1,0.181818181818182);
\draw[draw=black,fill=color0] (axis cs:0.1,0) rectangle (axis cs:0.15,0.218914185639229);
\draw[draw=black,fill=color0] (axis cs:0.15,0) rectangle (axis cs:0.2,0.266709928617781);
\draw[draw=black,fill=color0] (axis cs:0.2,0) rectangle (axis cs:0.25,0.316146540027137);
\draw[draw=black,fill=color0] (axis cs:0.25,0) rectangle (axis cs:0.3,0.316866619618913);
\draw[draw=black,fill=color0] (axis cs:0.3,0) rectangle (axis cs:0.35,0.389429763560501);
\draw[draw=black,fill=color0] (axis cs:0.35,0) rectangle (axis cs:0.4,0.395736175882745);
\draw[draw=black,fill=color0] (axis cs:0.4,0) rectangle (axis cs:0.45,0.418430034129693);
\draw[draw=black,fill=color0] (axis cs:0.45,0) rectangle (axis cs:0.5,0.482689747003995);
\draw[draw=black,fill=color0] (axis cs:0.5,0) rectangle (axis cs:0.55,0.523389830508475);
\draw[draw=black,fill=color0] (axis cs:0.55,0) rectangle (axis cs:0.6,0.534391534391534);
\draw[draw=black,fill=color0] (axis cs:0.6,0) rectangle (axis cs:0.65,0.565886287625418);
\draw[draw=black,fill=color0] (axis cs:0.65,0) rectangle (axis cs:0.7,0.592592592592593);
\draw[draw=black,fill=color0] (axis cs:0.7,0) rectangle (axis cs:0.75,0.644290657439446);
\draw[draw=black,fill=color0] (axis cs:0.75,0) rectangle (axis cs:0.8,0.697454545454545);
\draw[draw=black,fill=color0] (axis cs:0.8,0) rectangle (axis cs:0.85,0.737236215112321);
\draw[draw=black,fill=color0] (axis cs:0.85,0) rectangle (axis cs:0.9,0.780503550677857);
\draw[draw=black,fill=color0] (axis cs:0.9,0) rectangle (axis cs:0.95,0.838204592901879);
\draw[draw=black,fill=color0] (axis cs:0.95,0) rectangle (axis cs:1,0.886476426799007);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:-3.46944695195361e-18,0.12936200211491) rectangle (axis cs:0.05,0.0178040179312702);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.05,0.181818181818182) rectangle (axis cs:0.1,0.0691125543659061);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.1,0.218914185639229) rectangle (axis cs:0.15,0.118610623591105);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.15,0.266709928617781) rectangle (axis cs:0.2,0.169493837085039);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.2,0.316146540027137) rectangle (axis cs:0.25,0.219470826325132);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.25,0.316866619618913) rectangle (axis cs:0.3,0.270141142034665);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.3,0.389429763560501) rectangle (axis cs:0.35,0.320438113265641);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.35,0.395736175882745) rectangle (axis cs:0.4,0.370173216283043);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.4,0.418430034129693) rectangle (axis cs:0.45,0.419808872913745);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.45,0.482689747003995) rectangle (axis cs:0.5,0.469940078500584);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.5,0.523389830508475) rectangle (axis cs:0.55,0.519884739043349);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.55,0.534391534391534) rectangle (axis cs:0.6,0.56998676534683);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.6,0.565886287625418) rectangle (axis cs:0.65,0.620715723149354);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.65,0.592592592592593) rectangle (axis cs:0.7,0.67014245489384);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.7,0.644290657439446) rectangle (axis cs:0.75,0.719501734398641);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.75,0.697454545454545) rectangle (axis cs:0.8,0.76951999243823);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.8,0.737236215112321) rectangle (axis cs:0.85,0.820156561792749);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.85,0.780503550677857) rectangle (axis cs:0.9,0.87043899724836);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.9,0.838204592901879) rectangle (axis cs:0.95,0.920939462449954);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.95,0.886476426799007) rectangle (axis cs:1,0.977757448427582);
\addplot [semithick, red, dashed]
table {%
0 0
1 1
};
\end{axis}
\end{tikzpicture}
% This file was created with tikzplotlib v0.9.12.
\begin{tikzpicture}
\definecolor{color0}{rgb}{0.12156862745098,0.466666666666667,0.705882352941177}
\begin{axis}[
width=\textwidth,
height=.7\textwidth,
legend cell align={left},
legend style={
fill opacity=0.8,
draw opacity=1,
text opacity=1,
at={(0.03,0.97)},
anchor=north west,
draw=white!80!black
},
tick align=outside,
tick pos=left,
x grid style={white!69.0196078431373!black},
xlabel={Confidence},
xmin=0, xmax=1,
xtick style={color=black},
xtick={0,0.2,0.4,0.6,0.8,1},
xticklabels={0.0,0.2,0.4,0.6,0.8,1.0},
y grid style={white!69.0196078431373!black},
ylabel={Accuracy},
ymin=0, ymax=1,
ytick style={color=black},
ytick={0,0.2,0.4,0.6,0.8,1},
yticklabels={0.0,0.2,0.4,0.6,0.8,1.0}
]
\draw[draw=black,fill=color0] (axis cs:-3.46944695195361e-18,0) rectangle (axis cs:0.05,0.0433017591339648);
\draw[draw=black,fill=color0] (axis cs:0.05,0) rectangle (axis cs:0.1,0.088078967350038);
\draw[draw=black,fill=color0] (axis cs:0.1,0) rectangle (axis cs:0.15,0.13247470101196);
\draw[draw=black,fill=color0] (axis cs:0.15,0) rectangle (axis cs:0.2,0.162);
\draw[draw=black,fill=color0] (axis cs:0.2,0) rectangle (axis cs:0.25,0.195145631067961);
\draw[draw=black,fill=color0] (axis cs:0.25,0) rectangle (axis cs:0.3,0.261784511784512);
\draw[draw=black,fill=color0] (axis cs:0.3,0) rectangle (axis cs:0.35,0.280974949221395);
\draw[draw=black,fill=color0] (axis cs:0.35,0) rectangle (axis cs:0.4,0.317628374801482);
\draw[draw=black,fill=color0] (axis cs:0.4,0) rectangle (axis cs:0.45,0.368769298632554);
\draw[draw=black,fill=color0] (axis cs:0.45,0) rectangle (axis cs:0.5,0.46429879165141);
\draw[draw=black,fill=color0] (axis cs:0.5,0) rectangle (axis cs:0.55,0.55501130369254);
\draw[draw=black,fill=color0] (axis cs:0.55,0) rectangle (axis cs:0.6,0.637357414448669);
\draw[draw=black,fill=color0] (axis cs:0.6,0) rectangle (axis cs:0.65,0.669281834640917);
\draw[draw=black,fill=color0] (axis cs:0.65,0) rectangle (axis cs:0.7,0.694975230007077);
\draw[draw=black,fill=color0] (axis cs:0.7,0) rectangle (axis cs:0.75,0.732597623089983);
\draw[draw=black,fill=color0] (axis cs:0.75,0) rectangle (axis cs:0.8,0.748565965583174);
\draw[draw=black,fill=color0] (axis cs:0.8,0) rectangle (axis cs:0.85,0.807142857142857);
\draw[draw=black,fill=color0] (axis cs:0.85,0) rectangle (axis cs:0.9,0.850485436893204);
\draw[draw=black,fill=color0] (axis cs:0.9,0) rectangle (axis cs:0.95,0.910614525139665);
\draw[draw=black,fill=color0] (axis cs:0.95,0) rectangle (axis cs:1,0.97517200119653);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:-3.46944695195361e-18,0.0433017591339648) rectangle (axis cs:0.05,0.0202381901355929);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.05,0.088078967350038) rectangle (axis cs:0.1,0.0733544398650765);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.1,0.13247470101196) rectangle (axis cs:0.15,0.12411363109631);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.15,0.162) rectangle (axis cs:0.2,0.175409789383411);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.2,0.195145631067961) rectangle (axis cs:0.25,0.225348050067726);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.25,0.261784511784512) rectangle (axis cs:0.3,0.275195286756613);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.3,0.280974949221395) rectangle (axis cs:0.35,0.325973994903816);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.35,0.317628374801482) rectangle (axis cs:0.4,0.376399218483915);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.4,0.368769298632554) rectangle (axis cs:0.45,0.426100170804269);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.45,0.46429879165141) rectangle (axis cs:0.5,0.476206602292024);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.5,0.55501130369254) rectangle (axis cs:0.55,0.523720110033182);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.55,0.637357414448669) rectangle (axis cs:0.6,0.57415763192417);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.6,0.669281834640917) rectangle (axis cs:0.65,0.624607355427728);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.65,0.694975230007077) rectangle (axis cs:0.7,0.674909640775871);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.7,0.732597623089983) rectangle (axis cs:0.75,0.724874381146731);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.75,0.748565965583174) rectangle (axis cs:0.8,0.774502390144432);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.8,0.807142857142857) rectangle (axis cs:0.85,0.825419077520468);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.85,0.850485436893204) rectangle (axis cs:0.9,0.875472218261182);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.9,0.910614525139665) rectangle (axis cs:0.95,0.926426417762912);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.95,0.97517200119653) rectangle (axis cs:1,0.981677366968332);
\addplot [semithick, red, dashed]
table {%
0 0
1 1
};
\end{axis}
\end{tikzpicture}
% This file was created with tikzplotlib v0.9.12.
\begin{tikzpicture}
\definecolor{color0}{rgb}{0.12156862745098,0.466666666666667,0.705882352941177}
\begin{axis}[
width=\textwidth,
height=.7\textwidth,
legend cell align={left},
legend style={
fill opacity=0.8,
draw opacity=1,
text opacity=1,
at={(0.03,0.97)},
anchor=north west,
draw=white!80!black
},
tick align=outside,
tick pos=left,
x grid style={white!69.0196078431373!black},
xlabel={Confidence},
xmin=0, xmax=1,
xtick style={color=black},
xtick={0,0.2,0.4,0.6,0.8,1},
xticklabels={0.0,0.2,0.4,0.6,0.8,1.0},
y grid style={white!69.0196078431373!black},
ylabel={Accuracy},
ymin=0, ymax=1,
ytick style={color=black},
ytick={0,0.2,0.4,0.6,0.8,1},
yticklabels={0.0,0.2,0.4,0.6,0.8,1.0}
]
\draw[draw=black,fill=color0] (axis cs:-3.46944695195361e-18,0) rectangle (axis cs:0.05,0.234159779614325);
\draw[draw=black,fill=color0] (axis cs:0.05,0) rectangle (axis cs:0.1,0.163972286374134);
\draw[draw=black,fill=color0] (axis cs:0.1,0) rectangle (axis cs:0.15,0.285714285714286);
\draw[draw=black,fill=color0] (axis cs:0.15,0) rectangle (axis cs:0.2,0.0953461975028377);
\draw[draw=black,fill=color0] (axis cs:0.2,0) rectangle (axis cs:0.25,0.0990598486585646);
\draw[draw=black,fill=color0] (axis cs:0.25,0) rectangle (axis cs:0.3,0.210463733650416);
\draw[draw=black,fill=color0] (axis cs:0.3,0) rectangle (axis cs:0.35,0.286348501664817);
\draw[draw=black,fill=color0] (axis cs:0.35,0) rectangle (axis cs:0.4,0.379097093382808);
\draw[draw=black,fill=color0] (axis cs:0.4,0) rectangle (axis cs:0.45,0.426985981308411);
\draw[draw=black,fill=color0] (axis cs:0.45,0) rectangle (axis cs:0.5,0.475152129817444);
\draw[draw=black,fill=color0] (axis cs:0.5,0) rectangle (axis cs:0.55,0.556955380577428);
\draw[draw=black,fill=color0] (axis cs:0.55,0) rectangle (axis cs:0.6,0.604680581910183);
\draw[draw=black,fill=color0] (axis cs:0.6,0) rectangle (axis cs:0.65,0.631612492033142);
\draw[draw=black,fill=color0] (axis cs:0.65,0) rectangle (axis cs:0.7,0.70326409495549);
\draw[draw=black,fill=color0] (axis cs:0.7,0) rectangle (axis cs:0.75,0.761164274322169);
\draw[draw=black,fill=color0] (axis cs:0.75,0) rectangle (axis cs:0.8,0.903540033262057);
\draw[draw=black,fill=color0] (axis cs:0.8,0) rectangle (axis cs:0.85,0.906064209274673);
\draw[draw=black,fill=color0] (axis cs:0.85,0) rectangle (axis cs:0.9,0.637602179836512);
\draw[draw=black,fill=color0] (axis cs:0.9,0) rectangle (axis cs:0.95,0.798561151079137);
\draw[draw=black,fill=color0] (axis cs:0.95,0) rectangle (axis cs:1,0.838323353293413);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:-3.46944695195361e-18,0.234159779614325) rectangle (axis cs:0.05,0.0302444138402535);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.05,0.163972286374134) rectangle (axis cs:0.1,0.0725316161130647);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.1,0.285714285714286) rectangle (axis cs:0.15,0.125762406482089);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.15,0.0953461975028377) rectangle (axis cs:0.2,0.183911222545906);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.2,0.0990598486585646) rectangle (axis cs:0.25,0.220213881905843);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.25,0.210463733650416) rectangle (axis cs:0.3,0.274677219794376);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.3,0.286348501664817) rectangle (axis cs:0.35,0.324025766285225);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.35,0.379097093382808) rectangle (axis cs:0.4,0.375070166904836);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.4,0.426985981308411) rectangle (axis cs:0.45,0.4248526966913);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.45,0.475152129817444) rectangle (axis cs:0.5,0.47440601980529);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.5,0.556955380577428) rectangle (axis cs:0.55,0.524757606933123);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.55,0.604680581910183) rectangle (axis cs:0.6,0.574651827546783);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.6,0.631612492033142) rectangle (axis cs:0.65,0.626222473067642);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.65,0.70326409495549) rectangle (axis cs:0.7,0.676159902950187);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.7,0.761164274322169) rectangle (axis cs:0.75,0.72541985818834);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.75,0.903540033262057) rectangle (axis cs:0.8,0.77891449854362);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.8,0.906064209274673) rectangle (axis cs:0.85,0.816068536815689);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.85,0.637602179836512) rectangle (axis cs:0.9,0.876510673063003);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.9,0.798561151079137) rectangle (axis cs:0.95,0.928930321936127);
\draw[draw=black,fill=red,opacity=0.6] (axis cs:0.95,0.838323353293413) rectangle (axis cs:1,0.970025982150061);
\addplot [semithick, red, dashed]
table {%
0 0
1 1
};
\end{axis}
\end{tikzpicture}
\begin{thebibliography}{66}
\providecommand{\natexlab}[1]{#1}
\providecommand{\url}[1]{\texttt{#1}}
\expandafter\ifx\csname urlstyle\endcsname\relax
\providecommand{\doi}[1]{doi: #1}\else
\providecommand{\doi}{doi: \begingroup \urlstyle{rm}\Url}\fi
\bibitem[Amit \& Meir(2018)Amit and Meir]{amit2018meta}
Ron Amit and Ron Meir.
\newblock Meta-learning by adjusting priors based on extended pac-bayes theory.
\newblock In \emph{International Conference on Machine Learning}, pp.\
205--214. PMLR, 2018.
\bibitem[Andrieu et~al.(2003)Andrieu, De~Freitas, Doucet, and
Jordan]{andrieu2003introduction}
Christophe Andrieu, Nando De~Freitas, Arnaud Doucet, and Michael~I Jordan.
\newblock An introduction to mcmc for machine learning.
\newblock \emph{Machine learning}, 50\penalty0 (1):\penalty0 5--43, 2003.
\bibitem[Balandat et~al.(2020)Balandat, Karrer, Jiang, Daulton, Letham, Wilson,
and Bakshy]{balandat2020botorch}
Maximilian Balandat, Brian Karrer, Daniel~R. Jiang, Samuel Daulton, Benjamin
Letham, Andrew~Gordon Wilson, and Eytan Bakshy.
\newblock {BoTorch: A Framework for Efficient Monte-Carlo Bayesian
Optimization}.
\newblock In \emph{Advances in Neural Information Processing Systems 33}, 2020.
\bibitem[Bellemare et~al.(2017)Bellemare, Dabney, and
Munos]{bellemare2017distributional}
Marc~G Bellemare, Will Dabney, and R{\'e}mi Munos.
\newblock A distributional perspective on reinforcement learning.
\newblock In \emph{International Conference on Machine Learning}, pp.\
449--458. PMLR, 2017.
\bibitem[Bingham et~al.(2018)Bingham, Chen, Jankowiak, Obermeyer, Pradhan,
Karaletsos, Singh, Szerlip, Horsfall, and Goodman]{bingham2018pyro}
Eli Bingham, Jonathan~P. Chen, Martin Jankowiak, Fritz Obermeyer, Neeraj
Pradhan, Theofanis Karaletsos, Rohit Singh, Paul Szerlip, Paul Horsfall, and
Noah~D. Goodman.
\newblock {Pyro: Deep Universal Probabilistic Programming}.
\newblock \emph{Journal of Machine Learning Research}, 2018.
\bibitem[Blei et~al.(2017)Blei, Kucukelbir, and McAuliffe]{Blei_2017}
David~M. Blei, Alp Kucukelbir, and Jon~D. McAuliffe.
\newblock Variational inference: A review for statisticians.
\newblock \emph{Journal of the American Statistical Association}, 112\penalty0
(518):\penalty0 859–877, Apr 2017.
\newblock ISSN 1537-274X.
\newblock \doi{10.1080/01621459.2017.1285773}.
\bibitem[Blundell et~al.(2015)Blundell, Cornebise, Kavukcuoglu, and
Wierstra]{blundell-icml15a}
C~Blundell, J.~Cornebise, K.~Kavukcuoglu, and D.~Wierstra.
\newblock Weight uncertainty in neural network.
\newblock In F.~Bach and D.~Blei (eds.), \emph{Proceedings of the 32nd
International Conference on Machine Learning ({ICML}'15)}, volume~37, pp.\
1613--1622. Omnipress, 2015.
\bibitem[Brown et~al.(2020)Brown, Mann, Ryder, Subbiah, Kaplan, Dhariwal,
Neelakantan, Shyam, Sastry, Askell, et~al.]{gpt-3}
Tom~B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla
Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell,
et~al.
\newblock Language models are few-shot learners.
\newblock \emph{arXiv preprint arXiv:2005.14165}, 2020.
\bibitem[Chan et~al.(2018)Chan, Perrone, Spence, Jenkins, Mathieson, and
Song]{chan2018likelihood}
Jeffrey Chan, Valerio Perrone, Jeffrey~P Spence, Paul~A Jenkins, Sara
Mathieson, and Yun~S Song.
\newblock A likelihood-free inference framework for population genetic data
using exchangeable neural networks.
\newblock In \emph{NeurIPS}, 2018.
\bibitem[Chen \& Guestrin(2016)Chen and Guestrin]{xgboost}
Tianqi Chen and Carlos Guestrin.
\newblock Xgboost: A scalable tree boosting system.
\newblock In \emph{Proceedings of the 22nd ACM SIGKDD International Conference
on Knowledge Discovery and Data Mining}, KDD '16, pp.\ 785–794, New York,
NY, USA, 2016. Association for Computing Machinery.
\newblock ISBN 9781450342322.
\newblock \doi{10.1145/2939672.2939785}.
\bibitem[Chen et~al.(2020)Chen, Kornblith, Norouzi, and Hinton]{chen2020simclr}
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton.
\newblock A simple framework for contrastive learning of visual
representations.
\newblock In \emph{International conference on machine learning}, pp.\
1597--1607. PMLR, 2020.
\bibitem[Chen \& He(2021)Chen and He]{chen2021simsiam}
Xinlei Chen and Kaiming He.
\newblock Exploring simple siamese representation learning.
\newblock In \emph{Proceedings of the IEEE/CVF Conference on Computer Vision
and Pattern Recognition}, pp.\ 15750--15758, 2021.
\bibitem[Cranmer et~al.(2020)Cranmer, Brehmer, and Louppe]{cranmer2020review}
Kyle Cranmer, Johann Brehmer, and Gilles Louppe.
\newblock The frontier of simulation-based inference.
\newblock \emph{Proceedings of the National Academy of Sciences}, 117\penalty0
(48):\penalty0 30055--30062, 2020.
\bibitem[Cubuk et~al.(2020)Cubuk, Zoph, Shlens, and Le]{cubuk2020randaugment}
Ekin~D Cubuk, Barret Zoph, Jonathon Shlens, and Quoc~V Le.
\newblock Randaugment: Practical automated data augmentation with a reduced
search space.
\newblock In \emph{Proceedings of the IEEE/CVF Conference on Computer Vision
and Pattern Recognition Workshops}, pp.\ 702--703, 2020.
\bibitem[Dem\v{s}ar(2006)]{wilcox}
Janez Dem\v{s}ar.
\newblock Statistical comparisons of classifiers over multiple data sets.
\newblock \emph{J. Mach. Learn. Res.}, 7:\penalty0 1–30, December 2006.
\newblock ISSN 1532-4435.
\bibitem[Devlin et~al.(2018)Devlin, Chang, Lee, and Toutanova]{devlin2018bert}
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.
\newblock Bert: Pre-training of deep bidirectional transformers for language
understanding.
\newblock \emph{arXiv preprint arXiv:1810.04805}, 2018.
\bibitem[Finn et~al.(2017{\natexlab{a}})Finn, Abbeel, and Levine]{finn-icml17a}
C.~Finn, P.~Abbeel, and S.~Levine.
\newblock Model-agnostic meta-learning for fast adaptation of deep networks.
\newblock In D.~Precup and Y.~Teh (eds.), \emph{Proceedings of the 34th
International Conference on Machine Learning ({ICML}'17)}, volume~70, pp.\
1126--1135. Proceedings of Machine Learning Research, 2017{\natexlab{a}}.
\bibitem[Finn et~al.(2017{\natexlab{b}})Finn, Yu, Zhang, Abbeel, and
Levine]{finn-corl17a}
C.~Finn, T.~Yu, T.~Zhang, P.~Abbeel, and S.~Levine.
\newblock One-shot visual imitation learning via meta-learning.
\newblock In \emph{Proceedings of the 1st Annual Conference on Robot Learning
(Co{RL})}, volume~78, pp.\ 357--368. {PMLR}, 2017{\natexlab{b}}.
\bibitem[Fortuin et~al.(2021)Fortuin, Garriga-Alonso, Wenzel, Rätsch, Turner,
van~der Wilk, and Aitchison]{fortuin2021bayesian}
Vincent Fortuin, Adrià Garriga-Alonso, Florian Wenzel, Gunnar Rätsch, Richard
Turner, Mark van~der Wilk, and Laurence Aitchison.
\newblock Bayesian neural network priors revisited, 2021.
\bibitem[Garnelo et~al.(2018{\natexlab{a}})Garnelo, Rosenbaum, Maddison,
Ramalho, Saxton, Shanahan, Teh, Rezende, and Eslami]{garnelo2018CNP}
Marta Garnelo, Dan Rosenbaum, Christopher Maddison, Tiago Ramalho, David
Saxton, Murray Shanahan, Yee~Whye Teh, Danilo Rezende, and SM~Ali Eslami.
\newblock Conditional neural processes.
\newblock In \emph{International Conference on Machine Learning}, pp.\
1704--1713. PMLR, 2018{\natexlab{a}}.
\bibitem[Garnelo et~al.(2018{\natexlab{b}})Garnelo, Schwarz, Rosenbaum, Viola,
Rezende, Eslami, and Teh]{garnelo2018NP}
Marta Garnelo, Jonathan Schwarz, Dan Rosenbaum, Fabio Viola, Danilo~J Rezende,
SM~Eslami, and Yee~Whye Teh.
\newblock Neural processes.
\newblock \emph{arXiv preprint arXiv:1807.01622}, 2018{\natexlab{b}}.
\bibitem[Gijsbers et~al.(2019)Gijsbers, LeDell, Poirier, Thomas, Bischl, and
Vanschoren]{amlb2019}
P.~Gijsbers, E.~LeDell, S.~Poirier, J.~Thomas, B.~Bischl, and J.~Vanschoren.
\newblock An open source automl benchmark.
\newblock \emph{arXiv preprint arXiv:1907.00909 [cs.LG]}, 2019.
\newblock Accepted at AutoML Workshop at ICML 2019.
\bibitem[Gordon et~al.(2019)Gordon, Bronskill, Bauer, Nowozin, and
Turner]{gordon2018ml-pip}
Jonathan Gordon, John Bronskill, Matthias Bauer, Sebastian Nowozin, and Richard
Turner.
\newblock Meta-learning probabilistic inference for prediction.
\newblock In \emph{International Conference on Learning Representations}, 2019.
\bibitem[Hara et~al.(2017)Hara, Saitoh, and Shouno]{dropout_ensemble}
Kazuyuki Hara, Daisuke Saitoh, and Hayaru Shouno.
\newblock Analysis of dropout learning regarded as ensemble learning.
\newblock \emph{CoRR}, abs/1706.06859, 2017.
\bibitem[He et~al.(2016)He, Zhang, Ren, and Sun]{he2016resnet}
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.
\newblock Deep residual learning for image recognition.
\newblock In \emph{Proceedings of the IEEE conference on computer vision and
pattern recognition}, pp.\ 770--778, 2016.
\bibitem[Hendrycks et~al.(2019)Hendrycks, Mu, Cubuk, Zoph, Gilmer, and
Lakshminarayanan]{hendrycks2019augmix}
Dan Hendrycks, Norman Mu, Ekin~Dogus Cubuk, Barret Zoph, Justin Gilmer, and
Balaji Lakshminarayanan.
\newblock Augmix: A simple data processing method to improve robustness and
uncertainty.
\newblock In \emph{International Conference on Learning Representations}, 2019.
\bibitem[Hochreiter et~al.(2001)Hochreiter, Younger, and
Conwell]{hochreiter2001learning}
Sepp Hochreiter, A~Steven Younger, and Peter~R Conwell.
\newblock Learning to learn using gradient descent.
\newblock In \emph{International Conference on Artificial Neural Networks},
pp.\ 87--94. Springer, 2001.
\bibitem[Hoffman et~al.(2013)Hoffman, Blei, Wang, and Paisley]{svi2013hoffman}
Matthew~D. Hoffman, David~M. Blei, Chong Wang, and John Paisley.
\newblock Stochastic variational inference.
\newblock \emph{Journal of Machine Learning Research}, 14\penalty0
(4):\penalty0 1303--1347, 2013.
\bibitem[Hoffman et~al.(2014)Hoffman, Gelman, et~al.]{hoffman2014no}
Matthew~D Hoffman, Andrew Gelman, et~al.
\newblock The no-u-turn sampler: adaptively setting path lengths in hamiltonian
monte carlo.
\newblock \emph{J. Mach. Learn. Res.}, 15\penalty0 (1):\penalty0 1593--1623,
2014.
\bibitem[Izmailov et~al.(2021)Izmailov, Vikram, Hoffman, and
Wilson]{bnn_posteriors}
Pavel Izmailov, Sharad Vikram, Matthew~D. Hoffman, and Andrew~Gordon Wilson.
\newblock What are bayesian neural network posteriors really like?
\newblock \emph{CoRR}, abs/2104.14421, 2021.
\bibitem[Jordan et~al.(1999)Jordan, Ghahramani, Jaakkola, and
Saul]{jordan1999introduction}
Michael~I Jordan, Zoubin Ghahramani, Tommi~S Jaakkola, and Lawrence~K Saul.
\newblock An introduction to variational methods for graphical models.
\newblock \emph{Machine learning}, 37\penalty0 (2):\penalty0 183--233, 1999.
\bibitem[Kadra et~al.(2021)Kadra, Lindauer, Hutter, and
Grabocka]{kadra2021regularization}
Arlind Kadra, Marius Lindauer, Frank Hutter, and Josif Grabocka.
\newblock Regularization is all you need: Simple neural nets can excel on
tabular data, 2021.
\bibitem[Kandasamy et~al.(2019)Kandasamy, Neiswanger, Schneider, Poczos, and
Xing]{nat_with_bo}
Kirthevasan Kandasamy, Willie Neiswanger, Jeff Schneider, Barnabas Poczos, and
Eric Xing.
\newblock Neural architecture search with bayesian optimisation and optimal
transport, 2019.
\bibitem[Kim et~al.(2018)Kim, Mnih, Schwarz, Garnelo, Eslami, Rosenbaum,
Vinyals, and Teh]{kim2018ANP}
Hyunjik Kim, Andriy Mnih, Jonathan Schwarz, Marta Garnelo, Ali Eslami, Dan
Rosenbaum, Oriol Vinyals, and Yee~Whye Teh.
\newblock Attentive neural processes.
\newblock In \emph{International Conference on Learning Representations}, 2018.
\bibitem[Kingma \& Ba(2015)Kingma and Ba]{kingma2015adam}
Diederik~P Kingma and Jimmy Ba.
\newblock Adam: A method for stochastic optimization.
\newblock In \emph{ICLR (Poster)}, 2015.
\bibitem[Krizhevsky et~al.(2012)Krizhevsky, Sutskever, and Hinton]{alexnet}
Alex Krizhevsky, Ilya Sutskever, and Geoffrey~E Hinton.
\newblock Imagenet classification with deep convolutional neural networks.
\newblock In F.~Pereira, C.~J.~C. Burges, L.~Bottou, and K.~Q. Weinberger
(eds.), \emph{Advances in Neural Information Processing Systems}, volume~25.
Curran Associates, Inc., 2012.
\bibitem[Lake et~al.(2015)Lake, Salakhutdinov, and Tenenbaum]{lake2015omniglot}
Brenden~M Lake, Ruslan Salakhutdinov, and Joshua~B Tenenbaum.
\newblock Human-level concept learning through probabilistic program induction.
\newblock \emph{Science}, 350\penalty0 (6266):\penalty0 1332--1338, 2015.
\bibitem[LeCun et~al.(1989)LeCun, Boser, Denker, Henderson, Howard, Hubbard,
and Jackel]{convnet}
Y.~LeCun, B.~Boser, J.~S. Denker, D.~Henderson, R.~E. Howard, W.~Hubbard, and
L.~D. Jackel.
\newblock Backpropagation applied to handwritten zip code recognition.
\newblock \emph{Neural Computation}, 1\penalty0 (4):\penalty0 541--551, 1989.
\newblock \doi{10.1162/neco.1989.1.4.541}.
\bibitem[Liu et~al.(2018)Liu, Simonyan, and Yang]{liu2018darts}
Hanxiao Liu, Karen Simonyan, and Yiming Yang.
\newblock Darts: Differentiable architecture search.
\newblock In \emph{International Conference on Learning Representations}, 2018.
\bibitem[Liu \& Wang(2016)Liu and Wang]{liu2016stein}
Qiang Liu and Dilin Wang.
\newblock Stein variational gradient descent: A general purpose bayesian
inference algorithm.
\newblock \emph{arXiv preprint arXiv:1608.04471}, 2016.
\bibitem[Loshchilov \& Hutter(2016)Loshchilov and Hutter]{loshchilov2016sgdr}
Ilya Loshchilov and Frank Hutter.
\newblock Sgdr: Stochastic gradient descent with warm restarts.
\newblock \emph{arXiv preprint arXiv:1608.03983}, 2016.
\bibitem[Lueckmann et~al.(2021)Lueckmann, Boelts, Greenberg, Goncalves, and
Macke]{lueckmann2021benchmarking}
Jan-Matthis Lueckmann, Jan Boelts, David Greenberg, Pedro Goncalves, and Jakob
Macke.
\newblock Benchmarking simulation-based inference.
\newblock In \emph{International Conference on Artificial Intelligence and
Statistics}, pp.\ 343--351. PMLR, 2021.
\bibitem[MacKay(1992)]{mackay1992practical}
David~JC MacKay.
\newblock A practical bayesian framework for backpropagation networks.
\newblock \emph{Neural Computation}, 4\penalty0 (3):\penalty0 448--472, 1992.
\bibitem[Naeini et~al.(2015)Naeini, Cooper, and Hauskrecht]{ece}
Mahdi~Pakdaman Naeini, Gregory~F. Cooper, and Milos Hauskrecht.
\newblock Obtaining well calibrated probabilities using bayesian binning.
\newblock In \emph{Proceedings of the Twenty-Ninth AAAI Conference on
Artificial Intelligence}, AAAI'15, pp.\ 2901–2907. AAAI Press, 2015.
\newblock ISBN 0262511290.
\bibitem[Neal(1996)]{neal-bayes96a}
R.~Neal.
\newblock \emph{Bayesian Learning for Neural Networks}.
\newblock 1996.
\bibitem[Ortega et~al.(2019)Ortega, Wang, Rowland, Genewein, Kurth-Nelson,
Pascanu, Heess, Veness, Pritzel, Sprechmann, et~al.]{ortega2019meta}
Pedro~A Ortega, Jane~X Wang, Mark Rowland, Tim Genewein, Zeb Kurth-Nelson,
Razvan Pascanu, Nicolas Heess, Joel Veness, Alex Pritzel, Pablo Sprechmann,
et~al.
\newblock Meta-learning of sequential strategies.
\newblock \emph{arXiv preprint arXiv:1905.03030}, 2019.
\bibitem[Papamakarios \& Murray(2016)Papamakarios and
Murray]{papamakarios2016fast}
George Papamakarios and Iain Murray.
\newblock Fast $\varepsilon$-free inference of simulation models with bayesian
conditional density estimation.
\newblock In \emph{Advances in neural information processing systems}, pp.\
1028--1036, 2016.
\bibitem[Paszke et~al.(2019)Paszke, Gross, Massa, Lerer, Bradbury, Chanan,
Killeen, Lin, Gimelshein, Antiga, Desmaison, Kopf, Yang, DeVito, Raison,
Tejani, Chilamkurthy, Steiner, Fang, Bai, and Chintala]{NEURIPS2019_9015}
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory
Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban
Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan
Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu~Fang, Junjie Bai, and Soumith
Chintala.
\newblock Pytorch: An imperative style, high-performance deep learning library.
\newblock In H.~Wallach, H.~Larochelle, A.~Beygelzimer, F.~d\textquotesingle
Alch\'{e}-Buc, E.~Fox, and R.~Garnett (eds.), \emph{Advances in Neural
Information Processing Systems 32}, pp.\ 8024--8035. Curran Associates,
Inc., 2019.
\bibitem[Prokhorenkova et~al.(2018)Prokhorenkova, Gusev, Vorobev, Dorogush, and
Gulin]{catboost}
Liudmila Prokhorenkova, Gleb Gusev, Aleksandr Vorobev, Anna~Veronika Dorogush,
and Andrey Gulin.
\newblock Catboost: Unbiased boosting with categorical features.
\newblock In \emph{Proceedings of the 32nd International Conference on Neural
Information Processing Systems}, NIPS'18, pp.\ 6639–6649, Red Hook, NY,
USA, 2018. Curran Associates Inc.
\bibitem[Rasmussen \& Williams(2006)Rasmussen and Williams]{rasmussen-book06a}
C.~Rasmussen and C.~Williams.
\newblock \emph{Gaussian Processes for Machine Learning}.
\newblock The MIT Press, 2006.
\bibitem[Rasmussen \& Williams(2005)Rasmussen and Williams]{gp2005rasmussen}
Carl~Edward Rasmussen and Christopher K.~I. Williams.
\newblock \emph{Gaussian Processes for Machine Learning (Adaptive Computation
and Machine Learning)}.
\newblock The MIT Press, 2005.
\newblock ISBN 026218253X.
\bibitem[Ritchie et~al.(2016)Ritchie, Horsfall, and Goodman]{ritchie2016deep}
Daniel Ritchie, Paul Horsfall, and Noah~D Goodman.
\newblock Deep amortized inference for probabilistic programs.
\newblock \emph{arXiv preprint arXiv:1610.05735}, 2016.
\bibitem[Rothfuss et~al.(2021)Rothfuss, Fortuin, Josifoski, and
Krause]{rothfuss2021pacoh}
Jonas Rothfuss, Vincent Fortuin, Martin Josifoski, and Andreas Krause.
\newblock Pacoh: Bayes-optimal meta-learning with pac-guarantees.
\newblock In \emph{International Conference on Machine Learning}, pp.\
9116--9126. PMLR, 2021.
\bibitem[Ru et~al.(2021)Ru, Wan, Dong, and Osborne]{ru2021interpretable}
Binxin Ru, Xingchen Wan, Xiaowen Dong, and Michael Osborne.
\newblock Interpretable neural architecture search via bayesian optimisation
with weisfeiler-lehman kernels.
\newblock In \emph{International Conference on Learning Representations}, 2021.
\bibitem[Santoro et~al.(2016)Santoro, Bartunov, Botvinick, Wierstra, and
Lillicrap]{santoro2016meta}
Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy
Lillicrap.
\newblock Meta-learning with memory-augmented neural networks.
\newblock In \emph{International conference on machine learning}, pp.\
1842--1850. PMLR, 2016.
\bibitem[Snoek et~al.(2012)Snoek, Larochelle, and Adams]{snoek-nips12a}
J.~Snoek, H.~Larochelle, and R.~Adams.
\newblock Practical {B}ayesian optimization of machine learning algorithms.
\newblock In P.~Bartlett, F.~Pereira, C.~Burges, L.~Bottou, and K.~Weinberger
(eds.), \emph{Proceedings of the 25th International Conference on Advances in
Neural Information Processing Systems ({N}eur{IPS}'12)}, pp.\ 2960--2968,
2012.
\bibitem[Solomonoff(1997)]{SOLOMONOFF199773}
Ray~J Solomonoff.
\newblock The discovery of algorithmic probability.
\newblock \emph{Journal of Computer and System Sciences}, 55\penalty0
(1):\penalty0 73--88, 1997.
\newblock ISSN 0022-0000.
\newblock \doi{https://doi.org/10.1006/jcss.1997.1500}.
\bibitem[Vaswani et~al.(2017)Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez,
Kaiser, and Polosukhin]{vaswani-neurips17}
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones,
Aidan~N Gomez, \L~ukasz Kaiser, and Illia Polosukhin.
\newblock Attention is all you need.
\newblock In I.~Guyon, U.~V. Luxburg, S.~Bengio, H.~Wallach, R.~Fergus,
S.~Vishwanathan, and R.~Garnett (eds.), \emph{Advances in Neural Information
Processing Systems}, volume~30. Curran Associates, Inc., 2017.
\bibitem[Wainwright \& Jordan(2008)Wainwright and
Jordan]{wainwright2008graphical}
Martin~J Wainwright and Michael~Irwin Jordan.
\newblock \emph{Graphical models, exponential families, and variational
inference}.
\newblock Now Publishers Inc, 2008.
\bibitem[Wang \& Komatsuzaki(2021)Wang and Komatsuzaki]{gpt-j}
Ben Wang and Aran Komatsuzaki.
\newblock {GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model}.
\newblock \url{https://github.com/kingoflolz/mesh-transformer-jax}, May 2021.
\bibitem[Welling \& Teh(2011)Welling and Teh]{welling2011bayesian}
Max Welling and Yee~W Teh.
\newblock Bayesian learning via stochastic gradient langevin dynamics.
\newblock In \emph{Proceedings of the 28th international conference on machine
learning (ICML-11)}, pp.\ 681--688. Citeseer, 2011.
\bibitem[Wolpert \& Macready(1997)Wolpert and Macready]{wolpert1997nofreelunch}
David~H Wolpert and William~G Macready.
\newblock No free lunch theorems for optimization.
\newblock \emph{IEEE transactions on evolutionary computation}, 1\penalty0
(1):\penalty0 67--82, 1997.
\bibitem[Yoon et~al.(2018)Yoon, Kim, Dia, Kim, Bengio, and
Ahn]{yoon2018bayesian}
Jaesik Yoon, Taesup Kim, Ousmane Dia, Sungwoong Kim, Yoshua Bengio, and Sungjin
Ahn.
\newblock Bayesian model-agnostic meta-learning.
\newblock In \emph{Proceedings of the 32nd International Conference on Neural
Information Processing Systems}, pp.\ 7343--7353, 2018.
\bibitem[Zaidi et~al.(2020)Zaidi, Zela, Elsken, Holmes, Hutter, and
Teh]{zaidi2020neural}
Sheheryar Zaidi, Arber Zela, Thomas Elsken, Chris Holmes, Frank Hutter, and
Yee~Whye Teh.
\newblock Neural ensemble search for uncertainty estimation and dataset shift.
\newblock \emph{arXiv preprint arXiv:2006.08573}, 2020.
\bibitem[Zhou et~al.(2019)Zhou, Yang, Wang, and Pan]{bayes_nas}
Hongpeng Zhou, Minghao Yang, Jun Wang, and Wei Pan.
\newblock Bayesnas: {A} bayesian approach for neural architecture search.
\newblock \emph{CoRR}, abs/1905.04919, 2019.
\bibitem[Zoph et~al.(2018)Zoph, Vasudevan, Shlens, and Le]{zoph2018nasnet}
Barret Zoph, Vijay Vasudevan, Jonathon Shlens, and Quoc~V Le.
\newblock Learning transferable architectures for scalable image recognition.
\newblock In \emph{Proceedings of the IEEE conference on computer vision and
pattern recognition}, pp.\ 8697--8710, 2018.
\end{thebibliography}