"Do it my way!": Impact of Customizations on Trust perceptions in Human-Robot Collaboration
Extracted source text · arXiv 2310.18791
\documentclass[conference]{IEEEtran}
\IEEEoverridecommandlockouts
% The preceding line is only needed to identify funding in the first footnote. If that is unneeded, please comment it out.
\usepackage{cite}
\usepackage[table,xcdraw]{xcolor}
\usepackage[colorinlistoftodos,prependcaption,textsize=tiny]{todonotes}
\usepackage{amsmath,amssymb,amsfonts}
\usepackage{algorithmic}
\usepackage{graphicx}
\usepackage{textcomp}
\usepackage{xcolor}
\def\BibTeX{{\rm B\kern-.05em{\sc i\kern-.025em b}\kern-.08em
T\kern-.1667em\lower.7ex\hbox{E}\kern-.125emX}}
\newcommand\parv[1]{\textcolor{red}{PK: #1}}
\newcommand\simon[1]{\textcolor{blue}{SC: #1}}
\begin{document}
\title{”Do it my way!”: Impact of Customizations on Trust perceptions in
Human-Robot Collaboration\\
% {\footnotesize \textsuperscript{*}Note: Sub-titles are not captured in Xplore and
% should not be used}
% \thanks{Identify applicable funding agency here. If none, delete this.}
}
\author{\IEEEauthorblockN{Parv Kapoor}
\IEEEauthorblockA{\textit{Institute for Software Research} \\
\textit{Carnegie Mellon University}\\
Pittsburgh, U.S.A. \\
parvk@andrew.cmu.edu}
\and
\IEEEauthorblockN{Simon Chu}
\IEEEauthorblockA{\textit{Institute for Software Research} \\
\textit{Carnegie Mellon University}\\
Pittsburgh, U.S.A. \\
cchu2@andrew.cmu.edu}
\and
\IEEEauthorblockN{Angela Chen}
\IEEEauthorblockA{\textit{Robotics Institute} \\
\textit{Carnegie Mellon University}\\
Pittsburgh, U.S.A. \\
angelac2@andrew.cmu.edu}
}
\maketitle
\thispagestyle{plain}
\pagestyle{plain}
\begin{abstract}
Trust has been shown to be a key factor in effective human-robot collaboration. In the context of assistive robotics, the effect of trust factors on human experience is further pronounced. Personalization of assistive robots is an orthogonal factor positively correlated with robot adoption and user perceptions. In this work, we investigate the relationship between these factors through a within-subjects study (N=17). We provide different levels of customization possibilities over baseline autonomous robot behavior and investigate its impact on trust. Our findings indicate that increased levels of customization was associated with higher trust and comfort perceptions. The assistive robot design process can benefit significantly from our insights for designing trustworthy and customized robots.
\end{abstract}
\begin{IEEEkeywords}
Human-Robot Interaction, Trustworthy Robotics
\end{IEEEkeywords}
\section{Introduction}
\label{intro}
% \begin{enumerate} \color{red}
% \item Introduction needs to explicitly mention contributions
% \item add design principles extracted from results (maybe in discussion?)
% \end{enumerate}
\input{sections/intro.tex}
\section{Related Work}%
\label{related}
\input{sections/related.tex}
\section{System Overview}%
% \parv{will talk about setup, reinforcement learning agents, booked room, simulation and all the extraneous stuff}
\label{methods}
\input{sections/systemover.tex}
\section{User Study}%
\label{methods}
\input{sections/methods.tex}
% pilot study
% - pre-study questionaire
% - post-study questionaire
% - post-study interview
% SAM test
% -
\section{Results and Evaluation}%
% \parv{Angela's tables will come here}
\label{eval}
\input{sections/evaluation.tex}
\section{Discussion}%
\label{discuss}
\input{sections/discussion.tex}
\section{Future Work}
\label{future-work}
\input{sections/future-work.tex}
\section{Conclusion}
\label{conclusion}
\input{sections/conclusion.tex}
\bibliographystyle{IEEEtran}
% \bibliography{IEEEabrv,IEEEexample}
% \bibliographystyle{ieeetran}
\bibliography{references}
% \section{Ease of Use}
% \section{Appendix}
% \label{appendix}
% \input{sections/appendix.tex}
% \subsection{Maintaining the Integrity of the Specifications}
% The IEEEtran class file is used to format your paper and style the text. All margins,
% column widths, line spaces, and text fonts are prescribed; please do not
% alter them. You may note peculiarities. For example, the head margin
% measures proportionately more than is customary. This measurement
% and others are deliberate, using specifications that anticipate your paper
% as one part of the entire proceedings, and not as an independent document.
% Please do not revise any of the current designations.
% \section{Prepare Your Paper Before Styling}
% Before you begin to format your paper, first write and save the content as a
% separate text file. Complete all content and organizational editing before
% formatting. Please note sections \ref{AA}--\ref{SCM} below for more information on
% proofreading, spelling and grammar.
% Keep your text and graphic files separate until after the text has been
% formatted and styled. Do not number text heads---{\LaTeX} will do that
% for you.
% \subsection{Abbreviations and Acronyms}\label{AA}
% Define abbreviations and acronyms the first time they are used in the text,
% even after they have been defined in the abstract. Abbreviations such as
% IEEE, SI, MKS, CGS, ac, dc, and rms do not have to be defined. Do not use
% abbreviations in the title or heads unless they are unavoidable.
% \subsection{Units}
% \begin{itemize}
% \item Use either SI (MKS) or CGS as primary units. (SI units are encouraged.) English units may be used as secondary units (in parentheses). An exception would be the use of English units as identifiers in trade, such as ``3.5-inch disk drive''.
% \item Avoid combining SI and CGS units, such as current in amperes and magnetic field in oersteds. This often leads to confusion because equations do not balance dimensionally. If you must use mixed units, clearly state the units for each quantity that you use in an equation.
% \item Do not mix complete spellings and abbreviations of units: ``Wb/m\textsuperscript{2}'' or ``webers per square meter'', not ``webers/m\textsuperscript{2}''. Spell out units when they appear in text: ``. . . a few henries'', not ``. . . a few H''.
% \item Use a zero before decimal points: ``0.25'', not ``.25''. Use ``cm\textsuperscript{3}'', not ``cc''.)
% \end{itemize}
% \subsection{Equations}
% Number equations consecutively. To make your
% equations more compact, you may use the solidus (~/~), the exp function, or
% appropriate exponents. Italicize Roman symbols for quantities and variables,
% but not Greek symbols. Use a long dash rather than a hyphen for a minus
% sign. Punctuate equations with commas or periods when they are part of a
% sentence, as in:
% \begin{equation}
% a+b=\gamma\label{eq}
% \end{equation}
% Be sure that the
% symbols in your equation have been defined before or immediately following
% the equation. Use ``\eqref{eq}'', not ``Eq.~\eqref{eq}'' or ``equation \eqref{eq}'', except at
% the beginning of a sentence: ``Equation \eqref{eq} is . . .''
% \subsection{\LaTeX-Specific Advice}
% Please use ``soft'' (e.g., \verb|\eqref{Eq}|) cross references instead
% of ``hard'' references (e.g., \verb|(1)|). That will make it possible
% to combine sections, add equations, or change the order of figures or
% citations without having to go through the file line by line.
% Please don't use the \verb|{eqnarray}| equation environment. Use
% \verb|{align}| or \verb|{IEEEeqnarray}| instead. The \verb|{eqnarray}|
% environment leaves unsightly spaces around relation symbols.
% Please note that the \verb|{subequations}| environment in {\LaTeX}
% will increment the main equation counter even when there are no
% equation numbers displayed. If you forget that, you might write an
% article in which the equation numbers skip from (17) to (20), causing
% the copy editors to wonder if you've discovered a new method of
% counting.
% {\BibTeX} does not work by magic. It doesn't get the bibliographic
% data from thin air but from .bib files. If you use {\BibTeX} to produce a
% bibliography you must send the .bib files.
% {\LaTeX} can't read your mind. If you assign the same label to a
% subsubsection and a table, you might find that Table I has been cross
% referenced as Table IV-B3.
% {\LaTeX} does not have precognitive abilities. If you put a
% \verb|\label| command before the command that updates the counter it's
% supposed to be using, the label will pick up the last counter to be
% cross referenced instead. In particular, a \verb|\label| command
% should not go before the caption of a figure or a table.
% Do not use \verb|\nonumber| inside the \verb|{array}| environment. It
% will not stop equation numbers inside \verb|{array}| (there won't be
% any anyway) and it might stop a wanted equation number in the
% surrounding equation.
% \subsection{Some Common Mistakes}\label{SCM}
% \begin{itemize}
% \item The word ``data'' is plural, not singular.
% \item The subscript for the permeability of vacuum $\mu_{0}$, and other common scientific constants, is zero with subscript formatting, not a lowercase letter ``o''.
% \item In American English, commas, semicolons, periods, question and exclamation marks are located within quotation marks only when a complete thought or name is cited, such as a title or full quotation. When quotation marks are used, instead of a bold or italic typeface, to highlight a word or phrase, punctuation should appear outside of the quotation marks. A parenthetical phrase or statement at the end of a sentence is punctuated outside of the closing parenthesis (like this). (A parenthetical sentence is punctuated within the parentheses.)
% \item A graph within a graph is an ``inset'', not an ``insert''. The word alternatively is preferred to the word ``alternately'' (unless you really mean something that alternates).
% \item Do not use the word ``essentially'' to mean ``approximately'' or ``effectively''.
% \item In your paper title, if the words ``that uses'' can accurately replace the word ``using'', capitalize the ``u''; if not, keep using lower-cased.
% \item Be aware of the different meanings of the homophones ``affect'' and ``effect'', ``complement'' and ``compliment'', ``discreet'' and ``discrete'', ``principal'' and ``principle''.
% \item Do not confuse ``imply'' and ``infer''.
% \item The prefix ``non'' is not a word; it should be joined to the word it modifies, usually without a hyphen.
% \item There is no period after the ``et'' in the Latin abbreviation ``et al.''.
% \item The abbreviation ``i.e.'' means ``that is'', and the abbreviation ``e.g.'' means ``for example''.
% \end{itemize}
% An excellent style manual for science writers is \cite{b7}.
% \subsection{Authors and Affiliations}
% \textbf{The class file is designed for, but not limited to, six authors.} A
% minimum of one author is required for all conference articles. Author names
% should be listed starting from left to right and then moving down to the
% next line. This is the author sequence that will be used in future citations
% and by indexing services. Names should not be listed in columns nor group by
% affiliation. Please keep your affiliations as succinct as possible (for
% example, do not differentiate among departments of the same organization).
% \subsection{Identify the Headings}
% Headings, or heads, are organizational devices that guide the reader through
% your paper. There are two types: component heads and text heads.
% Component heads identify the different components of your paper and are not
% topically subordinate to each other. Examples include Acknowledgments and
% References and, for these, the correct style to use is ``Heading 5''. Use
% ``figure caption'' for your Figure captions, and ``table head'' for your
% table title. Run-in heads, such as ``Abstract'', will require you to apply a
% style (in this case, italic) in addition to the style provided by the drop
% down menu to differentiate the head from the text.
% Text heads organize the topics on a relational, hierarchical basis. For
% example, the paper title is the primary text head because all subsequent
% material relates and elaborates on this one topic. If there are two or more
% sub-topics, the next level head (uppercase Roman numerals) should be used
% and, conversely, if there are not at least two sub-topics, then no subheads
% should be introduced.
% \subsection{Figures and Tables}
% \paragraph{Positioning Figures and Tables} Place figures and tables at the top and
% bottom of columns. Avoid placing them in the middle of columns. Large
% figures and tables may span across both columns. Figure captions should be
% below the figures; table heads should appear above the tables. Insert
% figures and tables after they are cited in the text. Use the abbreviation
% ``Fig.~\ref{fig}'', even at the beginning of a sentence.
% \begin{table}[htbp]
% \caption{Table Type Styles}
% \begin{center}
% \begin{tabular}{|c|c|c|c|}
% \hline
% \textbf{Table}&\multicolumn{3}{|c|}{\textbf{Table Column Head}} \\
% \cline{2-4}
% \textbf{Head} & \textbf{\textit{Table column subhead}}& \textbf{\textit{Subhead}}& \textbf{\textit{Subhead}} \\
% \hline
% copy& More table copy$^{\mathrm{a}}$& & \\
% \hline
% \multicolumn{4}{l}{$^{\mathrm{a}}$Sample of a Table footnote.}
% \end{tabular}
% \label{tab1}
% \end{center}
% \end{table}
% \begin{figure}[htbp]
% \centerline{\includegraphics{fig1.png}}
% \caption{Example of a figure caption.}
% \label{fig}
% \end{figure}
% Figure Labels: Use 8 point Times New Roman for Figure labels. Use words
% rather than symbols or abbreviations when writing Figure axis labels to
% avoid confusing the reader. As an example, write the quantity
% ``Magnetization'', or ``Magnetization, M'', not just ``M''. If including
% units in the label, present them within parentheses. Do not label axes only
% with units. In the example, write ``Magnetization (A/m)'' or ``Magnetization
% \{A[m(1)]\}'', not just ``A/m''. Do not label axes with a ratio of
% quantities and units. For example, write ``Temperature (K)'', not
% ``Temperature/K''.
% \section*{Acknowledgment}
% The preferred spelling of the word ``acknowledgment'' in America is without
% an ``e'' after the ``g''. Avoid the stilted expression ``one of us (R. B.
% G.) thanks $\ldots$''. Instead, try ``R. B. G. thanks$\ldots$''. Put sponsor
% acknowledgments in the unnumbered footnote on the first page.
% \section*{References}
% Please number citations consecutively within brackets \cite{b1}. The
% sentence punctuation follows the bracket \cite{b2}. Refer simply to the reference
% number, as in \cite{b3}---do not use ``Ref. \cite{b3}'' or ``reference \cite{b3}'' except at
% the beginning of a sentence: ``Reference \cite{b3} was the first $\ldots$''
% Number footnotes separately in superscripts. Place the actual footnote at
% the bottom of the column in which it was cited. Do not put footnotes in the
% abstract or reference list. Use letters for table footnotes.
% Unless there are six authors or more give all authors' names; do not use
% ``et al.''. Papers that have not been published, even if they have been
% submitted for publication, should be cited as ``unpublished'' \cite{b4}. Papers
% that have been accepted for publication should be cited as ``in press'' \cite{b5}.
% Capitalize only the first word in a paper title, except for proper nouns and
% element symbols.
% For papers published in translation journals, please give the English
% citation first, followed by the original foreign-language citation \cite{b6}.
% \begin{thebibliography}{00}
% \bibitem{b1} G. Eason, B. Noble, and I. N. Sneddon, ``On certain integrals of Lipschitz-Hankel type involving products of Bessel functions,'' Phil. Trans. Roy. Soc. London, vol. A247, pp. 529--551, April 1955.
% \bibitem{b2} J. Clerk Maxwell, A Treatise on Electricity and Magnetism, 3rd ed., vol. 2. Oxford: Clarendon, 1892, pp.68--73.
% \bibitem{b3} I. S. Jacobs and C. P. Bean, ``Fine particles, thin films and exchange anisotropy,'' in Magnetism, vol. III, G. T. Rado and H. Suhl, Eds. New York: Academic, 1963, pp. 271--350.
% \bibitem{b4} K. Elissa, ``Title of paper if known,'' unpublished.
% \bibitem{b5} R. Nicole, ``Title of paper with only first word capitalized,'' J. Name Stand. Abbrev., in press.
% \bibitem{b6} Y. Yorozu, M. Hirano, K. Oka, and Y. Tagawa, ``Electron spectroscopy studies on magneto-optical media and plastic substrate interface,'' IEEE Transl. J. Magn. Japan, vol. 2, pp. 740--741, August 1987 [Digests 9th Annual Conf. Magnetics Japan, p. 301, 1982].
% \bibitem{b7} M. Young, The Technical Writer's Handbook. Mill Valley, CA: University Science, 1989.
% \end{thebibliography}
% \vspace{12pt}
% \color{red}
% IEEE conference templates contain guidance text for composing and formatting conference papers. Please ensure that all template text is removed from your conference paper prior to submission to the conference. Failure to remove the template text from your paper may result in your paper not being published.
\end{document}
\par As robots become more prevalent in society and play an increasingly meaningful
role in our daily lives, trust has become a key factor in effective collaboration
and long-term relationships between humans and robots. It has been shown
that trust increases users’ intention to use and it is negatively correlated with
user anxiety \cite{schule2022patients}. Trust is also a reliable
predictor of task outcome in human-robot collaboration \cite{herse2021using}. Furthermore, the
trust level can provide further insights into the design of future robots for safe and
effective collaboration. \cite{tomzcak2019let}
\par In the assistive/personal robots domain, one of the initial interactions people
have with the robot is customizing its default behavior per their personal
preferences. People tend to curate more personalized experiences through the
customization of the robot, including but not limited to setting safety operating
boundaries for robot manipulators, operation routines, and patterns. Recently, there has been a surge in personalized robots’ demand and they play increasingly pivotal roles in people’s daily lives. With increased human-robot
coexistence, robot designers and engineers realize the need to provide a more
diverse set of customization options for tighter user integration. Even
though the relationship between customization and trust is well-studied in the
context of digital devices and recommendation agents \cite{komiak2006effects}, currently, there’s no
related study in the area of human-robot interaction. We propose the following questions to investigate this relationship.
% \todo{Add bolded research questions:}
\par \textbf{RQ1: Does user customization of robot behavior affect perceived trust in robots' ability for assistive tasks?}
Additionally, in the field of psychology, A third-person perspective has been shown to regulate one's emotions \cite{wallace2016impact}. While we did not find any related work that links perspective with robotics, various attributes of first-person and third-person perspectives are studied extensively in the area of Virtual Reality (VR). In \cite{debarba2017}, it is stated that the third-person perspective (3PP) is compatible with body ownership when sensorimotor contingencies are present, while the first-person perspective (1PP) can foster the user to develop a strong sense of embodiment. In \cite{gorisse2017}, third-person perspective (3PP) has been shown be to associated with strong spatial awareness and environmental perception capacity, therefore, making the user less vulnerable. Hence, we are also interested in observing the impact of "perspective" on trust. We propose an additional question to explore this relationship.
\par \textbf{RQ2: Does user view perspective affect perceived trust in robots' ability for assistive tasks?}
\par In this paper, we explore user preferences for different customization levels of robotic behavior. We present two studies where 17 participants with various levels of robot use experience interact with a simulated assistive feeding task. Participants were given three levels of control over robot behavior and two perspectives through the simulator. We observe that users perceived the robot as more trustworthy and comfortable in both perspectives with increased levels of control. Additionally, we observe that perspectives did not have a statistically significant impact on trust but had an impact on comfort perceptions.
\subsection{Trust in Human-Collaboration Research}
Trust in the context of human-robot collaboration has been an important subject of interest\cite{xu2012trust} \cite{sadrfaridpour2016modeling}\cite{krausman2022trust}
\cite{hancock2011meta}, where some user studies focused on study designs derived from economic and psychology studies \cite{hsieh2020human}\cite{abubshait2021examining}, and some focused on directly measuring users' trust based on an existing robotic design \cite{schule2022patients}\cite{martelaro2016tell}. In most cases, post-study questionnaires are usually used to assess the user's trust level \cite{krausman2022trust}\cite{freedy2007measurement}. However, other measurements of trust have also been explored. For example, Krausman et al. (2022) \cite{krausman2022trust} underlined the different types
of trust and its indicative measures such as subjective, communication,
behavioral, and physiological in a team. In this work, we explored several definitions of trust in psychology \cite{schilke2021} to support our hypotheses. For our work, we will leverage existing work to design our trust elicitation questionnaire and interviews, and use measures such as intervention to indirectly measure trust.
\subsection{Personalization}
Personalization is critical in assistive tasks of daily functions for users with impairments \cite{alqasemi2005wheelchair}. It has been specifically investigated in the context of assistive feeding in \cite{bhattacharjee2020more}\cite{bhattacharjee2019towards}. In Bhattacharjee et al (2020) \cite{bhattacharjee2020more} studied user preferences for different levels of autonomy in feeding. Gordon et al (2020) \cite{gordon2020adaptive} created a simulated environment that adapts to user food preferences. In another line of work, Schultz et al (2022) \cite{schultz2022proof} presented a prototype robot-assisted self-feeding system that takes the eye and head movements for food preferences.
However, existing literature has not explored the relationship between trust and the level of customization in assistive robot applications like assistive feeding and drinking. To simplify the problem, We identified the key factor that directly affect the users' experience interacting with the robot: the level of control. Additionally, we also investigate the effect of viewing perspectives on trust perceptions. Neither of these have been studied in the context of trust in human-robot interactions. There is a large body of work in the area of virtual reality (VR), which studies the relationship between vulnerability, sense of embodiment, ownership, agency, and viewing perspectives \cite{debarba2017} \cite{gorisse2017}. We leveraged this work in viewing perspectives and the definition of trust in psychology to propose our hypothesis in the following sections, and explored the relationship between different levels of control, viewing perspectives, and trust.
\subsection{Simulator}
We use an assistive feeding task for our study. We modify a preexisting simulation environment (assistive gym \cite{erickson2020assistivegym}) which is a state of the art physics based simulator for physical robotic assistance. In order to make the scenarios hyper realistic we use textures from iGibson \cite{li2022igibson} simulator. We load the indoor household scenario from iGibson to mimic the traditional setup of a participant's household. We initialize the Jaco 2 robotic arm holding a spoon inside the environment. We assume that the food acquisition stage has already been performed. The task is succesful if the robotic arm is able to maneuver the spoon to the participant's mouth without dropping any simulated food particles.
% Participants were asked to sit while observing the UI of assistive gym \cite{erickson2020assistivegym} from a 27-inch monitor connected to a PC. In order to make the scenarios hyper realistic we use textures from iGibson \cite{li2022igibson} simulator.
\subsection{Modes of Control}
% According to related work in psychology about trust \cite{schilke2021}, trust is the willingness of one party (the trustor) to become vulnerable to another party (the trustee) on the presumption that the trustee will act in ways that benefit the trustor. The level of control is a direct interface that allows humans to express to the robot what is expected of them.
We train a baseline policy for autonomous control using proximal policy optimization (PPO) \cite{schulman2017proximal} algorithm which is a state of the art reinforcement learning algorithm for continuous action spaces. The environment provides high reward if all food particles are successfully fed to the human agent in the shortest time possible. There is a negative reward associated with dropping food particles from the spoon.
We discretize the level of control over baseline policy into three categories with increasing levels of complexity. The inputs from the participants are taken using the keyboard.
\begin{enumerate}
\item \textbf{No intervention control}: the participant has no control over baseline policy and the whole setup is fully autonomous
\item \textbf{Stop control}: the participant can stop the robot agent at any time by pressing 'S' and the arm will retract to its initial position. The participant can then press 'R' to resume and the agent will continue trying to feed the human agent.
\item \textbf{Granular control}: The participant can perform stop action but in this case also adjust the robot arm's position while its stopped by pressing one of the six designated keys to move it up, down, left, right, front and back. Once content, the participant can then resume the autonomous policy from the new adjusted position.
\end{enumerate}
\subsection{Perspective}
We investigated two main perspectives in our study to identify their impact on trust.
\begin{enumerate}
\item \textbf{Third Person Perspective (TPP)}: In this situation, we initialise the simulation view as a third person view of the robot agent feeding a human agent sat in the chair as illustrated in figure 4. We ask participants to imagine themselves as a caretaker for this human agent, monitoring robot behavior. \\
\item \textbf{First Person Perspective (FPP)}: In this situation, we initialise the simulation view as a first person view of the robot agent feeding the human agent as illustrated in figure 5. We asked the participants to imagine themselves as the human agent being fed by the robot.
\end{enumerate}
% \parv{what to do about this related section?}
% While we did not find any related work that links perspective with robotics, various attributes of first-person and third-person perspectives are studied extensively in the area of Virtual Reality (VR). In \cite{debarba2017}, it is stated that the third-person perspective (3PP) is compatible with body ownership when sensorimotor contingencies are present, while the first-person perspective (1PP) can foster the user to develop a strong sense of embodiment. In \cite{gorisse2017}, third-person perspective (3PP) has been shown be to associated with strong spatial awareness and environmental perception capacity, therefore, making the user less vulnerable. In addition, in \cite{debarba2017}, perspective-switching has been associated with the treatment of post-traumatic stress disorder (PTSD), in which one can develop a strong sense of embodiment of the virtual body in 1PP, and then switch to 3PP when the body is exposed to a threat, making participants less vulnerable in a stressful situation.
\begin{figure*}
\centering
\includegraphics[scale=0.3]{sections/user-study-instruction-flowchart-horizontal.png}
\caption{Flow chart for the user study}
\label{fig:flowchart}
\end{figure*}
% \parv{the descriptive statistics figures will come here about participants.}
\begin{table}[h]
\caption{Participant demographics information}
\label{table_I}
\begin{center}
\begin{tabular}{|c||c||c|}
\hline
Participant & Age & Expertise\\
\hline
P1 & 27 & Software Engineering \\
P2 & 29 & Software Engineering \\
P3 & 23 & Software Engineering \\
P4 & 22 & Software Engineering \\
P5 & 24 & Health Care \\
P6 & 44 & Software Engineering \\
P7 & 27 & Software Engineering \\
P8 & 29 & Human-Robot Interaction \\
P9 & 25 & Software Engineering \\
P10 & 23 & Software Engineering \\
P11 & 23 & Societal Computing \\
P12 & 23 & Software Engineering \\
P13 & 24 & Human-Computer Interaction \\
P14 & 26 & Human-Computer Interaction \\
P15 & 23 & Human-Computer Interaction \\
P16 & 23 & Software Engineering \\
P17 & 23 & Software Engineering \\
\hline
\end{tabular}
\end{center}
\end{table}
% \begin{figure}[h!]
% \centering
% \includegraphics[scale=0.6]{control-pref-post.png}
% \caption{Preferred level of autonomy in pre-study questionnaire}
% \label{fig:FPV}
% \end{figure}
% \begin{figure}[h!]
% \centering
% \includegraphics[scale=0.6]{perspective-pref-post.png}
% \caption{Preferred perspective in post-study questionnaire}
% \label{fig:FPV}
% \end{figure}
\subsection{Participants}
Seventeen participants (N=17) were recruited from Carnegie Mellon University Robotics Institute (RI), Institute for Software Research (ISR), and University of Pittsburgh Medical Center (UPMC). The study is a within-subject study. The participants have expertise spanning from software engineering, human-computer interaction, human-robot interaction and health care. Over half of the participants (9 out of 17) have previous experience interacting with a robot. 7 of the participants have direct or indirect experiences interacting with assistive devices.
% The majority of participants were graduate students with backgrounds in software engineering and robotics. The user study is expected to last about an hour.
% For conducting the user study, we will recruit participants via word-of-mouth,
% for example, by reaching out to classmates, friends, and acquaintances, which
% aligns with the regulation regarding the pilot study (without IRB approval).
% We are targeting to gather at least 30 participants based on availability.
\begin{figure}[h!]
% \centering
\includegraphics[width=0.9\columnwidth]{control-pref-pre.png}
\caption{Preferred level of autonomy in pre-study questionnaire}
\label{fig:FPV}
\end{figure}
\subsection{Design and Procedure}
\begin{table}[h]
\caption{Independent Variables of the User Study}
\label{table_I}
\begin{center}
\begin{tabular}{|c||c|}
\hline
Control & Perspectives\\
\hline
No Intervention Permitted & First-person View \\
Stop Action & 3rd-person View \\
Stop Action and Granular Control & \\
\hline
\end{tabular}
\end{center}
\end{table}
\subsubsection{Design}
The experiment was conducted in a controlled environment with participants facing the wall and the researchers instead of the windows to mitigate biases caused by environmental stimulation. The study was conducted indoors in a conference room at TCS hall, Carnegie Mellon University.
As presented in Table \ref{table_I}, we identified 2 independent variables for the case study. (1) the level of control over the robot's behavior and (2) the view perspectives. We have two dependent variables (1) the self-reported trust level for each trial condition and (2) the self-reported comfort level, which is hypothesized as a latent variable for the perceived trust.\\
\noindent{\textbf{Quantitative measurement}}: The quantitative measurements we are collecting are self-reported trust and comfort levels on a Likert scale. We also asked the participants to rank the level of different control levels after experiments with each perspective, and overall after all the trials, and rate their most preferred autonomy level and perspectives. In addition, we also ask the participants question "would you consider using our system again in the future?" in the post-study. The yes/no response of the latter question serves as a latent variable for the quality of the implementation of the system.
\noindent{\textbf{Qualitative measurement}}: The qualitative measures here we are collecting are divided into 3 parts (1) pre-study (2) post-trial and (3) post-study. In the pre-study, we ask participants to elaborate on their previous experience with robotics applications and assistive devices. In the post-trial survey, we try to elicit comments on their experiences with the particular condition. In the post-study questionnaire, we elicited the reason why the participants will use the system again in the future, and their feedback on improving the experience design and setup.
% \parv{Talk about DV here and also change the hypothesis framing}
Our first and second hypotheses are:\\
% \noindent{\textbf{H1: The effect of increased participant control on perceived trust will lead to higher reported trust scores.}} \\
\noindent{\textbf{H1: Trust is influenced by the level of participant control.}}
% \noindent{\textbf{H2: The effect of increased participant control on perceived comfort will lead to higher reported comfort scores.}}
\noindent{\textbf{H2: Comfort is influenced by the level of participant control.}}
Since the related work does show that the first-person view makes people more vulnerable, and the nature of assistive tasks fall is well suited for these situations. In lieu of this, We state our third and fourth hypotheses:\\
\noindent{\textbf{H3: The change of perspective has an impact on trust.}}\\
% \textbf{H3. The effect of the first-person view on perceived trust will lead to a higher number of interventions.} \\
\noindent{\textbf{H4: The change of perspective has an impact on comfort.}}
Intuitively, this is due to the case that the third-person perspective enables the participants to be less vulnerable and more detached from the assistive context. \\
We have explored latent variables of trust, and previous work has indicated that comfort level can be a latent variable of trust, therefore, we have a fifth hypothesis: \\
\noindent{\textbf{H5: Higher comfort levels are correlated with higher trust levels}}
% \textbf{H1: The effect of increased participant control on perceived trust will lead to lesser interventions.}
% By varying the granularity of the level of control humans are allowed to perform, we can
% https://en.wikipedia.org/wiki/Trust_(social_science)
% \textit{Tasks}: For this paper, we chose the two main tasks in assistive robotics research, namely: feeding, and drinking water. Both tasks involve various degrees of complexity and parameters that can be fine-tuned. In the case of feeding, it is simplified in the sense that the user can proactively approach the fork at the end of the robotic arm, whereas in the drinking case, the water cup has to physically touch the human subject with a varying angle. Both tasks involve training policies using rewards associated with strong spatial awareness and environmental perception capacity
% \todo{explain where we obtained the parameters from, citation required}
% (Up, Down, Left, Right, Stop)
\begin{figure}[h!]
\centering
\includegraphics[width=0.9\columnwidth]{sections/FPV.png}
\caption{First-person view of the feeding task in the simulator}
\label{fig:FPV}
\end{figure}
\begin{figure}[h!]
\centering
\includegraphics[width=0.9\columnwidth]{sections/TPV3.png}
\caption{Third-person view of the feeding task in the simulator}
\label{fig:TPV}
\end{figure}
%\includegraphics[]{sections/FPV.png}
%\includegraphics[]{sections/TPV.png}
We investigated 3 conditions for the level of control, 2 conditions for perspectives for the assistive feeding task. This lead to total of 6 scenarios. Additionally, we simulate 3 episodes per scenario to collect more data points and account for any bias. The conditions and parameters are summarized in table \ref{table_I}.
% \simon{Put the design}
% \textit{Control}: The level of control requires the overriding of the original mission/task policy trained using reinforcement learning. We used APIs within assistive-gym to associate keyboard input from the user to the action override when inputting granular controls like left, right, up, down, front, and back. \textit{Stop} action is relatively straightforward as it simply halts the current action of the robot until it is asked to resume.
% \textit{Perspectives}: We programmed cases to toggle between first-person view and third-person view of the scenario. Figures 1 and 2 illustrate these two views in our setup. We will disable the toggle button during the duration of the trial to prevent users from toggling between first-person and third-person view to ensure the accuracy of the measurement.
% \textit{Tasks}: We plan on investigating two tasks, assistive feeding, and drinking. The acquisition state is not part of the trial, as, upon the start of the task, the spoon and the bottle have already been acquired by the Jaco robot. On the Implementational level, the two tasks involve policy learning using a state of the art reinforcement learning algorithm called proximal policy optimization.
\subsubsection{Procedure}
% \parv{cite user study flow diagram}
The procedure is shown in figure \ref{fig:flowchart}. After welcoming the participants, each experiment session started with signing the consent form, demographic questionnaire, and a short introduction to the study. The questionnaire given prior to the study has three goals (1) to elicit demographic information (2) to understand prior experience interacting with robots and (3) to record study participants' preferred level of control. For the demographics questionnaire, we will ask questions like gender, age, education level, and area of study.
After the pre-study questionnaire, a pretrial was conducted to get the participants accustomed to the user interface and the simulator. We let the participants teleoperate with the robotic arm for feeding and drinking tasks using the keyboard.
The post-study questionnaire was given (1) after each trial and (2) after the completion of the entire user study. After each trial, the participants will be given a Likert-scale questionnaire in which they can rate the trust and comfort level that they feel towards the robot. Upon the completion of the study, the participants will be asked to pick the conditions with the highest trust and comfort levels.
A 15-minute semi-structured interview was conducted to conclude the study, mainly to elicit user feedback for the user study.
% \subsubsection{Survey}
% A survey will be conducted before and after the user study as well as after each trial. The form of the survey will be a combination of Likert-scale, yes/no, and subjective questions.
% The questionnaire given prior to the study has three goals (1) to elicit demographic information (2) to understand prior experience interacting with robots (3) to record study participants' preferred level of control. For the demographics questionnaire, we will ask questions like gender, age, education level, and area of study.
% To understand prior experience interacting with robots, we will ask participants whether they have interacted with a robot before, in particular, assistive robots, if the participants have prior experience developing robots, we will ask them about the number of years the participants have worked on robotics-related projects and will ask the robots to elaborate on their impression of the previous interactions.
% To record study participants' emotional state prior to the study, we use the Self Assessment Manikin (SAM) test, which measures emotions along three dimensions: Valence, Arousal, and Dominance. Valence relates to the hedonic tone and varies from negative to positive emotions; Arousal relates to bodily and mental activation and varies from calm to excited; Dominance relates to the degree of control, between being submissive or in control.
% \subsubsection{Interview}
% A 15-minute semi-structured interview will be conducted at the end of the user study. It will be open-ended with the goal of eliciting feelings towards the robot, including perceived safety and comfort level and subjective experience that may not have been captured by the questionnaire. In addition, we will ask user meta-questions regarding the design of the user study, (i.e. Are you satisfied with the case study? If we were to conduct this user study again, what would you like to be changed?) This can help us refine future user studies and improve the experiments. The interview will be recorded for future analysis.
% \subsubsection{Objective Metrics}
% We will also be collecting objective metrics to indirectly determine the trust level of the user. For example, one of the parameters that is prominently used in previous literature is the number of interventions from the user, which is negatively correlated with trust. In our case study, we will be recording the number of times the user requests the robot to stop (either through verbal communication to the experimenter, or the press action of using keyboard input). Furthermore, in the case that the user is allowed to maneuver the robot up, down, left, or right, we can also measure the distance from the spoon/drink on the robotic arm before the drinking or feeding action occurs.
% this section focuses on the plan of evaluation instead of putting the actual results here
% to be changed in the future.
% In our experiment, we used a mix-methods research methodology with surveys and interviews where the survey
% \parv{what was this line supposed to mean?}
% Please add the following required packages to your document preamble:
% \usepackage[table,xcdraw]{xcolor}
% If you use beamer only pass "xcolor=table" option, i.e. \documentclass[xcolor=table]{beamer}
% \begin{table}[]
% \begin{tabular}{c|c|c|c}
% Control & N & Mean & SD \\ \hline
% no & 30 & 2.3667 & 1.2452 \\ \hline
% stop & 30 & 3.3000 & 1.1492 \\ \hline
% granular & 30 & 3.4667 & 1.1666 \\ \hline
% \end{tabular}
% \caption{Descriptive statistics of the trust scores grouped by various levels of autonomy level}
% \label{tab:my-table}
% \end{table}
\subsection{Descriptive statistics}
All analyses were performed using statistical packages like \cite{seabold2010statsmodels} in Python. The data set consists of 90 data points (15 participants x 6 trials). A descriptive statistics for trust score and comfort score are as follows.
\begin{table}[h]
\begin{center}
\caption{Descriptive statistics of the trust scores and comfort scores}
\begin{tabular}{|c|c|c|c|}
\hline
Scores & N & Mean & SD\\
\hline
Trust & 90 & 3.044444 & 1.271212 \\
Comfort & 90 & 2.822222 & 1.285859 \\
\hline
\end{tabular}
\end{center}
\end{table}
The descriptive statistics of the trust score group by the control level are shown in TABLE III . A similar set of descriptive statistics of the comfort score are shown in TABLE 4.
\begin{table}[h]
\begin{center}
\caption{Descriptive statistics of the trust scores grouped by various levels of autonomy level}
\begin{tabular}{|c|c|c|c|}
\hline
Control & N & Mean & SD\\
\hline
no & 30 & 2.3667 & 1.2452 \\
stop & 30 & 3.3000 & 1.1492 \\
granular & 30 & 3.4667 & 1.1666 \\
\hline
\end{tabular}
\end{center}
\end{table}
\begin{table}[h]
\begin{center}
\caption{Descriptive statistics of the comfort scores grouped by various levels of autonomy level}
\begin{tabular}{|c|c|c|c|}
\hline
Control & N & Mean & SD\\
\hline
no & 30 & 1.8333 & 0.9855 \\
stop & 30 & 3.2000 & 1.1567 \\
granular & 30 & 3.4333 & 1.1043 \\
\hline
\end{tabular}
\end{center}
\end{table}
The descriptive statistics (mean and standard deviation) of the trust score grouped by perspective are shown in TABLE 5. A similar set of descriptive statistics of the comfort score are shown in TABLE 6.
\begin{table}[h]
\begin{center}
\caption{Descriptive statistics of the trust scores grouped by various levels of perspective}
\begin{tabular}{|c|c|c|c|}
\hline
Perspective & N & Mean & SD\\
\hline
1st & 45 & 2.8667 & 1.2898 \\
3rd & 45 & 3.2222 & 1.2411 \\
\hline
\end{tabular}
\end{center}
\end{table}
\begin{table}[h]
\begin{center}
\caption{Descriptive statistics of the comfort scores grouped by various levels of perspective}
\begin{tabular}{|c|c|c|c|}
\hline
Perspective & N & Mean & SD\\
\hline
1st & 45 & 2.6000 & 1.3212 \\
3rd & 45 & 3.0444 & 1.2239 \\
\hline
\end{tabular}
\end{center}
\end{table}
\subsection{Statistical testings}
To test Hypothesis H1-H4, we performed a repeated measure two-way ANOVA test on the trust score (Likert scale 1-5) and comfort score (Likert scale 1-5). The independent variables are control (three levels: no, stop, and granular) and perspective (two levels: 1st and 3rd).
The repeated measure ANOVA test for trust score shows control (F statistics = 28, p = 0.0001) has a statistically significant effect on trust under alpha = 0.05, whereas perspective (F statistics = 14 , p = 0.0757) doesn't have enough evidence to suggest it makes a significant effect on trust under alpha = 0.05. The interactive effect (F statistics = 28 , p = 0.2129) between control and perspective also concludes no significant effect under alpha = 0.05.
Fig. 5 depicts the relationship between two different perspectives fixed at each control level based on the mean trust score. We observe a general upward trend of trust as the level of control increases, while the discrepancy of trust closes as the level of control increases.
Fig. 6 depicts the relationship between three different control levels fixed at two different perspectives based on the mean trust score. We observe a general upward trend of trust as the perspective changes from the first person to the third person.
The repeated measure ANOVA test for comfort score shows control (F statistics = 28, p = 0.0000) has a statistically significant effect on comfort under alpha = 0.05, and perspective (F statistics = 14 , p = 0.0058) also has a statistically significant effect on comfort under alpha = 0.05. However, the interactive effect (F statistics = 28 , p = 0.1823) between control and perspective doesn't have enough evidence to suggest it makes a significant effect on trust under alpha = 0.05.
Fig. 7 depicts the relationship between two different perspectives fixed at each control level based on the mean comfort score. We observe an upward trend of comfort at the first person perspective. However, at the third person perspective, we notice an interesting drop of comfort as the control becomes more complicated ($stop \longrightarrow granular$).
Fig. 8 depicts the relationship between three different control levels fixed at two different perspectives based on the mean comfort score. We observe a general upward trend of comfort as the perspective changes from the first person view to the third person view.
To test Hypothesis H5, we ran Pearson's correlation test. Our test results (statistic=0.720, pvalue=1.314e-15) shows the two variables are positively and significantly correlated under the alpha threshold 0.05.
To summarize, % H1: control <-> trust
our \textbf{hypothesis H1 was correct} as the p-value (p = 0.0001) that we obtained through the two-way repeated ANOVA test between the control level and trust demonstrates participant control has a significant effect on trust.
% H2: control <-> comfort
Our \textbf{hypothesis H2 was correct}, as the p-value (p = 0.0000) that we obtained through the two-way repeated ANOVA test between the control level and comfort demonstrates participant control has a significant effect on comfort.
% H3: perspective <-> trust
Our \textbf{hypothesis H3 was incorrect}, as the p-value (p = 0.0757) that we obtained through the two-way repeated ANOVA test between perspective and trust demonstrates perspective does not have a significant effect on trust.
% H4: perspective <-> comfort
% \simon{refer to additional statistics (new statistics), through repeated measures ANOVA}
Our \textbf{hypothesis H4 was correct}, as the p-value (p = 0.0058) that we obtained through the two-way repeated ANOVA test between perspective and comfort demonstrates perspective has a significant effect on comfort.
Our \textbf{hypothesis H5 was correct}, as the p-value (p = 1.314e-15) that we obtained through the correlation test between the comfort level and the trust level demonstrates that there's a statistical significance between the correlation.
\begin{table}[h]
\begin{center}
\caption{Two-way ANOVA table on Trust}
\begin{tabular}{|c|c|c|c|c|}
\hline
& sum of sq & df & F & PR\\
\hline
Control & 13.6828 & 2.0000 & 28.0000 & 0.0001\\
Perspective & 3.6797 & 1.0000 & 14.0000 & 0.0757\\ ControlXPerspective & 1.6354 & 2.0000 & 28.0000 & 0.2129 \\
\hline
\end{tabular}
\end{center}
\end{table}
\begin{table}[h]
\begin{center}
\caption{Two-way ANOVA table on Comfort}
\begin{tabular}{|c|c|c|c|c|}
\hline
& sum of sq & df & F & PR\\
\hline
Control & 38.7885 & 2.0000 & 28.0000 & 0.0000\\
Perspective & 10.5660 & 1.0000 & 14.0000 & 0.0058\\
ControlXPerspective & 1.8097 & 2.0000 & 28.0000 & 0.1823\\
\hline
\end{tabular}
\end{center}
\end{table}
\begin{figure}
\centering
% \includegraphics[width=0.99\columnwidth]{Images/rosplane.png}
\includegraphics[width=0.9\columnwidth]{sections/Trust_Control.png}
\caption{Mean trust grouped by control} \label{fig:anova}
\end{figure}
\begin{figure}
\centering
% \includegraphics[width=0.99\columnwidth]{Images/rosplane.png}
\includegraphics[width=0.9\columnwidth]{sections/Trust_Per.png}
\caption{Mean trust grouped by perspective} \label{fig:anova2}
\end{figure}
\begin{figure}
\centering
% \includegraphics[width=0.99\columnwidth]{Images/rosplane.png}
\includegraphics[width=0.9\columnwidth]{sections/Comfort_Control.png}
\caption{Mean comfort grouped by control} \label{fig:anova3}
\end{figure}
\begin{figure}
\centering
% \includegraphics[width=0.99\columnwidth]{Images/rosplane.png}
\includegraphics[width=0.9\columnwidth]{sections/Comfort_Per.png}
\caption{Mean comfort grouped by perspective} \label{fig:anova4}
\end{figure}
% \begin{figure}[h!]
% \centering
% \includegraphics[scale=0.1]{sections/ANOVA_2.png}
% \caption{First-person view of the feeding task in the simulator}
% \label{fig:FPV}
% \end{figure}
% \textbf{Statistical Analysis}: Through the ANOVA test, we discovered that the more control given to the user, the more trust and comfort the user has towards the robot. Perceived trust has not been shown to have a significant correlation with perspective and neither does the control level. Trust is shown to be correlated with comfort level, indicating that comfort level can be an accurate predictor of trust, and last but not least, the 1st perspective has achieved a consistently lower level of trust than the 3rd person perspective.
% We hypothesized that the level of control given to the user and the observation perspectives will both affect the perceived trust of the robot.
% To verify these hypotheses, we use three-way ANOVA to explore the relationship between the independent variables (IV), including the level of control, the observation perspective, and tasks. We will also be analyzing users' ranking of the most trusted conditions, to see if certain conditions have elicited consistently high/low trust/comfort levels from the user.
% We will also use descriptive statistics like the median for participants' age and robot experience. Additionally, we will also include gender ratios in our final draft.
% This section is left empty on purpose. We will fill this section based on our results.
\subsection{Post-study quantitative feedback analysis}
Overall 13 participants (77\%) would use the system again in the future. A majority (65 percent) of our participants chose to have more granular control over a robot in our pre-study. After being exposed to six different control/perspective conditions, 70 percent of our participant chose to have more granular control. Roughly one third of the participants who chose granular control changed their answers. Some of their reasonings include "granular control was not intuitive" and "it is difficult to control the robot through simulations and the keyboard". The qualitative responses are insightful for us to construct our future study, where we could consider an alternative input and interface that is not through a keyboard and simulations. Instead, we could deploy a real robot (e.g., Jaco) and consider eye gaze and verbal cues for inputs.
Since we exposed our participants to the same six conditions, we performed repeated ANOVA on our data. However, even though our within subject study design allows us to analyze with more sample size (higher statistical power), we acknowledge that our analysis have the following short comings: (1) we treated our response as a continuous variable and made the assumption that it follows a normal distribution (2) our sample size is still too small. Due to the shortcomings, we believe the statistical results are not completely conclusive. Therefore, the qualitative feedback we got from our surveys were critical for us to validate our statistical results. Furthermore, the descriptive statistics (table 3-7) and the visualization of the trust and comfort scores (fig 5-8) provided more context for us to interpret our statistical results. For example, more control was rated with a higher trust score and comfort score on average. Therefore, it makes sense that control has an effect on either trust and comfort.
\subsection{Post-study qualitative feedback analysis}
Finally, based on the post-study interview, we found out that the decline in trust level was caused by mainly 3 reasons. (1) cognitive overload: granular control has 3 degrees of freedom, which results in a sudden increase in the cognitive load. (2) difficulty in seeing the relative position due to control perspective: during the experiment, the control key shortcut is designed based on the 1st person's perspective. Controlling the robot from a 3rd-person perspective requires doing the opposite of the control shortcuts. This increased difficulty may be contributing to the decrease in comfort (3) Concern about control hurting a 3rd person: Due to the fact in the 3rd-person perspective, the are liabilities and responsibilities for the participant to operate correctly instead in full autonomy and stop control, and liability is largely placed on the robot.
% From figure \ref{fig:anova}, we observe that as the level of control given to the user increases, the mean perceived trust consistently increases or stays steady. In addition, the third-person perspective can consistently achieve more trust than the first-person perspective except in the case of granular control, where the level of trust converges. Figure \ref{fig:anova2} showed an alternate view showing that the stop and granular control converge at the 3rd person perspective, which is consistent with what we have observed in figure \ref{fig:anova}.
% From figure \ref{fig:anova3} and figure \ref{fig:anova4}, we have observed a similar trend for the comfort level, except there's a decrease in terms of the comfort level of the user when using the granular control from a 3rd person perspective.
% hypothesis acceptance and rejection
% all the charts and tables
% item add some additional data from post questionaires and user study feedback
% The hypothesis
% \begin{enumerate}
% \item explain all the charts and tables mentioned by angela. //
% \item explictly mention all hypothesis and if we accept them or reject them //
%\item add some additional data from post questionaires and user study feedback
% \item bring up some meta comments from participants here
%
% \end{enumerate}
% \subsection{Participants}
% We recruited a total of 17 participants (N=17), with 10 males, and 7 females.
% \subsection{Threats to Validity}
% \textbf{Ordering and learning effects and fatigue: }
\begin{figure*}
\centering
\includegraphics[scale=0.1]{user-study.png}
\caption{The setup of a user study scenario}
\label{fig:flowchart}
\end{figure*}
% \textbf{Experimental Setup}: The participants of the study were subjected to the following biases (1) order effect (2) learning effects (3) novelty effect/fatigue. Due to technical reasons, the randomization was not done consistently, with the majority of the study conducted with a decreased level of autonomy, resulting in the ordering effects. Several participants explicitly mentioned that their ability to manipulate the robot had improved and that they were desensitized to the speed of the robot's arm movement during the course of the study. As we move along in the user study, both the experimenters and the participants experience novelty and fatigue effects. The participants tend to interact less with the simulator or give less feedback towards the latter part of the user study. For the experimenters, fatigue may result in inconsistencies in the level of elaboration on clarifying questions. Last but not least, due to space availability, the experiment was conducted in several different rooms, including an office setting, and several conference rooms varying in size in TCS Hall. Due to the fact that some of these places where we conducted the user study were relatively public, there were occasional interruptions from people coming in and out of the conference room which may cause distractions to the participants.
% \textbf{Technical Issues}: Due to the fact that the iGibson/Assistive Gym is constructed as a research prototype, there are a number of bugs
\textbf{Physical robot experiments}: We would like to conduct this user study again with a physical Jaco Robot, which increases the realism and conforms to the actual deployment scenario. We aim to use the lessons learned from running a study through simulators and find more interesting results through study redesign.
\textbf{Anthropomorphism}: During the user study, we have feedback from P6 regarding anthropomorphism, the subject mentioned that the robot looks more like an industrial robot rather than an assistive feeding robot which makes them feel uncomfortable. Adding an anthropomorphism component to the robotic arm may make the robot more approachable.
\textbf{Feedback mechanisms}: During the case study, we have feedback from P14 about the lack of diverse control input and feedback mechanisms, especially as the user of the assistive feeding robot may have mobility issues. In the future, in addition to the keyboard input mechanism, we would like to explore other input/feedback mechanisms, such as using eye gaze/blinking, and head tilting, to fulfill diverse use cases. In the current setup (especially in the case of no control), the robot has no way of processing feedback mechanisms from the users, and the lack of communication results in perceived parent-child-like power dynamics between the robot and the patient. We hypothesize that the additional feedback mechanism can make the human feel heard and respected, and preserve their dignity.
% quantitative results
To study the relationship between the control level given to the user and the perceived trust, we conducted a user study with 17 participants (N=17) from various disciplines using an assistive feeding task. Through our experiment, we used a mix-methods research methodology with surveys and interviews and discovered based on the statistical analysis that the more control given to the user, the more trust and comfort the user has towards the robot. Perceived comfort has been shown to have a significant correlation with perspective but perceived trust does not have a significant correlation with perspective. Trust is shown to be correlated with comfort level, indicating that comfort level can be an accurate predictor of trust. Finally, the 1st perspective has achieved a consistently lower level of trust than the 3rd person perspective. %Angela verified
% Generated by IEEEtran.bst, version: 1.14 (2015/08/26)
\begin{thebibliography}{10}
\providecommand{\url}[1]{#1}
\csname url@samestyle\endcsname
\providecommand{\newblock}{\relax}
\providecommand{\bibinfo}[2]{#2}
\providecommand{\BIBentrySTDinterwordspacing}{\spaceskip=0pt\relax}
\providecommand{\BIBentryALTinterwordstretchfactor}{4}
\providecommand{\BIBentryALTinterwordspacing}{\spaceskip=\fontdimen2\font plus
\BIBentryALTinterwordstretchfactor\fontdimen3\font minus
\fontdimen4\font\relax}
\providecommand{\BIBforeignlanguage}[2]{{%
\expandafter\ifx\csname l@#1\endcsname\relax
\typeout{** WARNING: IEEEtran.bst: No hyphenation pattern has been}%
\typeout{** loaded for the language `#1'. Using the pattern for}%
\typeout{** the default language instead.}%
\else
\language=\csname l@#1\endcsname
\fi
#2}}
\providecommand{\BIBdecl}{\relax}
\BIBdecl
\bibitem{schule2022patients}
M.~Sch{\"u}le, J.~M. Kraus, F.~Babel, and N.~Rei{\ss}ner, ``Patients' trust in
hospital transport robots: Evaluation of the role of user dispositions,
anxiety, and robot characteristics,'' in \emph{2022 17th ACM/IEEE
International Conference on Human-Robot Interaction (HRI)}.\hskip 1em plus
0.5em minus 0.4em\relax IEEE, 2022, pp. 246--255.
\bibitem{herse2021using}
S.~Herse, J.~Vitale, B.~Johnston, and M.-A. Williams, ``Using trust to
determine user decision making \& task outcome during a human-agent
collaborative task,'' in \emph{Proceedings of the 2021 ACM/IEEE International
Conference on Human-Robot Interaction}, 2021, pp. 73--82.
\bibitem{tomzcak2019let}
K.~Tomzcak, A.~Pelter, C.~Gutierrez, T.~Stretch, D.~Hilf, B.~Donadio, N.~L.
Tenhundfeld, E.~J. de~Visser, and C.~C. Tossell, ``Let tesla park your tesla:
driver trust in a semi-automated car,'' in \emph{2019 Systems and Information
Engineering Design Symposium (SIEDS)}.\hskip 1em plus 0.5em minus 0.4em\relax
IEEE, 2019, pp. 1--6.
\bibitem{komiak2006effects}
S.~Y. Komiak and I.~Benbasat, ``The effects of personalization and familiarity
on trust and adoption of recommendation agents,'' \emph{MIS quarterly}, pp.
941--960, 2006.
\bibitem{wallace2016impact}
S.~M. Wallace-Hadrill and S.~K. Kamboj, ``The impact of perspective change as a
cognitive reappraisal strategy on affect: A systematic review,''
\emph{Frontiers in Psychology}, vol.~7, p. 1715, 2016.
\bibitem{debarba2017}
\BIBentryALTinterwordspacing
H.~Galvan~Debarba, S.~Bovet, R.~Salomon, O.~Blanke, B.~Herbelin, and R.~Boulic,
``Characterizing first and third person viewpoints and their alternation for
embodied interaction in virtual reality,'' \emph{PLOS ONE}, vol.~12, no.~12,
pp. 1--19, 12 2017. [Online]. Available:
\url{https://doi.org/10.1371/journal.pone.0190109}
\BIBentrySTDinterwordspacing
\bibitem{gorisse2017}
\BIBentryALTinterwordspacing
G.~Gorisse, O.~Christmann, E.~A. Amato, and S.~Richir, ``First- and
third-person perspectives in immersive virtual environments: Presence and
performance analysis of embodied users,'' \emph{Frontiers in Robotics and
AI}, vol.~4, 2017. [Online]. Available:
\url{https://www.frontiersin.org/articles/10.3389/frobt.2017.00033}
\BIBentrySTDinterwordspacing
\bibitem{xu2012trust}
A.~Xu and G.~Dudek, ``Trust-driven interactive visual navigation for autonomous
robots,'' in \emph{2012 IEEE International Conference on Robotics and
Automation}.\hskip 1em plus 0.5em minus 0.4em\relax IEEE, 2012, pp.
3922--3929.
\bibitem{sadrfaridpour2016modeling}
B.~Sadrfaridpour, H.~Saeidi, J.~Burke, K.~Madathil, and Y.~Wang, ``Modeling and
control of trust in human-robot collaborative manufacturing,'' in
\emph{Robust intelligence and trust in autonomous systems}.\hskip 1em plus
0.5em minus 0.4em\relax Springer, 2016, pp. 115--141.
\bibitem{krausman2022trust}
A.~Krausman, C.~Neubauer, D.~Forster, S.~Lakhmani, A.~L. Baker, S.~M. Fitzhugh,
G.~Gremillion, J.~L. Wright, J.~S. Metcalfe, and K.~E. Schaefer, ``Trust
measurement in human-autonomy teams: Development of a conceptual toolkit,''
\emph{ACM Transactions on Human-Robot Interaction}, 2022.
\bibitem{hancock2011meta}
P.~A. Hancock, D.~R. Billings, K.~E. Schaefer, J.~Y. Chen, E.~J. De~Visser, and
R.~Parasuraman, ``A meta-analysis of factors affecting trust in human-robot
interaction,'' \emph{Human factors}, vol.~53, no.~5, pp. 517--527, 2011.
\bibitem{hsieh2020human}
T.-Y. Hsieh, B.~Chaudhury, and E.~S. Cross, ``Human-robot cooperation in
prisoner dilemma games: people behave more reciprocally than prosocially
toward robots,'' in \emph{Companion of the 2020 ACM/IEEE International
Conference on Human-Robot Interaction}, 2020, pp. 257--259.
\bibitem{abubshait2021examining}
A.~Abubshait, C.~G. McDonald, and E.~Wiese, ``Examining social cognition with
embodied robots: Does prior experience with a robot impact
feedback-associated learning in a gambling task?'' \emph{Journal of
Cognition}, vol.~4, no.~1, 2021.
\bibitem{martelaro2016tell}
N.~Martelaro, V.~C. Nneji, W.~Ju, and P.~Hinds, ``Tell me more designing hri to
encourage more trust, disclosure, and companionship,'' in \emph{2016 11th
ACM/IEEE International Conference on Human-Robot Interaction (HRI)}.\hskip
1em plus 0.5em minus 0.4em\relax IEEE, 2016, pp. 181--188.
\bibitem{freedy2007measurement}
A.~Freedy, E.~DeVisser, G.~Weltman, and N.~Coeyman, ``Measurement of trust in
human-robot collaboration,'' in \emph{2007 International symposium on
collaborative technologies and systems}.\hskip 1em plus 0.5em minus
0.4em\relax IEEE, 2007, pp. 106--114.
\bibitem{schilke2021}
\BIBentryALTinterwordspacing
O.~Schilke, M.~Reimann, and K.~S. Cook, ``Trust in social relations,''
\emph{Annual Review of Sociology}, vol.~47, no.~1, pp. 239--259, 2021.
[Online]. Available: \url{https://doi.org/10.1146/annurev-soc-082120-082850}
\BIBentrySTDinterwordspacing
\bibitem{alqasemi2005wheelchair}
R.~M. Alqasemi, E.~J. McCaffrey, K.~D. Edwards, and R.~V. Dubey,
``Wheelchair-mounted robotic arms: Analysis, evaluation and development,'' in
\emph{Proceedings, 2005 IEEE/ASME International Conference on Advanced
Intelligent Mechatronics.}\hskip 1em plus 0.5em minus 0.4em\relax IEEE, 2005,
pp. 1164--1169.
\bibitem{bhattacharjee2020more}
T.~Bhattacharjee, E.~K. Gordon, R.~Scalise, M.~E. Cabrera, A.~Caspi, M.~Cakmak,
and S.~S. Srinivasa, ``Is more autonomy always better? exploring preferences
of users with mobility impairments in robot-assisted feeding,'' in \emph{2020
15th ACM/IEEE International Conference on Human-Robot Interaction
(HRI)}.\hskip 1em plus 0.5em minus 0.4em\relax IEEE, 2020, pp. 181--190.
\bibitem{bhattacharjee2019towards}
T.~Bhattacharjee, G.~Lee, H.~Song, and S.~S. Srinivasa, ``Towards robotic
feeding: Role of haptics in fork-based food manipulation,'' \emph{IEEE
Robotics and Automation Letters}, vol.~4, no.~2, pp. 1485--1492, 2019.
\bibitem{gordon2020adaptive}
E.~K. Gordon, X.~Meng, T.~Bhattacharjee, M.~Barnes, and S.~S. Srinivasa,
``Adaptive robot-assisted feeding: An online learning framework for acquiring
previously unseen food items,'' in \emph{2020 IEEE/RSJ International
Conference on Intelligent Robots and Systems (IROS)}.\hskip 1em plus 0.5em
minus 0.4em\relax IEEE, 2020, pp. 9659--9666.
\bibitem{schultz2022proof}
J.~R. Schultz, A.~B. Slifkin, H.~Yu, and E.~M. Schearer, ``Proof-of-concept: A
hands-free interface for robot-assisted self-feeding,'' in \emph{2022
International Conference on Rehabilitation Robotics (ICORR)}.\hskip 1em plus
0.5em minus 0.4em\relax IEEE, 2022, pp. 1--6.
\bibitem{erickson2020assistivegym}
Z.~Erickson, V.~Gangaram, A.~Kapusta, C.~K. Liu, and C.~C. Kemp, ``Assistive
gym: A physics simulation framework for assistive robotics,'' \emph{IEEE
International Conference on Robotics and Automation (ICRA)}, 2020.
\bibitem{li2022igibson}
C.~Li, F.~Xia, R.~Mart{\'\i}n-Mart{\'\i}n, M.~Lingelbach, S.~Srivastava,
B.~Shen, K.~E. Vainio, C.~Gokmen, G.~Dharan, T.~Jain \emph{et~al.}, ``igibson
2.0: Object-centric simulation for robot learning of everyday household
tasks,'' in \emph{Conference on Robot Learning}.\hskip 1em plus 0.5em minus
0.4em\relax PMLR, 2022, pp. 455--465.
\bibitem{schulman2017proximal}
J.~Schulman, F.~Wolski, P.~Dhariwal, A.~Radford, and O.~Klimov, ``Proximal
policy optimization algorithms,'' \emph{arXiv preprint arXiv:1707.06347},
2017.
\bibitem{seabold2010statsmodels}
\end{thebibliography}