This is an open access article distributed under the terms of the Creative Commons
Attribution License (Most mental disorders are associated with impairments in social functioning. Social difficulties are both diagnostic criteria for several disorders such as Autism, Schizophrenia, Social Anxiety Disorder, or Personality Disorders (Kennedy & Adolphs, 2012; Skodol et al., 2002) and also constitute risk factors for developing, sustaining, and exacerbating clinical symptoms (Cacioppo, Hawkley, & Thisted, 2010; Fowler, Allen, Oldham, & Frueh, 2013; Hawkley & Cacioppo, 2010). Though social functioning is complex and challenging to assess, several recent methodological advancements may allow to directly study social-interactive behavior and its underlying (neural) mechanisms in more ecologically valid ways. The goal of this update article is to delineate these developments and their relevance for understanding mental disorders.
Deficits in social functioning can be based on impairments in the underlying social affective and cognitive processes (Amodio & Frith, 2006) that range from basic social attention and memory to empathy and theory of mind (ToM; also termed mentalizing; Happé, Cook, & Bird, 2017; Kanske, 2018). While empathy allows access to other minds via directly sharing other persons’ emotional states (de Vignemont & Singer, 2006), mentalizing enables the understanding of others through abstract inference of their thoughts and beliefs (Frith & Frith, 2005). The typical approach to studying these phenomena in experimental clinical psychology and social neuroscience has been criticized as assessing individual minds as detached observers (Fuchs & De Jaegher, 2009). Paradigms ask participants, for instance, to predict the behavior of cartoon characters, classify emotions of static pictures of eyes, or judge the trustworthiness of face photographs. While these tasks have certainly provided valuable insight into the mechanisms of social functioning, they lack the reciprocal nature of full-blown interactions (Bird et al., 2010; Dziobek et al., 2008; Kanske, Böckler, Trautwein, & Singer, 2015; von dem Hagen, Stoyanova, Rowe, Baron-Cohen, & Calder, 2014; Walter et al., 2009).
In contrast to this observer account of social cognition, recent developments in philosophy, experimental psychology, and neuroscience call for a second-person account that engages participants in real dynamic interactions (e.g., De Jaegher, Di Paolo, & Gallagher, 2010; Gallagher, 2008; Konvalinka & Roepstorff, 2012; Schilbach et al., 2013). Researchers argue that making sense of another person during an embodied and ongoing social interaction occurs implicitly by making use of enactive perception that takes the context of a shared, intersubjective world into account. In real-time social interaction, implicit processes seem to be more relevant than the explicit forms that have been especially emphasized in previous research (Schilbach, 2016). Accordingly, the full extent of deficient social functioning in psychopathology may only manifest in complex social interactions. For instance, autistic individuals report more problems with direct and immediate social interactions than situations involving slow-paced interactions (e.g., email) or social observation.
Recent research has advanced paradigms originating from an observer account of social interaction towards implementing interactions (or at least interactive elements) with one or more real other persons. In addition to examining real, reciprocal interaction, authors suggested that even the potential for reciprocal interaction constitutes an important step forward, as one becomes actively engaged through another person that is experienced as active and salient (Krach, Müller-Pinzler, Westermann, & Paulus, 2013; Risko, Richardson, & Kingstone, 2016) – a process that is also referred to as social immersion. Among the novel approaches, research on gaze behavior has taken a leading role. One prominent paradigm makes use of anthropomorphic virtual characters who respond in a contingent way to participants’ eye movements, resulting in reciprocal interaction (Wilms et al., 2010). Comparably, Redcay et al. (2010) developed a paradigm using live video-feed that allows for gaze based face-to-face interaction between an experimenter outside and a participant inside a magnetic resonance imaging (MRI) scanner. Yet another setup even enables two participants to interact via live video-feed while simultaneously tracking their eye-movements (Hessels, Cornelissen, Hooge, & Kemner, 2017). Such a dual eye-tracking method has also been implemented in setups wherein two individuals are lying in MRI scanners (e.g., Saito et al., 2010), enabling the simultaneous acquisition of brain activation of two interacting persons (referred to as hyperscanning). Other approaches rely less on technical means but establish real live interaction between participants with, for instance, free eye contact or structured conversations while measuring the allocation of visual attention or indicators of arousal (e.g., Freeth, Foulsham, & Kingstone, 2013; Myllyneva, Ranta, & Hietanen, 2015). While these paradigms sometimes differ substantially in the way they operationalize social interactions, they can be classified along different dimensions, such as complexity of interaction, temporal dynamics, social presence, and embodiment (Table 1).
Table 1
Classification of Studies
| Task / Study | Social presence | Interaction complexity | Temporal dynamics | Embodiment | Disorder | Measure |
|---|---|---|---|---|---|---|
| Joint attention (Oberwelland et al., 2017; Redcay et al., 2013; including dual eye-tracking: Bilek et al., 2017; Tanabe et al., 2012) | Yes | Medium low | High | Medium high - visual information | Autism / Borderline Personality Disorder | Neural activity & gaze behavior |
| Mutual eye-gaze (Hessels et al., 2018; Myllyneva et al., 2015) | Yes | Low | High | Medium high - visual information | Autism / Social Anxiety Disorder | Gaze behavior |
| Listening to short stories (Rice & Redcay, 2016) | Yes vs. no (experimental condition) | Low | Low | Medium - auditory information | Autism | Neural activity |
| Listening to short stories (von dem Hagen & Bright, 2017) | Yes vs. no (experimental condition) | No interaction | - | High - visual + auditory information | Autism | Gaze behavior |
| Live face-to-face interaction (Freeth & Bugembe, 2019; Freeth et al., 2013; Hanley et al., 2015; Hanley et al., 2014; Magrelli et al., 2013; Nadig et al., 2010) | Yes | High | High | Very high - physically present | Autism | Gaze behavior |
| Live interaction (Fitzpatrick et al., 2017a, 2017b; Romero et al., 2018) | Yes | High | High | Very high - physically present | Autism | Movement kinematics |
| Performance task, audience present (Chib et al., 2018; Müller-Pinzler et al., 2015) | Yes vs. no (experimental condition) | No interaction | - | Medium high - visual information | Social Anxiety Disorder | Neural activity |
| Touch anticipation (Scalabrini et al., 2017) | Yes | Low | High | Medium - tactile information | Narcissistic Personality Disorder | Neural activity |
| Conversation (Takei et al., 2013) | Yes | High | High | Very high - physically present | Schizophrenia | Neural activity |
Note. The table summarizes how the reviewed studies (clustered by the kind of task) vary on features of social interaction. These features encompass the social presence of the interaction partner or audience in a mutual social situation; the interaction complexity expresses how much information is transferred during the interaction; the temporal dynamics of the interaction provides information about how quick responses have to be integrated and reacted upon; and the embodiment of the interaction partner expresses how rich s/he is perceived by the participant.
This update article shortly reviews recent laboratory studies that adopt a second-person account of social interaction within clinical experimental psychology and clinical social neuroscience. We want to delineate how novel and ecologically valid measures have helped to gain new insight into impairments of social interaction, with a particular focus on studies employing advanced experimental and methodological approaches. This kind of research in the context of mental disorders is still scarce and we will focus on Autism Spectrum Disorders, Personality Disorders, Social Anxiety Disorder, and Schizophrenia.
Autism Spectrum Disorder
Autism Spectrum Disorder (ASD) is a developmental disorder manifested particularly in persistent patterns of deficient social interactive and communicative behavior (e.g., irregular eye contact, behavioral inflexibility in social contexts) (American Psychiatric Association [APA], 2013).
Deficits in joint attention are one of the core impairments in ASD (Dawson, Bernier, & Ring, 2012). Establishing joint attention – for instance, by directing or following another’s gaze to an object – is a simple, but inherently interactive process (Redcay, Kleiner, & Saxe, 2012) that can easily be implemented in truly interactive settings. Using a live video-feed, Tanabe and colleagues (2012) employed a dual eye-tracking joint attention task in dyads of ASD and typically developed (TD) participants during MRI hyperscanning. ASD participants showed reduced accuracy at detecting gaze direction, corresponding with reduced neural activation in the left occipital pole, suggesting altered early visual gaze processing. Furthermore, mixed pairs of TD and ASD participants revealed lower neural synchronization in the right inferior frontal gyrus (IFG) than TD-TD pairs, which the authors attributed to problems integrating self- and other-oriented attention in ASD participants. In another joint attention study using a live video-feed, ASD participants did not differ behaviorally from TD participants (Redcay et al., 2013). However, on the neural level, ASD participants (in contrast to TD participants) did not show differential activity between social and non-social conditions in the dorsomedial prefrontal cortex (dmPFC) and left posterior superior temporal sulcus (lpSTS), which might play a role in mutual engagement with a social partner. Similar patterns were observed using virtual interacting avatars (Oberwelland et al., 2017).
Beyond simple joint attention, one study used task-independent dual eye-tracking, instructing participants to look at each other for five minutes (Hessels, Holleman, Cornelissen, Hooge, & Kemner, 2018). Pairs high in autism displayed less two-way eye gaze (i.e., eye contact), but, interestingly, more one-way eye gaze (only one participant looking in the eyes of the other). The interactive nature of this study design could provide support for the so-called gaze aversion model (i.e., avoidance of eye contact) over the gaze indifference model (i.e., insensitivity to others’ eyes) (Moriuchi, Klin, & Jones, 2017).
Directly targeting the role of true interaction, some studies investigated how different degrees of ecological validity differentially influence behavior along autistic traits. Rice and Redcay (2016) implemented a simulated live interaction between participants in an MRI scanner and an experimenter, examining how brain activity is altered depending on whether participants think speech is addressed to them live versus pre-recorded. Increasing scores in subclinical autism went along with reduced differential dmPFC activation for live compared to pre-recorded speech, presumably reflecting lower perceived liveness of the speaker in high autistic individuals. Similarly, von dem Hagen and Bright (2017) manipulated participants’ belief whether the video of a person telling a story was pre-recorded or live. While these different beliefs resulted in modulated attention towards the eye region in low autistic individuals, they did not affect the attention of persons with high autistic traits. Using mobile eye-tracking, Freeth et al. (2013) involved participants in a structured conversation with an experimenter whose social presence varied (live face-to-face interaction versus pre-recorded video). During the pre-recorded video ‘interaction’, the amount of time looked at the experimenter correlated negatively with subclinical autistic traits, whereas there was no such correlation in the face-to-face interaction. These studies suggest that individuals with ASD display reduced sensitivity to the cues of online versus offline interaction compared to TD individuals.
Several other studies have used mobile eye-tracking in the context of a more natural social environment. During a semi-structured conversation, children with ASD looked less to the face of the experimenter (particularly to the eyes) than children without ASD (Hanley et al., 2014; Magrelli et al., 2013). Children were mostly listening in Hanley and colleagues’ (2014) study, and this pattern of reduced looking at their interaction partner’s face was not found when children were primarily speaking (Nadig, Lee, Singh, Bosshart, & Ozonoff, 2010). During a structured face-to-face conversation, adults with ASD showed fewer fixations on the eyes and more fixations on the mouth as compared to TD adults, however, they showed no alterations in fixation on the face in general (Hanley et al., 2015). Similarly, Freeth and Bugembe (2019) found no difference in fixations on the face when the social partner’s gaze was averted. However, when participants were being looked at directly, individuals with ASD fixated the face for a shorter time than TD individuals. These interactive studies have helped to reveal factors that modulate social attention of autistic individuals, such as conversational phase or gaze direction of the interlocutor.
To capture nonverbal interpersonal behavior beyond eye gaze, several recent studies employed video- or device-based motion tracking (e.g., Fitzpatrick et al., 2017a; Romero et al., 2018). Using a motion-tracking device, Fitzpatrick and colleagues (2017a) implemented a battery of imitation and motor synchronization tasks to capture dynamical measures of synchronicity. Children with ASD showed reduced social synchronization abilities and had difficulties producing consistently timed movements over the course of an interaction. Interestingly, synchronization abilities correlated with performance on a false-belief ToM task (Fitzpatrick et al., 2017b). Romero and colleagues (2018) objectively quantified synchronization of whole-body movement from video recordings of live interactions, showing that complex whole-body synchronicity between children with ASD and clinicians was above chance level, and correlated negatively with ASD severity, that is, children with higher social-cognitive abilities exhibited more behavioral synchronicity. Such dynamic measures of interpersonal behavior and coordination provide interesting insights into more complex components of social interaction.
In summary, these results underline how ASD research benefits from implementing a second-person approach: Being addressed by a social partner modulates social attention in response to different contextual factors. Depending on the type of paradigm (e.g., joint attention, manipulation of the degree of ecological validity), ASD compared to TD individuals show reduced sensitivity to the presence versus absence of a real-interaction partner, both in neural and in behavioral measures. In addition, ASD participants revealed diminished levels of neural and behavioral synchronicity with TD interaction partners.
Personality Disorders
Personality disorders comprise a number of maladaptive behavioral patterns and cognitive styles (APA, 2013). Based on previous research, we will focus on Borderline Personality Disorder (BPD) and Narcissistic Personality Disorder (NPD).
BPD is characterized by unstable affect and self-image as well as impulsivity, accounting for severely impaired everyday social functioning (APA, 2013). Investigating social interactions through a joint attention task, Bilek and colleagues (2017) assessed live interacting dyads in MRI hyperscanning. Neural coupling at the site of right temporo-parietal junction (rTPJ), a core region for mentalizing processes, was lowest in BPD-healthy control (HC) dyads, which might be a cause for difficulties in social interactions in everyday life. Interestingly, coupling in dyads of remittent BPD and HC was at the level of HC-HC dyads, suggesting a state specificity or reversibility of low neural coupling in BPD.
NPD is characterized by the need for admiration, a lack of empathy as well as pronounced self-absorbedness. In subclinical narcissism, Scalabrini and colleagues (2017) reported higher scores on narcissistic grandiosity going along with reduced activation in the right anterior insula (rAI) in anticipation of touching a human hand. The rAI is a main structure of the so-called salience network that is assumed to switch attention away from internal towards external stimuli, indicating that narcissists might be less responsive to others and rather remain in self-reflective internal processes.
Taken together, though studies employing truly interactive tasks are still sparse in personality disorders, initial evidence indicates the power of these paradigms in revealing reduced interpersonal sensitivity in the respective populations.
Social Anxiety Disorder
The diagnostic criteria for Social Anxiety Disorder (SAD) include fear in social performance situations and the fear of behaving embarrassingly, leading to avoidance of the respective situations altogether (APA, 2013).
Experimental settings that use socially immersive environments in order to induce the feeling of being observed by others are particularly suitable to study social evaluative threat and embarrassment in SAD populations. In doing so, Müller-Pinzler and colleagues (2015) applied a value estimation task to investigate neural pathways of embarrassment. Participants were led to believe that feedback regarding their performance in the experimental task was shared with three confederates outside the scanner room. When feedback on their performance was made public, participants with higher levels of SAD showed heightened visual attention towards their observers’ faces, as well as increased activation in medial prefrontal cortex (mPFC) and the right fusiform face area, possibly indicating increased attention to others and mentalizing about how oneself is perceived by the audience. Similar results were reported for the performance in a motor task under observation (Chib, Adachi, & O’Doherty, 2018).
In a behavioral task using dual eye-tracking, pairs of participants were instructed to look at each other for five minutes (Hessels et al., 2018; see section on autism above). Pairs high in subclinical social anxiety were engaged in more frequent, but shorter one-way eye gaze than low social anxiety pairs. Myllyneva et al. (2015) had a person sitting opposite the participant with an LCD screen in between that could be either transparent or opaque. When the switching between transparent and opaque was computer controlled, both SAD adolescents and controls showed higher arousal to direct gaze than to averted gaze by the other person. However, when participants were forced to initiate the social interaction themselves by controlling when the screen turned transparent/opaque, this difference only remained in individuals with SAD. Hence, self-initiated interaction reduced direct-gaze related arousal in healthy participants, but not in SAD participants.
Overall, these results corroborate clinical descriptions of SAD regarding a higher concern of one’s public appearance, resulting in increased neural activation of areas associated with mentalizing and face processing. The possibility to actively initiate contact with an interaction partner could reveal different arousal patterns in response to direct (versus averted) gaze between SAD patients and healthy controls.
Schizophrenia
Schizophrenia (SCZ) is a mental disorder characterized by profound changes in behavior, communication, and cognition, with symptoms including hallucinations, disorganized speech and behavior, and delusions that greatly impair interpersonal functioning (APA, 2013; Tandon et al., 2013).
To investigate neural activation in SCZ during live face-to-face conversation, Takei and colleagues (2013) used functional near infrared spectography (fNIRS) in a sample of SCZ patients and controls to investigate neural patterns during live interaction. Specifically, participants and experimenters spoke for fixed intervals about a previously specified topic. SCZ participants exhibited less appropriate speech, lower production of new topics, and spoke less overall. On a neural level, SCZ participants showed decreased activity in bilateral temporal lobes and right inferior frontal gyrus, co-varying with negative symptoms and disorganization, where the authors suggest a causal role of these brain areas.
While the field of SCZ research is still underrepresented regarding the implementation of truly interactive paradigms, these results show an interesting trend that could not have been revealed in other, less interactive tasks.
Conclusion
A shift towards applying a second-person account has clearly benefitted research on social interaction in psychopathology, with the case of autism taking a prominent role. Several studies showed profound differences in behavioral and neural measures during actual social interactions, as compared to engaging participants as mere observers. This pattern suggests that the full extent and the nature of impairments in social functioning unfolds only in complex social interactions.
Furthermore, many social phenomena are inherently interactive and can therefore only manifest themselves in paradigms implementing real dynamic interactions. The second-person account aims at capturing the underlying mechanisms of these phenomena in their entirety. A few published studies employed hyperscanning with dyads consisting of healthy and psychopathological participants during live interaction, enabling the investigation of co-activation patterns and synchronization of brain activity. However, the possibilities to interact while lying in an MRI scanner are highly restricted and paradigms used in this context are limited in their degree of ecological validity. Likewise, the implementation of paradigms employing more complex social interactions introduces new methodological problems, such as complexity of data and reduced experimental control. Here, economic games offer the chance to study social interactions in a controlled environment but with limited flexibility within the interaction. The use of virtual reality bears potential to regain experimental control as the behavior of a virtual character can be manipulated gradually. Usually, this comes at the price of the participants being aware that they interact not with another human but a virtual agent as has been done in experimental studies addressing Autism and psychotic symptoms (e.g., Forbes, Pan, & Hamilton, 2016; Veling, Pot-Kolder, Counotte, van Os, & van der Gaag, 2016). This problem could, however, be overcome by applying a cover story making participants believe that they interact with a virtual avatar that is controlled by another human (e.g., Wilms et al., 2010).
Outside the scanner, paradigms with real life face-to-face interaction allow assessing the embodied and implicit nature of interactions. The use of technical means such as mobile eye-tracking or motion capturing devices as well as advanced analytical methods represent an advancement in objective quantification of social interactions. However, it should be noted that although these advanced methods enhance the ecological validity of social interactions in the laboratory, assessing factors like emotions or the dynamics of interpersonal relations remains challenging. Here, field methods and self-report measures are still the means of choice: ecological momentary assessment (EMA) – the collection of various types of data via portable technical devices – has the advantage of capturing real-life social interactions while (or shortly after) they are happening. Furthermore, they can be complemented with more objective measures such as the electronically activated recorder (EAR). Here, participants wear a portable audio recorder that periodically records the acoustic environment, allowing for the analysis of, for instance, the words or prosody used during social interaction or the number of interaction partners.
In conclusion, paradigms employing a second-person approach to the study of social interactions in mental disorders have yielded promising results. While research using truly interactive paradigms is still in its infancy, it holds great potential for clinical research on social interaction.