Our everyday life includes more than one social interaction, which are based mainly on human communication. The latter consists of human emotional and behavioral informations, expressed by voice, facial expressions or body movements. However, a social interaction is not limited between humans and thus, Human-Computer and Human-Robot Interactions (HRI) are meeting their golden point of research. The multidisciplinary area of computer graphics, neuroscience, psychology, and artificial intelligence is trying to explore, decipher, decode and model the human emotional behaviors and expressions to enhance the field of social interactions between humans and technology. Toward this direction, the goal of this research is to delve deeper into the features and the nature of Human – Human (H-H) and Human – NonHuman (H-NH) social interactions, providing a detailed validated assessment of how humans react towards technology but also how the latter affects humans.  

In the first place, we examined the effect of the well-used virtual environments in the brain, identifying possible differences in perception between the virtual and the real world. An EEG device was used to capture participants’ brain activity in different brain areas examining motor, cognitive and other function of the users and a questionnaire was used to evaluate psychological factors and the sense of presence. Our results enhanced the current literature by revealing new brain states involved in virtual environments, like frontal theta state and centra alpha state and they highlighted the importance of the graphics content revealing a difference in the occipital area.  

The second part of our work is concentrated on the broad area of HRI, consisting of two different experiments. The first one concerned the identification of the effects human-humanoid interaction can have on human emotional states and behaviors, through a physical interaction with a robot, an identical human and a human. This research was supported by EEG and audio recordings as well as a questionnaire indicating that the human brain does understand visually and auditorily the difference between a robot and an identical human but the levels of concentration and motivation remain higher during HRI. 

The second experiment led us to multidisciplinary in-depth documentation, analysis, and comparison between H-H and H-NH interactions recording brain activity, muscles activity, body movements, voice, and emotional states. A dataset was created with 40 participants interacting with three different agents (human, virtual human and a robot) under the same scenario. The robot was also tested under four different roles. Human emotional and behavioral patterns were extracted and compared, providing valuable insights regarding the role of robots and the effort for humanization. The role of physical presence was also assessed.  Up-to-date researches show us that the need is focused on designing social agents in a more human-like way behaviorwise and not in terms of appearance and our study complemented the above concluding that it is the reactions of the agents that trigger the different human responses and not the appearance alone. Lastly, we developed a model that can recognize and classify the human voice based on the nature of the interlocutor with a score of 82%.   

This thesis bridge the gap among studies that have examined the role of human – likeness in nonhuman agents, studies that have examined human-human interactions alone and the ones that have analyzed and compared human reactions during nonhuman social interactions. Further research with more extended studies is required to shed more light to this broad area of H-NH interaction, revealing human reactions that can guide the design of nonhuman agents, can elicit better cognitive and emotional responses and ensure a higher level of engagement, respecting the human needs.