This research project aims to develop an integrative approach for building an embodied conversational agent (ECA) that is able to engage conversationally with human users, and capable of behaving according to social norms in terms of facial display and gestures.
Short Description of the Project
In general, our research into ECAs focuses on the attributes of autonomy and believability. To achieve autonomy, the architectural design aims to couple the agent’s perception and action appropriately, and to hide its internal mechanism. To achieve believability, we pay careful attention to the design of verbal and nonverbal interaction and to the attainment of flows of behavior that help to establish social facts within the virtual world.
In particular, the work concentrates on the development of the following key aspects of ECA technology:
- Three-layered Agent Architectures: Divide the agent into layers to support autonomous behaviors. ECAs will be able to react to actually detected situations, as well as to plan a course of action towards achieving its goals.
- Schema-base Discourse model: Provide a unified discourse framework where conversation about individual topics can be modeled. Under this framework, ECAs will able to narrate about a specific topic, as well as to respond appropriately to user queries.
- Generation and coordination of multimodal behaviors: Instantiate multimodal behaviors such as deictic gestures, turn taking, attention, and nonverbal feedback. We also need to devise an approach to coordinate the modalities that are generated.
We adopt this integrative approach in the design and development of an ECA called Elva. Elva is an embodied tour guide that inhabits an interactive art gallery to offer guidance to the human users. A user study has been performed to measure user satisfaction and agent believability. The human computer interaction evaluation methods are exploited to cover novel embodied conversational settings.
Elva welcomes user at entrance to Ng Eng Teng Gallery
A Virtual Tour with Elva
At present, the gallery houses a virtual exhibition “Configuring the Body: Form and Tenor,” which utilizes the existing content in the Ng Eng Teng Gallery of NUS Museums.
The following episode illustrates a dialog situation. Elva invited a user a guess the meaning of sculpture called untitled.
1. <Elva> This sculpture is "untitled". Can you make a guess what it looks like?
There was a short silence.
2. <Elva> Any guess, Yuan?
3. <Yuan> is it a bone?
4. <Elva> Bone. That is interesting. More?
5. <Yuan> i am not sure.
6. <Elva> Ok, let me give you a hint!
7. <Elva> Can you see it as a human body?
8. <Yuan> not really.
9. <Elva> Well, Yuan. Try to rotate the sculpture to have a better view.
User clicked on sculpture. The sculpture started to rotate.
Elva stood by and witnessed.
Silence lasted for in 8 seconds.
10. <Elva> To me, it is a female body with the head and limbs omitted, namely a "torso".
Elva continued to elaborate on the sculpture.
The following episode depicts how Elva uses deictic gestures to direct a user’s attention.
Elva stands at the sculpture, “Oh my bump”. She is going to explain to figures on the wall.
1. <Elva> Now Yuan, please look at the wall beside this sculpture.
2. These figures are extracted from a sketch book, where Eng Teng experimented on the "torso-to-face" concept.
Elva walks towards the figures on the wall, and stands just in front of the wall.
3. <Elva> As you can see on the right most, a human body is displayed.
As seen in the Figure above, Elva raised her hand and point to the right most of the wall-figures.
4. <Elva> The body gradually transforms to a face, as you can see on the left most.
As seen in the Figure above, Elva gradually moved her hand towards the left most of the wall-figures.
5. <Elva> Notice that from right to left, the interpretation of the human body becomes more and more abstract.
6. <Elva> Interestingly, the idea of "torso-to-face" comes from the Eng Teng's life drawing experience.
As seen in the Figure above, after Elva narrated about the wall-figures, she turned to the user.
Yuan, X. & Chee, Y. S. (2003). Embodied tour guide in an interactive virtual art gallery. In Proceedings of the International Conference on CyberWorlds 2003, pp. 432437. Los Alamitos, CA: IEEE Computer Society.
Yuan, X. & Chee, Y. S. (accepted, 2004). Design and evaluation of Elva: An embodied tour guide in an interactive virtual art gallery. To appear in Journal of Computer Animation and Virtual Worlds.