TU Bergakademie Freiberg - Institute for Informatics - Virtual Reality and Multimedia Group


  Virtual Workers Virtual Workers

Virtual Workers

Interactive Digital Humans in Virtual Prototyping




Duration
2005 - 2009 (follow-up research to the Virtuelle Werkstatt project)


Description
The project ´Virtual Workers´ aims to develop novel methods in Virtual Reality (VR) based virtual prototyping through the inclusion of semi-autonomous, intelligent digital humans. A main goal concerns the transmission of operating, maintenance, and assembly procedures performed by a human VR user to digital humans of different body shapes and sizes. In this way, not only the effort required for the generation of 3D animations will be greatly reduced; furthermore, functional and ergonomic verification of virtual prototypes can be accomplished in a generalized manner. The following individual goals are pursued:
  • From digital avatars to autonomous, AI-enhanced virtual workers: Computer graphical digital humans will be extended with simulated sensors and an Artificial Intelligence-based control architecture with deliberative and reactive layers. The resulting ´virtual workers´ will be capable of manipulating the virtual prototypes in an autonomous fashion.
  • From virtual prototype evaluation by one VR user to the evaluation by many virtual workers: The virtual workers will learn to perform operating, maintenance, and assembly procedures by imitation of human VR users. The learnt procedures are stored as high-level plans that can be adapted on-the-fly to virtual humans of different sizes.
  • Implementation of an integrated VR system supporting the design, evaluation, and variation of virtual prototypes: Operating, maintenance, and other procedures are performed either interactively by the human VR user, or - through the application of imitation learning techniques - by virtual workers of different sizes.

 

See also
The action capture page provides more information on an VR-based approach to imitation learning, by which the virtual workers learn to repeat interactively demonstrated procedures on virtual prototypes.
Our work on grasp recognition compares a variety of classification methods for hand shape recognition during grasping. It shows, that a reliable classification of grasp types can be achieved even with uncalibrated data gloves.

Team
Prof. Dr.-Ing. Bernhard Jung
Dipl.-Inform. Guido Heumer
Dr. Arnd Vitzthum
Dipl.-Inform. Heni Ben Amor
Dipl.-Inform. Mathias Weber

Media Images and videos

Publications
B. Jung, H. Ben Amor, G. Heumer, A. Vitzthum. Action Capture: A VR-based Method for Character Animation. In Virtual Realities. Dagstuhl Seminar 2008. Springer. To appear 2010.

A. Vitzthum, H. Ben Amor, G. Heumer, B. Jung. Action Description for Animation of Virtual Characters. In 6. Workshop "Virtuelle und Erweiterte Realität" der GI-Fachgruppe VR/AR. 2009. 257-268. (*Best Paper Award) PDF

S. Ikemoto, H. Ben Amor, T. Minato, H. Ishiguro & B. Jung. Physical Interaction Learning: Behavior Adaptation in Cooperative Human-Robot Tasks Involving Physical Contact. RO-MAN 2009 - 18th IEEE International Symposium on Robot and Human Interactive Communication. 2009. (*CoTeSys Cognitive Human-Robot Interaction Best Paper Award). PDF

H. Ben Amor, E. Berger, D. Vogt & B. Jung. Kinesthetic Bootstrapping: Teaching Motor Skills to Humanoid Robots through Physical Interaction. KI 2009: 32nd Annual Conference on Artificial Intelligence. 2009. 492-499. PDF

F. Gommlich, G. Heumer, B. Jung & A. Vitzthum. Simulation of Articulated Standard Control Actuators in Dynamic Virtual Environments. In Proc. IEEE Virtual Reality 2009. 269-270. PDF

H. Ben Amor, G. Heumer, B. Jung & A. Vitzthum. Grasp Synthesis from Low-Dimensional Probabilistic Grasp Models. Journal of Computer Animation and Virtual Worlds, vol. 19 (3-4), 2008. 445-454. (*Best Paper Award at Computer Animation and Social Agents Conference - CASA 2008) DOI: 10.1002/cav.252

G. Heumer, H. Ben Amor & B. Jung. Grasp Recognition for Uncalibrated Data Gloves - A Machine Learning Approach. Presence - Teleoperators and Virtual Environments, vol 17(2), April 2008. MIT Press. DOI: 10.1162/pres.17.2.121

G. Heumer, H. Ben Amor, M. Weber & B. Jung. Grasp Recognition with Uncalibrated Data Gloves – A Comparison of Classification Methods. Proceedings IEEE Virtual Reality 2007, pp. 19 - 26. (*Honorable Mention) PDF

H. Ben Amor, M. Weber, G. Heumer & B. Jung. Coordinate System Transformations for Imitation of Goal-Directed Trajectories in Virtual Humans. In Virtual Environments 2007. IPT EGVE 2007. 13th Eurographics Symposium on Virtual Environments. Short Papers and Posters. 2007. 45-46. PDF

M. Weber, G. Heumer, H. Ben Amor & B. Jung. An Animation System for Imitation of Object Grasping in Virtual Reality. Proceedings ICAT 2006 - 16th International Conference on Artificial Reality and Teleexistence. 2006. PDF

B. Jung, H. Ben Amor, G. Heumer & M. Weber. From Motion Capture to Action Capture: A Review of Imitation Learning Techniques and their Application to VR-based Character Animation. Proceedings VRST 2006 – Thirteenth ACM Symposium on Virtual Reality Software and Technology, 2006, pp. 145-154.  PDF

H. Ben Amor, S. Ikemoto, T. Minato, B. Jung & H. Ishiguro. A Neural Framework for Robot Motor Learning based on Memory Consolidation. ICANNGA-07 – International Conference on Adaptive and Natural Computing Algorithms. 2007, pp. 461-468.   (*Best Young Researcher Paper Award)   PDF

G. Heumer, M. Weber, H. Ben Amor & B. Jung. Calibration-free Recognition of Grasp Types – A Comparison of Classification Methods. Proceedings Dritter Workshop Virtuelle und Erweiterte Realität der GI-Fachgruppe VR/AR. 2006.

H. Ben Amor, S. Ikemoto, T. Minato & H. Ishiguro. Learning Android Control using Growing Neural Networks. Proceedings of JSME Robotics and Mechatronics Conference ROBOMEC, 2006.

M. Weber, G. Heumer & B. Jung: Towards Grasp Learning in Virtual Humans by Imitation of Virtual Reality Users. In Processings Zweiter Workshop Virtuelle und Erweiterte Realität der GI-Fachgruppe VR/AR. Shaker-Verlag. 2005.  PDF

M. Weber, T. Pfeiffer & B. Jung: Pr@senZ - P@CE: Mobile Interaction with Virtual Reality. In Proceedings Mobile HCI - 7th International Conference on Human Computer Interaction with Mobile Devices and Services. 2005. PDF

T. Pfeiffer, M. Weber & B. Jung: Ubiquitous Virtual Reality - Accessing Shared Virtual Environments through Videoconferencing Technology. Proceedings Theory and Practice of Computer Graphics Conference. Eurographics, 2005, pp. 209-216. (*Ken Brodlie Price for Best Paper) PDF

Funding Research in the Virtual Workers is funded by the Deutsche Forschungsgemeinschaft (DFG)



April 2010