Ernst Kruijff research

Research overview

My research focuses at the investigation, design and development of 3D User Interfaces (3DUIs). Initially, I performed studies on the usage of virtual reality (VR) technology for local and distributed engineering applications. Specifically, at [UU] [BUW] I explored cooperation aspects in the conceptual design process of architects. The research resulted at [BUW] in the development supervision of a VR aided design tool (initiated by Dr. Holger Regenbrecht), and the establishment of several experiments on distributed 3D design between premier architectural schools, the so called Virtual Design Studios. In addition, I performed several empirical studies on theoretical aspects of 3DUIs, notably on spatial knowledge effects of using virtual environments in architectural design processes. Followed up by my collaboration with several foreign researchers (Bowman, LaViola,Poupyrev,Mine), I formalized theoretical and practical knowledge on 3DUIs in a single source. I presented the material at some prominent conferences, which resulted into the authoring of a book on 3DUIs, released in 2004 by Addison-Wesley. Through my work at the scientific institutions [GMD] [FHG] and [TUG], from 1999 on an additional main focus has been the design, development and evaluation of 3D user interface devices and displays, in close relation to studies on human factors and ergonomics. Over the years, I accomplished six input devices and display systems. Hereby, I did not only study purely 3D devices and techniques, but also the mixing of techniques through hybrid interfaces. I applied theoretical and practical knowledge on 3DUIs during scientific coordination of several large international projects managed under own direction, including the public funded projects ProViT and DHX. Both projects studied the usage of distributed virtual environments, either for engineering purposes, or for sharing digitalized cultural heritage, and included strong industrial involvement.

Research skills and results summary

• Scientific studies in the field of 3D user interfaces
  Within the framework of scientific projects, I have performed detailed user studies including comprehensive human factors and ergonomics analyses, and carried out empirical evaluations. Major topics I investigate are multisensory interfaces and sensory integration, hybrid interfaces and pseudo-haptics. These studies resulted in a multitude of scientific publications, and the design and development of 6 interaction and display devices.
  
  
• Formalization of theoretical issues of 3D user interfaces
  Throughout scientific projects and the cooperation with several international researchers (Bowman, LaViola, Poupyrev, Mine) I have spent a great amount of effort on formalizing the field of 3D user interfaces, with a specific focus on wayfinding and system control methods. The formalization has been presented at multiple conferences (ACM SIGGRAPH, IEEE VR) and published in the reference book "3D User Interfaces - Theory and Practice".
  
  
• Scientific coordination and management of research projects,
  I was responsible for scientific and manegerial coordination of several large public-funded and internal projects, defining the main scientific directions and taking over some of the scientific studies and developments. I coordinated teams of up to 8 people, the total cost volume of the projects was around 10 million euros.
  
  
• User and task analysis, system specification of complex systems
  Especially within the large public funded projects, I have performed multiple user and task analyses, and defined several complex system architectures, mostly focusing as integrating distributed systems components.
  
  

Interests
3D User interfaces, Virtual / Mixed Reality, haptics and pseudo-haptics, unconventional human computer interfaces, human factors and ergonomics, multisensory integration, tangible interfaces, biofeedback, neuroscience, virtual storytelling.

Positions

Professional experience

• Co-organizer ACM VRST 2005, eCAADe2000
• Marketing of Technology at [FHG] and [VS]
• Review activities conferences: IEEE Virtual Reality (VR), Immersive Projection Technology (IPT) Symposium, Graphics Interface (GI), Eurographics Workshop on Virtual Environments (EGVE), Interactive 3D Graphics (I3D), and the International Symposium on Mixed and Augmented Reality (ISMAR).
• Review activities journals: Springer Journal Virtual Reality, ASME Journal of Computing And Information Science In Engineering
• International Programm Committee member WSCG 2004-2006.
• Reviewer for Elsevier Scientific
• Reviewer for the Dutch Government (NWO)
• Academical member of ACM and igroup.
• Founder of the Collaborative Virtual Design Environments (CVDE) mailinglist and archive (mailinglist inactive). I believe I also coined the word "CVDE".
  An older overview of virtual reality supported conceptual design tools can be found here.
• Teaching experience in multiple Virtual Reality and visualisation courses
  

Projects

Ph.D.
Unconventional 3D User Interfaces for Virtual Environments
Institute for Computer Graphics, Graz University of Technology

My Ph.D. focuses at the investigation and development of novel and partly truly unconventional spatial interaction techniques for virtual environments (VE). Based on the examination of the human potential of all psycho-physiological systems, it provides an overview of possibilities in which the human body can be used for interaction in a VE, ranging from eye-based input and pseudo-haptics, up to the usage of biosignals. The study is complemented by a series of studies specifically focusing at the development and testing of novel interaction techniques and devices.
(supervision Prof. Dieter Schmalstieg, Jun. Prof. Steffi Beckhaus)

Public-funded projects (scientific coordination)
At [FHG] I was the main scientific coordinator of the following public-funded projects:

• DHX
  Digital Ecological and Artistic Heritage Exchange - transcontinental guidance and exploration in globally shared cultural heritage (EU IST - 2001 - 33476 ). 2002-2005.
  The DHX (Digital Ecological and Artistic Heritage Exchange) project established a networked virtual reality infrastructure and content development environment for museums and cyber theatres, for mutual exchange of digital cultural and natural heritage. A consortium of European and Asian partners participated for transcontinental shared immersive experience in a global scale using high-bandwidth trans Euro-Asian networks.
  The main components of the project included networked audio-videoconferencing, virtual guides using advanced virtual storytelling methods, multi-person interaction methods, image-based rendering techniques, computer-vision based scene reconstruction. Project results are being used at several locations, including the Natural History Museum of Crete in Heraklion, and the Beethoven Haus in Bonn.
  
  Within the project, I especially performed multiple user and tasks analyses, defined the overall system architecture, and defined the evaluation mechanisms. Furthermore, I have taken care of the interaction mechanisms for remote virtual guidance. The developments I made, coming out of these research activities, included the Capsa Arcana console construction (more info here) and the iWoody technology (together with Dr. Sergey Matveyev). Further information can be found here.
  
  Related publications:
  • Kruijff, E., Conrad, S., Palamidese, P., Mazzoleni, P., Hasenbrink, F., Suttrop, M., Kwon, Y-M., Remote Virtual Guidance in Immersive Museum Applications. VSMM2004, Gifu, Japan, 2004.
  • Conrad, S., Kruijff, E., Suttrop, M., Hasenbrink, F., Lechner, A. A Storytelling Concept for Digital Heritage Exchange in Virtual Environments. In proceedings of the International Conference on Virtual Storytelling (Poster), Toulouse, France, 2003.
  • Goebel,M., Kruijff,E., Matveyev, S., Severin,I., Simon,A., Tikhonravov,A. Matveyev, K.,Kozlov, I. Palamidese, P., Carreras, F., Massei, L. Museums and Virtual Environments. EVA 2003, Moscow, 2003.
  • Kruijff, E., Severin, I. Transcontinental guidance and exploration of digital cultural heritage (DHX). In proceedings of CIDOC/ADIT 2003, St. Petersburg, Russian Federation, 2003.
  • Goebel, M., Kruijff, E., Mostafawy, S, Mine, M. Storytelling in Virtual Environments - edutainment in Digital Heritage Exchange (DHX) scenarios. Workshop at the 8th International Conference on Virtual Systems and Multimedia (VSMM), 2002, Gyeongju, Korea.

• ProViT
  Digital Product Development with Virtually Cooperating Teams using Optical Networks (BMBF). 2001-2004.
  The ProViT project provided a work environment for engineers which allows design review via distributed virtual environments. The scenarios encapsulate data from machinery, airplane and ship industry to be shared over high-bandwidth networks. The project among others included new approaches to immersive teleconference / Telepresence systems (including advanced AV-techniques), integration of 2D and 3D information repositories and interaction possibilities, new session- and networkmanagement techniques, and the application of physical simulation techniques. Further information can be found here.
  
  Within the project, I coordinated the user and task analysis of multiple scenarios, and created the overall system architecture. Furthermore, I was highly involved in setting up and creating the hybrid interaction mechanisms used to interact with 2D and 3D display systems.

  
  
  Related publications:

  • Cao, W., Gaertner, H., Conrad, S., Kruijff, E. , Langenberg, D., Schultz, R. Digital Product Development in a Distributed Virtual Environment. VRAI 2003, Tianjin, Proceedings of the SPIE, Volume 5444, pp. 322-326 , 2003.
  • Kruijff, E., Conrad, S., Mueller, A. Flow of action in mixed interaction modalities. In proceedings of HCI International 2003, Heraklion, Greece, 2003.
  • Mueller, A., Conrad, S., Kruijff, E.. Multifaceted interaction with a Virtual Engineering Environment using a Scenegraph-oriented Approach. In proceedings of WSCG 2003 (Poster), Plzen, Czech Republic, 2003.

   

Public-funded projects (participation)

• At [TUG] I cooperated in the Liverplanner project
  (Austrian science fund FWF under grants P17066 and Y193)
  Within the liverplanner project, a novel system for interactive visualization and manipulation of medical datasets for surgery planning based on a hybrid VR / Tablet PC user interface was created . The goal of the system is to facilitate efficient visual inspection and correction of surface models generated by automated segmentation algorithms based on x-ray computed tomography scans, needed for planning surgical resections of liver tumors.
  
  Within the project I developed a new input device called Eye of Ra , and prepared and performed a large user evaluation (together with Alexander Bornik and Thomas Pock). Further information can be found here.
  
  Related publication:

  • Bornik, A., Beichel, R., Kruijff, E., Reitinger, B., Schmalstieg. A Hybrid User Interface for Manipulation of Volumetric Medical Data. In proceedings of Symposium on 3D user interfaces, IEEE Virtual Reality Conference, 2006.

• At [BUW] I participated in the EU-funded projects AVOCAAD and ACCOLADE

Internal projects at [FHG]
Development of interaction techniques and devices. More information under technology section.

Internal projects at InfaR [BUW]

• Emperical experiments and usability studies of three-dimensional user interfaces, input devices and interaction techniques, multiple requirement analysis studies. The longest study investigated wayfinding issues for navigation in complex virtual architectural environments, together with Stefan Hansen.
  
  Related publications:

  • Kruijff, E., Donath, D. Supporting Shared Architectural Understandig - Spatial knowledge transfer within Virtual Environments. Proceedings of ACCOLADE 2000 workshop, Bruxelles, 2000.
  • Donath,D., Kruijff,E., Regenbrecht, H. Spatial knowledge implications by using a Virtual Environment during design review. Proceedings of ACADIA 1999, Salt Lake City, 1999.
    

• Project participation/coordination DAAD-RGC funded project (PhD research grant) "the influence of virtual environments on the design of (digital) architectural expressions", together with Marc Aurel Schnabel, and Prof. Thomas Kvan, Hong Kong University. The study focused at the influence and the relevance of Virtual Environments (VE) on the perception of space and design of architectural structures, allowing architects to develop new tools and idioms to express and create space and volume.
  
  Related publication:

Research
Research overview
Research skills / results Positions
Professional experience

Projects
Ph.D.
Public funded projects
Internal projects