Research Overview
I am a Professor in the Department of Computer Science at the University of Minnesota Duluth (UMD). I teach a variety of classes on computer science and conduct research activities with both undergraduate and graduate students. My research interests focus on a combination of developing simulation codes and effective interaction related to environment simulation and learning environments. I have been specifically interested in exploring effective use of virtual, augmented or mixed reality environments for education and learning. I also work on software related to large-scale simulation of urban micro-climates to investigate the impact of the built environment on wind flow, pollution dispersion and energy use. I have worked on projects related to visual perception in virtual environments investigating how humans perceive and interact with simulated 3D worlds.
Currently Active Projects
Tarrytown - Learning in Immersive Virtual Environments
This interdisciplinary project spans Computer Science, Communications and the Digital Arts working with PIs Edward Downs and Betsy Hunt. Working with the PIs, students from CS, Communication and the Arts have developed an interactive, virtual environment in which to investigate aspects of learning. The immersive environment visuals and story is based on Tarrytown and the Old Dutch Church of Sleepy Hollow Cemetery https://sleepyhollowcemetery.org/. The goal of this project is to create stimulus material that can be used to experimentally test learning in VR, as well as to create an interactive VR experience for students and members of the community to enjoy. The multidisciplinary nature of this project showcases how art, social science, computer science, and digital production can work synergistically to advance what we know about learning in VREs.
QES - Quick Environmental Simulation
A collaborative project including researchers at the University of Utah, the Quick Environmental Simulation (QES) code is a low-computational-cost framework designed to compute high-resolution wind and concentration fields in complex atmospheric-boundary-layer environments. QES is written in C++ and NVIDIA's CUDA to utilize accelerated computation. Documentation for QES is maintained here: https://qes-documentation.readthedocs.io/en/latest. The publicly available release of this software is available on github: https://github.com/UtahEFD/QES-Public.
Exploring Radiance Field Content Generation for Virtual Reality - IEEE VR 2024
Work with Graduate Student Asif Sijan, presented as a poster at IEEE VR 2024 - Generating content for virtual reality takes effort and expanding how content can be generated has the potential to increase participation by a wider range of users well outside virtual reality researchers and programmers. This work explores some current content acquisition and generation techniques for replicating real world 3D scenes and objects to better understand the practical use and application of these techniques. The techniques explored focus on the recent emergence of radiance field-based methods and their potential to replace previous scene/object-generating techniques.
Selected Past Projects
Grave Errors - Exploring Motion Mechanics in Virtual Reality Learning Environments
This project explored the affordances that locomotion type (teleportation or natural walking) may impart on learning within an immersive virtual reality environment. The study utilized a fully immersive virtual cemetery modeled after Edgar Lee Masters’ (1915) fictitious Spoon River Anthology to assess how motion mechanics related to participant navigability would influence learning outcomes, spatial updating and concepts related to spatial reconstruction.
Designing Immersive Virtual Environments for Cognitive Learning and Spatial Memory Tasks
Virtual reality provides a realistic way to learn at a flexible progression and to develop skills that could be difficult to grasp in the real world. Our hypothesis is that there are certain VR affordances that educators and developers can leverage to build simulated learning experiences that can transform education and training activities. We investigated the role that natural locomotion and teleportation may have on cognitive and spatial information processing in a virtual environment. The learning space is a virtual cemetery, and it consists of thirteen tombstones with stories about the lives of the residents of spoon river, a fictional town mentioned in Spoon River Anthology by Edgar Lee Masters. (2019)
Effects of Avatar Hand-size Modifications on Size Judgments of Familiar and Abstract Objects in Virtual Reality
Many research studies have investigated spatial understanding within virtual envi- ronments, ranging from distance estimation, size judgments, and perception of scale. Eventually, this knowledge will help us to create virtual environments that better match our spatial abilities within natural environments. To further understand how people interpret the size of virtual objects, we present an experiment that utilizes a proprioceptive-based size estimation measure designed to elicit a three-dimensional judgment of an object’s size using a box-sizing task. (2019)
GEnUSiS - Urban Environmental Simulation
Simulating complex interactions between urban cities and environmental aspects of pollution dispersion and energy use, while developing immersive and interactive decision support systems for communicating simulation results.
Visualization
Comparing Interface Affordances for Controlling a Push Broom in VR
This thesis explores how VR controller interfaces affect how participants hold a virtual push broom in VR. We aim to understand how the affordances provided by current VR controllers and a custom broom VR controller impact user hand grip in a visual VR broom task. We compare hand grip in two VR conditions against hand placement of a real push broom without VR. The goal is to understand the roles that interaction interfaces have on recreating physically accurate actions in VR training scenarios. (2018)
A Ray-Traced Sampling Framework for Visualizing and Improving Material Detail in Urban Energy Simulations with QUIC EnvSim
The hypothesis that this thesis works on is that adding micro level material details to buildings at a scale smaller than the domain discretization, will affect the energy balance calculations in the urban environment. This paper extends on QUIC EnvSim(QES) which is an extendable framework for urban climate modeling. The new model uses visual information about buildings to create more detailed calculation of material properties such as albedo, emissivity, and diffuse fraction with the urban microclimate simulation. (2015)
TPAWT Integration Project
Developing a stereo-projection virtual environment at UMD that integrates with the University of Utah's TPAWT (TreadPort Active Wind Tunnel) to explore locomotion and haptic terrain rendering with a robotic shoe device. We built a simulation of walking on Mars as part of this project to explore locomotion activities using an L-shaped projection screen. This project coordinated development with Utah's Treadport and TPAWT (treadmill-based active wind tunnels).
EnvSim - A rapid radiation transfer model for complex urban domains
This paper presents a novel, physically-based, building-resolving radiation transfer model (QESRadiant) that utilizes ray tracing techniques accelerated with graphics processing units (GPUs). QESRadiant builds on computer graphics methods, incorporating approaches for global illumination and light transport. Tests show that our methods can rapidly simulate the radiation balance for millions of surfaces with unique shapes and properties using a single consumer-class workstation.
A high performance framework for coupled urban microclimate models
This paper introduces QUIC EnvSim (QES), a scalable, high performance framework for coupled urban microclimate models. QES allows researchers to develop and modify such models, in which tools are provided to facilitate input/output communications, model interaction, and the utilization of computational resources for efficient simulations. Common functionality of urban microclimate modeling is optimally handled by the system. By employing Graphics Processing Units (GPUs), simulations within QES can be substantially accelerated. Models for computing view factors, surface temperatures, and radiative exchange between urban materials and vegetation have been implemented and coupled into larger, more sophisticated simulations.These models can be applied to complex domains such as large forests and dense cities. (2014)
Energy transfer ray tracing with OptiX
QUIC Energy is an energy modeling system for urban environments. Our research group has developed QUIC Energy as a part of a set of GPU-assisted tools with a common goal of increasing knowledge relating urban organization and design with environmental concerns. We hypothesize that it is possible to optimize urban organization, building placement, and material selection minimizing building energy consumption for heating and cooling, as well as minimizing air pollution exposure. Our work focuses on the interactions between urban structures and surroundings. Using this information, we are able to investigate potential strategies for optimization along a number of variables. (2012)
Self-Avatars and IR-based Position Tracking in Virtual Environments using Microsoft Kinects
The purpose of this work is to develop a means for providing users with real time visual body feedback when interacting in Virtual Environments using Microsoft Kinect sensors. The advent of the Microsoft Kinect provided the research community with an inexpensive but versatile piece of equipment that can be used to obtain real time 3D information of the physical world. Our setup uses multiple Microsoft Kinects to capture a 3D point cloud of a participant in the Virtual Environment, constructs a 3D mesh out of the point cloud, and reprojects the 3D data into a Virtual Environment so that a user can experience his/her own body moving when looking through an immersive display device. (2012)
Mobile Language Learning Group (MLLG)
Interdisciplinary collaboration bringing together computer science students, foreign language students, and graphic design students to develop language learning applications.
Interface
Accelerating urban Lagrangian dispersion simulations
In this paper, we describe a novel application of an existing Lagrangian dispersion modeling system to achieve real-time simulation and visualization of an urban plume that a user can interact with in a virtual environment (VE) through the utilization of commodity graphics hardware, utilizing the highly parallel computational capabilities available on graphics processing units (GPU).
Effects of HMD Properties on Distance Judgments in Virtual Reality
his article describes an experiment in which distance judgments based on normal real-world and HMD viewing are compared with judgments based on real-world viewing while wearing two specialized devices. One is a mock HMD, which replicated the mass, moments of inertia, and field of view of the HMD and the other an inertial headband designed to replicate the mass and moments of inertia of the HMD, but constructed to not restrict the field of view of the observer or otherwise feel like wearing a helmet.
Ribbon networks for Modeling Paths of Autonomous Agents in Virtual Reality
This paper presents the environment description framework (EDF) for modeling complex networks of intersecting roads and pathways in virtual environments. EDF represents information about the layout of streets and sidewalks, the rules that govern behavior on roads and walkways, and the locations of agents with respect to navigable structures. The framework serves as the substrate on which behavior programs for autonomous vehicles and pedestrians are built.
Student Opportunities
I encourage computer science students to take on interdisciplinary projects across our campus. If you're interested in joining these teams, please contact me!