Building-scale VR is a virtualized indoor area of an actual building that enables accurate tracking of the position and orientation of humans throughout the indoor area of the building. The technologies that make up the building-scale VR include:

1. 3D map creation technology that faithfully reproduces the 3D shape of an actual building
2. Technology for automatically displaying a 3D map as a point cloud in a virtual space
3. Technology that tracks the pose (position and orientation) of people moving around the entire indoor of a building
4. Technology for placing and transforming movable objects on a 3D map
5. Technology for displaying multiple people in a virtual space and all of them share the same information as in real life
6. Interface technology for operating objects with both hands in virtual space
7. Technology that reflects changes in the real-world situation in virtual space in real time
8. Technology that automatically generates environmental sounds and vibrotactile haptics from textures for real world objects and presents them in virtual space
9. Technology for simulating human behaviors and physical phenomena in virtual space

A typical application of building-scale VR is simulation of disaster situations and training using them. Not only disaster situations, but also various dangerous situations can be simulated so that multiple people can experience them at the same time. In the simulation of a disaster situation, moving or transforming an indoor object. For example, in the event of an earthquake, furniture will vibrate and move. In the case of a fire, furniture may burn and become deformed. For such a simulation, we have placed a 3D model on the point cloud map so that it can be operated individually.

There are several advantages to giving a presentation in a virtual space. First of all, even if you participate from a distance, you can get the effect of listening to the presentation in front of you, and since all information is digitized in the virtual space, you can record and reproduce it. It is possible to easily rehearse by expressing the audience with a 3D model + animation. Also, by using the building scale VR, you can give a presentation in the same situation as in a real conference hall. In addition, we are conducting research on presentation rehearsals with AI avatars as spectators and researching a mechanism for evaluating the presentation. Along with discussion, presentation is one of the important skills, so you need to be able to analyze it effectively and train effectively. Specifically, the presenter's speaking style, body and line-of-sight movements, slide contents and pointing movements are analyzed to calculate a score. Also, the reaction of the AI avatar that is the audience is determined based on the hourly evaluation value of the presentation evaluated by each avatar and displayed in real time.

Cyber-Physical Education is a new teaching and learning methodology. As a result, educational and learning activities in classrooms built with VR technology will be as effective as or even more effective than those in the real world. Moreover, the result of the activity in the virtual space can be valuable in the real world. First, we will use building-scale VR to virtualize various indoor facilities that are learning environments, and then create learning and experiment environments in virtual environments. At this time, virtualize STEM (Science, Technology, Engineering, Mathematics) educational materials and systematize knowledge about them. Based on this teaching material and knowledge, we can support learning. At this time, the VR agent plays the role of the teacher by becoming the conversation partner of the learner. A VR agent is an AI that controls a VR avatar, not a human. In this research, we apply a combination of a script-based dialogue module and a chat dialogue module for responding to off-topic questions. In addition, a mechanism that records and reproduces the activity of the learner enables another learner to relive the experience of one learner. Also, in order to ensure that learning outcomes are not limited to the virtual world, we will materialize them as something that can be used in the real world and enable them to confirm their value. All of these knowledge, experiences and achievements are saved as data that can be analyzed and used to evaluate the quality of the learner's activity using the machine learning mechanism.