ISO/IEC

This document describes guidelines for developing education and training systems using VR/AR/MR technology. It defines VR/AR/MR based information modelling that can be used for education and training systems. It provides procedures and methods to be used when developing 3D VR/AR/MR based education and training systems using ISO/IEC JTC 1 standards. It also provides a systematic approach to developing VR/AR/MR based applications for systems integration areas. This work will:

- define concepts of VR/AR/MR based education and training.

- define an information modelling architecture for VR/AR/MR based education and training systems.

- specify standards based functional components for VR/AR/MR based education and training systems.

- specify framework components for implementing VR/AR/MR based education and training systems.

- include use cases for VR/AR/MR based education and training systems based on the information modelling architecture.

Device hardware technology for VR/AR/MR based education and training systems is excluded from this draft.

ISO/IEC 19776-3:2015, Extensible 3D (X3D), defines a system that integrates 3D graphics and multimedia. Conceptually, each X3D file is a 3D time-based space that contains graphic and aural objects that can be dynamically modified through a variety of mechanisms. This part of ISO/IEC 19776 defines a mapping of the abstract objects in X3D to a specific X3D encoding written out in a compact binary form. Each X3D file encoded using the Compressed binary encoding:

a. supports all of the purposes of X3D files defined in the X3D abstract specification ISO/IEC 19775; and

b. encodes X3D constructs in a compressed binary format, taking advantage of geometric and information-theoretic compression techniques.

X3D files encoded using the Compressed binary encoding may be referenced from files using other X3D encodings, and may itself reference other X3D files encoded using other X3D encodings. Sets of X3D files that use multiple encodings can only be processed by browsers that support all of the utilized X3D encodings.

ISO/IEC 19775-2:2015, Extensible 3D (X3D), defines a system that integrates 3D graphics and multimedia. Conceptually, each X3D file is a 3D time-based space that contains graphic and aural objects that can be dynamically modified through a variety of mechanisms. This part of ISO/IEC 19776 defines a mapping of the abstract objects in X3D to a specific encoding using the technique defined in ISO/IEC 14772 ? Virtual reality modeling language (VRML). Each Classic VRML-encoded X3D file:

a. supports all of the purposes of X3D files defined in ISO/IEC 19775; and

b. encodes X3D constructs in Classic VRML format.

A Classic VRML-encoded X3D file may be referenced from files using other encodings and may itself reference X3D files encoded using other encodings. Such files can only be processed by browsers which conform to all of the utilized encodings.

This document specifies syntax and semantics of the data formats for interaction devices by providing a standardized format for interfacing actuators and sensors by defining XML schema-based language named Interaction Information Description Language (IIDL). IIDL provides a basic structure with common information for communication with various actuators and sensors in consistency. Device Command Vocabulary (DCV) is defined to provide a standardized format for commanding individual actuator, and Sensed Information Vocabulary (SIV) is defined to provide a standardized format for holding information from individual sensors either to get environmental information from real world or to influence virtual world objects using the acquired information on the basis of IIDL.

The technologies specified in this document are description languages and vocabularies which describe sensorial effects. The adaptation engine is not within the scope of this document (or the ISO/IEC 23005 series). This document specifies syntax and semantics of the tools describing sensory information to enrich audio-visual contents: Sensory Effect Description Language (SEDL) as an XML schema-based language which enables one to describe a basic structure of sensory information; Sensory Effect Vocabulary (SEV), an XML representation for describing sensorial effects such as light, wind, fog, vibration, etc. that trigger human senses.

The technologies of this document specified are:

- Description languages and vocabularies to characterize devices and users;

- Control information to fine tune the sensed information and the actuator command for the control of virtual/real worlds, i.e., user's actuation preference information, user's sensor preference information, actuator capability description, and sensor capability description. The adaptation engine is not within the scope of this document. This document specifies syntax and semantics of the tools required to provide interoperability in controlling devices (actuators and sensors) in real as well as virtual worlds: Control Information Description Language (CIDL) as an XML schema-based language which enables one to describe a basic structure of control information;

- Device Capability Description Vocabulary (DCDV), an XML representation for describing capabilities of actuators such as lamps, fans, vibrators, motion chairs, scent generators, etc.;

- Sensor Capability Description Vocabulary (SCDV), interfaces for describing capabilities of sensors such as a light sensor, a temperature sensor, a velocity sensor, a global position sensor, an intelligent camera sensor, etc.;

- Sensory Effect Preference Vocabulary (SEPV), interfaces for describing preferences of individual user on specific sensorial effects such as light, wind, scent, vibration, etc.; and

- Sensor Adaptation Preference Vocabulary (SAPV), interfaces for describing preferences on a sensor of an individual user on each type of sensed information.

The objective of this document is to propose an extension to the existing standard for the information model for representing the mixed and augmented reality scene/contents description, namely:

1) Extending the existing and conventional constructs for representing the virtual reality scene graph and structure such that a comprehensive range of mixed and augmented reality contents can also be represented.

2) As part of the extension, representing physical objects in the mixed and augmented reality scene targeted for augmentation.

3) As part of the extension, representing physical objects as augmentation to other (virtual or physical) objects in the mixed and augmented reality scene.

4) Providing ways to spatially associate aforementioned physical objects with the corresponding target objects (virtual or physical) in the mixed and augmented reality scene.

5) Other necessary functionalities and abstractions that will support the dynamic MAR scene description such as event/data mapping, and dynamic augmentation behaviours.

6) Describing the association between these constructs and the MAR system which is responsible for taking and interpreting this information model and rendering/presenting it out through the MAR display device.

The document also provides definitions for terms as related to these MAR content informational components and their attributes. The target audience of this document are mainly MAR system developers and contents designers interested in specifying MAR contents to be played by an MAR system or browser. The standard will provide a basis for further application standards or file formats for any virtual and mixed reality applications and content representation. The extension will be self-contained in the sense that it is independent from the existing virtual reality information constructs, focusing only on the mixed and augmented reality aspects. However, this document only proposes the information model, and neither promotes nor proposes to use a specific language, file format, algorithm, device, implementation method, and standard. The proposed model is to be considered as the minimal basic model that can be extended for other purposed in actual implementation.

ISO/IEC 14772, the Virtual Reality Modeling Language (VRML), defines a file format that integrates 3D graphics and multimedia. Conceptually, each VRML file is a 3D time-based space that contains graphic and aural objects that can be dynamically modified through a variety of mechanisms. This part of ISO/IEC 14772 defines a primary set of objects and mechanisms that encourage composition, encapsulation, and extension. The semantics of VRML describe an abstract functional behaviour of time-based, interactive 3D, multimedia information. ISO/IEC 14772 does not define physical devices or any other implementation-dependent concepts (e.g., screen resolution and input devices). ISO/IEC 14772 is intended for a wide variety of devices and applications, and provides wide latitude in interpretation and implementation of the functionality. For example, ISO/IEC 14772 does not assume the existence of a mouse or 2D display device. Each VRML file:

a. implicitly establishes a world coordinate space for all objects defined in the file, as well as all objects included by the file;

b. explicitly defines and composes a set of 3D and multimedia objects;

c. can specify hyperlinks to other files and applications; and

d. can define object behaviours.

An important characteristic of VRML files is the ability to compose files together through inclusion and to relate files together through hyperlinking. For example, consider the file earth.wrl which specifies a world that contains a sphere representing the earth. This file may also contain references to a variety of other VRML files representing cities on the earth (e.g., fileparis.wrl). The enclosing file, earth.wrl, defines the coordinate system that all the cities reside in. Each city file defines the world coordinate system that the city resides in but that becomes a local coordinate system when contained by the earth file. Hierarchical file inclusion enables the creation of arbitrarily large, dynamic worlds. Therefore, VRML ensures that each file is completely described by the objects contained within it. Another essential characteristic of VRML is that it is intended to be used in a distributed environment such as the World Wide Web. There are various objects and mechanisms built into the language that support multiple distributed files, including:

a. in-lining of other VRML files;

b. hyperlinking to other files;

c. using established Internet and ISO standards for other file formats; and

d. defining a compact syntax.

This document specifies the syntax, semantics and decoding for visual volumetric media using video‑based coding methods. This document also specifies processes that can be needed for reconstruction of visual volumetric media, which can also include additional processes such as post‑decoding, pre-reconstruction, post‑reconstruction and adaptation.

This document specifies:

- physical and material parameters of virtual or real objects expressed to support comprehensive haptic rendering methods, such as stiffness, friction and micro-textures; and

- a flexible specification of the haptic rendering algorithm itself.

It supplements other standards that describe scene or content description and information models for virtual and mixed reality, such as ISO/IEC 19775 and ISO/IEC 3721-1.

This document specifies an image-based representation model that represents target objects/environments using a set of images and optionally the underlying 3D model for accurate and efficient objects/environments representation at an arbitrary viewpoint. It is applicable to a wide range of graphic, virtual reality and mixed reality applications which require the method of representing a scene with various objects and environments. This document:

(1) defines terms for image-based representation and 3D reconstruction techniques;

(2) specifies the required elements for image-based representation;

(3) specifies a method of representing the real world in the virtual space based on image-based representation;

(4) specifies how visible image patches can be integrated with the underlying 3D model for more accurate and rich objects/environments representation from arbitrary viewpoints;

(5) specifies how the proposed model allows multi-object representation; and

(6) provides an XML based specification of the proposed representation model and an actual implementation example (see Annex A).

The technologies of this document specified are description languages and vocabularies to describe virtual world objects. The adaptation engine is not within the scope of this document. This document specifies syntax and semantics of the tools used to characterize a virtual world object related metadata: Virtual World Object Characteristics (VWOC) as an XML Schema-based language which enables one to describe a basic structure of avatars and virtual world objects in virtual environments.