As augmented reality (AR) becomes increasingly integrated into training, maintenance, and education, the demand for tools that simplify AR application creation grows. However, designing complex, interactive AR experiences remains a challenge, often requiring advanced programming skills. To bridge this gap, we developed M2AR, a web-based metamodeling environment specifically designed for enhancing traditional and new modeling languages with 3D spatial capabilities. An application area where that makes sense is the definition of AR applications, e.g., with the newly developed Augmented Reality Workflow Modeling Language (ARWFML). Thereby, M2AR enables users to design and execute AR applications with ease – no coding required.
What is M2AR?
M2AR (short for “Model to Augmented Reality”) is a 3D-enhanced, web-based environment for modeling AR workflows. Using WebXR and the powerful three.js JavaScript library, M2AR allows users to model AR applications directly in their web browsers. By leveraging the potential of the ARWFML, users can define AR workflows, by defining scenes, and interactions visually, making the creation of AR applications accessible to users from all backgrounds.
Why 3D Modeling Matters in AR Design
AR inherently operates in a 3D space, making a 3D modeling environment essential for accurate representation and positioning of virtual elements. M2AR’s 3D modeling capabilities allow users to visualize object position, rotation, scale, depth, and interaction in a spatially accurate manner. This ensures that the virtual content created during model-time aligns seamlessly with the real world when the AR application is executed, enhancing creation, usability and effectiveness of the AR application.
Key Components of ARWFML (Augmented Reality Workflow Modeling Language)
At the heart of M2AR is the AR Workflow Modeling Language (ARWFML), a domain-specific visual modeling language designed to address the unique challenges of AR applications.
Figure 1: Language overview of the ARWFML, example of the definition of an AR application for color brick assembly.
ARWFML provides three scene types to handle different aspects of AR applications. All of these scene types are necessary to model a new AR application:
1. ObjectSpace: Defines the AR environment by specifying real-world detectables (e.g., markers or real objects) and virtual augmentations (3D visualizations that can be shown in AR). This layer also manages the spatial relationship between real and virtual components, making it easier to align augmentations with the physical world. The ObjectSpace model does not consider when and how virtual content is visualized in a potential AR application, but rather defines the content that is available for later use.
Figure 2: Example of an ObjectSpace model definition in the M2AR metamodeling client.
2. Statechange: Handles dynamic changes within the AR environment, allowing users to specify transformations like positions, rotation, or visibility changes for augmentations defined within ObjectSpace. This is particularly useful for creating interactive and adaptable AR experiences. The time, when a Statechange is triggered, is not defined here, but at a later stage.Figure 3: Example of an Statechange model definition in the M2AR metamodeling client.
Figure 3: Example of an Statechange model definition in the M2AR metamodeling client.
3. FlowScene: Defines the overall workflow of the AR application, incorporating conditions and triggers that dictate how and when Statechanges are triggered, e.g., when and how objects appear or change. By modeling these workflows, users can build complex applications based on user actions or external conditions, the detection of a real-world object, or a voice command.
Figure 4: Example of an FlowScene model definition in the M2AR metamodeling client.
Note that some works have explored the opposite direction, i.e. using AR editors to create models or even virtual reality environments for modeling.
Execution of ARWFML Models as an AR Application
The AR Engine is a key feature of M2AR, responsible for the execution of AR applications based on the defined ARWFML models. By supporting WebXR standards, the AR Engine ensures compatibility across a wide range of devices, from smartphones and tablets to AR headsets. It interprets the ARWFML models in real-time, allowing users to experience their workflows in an immersive environment.
Figure 5: Example of the resulting AR application based on the color brick use case above.
Video 1: Example of the execution of an AR application modeled with M2AR.
Future-Proof and Accessible
M2AR’s foundation on WebXR and web-based technologies makes it a future-proof tool, allowing users to create and execute AR applications on a variety of devices without additional installations. This accessibility is central to our goal of democratizing AR application design, making it easier for users to create AR content for diverse applications.
AR at Design-Time
Using the same technology for design-time and run-time allows us to use the right environment at the right time. For example, when modeling how different 3D objects will be placed during the execution of an AR application (Statechanges), it would be useful to conduct this task directly in AR. Thus, the M2AR metamodeling platform allows for the easy change from desktop- to immersive environments, e.g., on a head-mounted display like the Meta Quest 3.
Video 2: Example of the Statechange definition in AR.
Example Use Case: IKEA Furniture Assembly
Surely you know how complicated it can be to translate the cryptic images from the paper-based 2D manual to your real living room, while trying to assemble a new piece of furniture. Thus, an AR application showing you in the real world where to place the next object based on your current progress would be a welcomed help.
The figure below shows the models needed for defining an AR application for guiding a user through the assembly of an IKEA bedside table. The image on the top left shows a screenshot of M2AR being used to set up the ObjectSpace model. This model includes different Detectables (markers), which are used to detect the different furniture parts. Each Detectable is represented by a unique image, and one of them, labeled “Origin IKEA,” is marked as the origin point. This origin is referenced by the FlowScene model, helping anchor the entire AR experience. Other Detectables are linked to different Conditions in the FlowScene to guide each step of the assembly process.
The ObjectSpace also includes six Augmentations – virtual parts that represent each piece of the real bedside table in 3D. These parts are uploaded as 3D models, giving the user a realistic preview of each component in the AR assembly guide.The lower left image shows the setup of a StateChange model, which control how the AR parts change as the assembly progresses. For example, this StateChange, “Rotate MiddlePlate,” shows the “MiddlePlate” part and the “Leg 1”, after the entire construction has been rotated.
After creating the ObjectSpace, FlowScene, and seven StateChanges, all models can be executed as an AR application, providing users with an interactive, step-by-step assembly experience, e.g., as shown in the lower right image.
Figure 6: ARWFML models and the result as AR application for the IKEA use case. ObjectSpace model (top left). FlowScene model (top right). Statechange “Rotate MiddlePlate” (lower left). Visualization of the “Rotate MiddlePlate” Statechange during the execution of the AR application (lower right).
References
- Muff, Fabian and Hans-Georg Fill. “Initial Concepts for Augmented and Virtual Reality-based Enterprise Modeling.” ER Demos/Posters (2021). https://ceur-ws.org/Vol-2958/paper9.pdf.
- Muff, Fabian and Hans-Georg Fill. “A Domain-Specific Visual Modeling Language for Augmented Reality Applications Using WebXR.” International Conference on Conceptual Modeling, 2023. https://doi.org/10.1007/978-3-031-47262-6_18.
- Muff, Fabian and Hans-Georg Fill. “M2AR: A Web-based Modeling Environment for the Augmented Reality Workflow Modeling Language.” ACM/IEEE International Conference on Model Driven Engineering Languages and Systems, 2024. https://doi.org/10.1145/3652620.3687779.
- Muff, Fabian and Hans-Georg Fill. “Multi-Faceted Evaluation of Modeling Languages for Augmented Reality Applications – The Case of ARWFML.” Conceptual Modeling. ER 2024. Lecture Notes in Computer Science, vol 15238, 2024. https://doi.org/10.1007/978-3-031-75872-0_5 .
- Muff, Fabian. Metamodeling for Extended Reality. 1st ed., Springer Cham, 2025. ISBN: 978-3-031-76761-6 (Hardcover), 978-3-031-76762-3 (eBook).
Fabian Muff is a senior researcher and lecturer at the University of Fribourg, Switzerland and a member of the Research Group Digitalization and Information Systems. He holds a PhD from the University of Fribourg in business informatics. His research activities focus on the combination of conceptual modeling and extended reality technologies, as well as the development of a new 3D-enabled metamodeling platform.
Recent Comments