Uncertainty and inconsistency are inherent aspects of the development of any complex system. In practice, both must be tolerated to some extent so as not to inhibit engineers in their work, especially during the early stages of development. Tolerance becomes particularly relevant in industrial environments due to some of their practicalities, such as the common use of rather informal model designs, the existence of very large sets of loosely connected artifacts, and the difficulty of precisely defining consistency rules among them. Furthermore, the number of inconsistencies to be managed in industrial systems is often huge, so mechanisms are needed to prioritize which ones need to be resolved in each sprint and which ones can still wait.

To address these issues, in our recent paper “Uncertainty-aware consistency checking in industrial settings” (preprint) to be presented at the upcoming MODELS 2023 conference, we have proposed the explicit representation of the doubts that engineers may have about the system elements at each step of the development process, and showed how this uncertainty information can be used to prioritize the system inconsistencies. Interestingly, we discovered that making the doubts explicit also allows engineers uncovering some potential inconsistencies that otherwise might remain hidden.

We validated our proposal in two industrial settings that use model-based engineering practices, with very successful results. One of these companies uses SysML models to design and develop critical systems in C++, and models and code should be kept consistent so that, on the one hand, the design from the models is followed and, on the other hand, the models can be trusted to document the implementation. The second company uses informal models to represent the components comprising variants of a product-line based architecture, which should be kept consistent with the possible relationships among them in order to automatically develop the products they sell and maintain (in this case, components for embedded systems).

Our work addresses the following research questions:

1. What types of uncertainty and inconsistency affect the elements of these systems models?
2. How to explicitly represent the uncertainty in these models?
3. How to combine multiple uncertainties across multiple development artefacts, and how do they relate to the system consistency rules?
4. How to reason about uncertainty and inconsistency in order to (i) prioritize the inconsistencies to resolve, and (ii) help engineers in refining both their understanding and their models of the system under development?

Expressing uncertainty as doubts using subjective logic

We propose to express uncertainty as a degree of doubt that a user has about a given aspect of the system or about the model elements that are used to represent the system and guide its development and evolution. Doubts can refer to the occurrence of an element in a given design or configuration, its availability, or its required features or properties (e.g., version or variant). Engineers can also express doubts about the contents of a design model or of a container-type model element.

Subjective logic is a formalism very well suited for expressing user opinions in terms of degrees of belief (i.e., the degree of belief that a statement is true), disbelief (degree of belief that it is false), and uncertainty (i.e., the degree of uncommitted belief), such that belief+disbelief+uncertainty = 1. Opinions in subjective logic have been traditionally used to express degrees of confidence or trust. In this work, we use opinions to represent doubts. Unfortunately, expressing three nuanced values (belief, disbelief and uncertainty) thoughtfully for each uncertain element is quite challenging in practice for software engineers. Therefore, we propose a lightweight notation, trading intricacy for practicality.

Symbol	Meaning	Subjective logic opinion (belief, disbelief, uncertainty)
!	Complete certainty	(1.0, 0.0, 0.0)
~	Somewhat uncertain	(0.95, 0.025, 0.025)
?	Uncertain	(0.8, 0.1, 0.1)
??	Very uncertain	(0.5, 0.25, 0.25)

 

The figure below shows an example of an informal diagram representing the reference architecture of a product family with annotated doubts. Doubt expressions start with the initials of an engineer expressing the doubt, followed by the doubt symbol (!, ~, ?, or ??) and its type (Occurrence, Availability, Properties or Contents), the text provides a rationale for the doubt.

In the paper we show how this notation can be used with different types of models, ranging from very informal graphical ones such as the one shown in Figure 1, to more rigorous ones either graphical (SysML) or textual (C++ code), and written at different levels of abstraction.

Combining doubts

The use of subjective logic enables the definition and implementation of very interesting operations, such as the combination of doubts by multiple engineers or their propagation through the connected development artefacts, with the goal of obtaining a single value of doubt about each element.

Propagation of doubts relies on the relationships between various artefacts, and doubt combination is achieved by using the fusion operators defined in Subjective logic. In this way, all elements directly or indirectly affected by a kind of belief uncertainty get annotated with their corresponding doubts.

Given a set of models with annotated doubts and a set of consistency rules that specify how two or more elements from different must be related for the models to be consistent, our implementation produces a list of the inconsistencies detected, priotirized by their degree of doubt (!, none,~, ?, or ??). The more doubt, the lower the priority of resolving the inconsistency.

Using doubts to prioritise and discover inconsistencies

At the core of our approach lies the assumption that inconsistencies including elements with a high degree of doubt are of less importance to resolve than those inconsistencies with a high degree of certainty. Elements with a high degree of doubt are likely to be changed in the near future. In contrast, elements with a high degree of certainty are assumed to be correct and, therefore, are not expected to change. Any inconsistency involving these highly certain elements contradicts that assumption and implies the need for a change.

Based on this principle, inconsistencies can be priorized according to their degrees of certainty and uncertainty. For example, inconsistencies affecting very certain elements (i.e., annotated with the “!” symbol) should be resolved first. The second order of priority includes those inconsistencies without annotated doubts. The remaining inconsistencies involve elements with some degree of doubt, since they are less certain and might later change. They can be prioritized from least to most doubt: ~, ?, ??.

Results and final remarks

We have implemented our ideas and the initial results are encouraging in the two industrial settings that we have used to validate the proposal. In general, we envision that our approach can be very useful in model-based software environments where development artifacts must be kept consistent, modelers work in short development cycles, and mechanisms must be in place to identify, prioritize and resolve those inconsistencies with higher priority after each sprint.Those of lower priority will probably resolve themselves in the course of refining the models and clarifying existing uncertainties, or become mature enough to be resolved in subsequent sprints.

We are currently working on further developing this approach. The work ahead of us includes evaluating its applicability, scalability and usability in more industrial settings. We are also investigating further usages of our proposal. If you are interested in our work and would like to participate in its evaluation, or in its application in your company, please do not hesitate to contact us!