Dealing with non-functional requirements (NFRs) has posed a challenge onto software engineers for many years. Over the years, many methods and techniques have been proposed to improve their elicitation, documentation, and validation. Knowing more about the state of the practice on these topics may benefit both practitioners’ and researchers’ daily work. This paper (free pdf version) presents some of the findings of an empirical study based on 13 interviews with software architects. It addresses questions such as:
- who decides the NFRs,
- what types of NFRs matter to architects,
- how are NFRs documented,
- and how are NFRs validated.
The paper is co-authored with David Ameller, Claudia Ayala and Xavier Franch (from the GESSI group). The paper was originally published at the RE’2012 but we just learnt that we got the 10 years Most Influential Paper Award for this work so an (updated) version will be presented at this year’s Requirements Engineering conference (RE’2022).
With the excuse of the award, I thought it was worth to bring back the main contributions of the paper. And of course, I’ll be updating the post in a few weeks to elaborate on our perspectives on the topic 10 years after (stay tuned!). Keep reading for a summary of the work, or use the links above to read the full paper. And if you’re interested in the topic of NFRs, I recommend you to also take a look at our study on how NFRs are integrated in model-driven development projects.
Non-functional requirements (NFRs) express desired qualities of the system to be developed. They refer both to observable qualities such as system performance, availability and dependability, and also to internal characteristics concerning, e.g., maintainability and portability.
Over the years, a common claim made by software engineers is that it is not feasible to produce a software system that meets stakeholders’ needs without taking NFRs into account.
As such, research papers on software architecture often include sentences like:
- “[NFRs] play a critical role during system development, serving as selection criteria for choosing among myriads of alternative designs
- “the rationale behind each architecture decision is mostly about achieving certain NFRs”
- “quality attribute requirements strongly influence a system’s architecture”
but the problem is that there is very little evidence that this is really how software architects work in practice. Therefore, we set out to see whether we could confirm or not this assumption.
Research method: A survey for software architects
We carried out an exploratory study using a qualitative research approach. The study was conducted over our local network of software architects. We used semi-structured interviews for gathering information. Each interview took about one hour and was audio-taped and prepared for analysis. The respondents held different positions in the organizations and were in charge of architectural tasks in at least the project they based their answers on
Based on the analysis of the answers, we were able to draw some observations about the use and impact of NFRs in industrial practice, align them with the results of previous empirical studies, and discuss possible actions that could eventually help to improve the state of practice in the field.
Some of the observations that can be derived from the analysis of the interviews’ data were quite surprising. Let’s highlight some of them (again, refer to the paper for a full description and discussion of the findings):
- In fact, nobody had a permanent software architect role. Instead, they played the role of software architect (and others simultaneously) depending on the project. It was difficult to enumerate the specific architects’ tasks as these overlapped other roles’ tasks
- There is a lot of confusion regarding the exact meaning of several non-functional requirements. Architects did not share a common vocabulary for types of NFRs and showed some misunderstandings.
- The two most important types of technical NFRs for architects were performance and usability.
- Architects considered non-technical NFRs to be as relevant as technical NFRs. NFRs clearly influence the choice of the architecture.
- The software architect, and not the client, is the one that decides which NFRs are needed. The client may be the domain expert but it is not typically the one in charge of proposing NFRs. The architect himself adds them.
- NFRs were not often documented, and even when documented, the documentation was not always precise and usually become desynchronized.
- System compliance with the NFRs is hardly ever verified. Architects do say the system complains with them but there is no validation of the claims.
- No tool support to manage the NFRs is used.