The modeling book
Site Sponsor
Modeling effectively : insights from empirical studies – Upcoming talk
Probably this title of this talk should be “the proof you all been waiting for”!
Next Thursday, March 31st. in Nantes, Michel Chaudron will give the talk “Modeling effectively : insights from empirical studies” as part of the Jeudi des Modèles series (you can register target="_blank"> here ).
- What is the state of UML modeling in practice?
- How can we assess the quality of UML models?
- How do engineers look at UML models?
- Do UML models actually help in creating better software?
Michel will review all these topics during the talk. As he says, “modeling is a common part of modern day software engineering practice. Little evidence is known about how models are made and how they help in producing better software. In this talk I will present highlights from the last decade of research that I have performed in the area of software modeling using UML.”
We all know that one of the main criticisms to modeling is the lack of empirical studies that show its supposed benefits in practice. Let´s see if Michel can settle this issue once and for all (ok, maybe that´s too ambitious, let´s say: “let´s see if Michel sheds some light on this issue”)
Don´t miss it! (if you believe in modeling, this talk will give you the tools to convince others, if you are a modeling skeptic then maybe this talk will help you to see the light 
)
If you enjoyed this post you can subscribe to this Software Modeling blog , to the portal’s mailing list , follow me ON twitter, participate IN the forums AND/OR CHECK the most popular posts . AND if you really liked it help me pass it ON TO others by sharing the post USING the links below
Since it sounds like Michel is going to say that UML modeling improves productivity, it would be interesting to hear his response to three studies, all of which show UML reducing productivity (time to completion increases by 15% – 48%). I review the results in my blog: Using UML takes 15% longer than just coding.
WJ Dzidek, E Arisholm, LC Briand: A Realistic Empirical Evaluation of the Costs and Benefits of UML in Software Maintenance, IEEE Transactions on Software Engineering, Vol 34 No 3, May/June 2008
Of course, modeling in general can either reduce or increase productivity – it all depends on how good a fit your modeling language, generators and tools are to your domain. With Domain-Specific Modeling, empirical studies consistently show 500%-1000% productivity increase: Domain-Specific Modeling: MDD that works lists 7 published studies.
Well I cannot speak for him (maybe he surprises us
) but after his presentation we will post the slides or video online.
Although we should maybe start by asking the authors of the paper you cite for his opinion on the subject. I´ve seen this paper used to both claim that UML improves productivity and claim that it worsens it. When discussing, we should also distinguish what we mean by productivity (of the initial release? including maintenance over a long period of time?)
I´m afraid everybody is playing with the numbers a little bit depending on his/her point of view and this does not really help to clarify the situation.
To me it looks very surprising that modeling with UML is so bad and instead modeling with DSLs is so good (specially because I believe that UML can behave as well as a DSL in many domains ) but I don´t have the background on empirical methods (nor the time) to carefully read all papers and see if the studies are rigorous enough, and if so, if they are representative enough of the different situations we may encounter.
In my opinion, it would help if we had more studies specially studies not coming from companies that sell UML or DSL tools to give more credibility to the results
One can use UML in many different ways.
I can see a wide range of results in terms of productivity depending on what “mode” you are using UML. Needless to say, I’d expect DSM-comparable gains WITH executable UML (UML AS programming LANGUAGE), AND much less promising gains (<30%) WITH PARTIAL code generation (UML AS blueprint). I don’t think the benefits of UML as sketch are direct enough to be measured IMO.
In my experience, UML has been typically applied with partial code generation from PSM models, which gives roughly 40% productivity improvement maximum. This is excluding the learning/adoption costs of new method/methodology. My industrial colleagues typically expect productivity losses during the first UML-powered development effort. They also expect that situation will gradually improve with next development. This all within the development phase.
Yes, technically and theoretically it is possible to have DSLs on basis of UML. In practice, there is one big problem: To put it in black and white, DSL and UML experts have two 2 different core values: customization and standardization. And while I have seen a lot of gray situations, a person with one core value can bent only so much towards the other. The same core value conundrum also affects tool vendors. I found that many UML tools could not allow agile development and evolution of good quality DSLs. In practice DSM/DSLs turn out to be more about supporting organization’s way OF working (inside-out), while MDA/UML IS MORE about tool introduction (outside-IN).
IMO standardization mindset AND inflexible UML tools ARE holding UML back.
I think what you mention about ” My industrial colleagues typically expect productivity losses during the first UML-powered development effort” is absolutely true but it´s true with any new technology that you introduce in an organization. The development team needs some time to learn how to use it properly
Yes, indeed, it is true for any change. However, UML is not only a new technology, but also a new way of working involving modeling. Hence UML introduction (or for that matter introduction of any modeling) will have steeper and longer learning curve. Such a curve that it may neutralize UML wins (up to 30%) in first development.
Steve says “Since it sounds like Michel is going to say that UML modeling improves productivity, … “. This is really jumping to conclusions.
There are only some thing that I will say at this point (as I do not want to give away the content of my presentations) is that I aim to be objective in reporting the results and that I am not dogmatic about UML.
I was part of the opposition of James Dzidek’s ph.d. thesis IN Oslo, so I know that WORK fairly well (actualy _I_ had TO present part OF it AT the defense). The authors OF the paper indeed see the results AS being IN favour OF USING UML.
There ARE certainly successes IN USING DSL’s. The scientific studies you report however all have been authored by the people that work for a company that also sell DSL-tooling. In scientific work in medicine, such papers would not get published. (Which is not to say that they could not be true). At the same time it would be fair to say that there is also a number of projects that tried to use DSL’s that failed, OR overshot budget AND TIME. This IS always an issue WHEN reporting ON CASE studies – esp. regarding new technologies: ONLY the positive results GET published.
I look forward TO a lively discussion ON Thu-afternoon
(how much TIME do we have?)
[...] paradigm to create software (well, even know we are still discussing about the benefits of modeling ). LF The focus is not modeling itself. We wanted to deliver software faster [...]
[...] Does software modeling pay? – Empirical studies in UML (presentation) The announcement of Michel Chaudron’s talk AT the Jeudi des Modèles conference series: Modeling effectively – insights FROM empirical studies raised SOME interesting discussions . [...]
You can find them here
[...] the question (in fact, one of the most popular discussion points in the portal, see, for instance, this and this previous posts) is whether MDD pays off at some point in [...]
[...] to the (true, few) scientific empirical studies on the effectiveness of software modeling (see Michel Chaudron’s work as an example) but now I have an even anser to give you: “Empirical Evidence of the Value of Version [...]