The modeling book
Site Sponsor
MDD pays off in the mid-term: an industrial experiment
Glass’s Law (discovered here ) states that any new tool or technology makes you INITIALLY less productive.
MDD is not different so nobody should expect to see the benefits of applying MDD in his first project. But the question (in fact, one of the most popular discussion points in the portal, see, for instance, this and this previous posts) is whether MDD pays off at some point in time.
In this sense, this paper: “Generating blogs out of product catalogues: an MDE approach” by Oscar Díaz AND Felipe M. Villoria from the ONEKIN group presents an interesting experiment that sheds some light on this issue.
They compare the cost (in labour hours) of developing a “catalogue blog” application (a specific kind of blog aimed at being “a conduit for customer feedback as well as fostering community construction around products”) both through manual coding or code generation.
As shown in the following figure, the break-even point is at around 50 projects since their empirical study showed that the cost of manually creating and configuring the blog application was about 2h while the upfront investment to define the MDD process to generate them automatically (starting from an initial blog model where the user can indicate the desired characteristics for the blog) was around 80 hours plus a very small increment for each generated blog.
Of course, the break-even point depends a lot on the complexity of the application under consideration. The more complex the application the more we can gain if we automate its development process. I’d say this experiment is almost a worst case scenario for MDD since the manual coding alternative is really quick. In general I expect a better ROI for MDD. Anyway, a nice experiment that I think it is worth to share with you. Opinions?
If you enjoyed this post you can subscribe to this Software Modeling blog , to the portal’s mailing list or follow me on twitter. And if you really liked it help me pass it on to others by sharing the post USING the links below. Don’t forget to check our consulting and code-generation services either!!
i think the CASE oversimplifies the problem.
it assumes the requirements ARE fixed upfront.
what if requirements constantly change? how would the “cost curve” look LIKE?
The MDD method may be very costless IN terms OF new projects WITH exact requirements. What if requirements change often,even if slightly…here AND there etc. that still need effort IN manual way but most programmers ARE used TO do anyway: they just patch the code, it’s right there! With MDD, you have several possible places to make change: the model, “annotate/tweak/extend” the model, the templates, post processor, or just hand-patch it after generation? Not to mention the needed skillset to make these changes, the still rare developers who are used to MDD means MDD coders are expensive from HR perspective.
note that I am far from against MDD. i just am very interested to know a more practical, “close to real world mess” kind of project where MDD shows its paws. We all know world of software is far from perfect smooth landscape, I really want to know how MDD can (and SHOULD!) work in these rough “offroad” areas.
Hi,
nice experience and result indeed.
Let me also point to one of our first industrial experiences where we applied WebML and WebRatio to an industrial case inside Acer worldwide.
A paper reports our results, which are aligned with the ones you mention:
http://dbgroup.como.polimi.it/brambilla/acerFPcase
The core thing is that MDD scales much better than traditional development, especially in maintenance.
Other brief case studies are available here:
http://www.webratio.com/portal/contentPage/en/Case Studies
Hi Hendy,
as I mention in my comment, I think that evolution and maintenance is actually one of the strengths of MDD approaches! (see above)
Thanks FOR the resources you linked.
I will study those.
Glass’s law is just the observation that with any new tool, there’s a period OF learning. The 1999 paper (http://www.cparity.com/projects/AcmClassification/samples/293481.pdf) FOUND productivity increases WITH new technologies TO be FROM -90% TO 570%. Unsurprisingly, SOME OF the biggest productivity increases came WHEN changing the LANGUAGE rather than changing the tool. That’s confirmed by many results for Domain-Specific Languages and Domain-Specific Modeling, e.g. http://www.metacase.com/blogs/stevek/blogView?showComments=true&entry=3446309083.
The figures you quote don’t seem TO MATCH the graph: FROM the graph it looks LIKE 80 hours (NOT 100) were spent setting up the MDD solution, AND FROM there it took 12 hours TO build 50 blogs. The total OF 92 hours was equal TO the TIME TO build 50 blogs manually. That gives a productivity increase factor OF just over 750% – RIGHT IN the middle OF the range OF our own experience OF 5-10x.
Admittedly, FOR such a small DOMAIN I would expect setting up the MDD solution WITH MetaEdit+ would ONLY take a few hours, NOT 80, but I don’t know what language workbench the authors were using. For those who are interested in reducing the 80 hours to setup an MDD solution, as well as reducing the time per application, take a look at the Language Workbench Competition, http://www.languageworkbenches.net/submissions.html.
When measuring these things, you have to look not just at the number of applications produced, but also the time budgeted. If you only need to build 10 blogs, there’s obviously NO point IN this example. However, if you ONLY have one man-MONTH TO spend (a pretty small project), you can build either 87 blogs manually OR 333 WITH MDD (including setting up the MDD solution).
In the REAL world, there ARE very few finished applications that would take less than 2 hours TO build manually. AT least WITH DOMAIN-Specific Modeling, we expect payback within the FIRST man-MONTH. Measured IN the SIZE OF the REAL applications IN around 100 cases we’ve been involved in, payback is generally around 3 applications. An end-user product is often composed of many applications (e.g. a phone has a phone book, calendar, text messaging, calling, call waiting etc.).
First, thanks for the correction on the numbers. I updated the text to clarify that.
Secondly, it’s great TO see that IN your experience payback IS generally around 3 applications. Much MORE convincing FOR new users!
Fully agree WITH Marco: addressing changes IN requirements IS WHERE MDD shines.
> it assumes the requirements ARE fixed upfront.
This IS NOT an assumption OF MDD. Much TO the contrary. MDD deals WITH DOMAIN AND technical requirements IN very different ways: DOMAIN reqs ARE dealt WITH by the model, technical reqs ARE dealt WITH via generation (transformations/templates). Changes IN a TRUE MDD application, if affecting generated code, should be made AT the model LEVEL (DOMAIN requirements changed), OR AT the transformation LEVEL (technical requirements changed).
The issues you raised ARE indeed REAL, but that IS typical OF code generation IN a non-MDD way, AND MDD cannot be blamed FOR that. MDD implies the models ARE the REAL thing, AND generated code IS object code. Changing object code IS OF course silly.
[...] As we discussed in this previous post, MDE pays off (in the mid-term) [...]