As all of you, I’ve also been playing with ChatGPT. I’ve tried it to generate code and models from simple prompts. With impressive but mixed results. ChatGPT will NOT definitely replace programmers. Even less modelers. But it still may revolutionize software development, especially, model-driven software development. In this post, I explain how.
Firsts experiments with ChatGPT
For instance, regarding code generation, I ask it to create a simple WordPress plugin (as this seemed indeed possible) to display my most popular slideshare presentations in any page thanks to a new WP shortcode. The result was a piece of useful but incomplete code with some small errors regarding the use of the slideshare API due to mixing up API versions. I was able to improve the results by chatting with ChatGPT. But of course, I could only do so because I have enough experience in building WP plugins to know what was missing.
When it comes to generating models, ChatGPT confessed it cannot draw diagrams but I was pleasantly surprised to see that ChatGPT knows the PlantUML syntax so we can use it to create textual models. You can even teach ChatGPT new syntaxes via examples. Again, the results were useful but far from perfect. If you ask ChatGPT to create a class diagram for a purchasing system you get a mostly correct PlantUML model (sometimes it makes some mistakes with the syntax of associations) that could be used as inspiration for the real model you’ll need to create. To me, it looks mostly useful as a way to double-check whether you’re missing something (e.g. you may see in the ChatGPT model an attribute you didn’t have and that, indeed, it should be part of your model). And, since every time you ask, you get an entirely different result, it could also be used to see variations of models in the domain that could potentially propose some interesting alternative design suggestions.
Will LLMs replace programmers? And modelers?
With ChatGPT and the growing popularity of all types of Large Language Models (LLM) trained to perform many different tasks, including all types of code-related ones such as writing code based on an initial prompt, many people started to wonder whether this was the end of programming as a profession.
Clearly, this is not the case. So far, LLMs excel only at generating boilerplate code while making mistakes at more complex tasks. And if you’re a programmer, I do hope your job involves more than writing repetitive boilerplate code!!!! As Kai-Fu Lee puts it, the more creativity a job requires, the more humans excel at it and the worse AI systems perform.
Moreover, any type of complex program generation would require a very precise specification of what needs to be created. And that’s our job, the job of the modeling experts. So we are even safer than programmers!
From models to prompts (or my belief that LLMs will replace code generators)
This idea of a component that excels at generating repetitive code from models sounds familiar, doesn’t it? In model-driven development, this is what we know as code generators: model to text transformations that apply a set of rules on an input model to generate code.
So, hear me out, my prediction is that we will soon see LLMs-based generators in low-code platforms. You’d still create the models but then those formal specifications and software models would be translated into a set of precise prompts for an underlying LLMs in charge of generating the actual code. Same as current rule-based and manually written code generators, there is so much you can generate and there will always be some manual coding involved as soon as you need something not completely typical. But again, this is already a limitation of current generators not a new one imposed by the use of LLMs.
There are obvious drawbacks to this idea. To begin with, the output will not be deterministic. I’m pretty sure that with some proper prompt engineering we would manage to get always similar results but this could be an issue. Another issue is that we can never be 100% sure of the quality of the generated code. “Traditional” code generators can be certified, LLM-based ones no (at least not now). So forget about using these LLM-based low-code platforms for critical applications (but I’d argue low-code platforms do NOT target these type of applications and focus on more consumer-oriented data-driven ones anyway).
But there are also obvious benefits. The biggest one would be the capacity to target any platform immediately and at no additional cost. Now you don’t need to create (and maintain!) a new generator for every platform you want to generate code for. With a single LLM you can target all of them. And as long as you keep updating the LLM version you’re using, you’ll be able to immediately target new platforms and new versions of such platforms to always generate up-to-date code. And once you embed an LLM into your platform, you could use the same LLM for other tasks beyond code-generation such as the validation of the models via a verbalization in natural language to discuss them with the client (something that right now takes a lot of effort and requires some non-trivial model-to-model transformations).
How do YOU see the role of LLMs in model-driven development?
Do you agree with my perspective? Do you have a more (or less) ambitious one? What’s your take on the future role of LLMs in model-driven engineering? What other tasks in software engineering (and, of course, model-driven engineering) will be reduced to prompt engineering? Looking forward to read your comments!