{"id":8467,"date":"2023-08-08T05:31:28","date_gmt":"2023-08-08T05:31:28","guid":{"rendered":"https:\/\/modeling-languages.com\/?p=8467"},"modified":"2023-08-08T05:31:28","modified_gmt":"2023-08-08T05:31:28","slug":"model-driven-prompt-engineering","status":"publish","type":"post","link":"https:\/\/modeling-languages.com\/model-driven-prompt-engineering\/","title":{"rendered":"Model-driven Prompt Engineering"},"content":{"rendered":"

Generative artificial intelligence (AI) systems are capable of synthesizing complex content such as text, source code or images according to the instructions described in a natural language prompt. The quality of the output depends on crafting a suitable prompt<\/strong>. This has given rise to prompt engineering, the process of designing natural language prompts to best take advantage of the capabilities of generative AI systems<\/strong>.<\/p>\n

Through experimentation, the creative and research communities have created guidelines and strategies for creating good prompts. However, even for the same task, these best practices vary depending on the particular system receiving the prompt.<\/strong> Moreover, some systems offer additional features using a custom platform-specific syntax, e.g., assigning a degree of relevance to specific concepts within the prompt.<\/p>\n

We propose applying model-driven engineering to support the prompt engineering process<\/strong>. Using a domain-specific language (DSL), we define platform-independent prompts<\/strong> that can later be adapted to provide good quality outputs in a target AI system. The DSL also facilitates managing prompts by providing mechanisms for prompt versioning and prompt chaining. Tool support is available<\/a> thanks to a Langium<\/a>-based Visual Studio Code plugin.<\/p>\n

This work will be presented at the upcoming Models’23.<\/a> Meanwhile, you can read the full paper<\/a> or the extended summary below.<\/p>\n\n

Challenges of Prompt Engineering<\/h2>\n

A challenge of prompt engineering is the fact that it is essen-tially a platform-specific process. Prompts that perform well in a particular system may under-perform in other systems or even in different versions of the same system. For instance, the release of version 2 of the Stable Diffusion open-source text-to-image system caused user complaints that prompts previously considered as \u201cgood\u201d were producing worse results. Two changes motivating the performance gap were different training datasets and the replacement of a component of the AI system (the encoder). These types of changes are expected each time a new version is released.<\/p>\n

Moreover, some advanced aspects of prompting such as negative information, assigning weights to prompt fragments or setting platform-specific flags have a different degree of support and use a different syntax in each generative AI system.<\/p>\n

Benefits of Model-Driven Engineering to Prompt Engineering<\/h2>\n

To address these challenges, we propose to apply model-driven engineering (MDE) principles and techniques to prompt engineering. Our goals are:<\/p>\n

    \n
  1. Facilitating the definition of good prompts for different generative AI systems<\/strong>;<\/li>\n
  2. Facilitating the migration of a prompt<\/strong> from one platform to another without losing effectiveness; and<\/li>\n
  3. Enabling knowledge management of prompt-related tasks<\/strong> (such as documentation, traceability or versioning). In particular, we offer the following contributions:<\/li>\n<\/ol>\n

    Beyond these core benefits, we believe MDE can also bring additional benefits such as:<\/p>\n