Turing test for code-generation tools

Tweet about this on TwitterShare on FacebookBuffer this pageShare on RedditShare on LinkedInShare on Google+Email this to someone

The other day I saw this freelance job consisting in generating MySQL DDL scripts from UML class diagrams for around 55$ (average bid).

Immediately I thought that this would be an easy way to make some money if a freelance had access to a code-generation tool able to generate MySQL scripts from UML diagrams automatically (e.g. my own UMLtoDB service ๐Ÿ™‚ )

A good question is then, if this solution would be accepted. Imagine that you are the “client” asking for this job. Would you trust enough code-generation tools to accept (and pay) for the job if the tool and not a human has written the code? or would you feel cheated? Do you believe that automatically generated code can have the same quality of the programmer-generated one? At least for this scenario (UML to DB)?

To solve all these issues Iยดd like to propose a turing test for code-generation tools . Tools passing the test would have proved the quality of the code they generate is comparable to that of humans and could be used instead of them. Similarly to the classic turing test for AI , the turing test for code-generation tools reads as follows:

A human judge examines the code generated by one programmer and one code-generation tool for the same formal specification. If the judge cannot reliably tell the tool from the human, the tool is said to have passed the test

I know my definition may not be very precise but I’ve the feeling that we will make a huge step towards the massive adoption of MDD tools once tools can pass this test. Note that, passing the test does not imply the tool can generate high quality code, just that tool generates code comparable to the kind of code produced by humans. I suspect that sometimes, adoption can be more linked to psychological aspects (kind of feeling the code “familiar”) than quality ones.

This looks like a suitable target for our cognification of model-driven engineering initiative!

Tweet about this on TwitterShare on FacebookBuffer this pageShare on RedditShare on LinkedInShare on Google+Email this to someone
Comments
  1. Anonymous
  2. jordi
  3. Anonymous
  4. jordi
  5. Angelo
  6. Anonymous

Reply

Your email address will not be published. Required fields are marked *