Recently, DeepMind announced a breakthrough in AI for code generation. AlphaCode, which uses large-scale transformer models to generate code, was pitted against humans in a series of programming contests run by Codeforces. Contestants were asked to design novel solutions to abstract problems, which required a blend of natural language understanding, formal logic, creative algorithm design, and coding. In an unprecedented outcome for AI systems, AlphaCode’s performance was in line with the median human competitor.
This is the latest in a flurry of developments in the field – and has captured headlines just as OpenAI’s Codex launch did last summer.
What are the implications? Will we start to see real-world computer programs being synthesised automatically – and what could this mean for the global software economy? And more presently, how do we evaluate the startups that are at the forefront of building AI-for-code products?
The opportunity: from workflow efficiency to full program synthesis
In recent years, the ML community has made great progress in understanding and generating natural language. Programming is one of the more promising applications for large language models and deep learning techniques. As demand for talent continues to outstrip supply, there is an opportunity to build tools that make developers more effective, or give non-developers tools to help them create software – hence the rise of no-code / low-code tooling in recent years.
As natural language now interacts with code, it is easy to imagine highly valuable applications emerging. A code-to-documentation tool could make it easier for collaborators to understand what a piece of code does. A highly effective general-purpose transpiler could help us shift entire codebases out of obsolete or deprecated languages – perhaps alleviating the bottleneck of COBOL engineers to maintain legacy infrastructure, for example. And if an IDE can autocomplete snippets of code from natural language prompts, shaving seconds from a developer’s workflow at scale, this might translate into highly valuable productivity improvements. One might perhaps build a virtual coding tutor, answering students’ natural language queries with working code. Each of these could lead to a multi-billion-dollar revenue opportunity if adopted at global scale.
OpenAI’s Codex has been a notable pioneer in improving the developer workflow. Codex is a descendant of GPT-3, trained on natural language as well as publicly available code sources like GitHub. Launched last summer, it helps users translate natural language to functioning code in over a dozen languages. Applications like GitHub’s Copilot, as well as a crop of early-stage startups, are using Codex to offer ‘virtual pair programmers’, query generators, docstring writers, and other tools to save developer time and minimise errors. These have been met with acclaim, albeit with some reservationsabout code quality, security, bias, and IP issues.
Program synthesis is a much more substantial challenge, but with potentially further-reaching implications. DeepMind is now demonstrating that AI systems can generate entirely novel programs based on problems described in natural language – even when they cannot duplicate solutions seen before, and cannot try out every potentially viable algorithm. Yet we are very much in the early phases of program synthesis, and there will long remain a need for humans to apply context and creativity, as they break down abstract business problems into machine-readable ones. We remain some distance from code generation systems that truly pass the Turing Test.
Commercialising AI for code: key questions for startups
There is a rising groundswell of startups operating at the interface between natural language and code. As we evaluate them, our three biggest questions are:
In conclusion – we are in the very early stages of the adoption curve, and there are many open questions for early-stage founders to navigate today. An enthusiastic futurist might foresee a scenario in which AI replaces human programmers altogether. We are far from this endgame – but in the short term, it’s clear that there are new ways to give ‘superpowers’ to developers and alleviate some of their capacity constraints. The founders who enable this could become a vital part of the way we build software over the coming decades.
If you’re one of them, we’d love to chat: please get in touch.
With thanks to the founders and experts who joined our ‘AI for Code’ roundtable session.