After six months with Copilot

I've been using Github Copilot daily: an AI-powered code autocompletion tool that seems to have a life of its own.

A real case searching for a currency conversion API.

GitHub Copilot is available as an extension for Visual Studio Code, Neovim, and JetBrains. It sends the code and context (programming language, entire project code, file names) to OpenAI's servers, where it's completed and sent back as a suggestion. Pressing Tab adds the suggestion to the code, while Esc discards it.

It's based on an AI called Codex, a descendant of GPT-3 tailored for programming languages and trained on all public domain repositories on Github. This includes code, comments, and documentation in every imaginable programming language.

Using it for the first time is exciting. Writing a function name and having it generate not only the code but also a valid link to a currency conversion API feels like something out of a sci-fi novel. However, after the initial hype, it's easy to see that the tool isn't perfect. It needs guidance to achieve the desired result, and you need to understand what it's writing to maintain control.

For example, at second 0:30 in the previous video, it tries to use a paid service, so I define a URL for the European Central Bank to give it a clue about which tools to use. Even then, the generated code isn't perfect, and I have to manually correct the regex to parse the data.

The death pyramids are one of its most typical errors.

Still, I decided to keep the plugin active for the last 5 months, and having tried it with many different languages: Python, JS, CSS, C++, PHP, SQL, Arduino, VBS, OpenScad, GLSL… it's been a radical change in my usual way of programming. One of those leaps that happen few times in life.

When I was young, my father bought Computer Hoy every month, and I read it from cover to cover. In one edition, there was a guide that said something like "learn to optimize your tasks with Visual Basic," and for some reason, I followed it. When I saw the magic of thinking up an idea, describing it to a computer, and seeing it come to life, I felt the first of those leaps.

Since then, I started programming in VBS with nothing more than that guide and some photocopies someone got me. My way of solving my needs was to look at those 20 photocopied pages over and over until I found an answer. When the sheets were worn and yellowed, the internet came home, and the next leap happened.

Programming became very different. Now my knowledge was not limited to a few sheets, but to the complete reference of Visual Basic or any other language I wanted to learn. And between online references, forums, and email threads, I kept programming until I started University, which was when I learned what an IDE with debugger was.

That leap was also significant. Now I could write in a text editor that colored my code, autocompleted some methods, allowed me to navigate the language reference with just Ctrl+Space, and see line by line how my application's state changed. And on top of that, the forum was Stack Overflow.

Today I still program like that, but I have another ace up my sleeve: the ability not to look at an API if I don't remember something, to let the IDE finish my code following the same style as my codebase, and in general to focus on the fun part of programming instead of struggling with the APIs of Matplotlib, Puppeteer, or PyTorch.

Programming image recognition with a webcam on a Raspberry with just kindly asking for it in a comment.

When Copilot came out, all opinions after a week of use were that it would replace programmers and that it was a magical tool, so I wanted to wait until I had tested it for a good time to understand its place in the programmer's toolbox. And my conclusion is that it's very, very useful if used with some philosophy.

It's easy to get carried away by the solutions Copilot proposes (after all there's no Oscar for best code). But as it's not perfect, without knowing what it's doing, it's easy to write dysfunctional code. Or worse: functional but with hard-to-detect vulnerabilities. Something similar to indiscriminate copy-pasting from Stack Overflow.

It's clear that there's a trend towards increasingly assisted programming. And if the future lies in tools like Copilot, we'll have to tackle challenges like avoiding the distraction of recommendations or promoting critical thinking in the face of suggestions that imply bad practices and outdated approaches; something similar to what happens with AIs trained on real datasets and their acquired biases.

For now, we can try it out to better understand its implications. If you've read this far and are interested, you'll be glad to know that telemetry is only used to see which solutions are accepted, but the code that is written does not feed the model. Being a Windows product, they're probably fishing to catch us later with a subscription service, so you can sign up for the beta before it's too late.

Every video in this post was converted to webm like this, avoiding looking 100 times to the FFMPEG reference.

Join the discussion

Get a mail