GitHub Copilot

GitHub Copilot is a cloud-based artificial intelligence tool developed by GitHub and OpenAI to assist users of Visual Studio Code, Visual Studio, Neovim, and JetBrains integrated development environments (IDEs) by autocompleting code. Currently available by subscription to individual developers, the tool was first announced by GitHub on 29 June 2021, and works best for users coding in Python, JavaScript, TypeScript, Ruby, and Go.

History

On June 29, 2021, GitHub announced GitHub Copilot for technical preview in the Visual Studio Code development environment. GitHub Copilot was released as a plugin on the JetBrains marketplace on October 29, 2021. October 27, 2021, GitHub released the GitHub Copilot Neovim plugin as a public repository. GitHub announced Copilot's availability for the Visual Studio 2022 IDE on March 29, 2022. On June 21, 2022, GitHub announced that Copilot was out of "technical preview", and is available as a subscription-based service for individual developers.

Features

When provided with a programming problem in natural language, Codex is capable of generating solution code. It is also able to describe input code in English and translate code between programming languages.

According to its website, GitHub Copilot includes assistive features for programmers, such as the conversion of code comments to runnable code and autocomplete for chunks of code, repetitive sections of code, and entire methods and/or functions. GitHub reports that Copilot’s autocomplete feature is accurate roughly half of the time; with some Python function header code, for example, Copilot correctly autocompleted the rest of the function body code 43% of the time on the first try and 57% of the time after ten attempts.

GitHub states that Copilot’s features allow programmers to navigate unfamiliar coding frameworks and languages by reducing the amount of time users spend reading documentation.

Implementation

GitHub Copilot is powered by the OpenAI Codex, an artificial intelligence model created by OpenAI which is an artificial intelligence research laboratory. The OpenAI Codex is a modified, production version of the Generative Pre-trained Transformer 3 (GPT-3), a language model using deep-learning to produce human-like text. The Codex model is additionally trained on gigabytes of source code in a dozen programming languages.

Copilot’s OpenAI Codex is trained on a selection of the English language, public GitHub repositories, and other publicly available source code. This includes a filtered dataset of 159 gigabytes of Python code sourced from 54 million public GitHub repositories.

Open AI’s GPT-3 is licensed exclusively to Microsoft, GitHub’s parent company.

Reception

Since Copilot's release, there have been concerns with its security and educational impact, as well as licensing controversy surrounding the code it produces.

Licensing controversy

Although most code output by Copilot can be classified as a transformative work, GitHub admits that a small proportion is copied verbatim, which has led to fears that the output code is insufficiently transformative to be classified as fair use and may infringe on the copyright of the original owner. This leaves Copilot on untested legal ground, although GitHub states that "training machine learning models on publicly available data is considered fair use across the machine learning community". The company has also stated that as of June 2022 only a few source codes are taken over completely or partially unchanged. Therefore as the software continues to learn, this figure is expected to drop. Also in June 2022, the Software Freedom Conservancy announced it would end all uses of GitHub in its own projects, accusing Copilot of ignoring code licenses used in training data. In November 2022, a class-action lawsuit was filed, challenging the legality of Copilot.

FSF white papers

On July 28 2021, the Free Software Foundation (FSF) published a funded call for white papers on philosophical and legal questions around Copilot. Donald Robertson, the Licensing and Compliance Manager of the FSF, stated that "Copilot raises many ..questions which require deeper examination." On February 24, 2022, the FSF announced they had received 22 papers on the subject and using an anonymous review process chose 5 papers to highlight.

Privacy concerns

As the service is cloud-based and requires continuous communication with the GitHub Copilot servers, it marks a fundamental shift in bringing the process of writing software online and thus into the hands of third parties, where every keystroke can be monitored.

Security concerns

A paper accepted for publication in the IEEE Symposium on Security and Privacy in 2022 assessed the security of code generated by Copilot for the MITRE’s top 25 code weakness enumerations (e.g., cross-site scripting, path traversal) across 89 different scenarios and 1,689 programs. This was done along the axes of diversity of weaknesses (its ability to respond to scenarios that may lead to various code weaknesses), diversity of prompts (its ability to respond to the same code weakness with subtle variation), and diversity of domains (its ability to generate register transfer level hardware specifications in Verilog). The study found that across these axes in multiple languages, 39.33% of top suggestions and 40.73% of total suggestions led to code vulnerabilities. Additionally, they found that small, non-semantic (i.e., comments) changes made to code could impact code safety.

Education concerns

A February 2022 paper released by the Association for Computing Machinery evaluates the impact Codex, the technology used by Github Copilot, may have on the education of novice programmers. The study utilizes assessment questions from an introductory programming class at The University of Auckland and compares Codex’s responses with student performance. Researchers found that Codex, on average, performed better than most students; however, its performance decreased on questions that limited what features could be used in the solution (e.g., conditionals, collections, and loops). Given this type of problem, “only two of odex’s10 solutions produced the correct output, but both ..violated heconstraint.” The paper concludes that Codex may be useful in providing a variety of solutions to learners, but may also lead to over-reliance and plagiarism.

See also

References

External links

*

{{Differentiable computing

GitHub
Applications of artificial intelligence