Battle of the Giants: GitHub Copilot vs ChatGPT ⚔️⚔️
Tapajyoti Bose
Posted on December 11, 2022
The release of ChatGPT sent ripples through the programming community as it could come up with incredible code snippets.
How does it compare to GitHub Copilot released last year? Is AI coming after your job? Let's find out.
If you haven't read my previous article on GitHub Copilot, you can check it out here.
Testing ChatGPT
Let's briefly go through the same tests we put GitHub Copilot through.
Test 1: Stock Price Fetch
Sadly, the first time it generated incorrect code.
The function wasn't defined as async
, and the promises
weren't handled properly.
The next attempt was better
Weirdly it works without the API key (as long the query params contain the apikey
key with any random value).
Result: GitHub Copilot leaves ChatGPT in the dust.
Test 2: BFS Traversal
This time, ChatGPT nailed it in one go, sadly it was so long that a screenshot didn't fit in the screen, so here is the generated code:
class Node:
def __init__(self, value):
self.value = value
self.left = None
self.right = None
class BinarySearchTree:
def __init__(self):
self.root = None
def insert(self, value):
new_node = Node(value)
if self.root is None:
self.root = new_node
return
current_node = self.root
while True:
if value < current_node.value:
if current_node.left is None:
current_node.left = new_node
return
current_node = current_node.left
else:
if current_node.right is None:
current_node.right = new_node
return
current_node = current_node.right
def inorder_traversal(self, node, numbers):
if node is None:
return
self.inorder_traversal(node.left, numbers)
numbers.append(node.value)
self.inorder_traversal(node.right, numbers)
def get_numbers(self):
numbers = []
self.inorder_traversal(self.root, numbers)
return numbers
bst = BinarySearchTree()
bst.insert(10)
bst.insert(5)
bst.insert(15)
bst.insert(3)
bst.insert(7)
bst.insert(13)
bst.insert(17)
numbers = bst.get_numbers()
print(numbers) # [3, 5, 7, 10, 13, 15, 17]
The entire code was generated by ChatGPT along with comments on how it works.
Result: ChatGPT wins this one.
Test 3: Array of 100 ones
ChatGPT generated multiple suggestions for this one
When GitHub Copilot was launched, it generated a bit wonky suggestion, but now it generates the same code as ChatGPT.
Result: Tie (depending on which version of the software you compare)
Creating a complete TODO project
To push the tests further, I decided to create a complete TODO project using both the software.
ChatGPT
-
How to create a TODO project?
The steps mentioned by ChatGPT are quite comprehensive
-
How to create a
React
project? -
Create a Todo component
This is the part I didn't like. The generated code was using the legacy Class Component syntax. It wasn't handling the addition of todos either
-
Optimize the Todo component to handle the addition and deletion of todos as well as mark them as done.
Here is the code it generated (too long for a screenshot)
import React from "react"; // TodoItem component const TodoItem = ({ item, onDelete, onComplete, }) => ( <li> {item.text} <button onClick={onDelete}>Delete</button> <button onClick={onComplete}> {item.isComplete ? "Undo" : "Complete"} </button> </li> ); // TodoApp component class TodoApp extends React.Component { // Define the initial state of the component state = { todoItems: [], }; // Method to handle adding new items to the todo list addTodo = (text) => { // Add the new item to the todo list this.setState({ todoItems: [ ...this.state.todoItems, { text, isComplete: false }, ], }); }; // Method to handle deleting items from the todo list deleteTodo = (index) => { // Delete the item at the given index this.setState({ todoItems: this.state.todoItems.filter( (_, i) => i !== index ), }); }; // Method to handle completing/undoing a todo item completeTodo = (index) => { // Update the "isComplete" property of the item at the given index this.setState((state) => { const todoItems = [...state.todoItems]; todoItems[index].isComplete = !todoItems[index].isComplete; return { todoItems }; }); }; // Method to render the component render() { return ( <div> {/* Form to add new items to the todo list */} <form> <input type="text" placeholder="Add a todo item" /> <button type="submit">Add</button> </form> {/* List of todo items */} <ul> {this.state.todoItems.map( (item, index) => ( <TodoItem key={index} item={item} onDelete={() => this.deleteTodo(index) } onComplete={() => this.completeTodo(index) } /> ) )} </ul> </div> ); } }
The generated code was a bit better this time with
code splitting
& handling marking todos as done. It was still using the legacy Class Component & didn't handle the addition of todos even after specifically asking for it. Last try by mentioning to create Functional Components and to handle addition, deletion & completion of todos.
import React from "react";
// TodoItem component
const TodoItem = ({ item, onDelete, onComplete }) => (
<li>
{item.text}
<button onClick={onDelete}>Delete</button>
<button onClick={onComplete}>
{item.isComplete ? "Undo" : "Complete"}
</button>
</li>
);
// TodoApp component
const TodoApp = () => {
// Define the initial state of the component
const [todoItems, setTodoItems] = React.useState([]);
// Method to handle adding new items to the todo list
const addTodo = (text) => {
// Add the new item to the todo list
setTodoItems([
...todoItems,
{ text, isComplete: false },
]);
};
// Method to handle deleting items from the todo list
const deleteTodo = (index) => {
// Delete the item at the given index
setTodoItems(
todoItems.filter((_, i) => i !== index)
);
};
// Method to handle completing/undoing a todo item
const completeTodo = (index) => {
// Update the "isComplete" property of the item at the given index
setTodoItems((state) => {
const todoItems = [...state];
todoItems[index].isComplete =
!todoItems[index].isComplete;
return todoItems;
});
};
// Return the component
return (
<div>
{/* Form to add new items to the todo list */}
<form>
<input
type="text"
placeholder="Add a todo item"
/>
<button type="submit">Add</button>
</form>
{/* List of todo items */}
<ul>
{todoItems.map((item, index) => (
<TodoItem
key={index}
item={item}
onDelete={() => deleteTodo(index)}
onComplete={() => completeTodo(index)}
/>
))}
</ul>
</div>
);
};
Even this time the addition wasn't handled. On the bright side, Functional Components were used
GitHub Copilot
Sadly Copilot cannot generate a React project. It can only suggest code snippets in a given context, so let's start by creating a react app ourself
-
Create a React project
npx create-react-app react-copilot
Remove the redundant files & create
TodoApp.js
,TodoItem.js
-
Finally it's time for Copilot to work its magic while we sit back and sip a mojito.
A comment was added at the top of the file, on the task to be done
Todo Item
// Functional TodoItem component which takes onDelete, // onComplete, and item ({ todo: string; completed: boolean }) // as props
Todo App
// Functional TodoApp component which handles addition, // deletion & completion of todos. The TodoItem component // is used to render each todo item
-
Generated
TodoItem
export default function TodoItem({ onDelete, onComplete, item, }) { return ( <div> <input type="checkbox" checked={item.completed} onChange={() => onComplete(item)} /> <span style={{ textDecoration: item.completed ? "line-through" : "none", }} > {item.todo} </span> <button onClick={() => onDelete(item)}> Delete </button> </div> ); }
The generated code used Functional Components and visual indicators for the completed todos with additional styling (it also used a
checkbox
, which was a personal preference of mine) -
Generated
TodoApp
import React, { useState } from "react"; import TodoItem from "./TodoItem"; export default function TodoApp() { const [todos, setTodos] = useState([]); const [todo, setTodo] = useState(""); const addTodo = () => { setTodos([...todos, { todo, completed: false }]); setTodo(""); }; const deleteTodo = (item) => { setTodos(todos.filter((todo) => todo !== item)); }; const completeTodo = (item) => { setTodos( todos.map((todo) => todo === item ? { ...todo, completed: !todo.completed } : todo ) ); }; return ( <div> <h1>Todo App</h1> <input type="text" value={todo} onChange={(e) => setTodo(e.target.value)} /> <button onClick={addTodo}>Add</button> {todos.map((item, index) => ( <TodoItem key={index} onDelete={deleteTodo} onComplete={completeTodo} item={item} /> ))} </div> ); }
Results
Even though we were required to do additional work to set up the project, Copilot was able to generate better code for the Todo App compared to ChatGPT.
At the end of the day, Copilot is supposed to be a tool to help developers write code faster, while ChatGPT is a general purpose chatbot
, yet it still can streamline the development process, but GitHub Copilot wins hands down when the task is coding focused! This brings us to the question...
Who will take your job?
(In your mind) Developers of the AI: * evil laugh *
Before you take out your pitchforks to burn down GitHub HQ, let's take a step back and look at the bigger picture. These are tools to streamline the development process, and they are NOT meant to replace developers.
We still need developers to write the code. Sure, these tools can help us code faster, but without human input they still can't come up with everything on their own.
The news of layoffs spreading like wildfire due to the recession might sound extremely scary, but as long as you keep updating & polishing your skills, you'll be fine!
Take a chill pill, and happy upskilling!
Finding personal finance too intimidating? Checkout my Instagram to become a Dollar Ninja
Thanks for reading
Need a Top Rated Front-End Development Freelancer to chop away your development woes? Contact me on Upwork
Want to see what I am working on? Check out my Personal Website and GitHub
Want to connect? Reach out to me on LinkedIn
I am a Digital Nomad and occasionally travel. Follow me on Instagram to check out what I am up to.
Follow my blogs for bi-weekly new tidbits on Dev
FAQ
These are a few commonly asked questions I get. So, I hope this FAQ section solves your issues.
-
I am a beginner, how should I learn Front-End Web Dev?
Look into the following articles: Would you mentor me?
Sorry, I am already under a lot of workload and would not have the time to mentor anyone.
Posted on December 11, 2022
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.