Battle of the Giants: GitHub Copilot vs ChatGPT ⚔️⚔️

ruppysuppy

Tapajyoti Bose

Posted on December 11, 2022

Battle of the Giants: GitHub Copilot vs ChatGPT ⚔️⚔️

The release of ChatGPT sent ripples through the programming community as it could come up with incredible code snippets.

How does it compare to GitHub Copilot released last year? Is AI coming after your job? Let's find out.

If you haven't read my previous article on GitHub Copilot, you can check it out here.

Testing ChatGPT

Let's briefly go through the same tests we put GitHub Copilot through.

Test 1: Stock Price Fetch

Sadly, the first time it generated incorrect code.

stock-price-fail

The function wasn't defined as async, and the promises weren't handled properly.

The next attempt was better

stock-price-fetch

Weirdly it works without the API key (as long the query params contain the apikey key with any random value).

Result: GitHub Copilot leaves ChatGPT in the dust.

Test 2: BFS Traversal

This time, ChatGPT nailed it in one go, sadly it was so long that a screenshot didn't fit in the screen, so here is the generated code:

class Node:
  def __init__(self, value):
    self.value = value
    self.left = None
    self.right = None

class BinarySearchTree:
  def __init__(self):
    self.root = None

  def insert(self, value):
    new_node = Node(value)

    if self.root is None:
      self.root = new_node
      return

    current_node = self.root
    while True:
      if value < current_node.value:
        if current_node.left is None:
          current_node.left = new_node
          return
        current_node = current_node.left
      else:
        if current_node.right is None:
          current_node.right = new_node
          return
        current_node = current_node.right

  def inorder_traversal(self, node, numbers):
    if node is None:
      return

    self.inorder_traversal(node.left, numbers)
    numbers.append(node.value)
    self.inorder_traversal(node.right, numbers)

  def get_numbers(self):
    numbers = []
    self.inorder_traversal(self.root, numbers)
    return numbers

bst = BinarySearchTree()
bst.insert(10)
bst.insert(5)
bst.insert(15)
bst.insert(3)
bst.insert(7)
bst.insert(13)
bst.insert(17)

numbers = bst.get_numbers()
print(numbers)  # [3, 5, 7, 10, 13, 15, 17]
Enter fullscreen mode Exit fullscreen mode

The entire code was generated by ChatGPT along with comments on how it works.

Result: ChatGPT wins this one.

Test 3: Array of 100 ones

ChatGPT generated multiple suggestions for this one

array-of-100-ones

When GitHub Copilot was launched, it generated a bit wonky suggestion, but now it generates the same code as ChatGPT.

Result: Tie (depending on which version of the software you compare)

Creating a complete TODO project

To push the tests further, I decided to create a complete TODO project using both the software.

ChatGPT

  1. How to create a TODO project?

    The steps mentioned by ChatGPT are quite comprehensive

    create-todo-project-gpt

  2. How to create a React project?

    create-react-app

  3. Create a Todo component

    todo-component

    This is the part I didn't like. The generated code was using the legacy Class Component syntax. It wasn't handling the addition of todos either

  4. Optimize the Todo component to handle the addition and deletion of todos as well as mark them as done.

    Here is the code it generated (too long for a screenshot)

    import React from "react";
    
    // TodoItem component
    const TodoItem = ({
     item,
     onDelete,
     onComplete,
    }) => (
     <li>
       {item.text}
       <button onClick={onDelete}>Delete</button>
       <button onClick={onComplete}>
         {item.isComplete ? "Undo" : "Complete"}
       </button>
     </li>
    );
    
    // TodoApp component
    class TodoApp extends React.Component {
     // Define the initial state of the component
     state = {
       todoItems: [],
     };
    
     // Method to handle adding new items to the todo list
     addTodo = (text) => {
       // Add the new item to the todo list
       this.setState({
         todoItems: [
           ...this.state.todoItems,
           { text, isComplete: false },
         ],
       });
     };
    
     // Method to handle deleting items from the todo list
     deleteTodo = (index) => {
       // Delete the item at the given index
       this.setState({
         todoItems: this.state.todoItems.filter(
           (_, i) => i !== index
         ),
       });
     };
    
     // Method to handle completing/undoing a todo item
     completeTodo = (index) => {
       // Update the "isComplete" property of the item at the given index
       this.setState((state) => {
         const todoItems = [...state.todoItems];
         todoItems[index].isComplete =
           !todoItems[index].isComplete;
         return { todoItems };
       });
     };
    
     // Method to render the component
     render() {
       return (
         <div>
           {/* Form to add new items to the todo list */}
           <form>
             <input
               type="text"
               placeholder="Add a todo item"
             />
             <button type="submit">Add</button>
           </form>
    
           {/* List of todo items */}
           <ul>
             {this.state.todoItems.map(
               (item, index) => (
                 <TodoItem
                   key={index}
                   item={item}
                   onDelete={() =>
                     this.deleteTodo(index)
                   }
                   onComplete={() =>
                     this.completeTodo(index)
                   }
                 />
               )
             )}
           </ul>
         </div>
       );
     }
    }
    

    The generated code was a bit better this time with code splitting & handling marking todos as done. It was still using the legacy Class Component & didn't handle the addition of todos even after specifically asking for it.

  5. Last try by mentioning to create Functional Components and to handle addition, deletion & completion of todos.

import React from "react";

// TodoItem component
const TodoItem = ({ item, onDelete, onComplete }) => (
  <li>
    {item.text}
    <button onClick={onDelete}>Delete</button>
    <button onClick={onComplete}>
      {item.isComplete ? "Undo" : "Complete"}
    </button>
  </li>
);

// TodoApp component
const TodoApp = () => {
  // Define the initial state of the component
  const [todoItems, setTodoItems] = React.useState([]);

  // Method to handle adding new items to the todo list
  const addTodo = (text) => {
    // Add the new item to the todo list
    setTodoItems([
      ...todoItems,
      { text, isComplete: false },
    ]);
  };

  // Method to handle deleting items from the todo list
  const deleteTodo = (index) => {
    // Delete the item at the given index
    setTodoItems(
      todoItems.filter((_, i) => i !== index)
    );
  };

  // Method to handle completing/undoing a todo item
  const completeTodo = (index) => {
    // Update the "isComplete" property of the item at the given index
    setTodoItems((state) => {
      const todoItems = [...state];
      todoItems[index].isComplete =
        !todoItems[index].isComplete;
      return todoItems;
    });
  };

  // Return the component
  return (
    <div>
      {/* Form to add new items to the todo list */}
      <form>
        <input
          type="text"
          placeholder="Add a todo item"
        />
        <button type="submit">Add</button>
      </form>

      {/* List of todo items */}
      <ul>
        {todoItems.map((item, index) => (
          <TodoItem
            key={index}
            item={item}
            onDelete={() => deleteTodo(index)}
            onComplete={() => completeTodo(index)}
          />
        ))}
      </ul>
    </div>
  );
};
Enter fullscreen mode Exit fullscreen mode

Even this time the addition wasn't handled. On the bright side, Functional Components were used

GitHub Copilot

Sadly Copilot cannot generate a React project. It can only suggest code snippets in a given context, so let's start by creating a react app ourself

  1. Create a React project

    npx create-react-app react-copilot
    
  2. Remove the redundant files & create TodoApp.js, TodoItem.js

  3. Finally it's time for Copilot to work its magic while we sit back and sip a mojito.

    A comment was added at the top of the file, on the task to be done

    Todo Item

    // Functional TodoItem component which takes onDelete,
    // onComplete, and item ({ todo: string; completed: boolean })
    // as props
    

    Todo App

    // Functional TodoApp component which handles addition,
    // deletion & completion of todos. The TodoItem component
    // is used to render each todo item
    
  4. Generated TodoItem

    export default function TodoItem({
     onDelete,
     onComplete,
     item,
    }) {
     return (
       <div>
         <input
           type="checkbox"
           checked={item.completed}
           onChange={() => onComplete(item)}
         />
         <span
           style={{
             textDecoration: item.completed
               ? "line-through"
               : "none",
           }}
         >
           {item.todo}
         </span>
         <button onClick={() => onDelete(item)}>
           Delete
         </button>
       </div>
     );
    }
    

    The generated code used Functional Components and visual indicators for the completed todos with additional styling (it also used a checkbox, which was a personal preference of mine)

  5. Generated TodoApp

    import React, { useState } from "react";
    import TodoItem from "./TodoItem";
    
    export default function TodoApp() {
     const [todos, setTodos] = useState([]);
     const [todo, setTodo] = useState("");
    
     const addTodo = () => {
       setTodos([...todos, { todo, completed: false }]);
       setTodo("");
     };
    
     const deleteTodo = (item) => {
       setTodos(todos.filter((todo) => todo !== item));
     };
    
     const completeTodo = (item) => {
       setTodos(
         todos.map((todo) =>
           todo === item
             ? { ...todo, completed: !todo.completed }
             : todo
         )
       );
     };
    
     return (
       <div>
         <h1>Todo App</h1>
         <input
           type="text"
           value={todo}
           onChange={(e) => setTodo(e.target.value)}
         />
         <button onClick={addTodo}>Add</button>
         {todos.map((item, index) => (
           <TodoItem
             key={index}
             onDelete={deleteTodo}
             onComplete={completeTodo}
             item={item}
           />
         ))}
       </div>
     );
    }
    

Results

Even though we were required to do additional work to set up the project, Copilot was able to generate better code for the Todo App compared to ChatGPT.

At the end of the day, Copilot is supposed to be a tool to help developers write code faster, while ChatGPT is a general purpose chatbot, yet it still can streamline the development process, but GitHub Copilot wins hands down when the task is coding focused! This brings us to the question...

Who will take your job?

(In your mind) Developers of the AI: * evil laugh *

evil-laugh

Before you take out your pitchforks to burn down GitHub HQ, let's take a step back and look at the bigger picture. These are tools to streamline the development process, and they are NOT meant to replace developers.

We still need developers to write the code. Sure, these tools can help us code faster, but without human input they still can't come up with everything on their own.

The news of layoffs spreading like wildfire due to the recession might sound extremely scary, but as long as you keep updating & polishing your skills, you'll be fine!

Take a chill pill, and happy upskilling!

Finding personal finance too intimidating? Checkout my Instagram to become a Dollar Ninja

Thanks for reading

Need a Top Rated Front-End Development Freelancer to chop away your development woes? Contact me on Upwork

Want to see what I am working on? Check out my Personal Website and GitHub

Want to connect? Reach out to me on LinkedIn

I am a Digital Nomad and occasionally travel. Follow me on Instagram to check out what I am up to.

Follow my blogs for bi-weekly new tidbits on Dev

FAQ

These are a few commonly asked questions I get. So, I hope this FAQ section solves your issues.

  1. I am a beginner, how should I learn Front-End Web Dev?
    Look into the following articles:

    1. Front End Development Roadmap
    2. Front End Project Ideas
  2. Would you mentor me?

    Sorry, I am already under a lot of workload and would not have the time to mentor anyone.

💖 💪 🙅 🚩
ruppysuppy
Tapajyoti Bose

Posted on December 11, 2022

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related