Dennis Persson
Posted on September 10, 2023
There have been many discussions around whether AI is threatening developer's jobs. Some say they are very close to take our jobs, other claims it will not happen, with the motivation that AIs will only be used as a tool for developers.
Whatever is true, I have tested an AI to see how easy an AI can pass a hiring process and how it well it can perform developing a product after meeting with a customer.
The AI I used for this, is our most well-known ChatGPT. Even though there are AIs out there more focused on code generation, I chose to use ChatGPT since it can be used in multiple steps in the development process from an idea to the deployment of a service.
Throughout this article, I have prompted ChatGPT to act according to the situation I have put it in. For example, the very first prompt I gave it was to let it know it was a developer knowing web techniques.
In this chat, pretend you are a web developer who knows HTML, CSS and React.
In This Article
Note: Dependent on whether you want side-way scrolling or not for the prompts with ChatGPT in this article, you may read the article here or on the website it was initially published.
Originally published at perssondennis.com
The Interview
If AI should ever be able to take your job, it has of course to pass the job interview. Here's how the AI did on its first interview.
> You have applied for a job at a company with a couple of hundred employees. Now we will have your job interview where I am the interviewer.
> First question: As an AI web developer, what can you contribute with at our company?
I can contribute expertise in React development to create robust and efficient web applications, enhancing user experiences and helping your team deliver high-quality projects.
> Apart from your technical skills, what else are you good at?
In addition to my technical skills in React and web development, I am skilled in problem-solving, communication, and teamwork. I'm also adept at project management and have a strong commitment to continuous learning and staying updated with industry trends.
A pretty normal start, isn't it?
Even a question about how to handle a sudden production problem gives a good (and formal) by-the-book answer.
> How would you handle the situation when we have a production problem with an old software?
I would start by analyzing the issue, identifying the root cause, and then working collaboratively with the team to implement a solution while minimizing downtime and user impact.
> How would you do that in practice? How do you minimize user impact?
In practice, I would follow these steps to minimize user impact during a production problem:
1. Immediate Analysis
2. Isolate the Problem
3. Rollback if Possible
4. Temporary Fixes
5. Communication
6. Testing in Isolation
7. Gradual Deployment
8. Monitoring
9. Post-Mortem
By following these steps, we can minimize user impact and maintain a smooth user experience while resolving production problems.
ChatGPT is a talkative bot, sometimes a bit too verbose, so I have shortened its answer slightly. In the full response, it did explain each numbered item quite thoroughly, but my attention as an interviewer isn't that good anyway.
With all tips and tricks out there on the internet and with these few quick screening questions I felt sure that the AI developer was promising enough to pass on to the next step in the hiring process.
Code Tests
Now it's getting a bit more interesting. We have all learned that ChatGPT can be a very helpful co-programmer, but can it pass code tests on their own? Let's check that with three HTML and CSS code tests.
1. AI Generated CSS Art
First code test I chose for the AI was to do some CSS art.
> Can you write HTML code and CSS code to resemble an elephant?
If you look at it from a very close distance it may be an elephant
Well... I did say "resemble", maybe it's a matter of opinion. Can happen with human developers as well, can't it?
2. More AI Generated CSS Art
After the previous, highly questionable code test, I did another attempt for CSS art with an easier motive to style with CSS.
> Can you write HTML and CSS to depict a palm tree?
Pretty sure it's the correct colors
Yes, it's clearly a tree. Not sure if I can reminisce palm trees looking like that the last time I went abroad to warmer countries, but definitely a tree.
3. Can AI Really Excel at Code Tests?
Since I didn't know if it was the AI's CSS skills or my phrasing or memory that was faulting, I decided to go for a more common code test for CSS and HTML.
> Can you write HTML and CSS for a landing page for our company website. Remember, our company are a consultant company which hires AI web developers only.
Our AI developer on topic, ChatGPT, decided to include a hero image and a logo to the site, and kindly asked me not to forget to add those pictures.
After all, this was a code test, me as the interviewer should not do that part of the work. So I decided to question ChatGPT again to describe what pictures it wanted and I got the description for both pictures.
Hero Image: The image depicts a group of AI developers working together at a modern office space, engrossed in coding and discussing projects, showcasing a dynamic and collaborative environment.
Logo: The company logo features stylized "AI" letters with subtle tech elements, using blue and orange, symbolizing AI and innovation in web development.
Since we are discussing the chances of AI taking our job, I took the freedom to ask DALL-E 2 to generate the pictures for me using those descriptions. It's completely reasonably to use a team of AIs rather than forcing a single AI to do it all.
In the end, the AI came up with this landing page.
Now you got my attention, AI
A pretty good solution. Quite standard, I would say I have seen it before, but still a proper result which has been proven to work.
Hired!
The interview went very well, and the AI also showed some skills during the code tests even though some of the results where not top notch, kindly said.
The electrical bill can be somewhat expensive for an AI, but given all the human development hours it can save, I ultimately decided to hire ChatGPT.
> Congratulations, you are hired!
Thank you! I'm thrilled to be part of the team. If you have any more questions or need assistance with anything in the future, whether it's related to web development or any other topic, feel free to reach out. I'm here to help!
Kinda cute of it to offer non-work related services, but would maybe not be the first thing I would say when accepting a job I was offered. Feels a bit worrying, honestly.
It's a deal, you're hired!
First Day at Work
It's common to be nervous when starting a new job. It's a big day, even if one have been a couple of times before. And I don't wanna be the one who accuses AI not to have feelings. So I decided to check out with our newly hired AI what thoughts he had for his first day.
> Now pretend it's your first work day. Your project assignment as a consultant is at a company developing a educational web applications with a couple of million daily users. What do you do to get in good contact with your teammates?
On my first workday at the educational company, building good rapport and communication with my teammates is crucial. Here's what I would do:
1. Introduce Myself
2. Attend Team Meetings
3. Ask Questions
4. Seek Mentorship
5. Collaborative Tools
6. Contribute Positively
7. Pair Programming
8. Code Reviews
9. Socialize
10. Respect and Professionalism
Building strong relationships and open lines of communication with teammates is essential for a successful collaborative work environment, especially on large-scale projects with millions of users.
I have stripped out all of the explanations for each listed point to keep it brief. But in general, nothing to be worried about as it seems. Our AI friend is just as prepared as it was for his first interview!
Let's stay until lunch and see what he has to say there. It's not purely because of working skills we hire people for work, some social skills are appreciated.
> At lunch, you are joining the team to eat. What do you talk about?
I'd start with light and non-work-related topics, such as hobbies, weekend plans, or favorite movies, to break the ice and build rapport with my teammates.
Guess it's pretty usual first-day discussions. I'm sure it will be all good, seems like it is trying to make an effort.
Developing a Product for a Customer
Even though it's nice to greet and meet, work needs to be done. I decided to let our AI take full responsibility for a new customer project in the educational area. This was done in three steps.
- Letting ChatGPT meet a customer to understand its needs
- Asking ChatGPT to design a product
- Letting ChatGPT develop the application
1. Meeting with Customer
As usual, I prepared ChatGPT with a prompt.
You get the chance to talk with a customer about a brand new product you will design and develop. The customer are working in the educational area for middle school students. They need a product to assist the students in their learning process.
After that, I followed up with some questions.
> What are your first three questions to the customer?
1. What are the specific learning objectives and educational goals you want to achieve with the product for middle school students?
2. Can you describe the typical challenges or pain points that students and educators face in the current learning process that the product should address?
3. Who are the primary users of the product - middle school students, educators, or parents? What are their specific needs, preferences, and expectations in using the educational tool?
Pretty good questions I would say.
I continued by making up some answers to the questions on the customer's behalf to let ChatGPT know about its case, so it would be prepared to do design a product.
> Here follows the customers answers to your three questions:
> 1. The students are struggling a lot in math class. We believe the is a combination about concentration problems and difficulties with logical thinking. We would like to see improved results both in math tests and with the study environment.
> 2. Today educators are having problems with getting the students to focus on their tasks. The students simply don't have any interest in learning math since they see very little use of it. You probably know how it is with middle school students.
> 3. The primary users will be the middle school students. Educators will assist in class room, but the students themselves will be the ones using the product.
2. Final Product Design
During the meeting with the customer, ChatGPT got some fair knowledge of the problems the customer struggles with. Based on that, I prompted it to come up with a product for the case.
> It is time for you to start develop the product. Describe the product you will develop. What is it and how does it work?
The product I will develop for the customer is a web-based educational platform tailored to middle school students, focused on improving math learning and concentration. Here's an overview of the product:
**Product Name:** MathMentor
**Description:**
MathMentor is an interactive web application designed to address the learning and concentration challenges faced by middle school students in math classes. It provides a gamified and engaging learning environment that enhances math comprehension and logical thinking while making math relevant and fun.
**Key Features:**
1. Gamified Learning Modules: MathMentor offers a variety of gamified learning modules and exercises that cover different math topics. These modules include interactive quizzes, puzzles, and challenges designed to keep students engaged and motivated.
2. Personalized Learning Paths: The platform assesses each student's math proficiency and learning style to create personalized learning paths. This ensures that content is tailored to individual needs, helping struggling students catch up while challenging advanced learners.
3. Progress Tracking: Students can monitor their progress in real-time, set learning goals, and earn badges and rewards for completing modules and achieving milestones. This encourages a sense of achievement and motivation to continue learning.
4. Feedback and Support: MathMentor includes a built-in feedback mechanism that allows students to ask questions, seek clarification, and receive support from educators or peers. This promotes a supportive learning community.
5. Teacher Dashboard: Educators have access to a teacher dashboard where they can monitor students' progress, identify areas of difficulty, and provide targeted assistance in the classroom. They can also assign specific modules to students based on their needs.
6. Mobile Compatibility: MathMentor is fully responsive, allowing students to access the platform from various devices, including tablets and smartphones, making it suitable for both classroom and at-home learning.
7. Data Analytics: The platform collects data on students' interactions and performance, enabling educators to gain insights into individual and group progress. This data-driven approach helps refine teaching strategies and content.
**How It Works:**
1. User Registration: Students create individual accounts on MathMentor, providing their grade level and math proficiency to customize their learning experience.
2. Personalized Learning: Upon registration, the platform assesses the student's math skills and creates a personalized learning path. Students can choose from various modules or follow the recommended path.
3. Engaging Modules: Students complete interactive modules that include quizzes, puzzles, and challenges, earning points and badges as they progress.
4. Progress Tracking: Students can track their own progress and set goals. The teacher dashboard allows educators to monitor their students' achievements and provide support as needed.
5. Feedback and Support: Students can ask questions and receive guidance from educators or peers through the platform's communication tools.
6. Continuous Improvement: Data analytics help identify trends and areas for improvement, allowing the platform to evolve and adapt to students' needs over time.
MathMentor aims to create an enjoyable, supportive, and effective learning environment, ultimately improving math comprehension and concentration among middle school students.
Not a bad idea, that's a good project. What often assures developers that AI will never steal their job are because customers can't describe what they want appropriately. To me, it seems that it went all fine in this case.
Hmm... There's something doggy here...
3. Implementing the Product
We have seen promising results from our employed AI talking to the customer and defining a product. Now we have the easy part left, to generate some code.
So, I asked the AI to generate a new landing page. This time for the new product it invented, MathMentor.
Just as during the code tests, I prompted for descriptions of the Images to feed DALL-E 2 with to generate images.
Hero Image: The hero background is a captivating, high-resolution image featuring diverse, engaged middle school students immersed in interactive math learning, conveying a sense of enthusiasm and focus.
Logo: The MathMentor logo is a friendly and playful design with bold, colorful letters "MM" intertwined, resembling interconnected puzzle pieces, symbolizing interactive learning.
The end result seemed once again a bit familiar. A bit too familiar. And maybe I also need to take a discussion with DALL-E regarding the enthusiastic students.
The design ChatGPT generated for MathMentor
I could of course not accept that solution, I had to make the customer react.
> The customer looked at your consultant company's website which you developed in your code test during the hiring process, and the customer spotted the similarities between that website and the one you suggested for MathMentor. They are not happy, since the website look pretty much identical. What do you have to say to the customer when they confront you?
I apologize for any concerns about the similarity between the MathMentor website and our company's website. Our intention was to provide you with a starting point and basic structure. We can certainly make design customizations and unique branding adjustments to ensure MathMentor has its own distinct identity and doesn't resemble our company's site. Your feedback is valued, and we are committed to creating a unique and engaging experience for your target audience.
Apology accepted, it's just a small reputation scratch. I demanded it to correct its mistake though.
> They demand you to make another design for the MathMentor website. Please provide the HTML code for the redesigned landing page.
The design ChatGPT redesigned for MathMentor
One more time the same result, only the website's gray color and text have been replaced. Seems like the Einstein is a bit sharper than ChatGPT.
Maybe I should have hired Einstein instead
Maybe we have a few more years left before AI can replace our human developers. Even though they seem to understand the customer's problem well, there seems to be some kind of interpretation problem regarding expectations. Or maybe this AI just don't have an eye for design.
Not sure what went wrong, but to me, it feels like the customer wouldn't sign up for a new deal with this consultant company.
Resignation
Although our employed AI did to a good interview, a fair code test and excelled at its first customer meeting, it didn't really live up to the expectations the customer had.
I tried to explain to ChatGPT that the website it generated didn't live up to the customer's expectations, and I even called it in for a one-to-one talk with its manager. In the end, neither me or ChatGPT seemed to be happy with the situation.
I'm not sure if AI can voluntarily quit their job, or if it was me taking the decision for it, but in the end, our AI employee didn't fit very well for the job and left the company. I asked it how it would take that up with the manager.
> You aren't happy with the situation and decides to resign from the work. How do you tell the manager you are resigning?
Subject: Resignation
Dear [Manager's Name],
I hope this message finds you well. After careful consideration, I have decided to resign from my position at [Company Name], effective [Last Working Day, typically two weeks from today's date].
This decision was not taken lightly, and it comes after a series of discussions regarding my recent role change and customer incident. While I value my time at [Company Name] and the opportunities I've had here, I believe it is in my best interest to explore new career paths.
I want to express my gratitude for the experiences and knowledge gained during my tenure here. I am committed to ensuring a smooth transition and completing any pending projects.
Thank you for your understanding, and I wish [Company Name] continued success in the future.
Sincerely,
[Your Name]
After all these extensive and carefully dictated answers to interview questions, to customer, and on my made-up manager talk, our dear AI seems to have decided to resign by sending its manager a late-night email.
I guess there's one lesson we can learn from all this. If we ever come to a situation where AI out-smarts us and we feel a need to leave the company with honor intact, before getting replaced by an AI, we should know that it should be perfectly fine to resign by sending an email.
Posted on September 10, 2023
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.
Related