Through December, I spent a large amount of time doing two things: working my way through the 2020 Advent Of Code; and assessing applications for a job opening we had at Shamaazi. The two initially sound unrelated, but part of our hiring process involves sending candidates an engineering assignment. Similarly, some people use Advent Of Code as interview preparation.

It’s safe to say I spent most of December looking at or thinking about programming challenges 👀

I’ve been reflecting on ‘what makes a good engineering assessment’, and I’ve come to the conclusion that a good assessment – as part of recruiting – should pay careful attention to three main things.

1) Keep it simple, stupid (KISS) 😗

If you’re applying for a new programming job, changing career or trying to get your foot onto the ladder, then the chances are you’re applying to a few different companies. Even if you’re just applying to one though, it takes time. Time to write a CV, time to write a cover letter, research the company, send an application, prep for an interview. Engineering assignments are just one more thing that can eat away at your time.

Time is a limited resource.

Expecting candidates to spend hours decrypting sparse instructions, or spend hours going ‘above and beyond’ in implementing a solution just further reduces any free time they would have. Having lengthy tech-assessments, or expecting candidates to devote their lives to you just creates a false barrier to entry. Some of the best developers I know have lives, kids, families or hobbies outside of their jobs. When recruiting, you should look for well-rounded individuals, not just those who live and breathe software development.

So instead, let’s keep Tech Assessments short. Not simple, but short – short enough that they can be done in a small amount of time. Short so the instructions are easy to wrap your head around. Short so you don’t feel like you have to work all weekend just for a chance at an interview. Because let’s face it, nobody wants that.

2) Make sure it’s explicitly clear what you’re looking for 👀

Recently, I had a friend apply to a new job. He’s a fantastic engineer, works incredibly hard, comes up with brilliant ideas and really contributes towards creating a fun-loving, happy culture. In my eyes, he’s exactly the sort of developer you would want to work with.

As part of applying to a new job, he was asked to complete a short technical assessment. Nothing out of the ordinary here – he completed it pretty quickly and sent it over to me to check through. Together we checked through it and came away happy. It was a really neatly organised, clean and efficient solution to the problem he was given. So he submitted it.

A week went by. No response.

Another week went by. No response.

By this point, he began to get agitated. Maybe they had missed his email, maybe they had forgotten to look at his solution? Maybe the person intended to review it was on leave?

So he reached out to them, expressing his concerns, wondering what had happened. Why had he got no reply? They emailed him back a simple, short answer:

”your submission, while solving the given solution, did not go above and beyond and as a result, was not what we were looking for”

He told me this, and I was immediately outraged. How could they be so stupid to pass up on a talented engineer? Why would they want their engineers to show terrible self-control, to be constantly going off-piste? After some reflection, I realised the problem wasn’t just that they were looking for the wrong characteristics in their developers, but that they had never made it clear in their assignment what they wanted you to do. If they didn’t really care about the solution to the problem, but what you did above and beyond that, then why didn’t they say so?

So instead, let’s keep it explicitly clear what qualities we wish people to display, what areas they should focus on, and make sure it’s crystal clear what they should do. Communication, after all, is a key skill.

3) Make sure the challenge is representative 🏁

It’s very easy to forget when you’re screening candidates and reviewing tech assessments, but every single part of the recruitment process communicates what it’s like to work at a company. Job descriptions give a clear indication of the role, invitation emails should give a bit of a sense of the people who work at a company. Equally, a technical assessment should give an indication of the type of work you will be doing.

If an assignment is all about React hooks, then that’s a strong indication I’ll both need to learn React, and that I’d be writing a tonne of React at that job.

Sounds simple, right?

But people seem to forget this. Instead, they focus on obscure algorithms, copy-pasted katas, or completing contrived challenges –like Advent of Code, fun as it is.

So instead, let’s make technical assessments relevant. Rather than reaching for an obscure problem, what’s the most recent, complicated piece of code you had to write, or edit, or maintain? Can that be explained in a short brief? If so, let’s use that instead.

Closing thoughts 💭

Technical challenges are an incredibly useful tool to use as part of the hiring process. They can often give a much better indication of a candidate than any amount of phone screening, whiteboard-quizzing, or CV scrutinising can. But we have to be careful with them.

Through creating representative challenges, keeping technical assessments lightweight and mindful of time, and by making it explicitly clear what qualities a good candidate would show, we can create technical assessments that don’t feel irrelevant, don’t feel like a time-drain, and are much more useful to both the assessor and the assessee.

Want to know more of my thoughts on recruitment? Read about the qualities I believe developers should show

Do you want to know more about the problems people have in technical assessments? Would you like to know how to avoid these pitfalls when completing a technical assessment? Please get in touch and let me know.