Measuring our recruiting pipeline at Readify

21 October 2016Matt Hilton

One of the first things I wanted to do when stepping into this new role at Readify was to improve the way we track and analyse the effectiveness of changes we make to our recruiting approaches.

Having the role available at all was an acknowledgement that we have a continued struggle with our ability to find and hire the best talent. There are some things we do really well, and there are some that we could definitely improve upon.

As I was brainstorming areas I could focus on in my initial days in the role, here are some of the bigger ones that came to mind:

  • Enormous amount of time spent on tech interviews with a very low pass rate (20:1 in some cases).
    • Perception that the initial stages of our funnel aren't working
  • Little understanding of how changes to job ads or recruiting activities actually impact on our results
  • Limited visibility of the progress of our candidate pipeline

We've done a lot over the past few years to improve our systems and approaches, but we haven't always been able to directly demonstrate the results of these changes. We've relied mostly on indirect measures, so it's always been difficult to know what's working and what's not.

We have a process which isn't quite as effective as we'd like it to be, but we're not sure why.


Applying an Agile-inspired mindset to this problem, there are some very simple steps that can be taken to attempt to improve most situations such as these: inspect, then adapt.

Without measuring where we're at, we can't measure the impact of any changes we make.

... So I did some measuring.

But how do you know what to measure?

Business lens

You need to start from the business questions. Some of the potential questions I came up with that might be useful to answer in relation to our hiring processes were:

  • How do we improve the effectiveness of our hiring process?
    • Where do people "fall out" of the process?
    • Is it regrettable that they do?
    • If so, how do we prevent it?
  • How effective are our advertisements?
    • Which incoming candidate channels result in the most hires?
    • Where do we invest for the best ROI?
  • What is the Readify time-cost of putting a candidate through each stage of the process?
    • How does this translate to direct cost?
    • What can we stop doing right now to reduce burden/cost with minimal impact to the outcome?


From these questions, we can then derive some key metrics that will allow us to analyse the data we already have, glean some insights, and generate hypotheses.

One of the most obvious things to measure which we'd not done consistently was: "which percentage of all candidates manage to get to each stage of our process?". This is easily determined by pivoting our candidates' status-change records in the system to a single-row-per-candidate view that contains the earliest date/time registered for the candidate reaching that stage; null for "never reached". Something like this:

CandidateFirst interactionAttempted puzzlePassed puzzlePassed interviewHired

*not actually the stages we measure on, but gives you an idea

All of a sudden (upon years' worth of data, and layering some sneaky PowerBI on top), we have a beautiful visual representation of our recruiting "funnel" that we've often internally referred to, but never seen quite so dramatically illustrated:


The most obvious question when presented with this shape is: What on earth is happening between step 1 and step 2??? That's a 90% drop-off of candidates right there. Why is this the case? Is it an intentional effective screening of applicants that would fail the later stages anyway, or are we somehow attracting the wrong kind of candidate into the top of the funnel?

Interestingly, this shape is almost exactly the same when we slice the data by different time periods. Since our recruiting process took on its current structure, it hasn't seen much variation.

Other questions crop up when we slice by channel. We've been capturing the source channel for our incoming applications since February 2016, so it's quite interesting to quickly be able to see the differences in between the different channels. The overall shape is still the same (a funnel of reducing volume); its proportions are different.

So what do we do from here? Now that we have the most basic of baselines, two things:

  • Firstly, expose more metrics so we can reason about some of our other business questions.
  • Secondly, generate some hypotheses and design experiments to test them out.
    • (Then measure the results, of course ;))

Hypothesis & Experiment

One hypothesis is that the content of our job ads is not fully setting the right expectations about the role, which candidates don't discover until they click through to our recruiting portal and discover they have to write code as a first step.

Off the back of this (which is not a particluarly mind-blowing hypothesis. I came up with it at 5am on a train), I designed some new content for our Job Advertisements. The goal of this was to improve the conversion rate of our click-throughs, so that more of our candidates coming through the new ad have already had expectations set that they will need to write code as the first stage of our process.

We've posted the ad using a campaign code that will allow us to track applicants coming from this specific ad (across multiple sources), so we can then splice our measurements and judge the effectiveness of the ad over time.

How it goes? Yet to be seen. That can wait for a follow-up post :)

Content of the first iteration of the new job ad:

The company

At Readify, we exist to transform the business of delivering software. We do this based on real experience delivering successful, beautiful software solutions in all kinds of industries and environments. We make it happen, founded on beliefs such as:

  • Automated testing is not a nice-to-have, it's a must-have.
  • Continuous delivery is an achievable reality for all of our customers.
  • "Doing agile" is not useful. Continuous feedback and improvement (inspecting and adapting) is.
  • Our software communities are the lifeblood of our industry, and we have a responsibility to grow and influence them.

The role

We are a software consultancy. This means there are two key skill-sets you should be able to show us:


We ship code all the time. You will be asked to show you can write and deploy working code before we schedule an interview with you; this is necessary because our technical interviews are detailed and in-depth. We want to make sure you're truly interested in the role before spending this time. Operating at a Readify level, we expect you'll demonstrate a working knowledge of topics ranging from IoC and the .NET framework, to deployment and automated testing. We'll go broad and we'll go deep to find the extent of your skills & knowledge.


None of our roles are "just developer" roles - they're all client-facing, whether that's virtually (phone, email, Skype) or physically (on-site). Our consultants aren't just productive team members - they are influencers, coaches, mentors, teachers and business partners. We're about changing the business of software development, and change doesn't happen without awesome leadership, communication and influencing skills. We'll help you build these along the way, but you'll need a good base to start from.

The benefits


The number one benefit of working at Readify is the people you'll get access to. Joining us, you become part of a vibrant internal community, a huge support network where you can't help but learn new things, just by osmosis. Our people are big names in the software community, running user-groups, conferences, hack-days and learn-to-code camps. Some of our shining stars include:

  • Rob Moore: Testing enthusiast; Agility practitioner; high-five/fist-bump king.
  • Rahul Nath: Blogger; Azure ninja; all-round nice guy.
  • Mouna Neelakanta: Kids' coding mentor; UX believer; champion public speaker; 
  • Daniel Chambers: Lambda lover; Docker cheerleader; dry-wit distributor.
Career development

We have an industry-leading, feedback-centric, peer-based career progression approach. Our consultants write and maintain their own job descriptions in a living document we call MadSkillz. It's on GitHub. Check it out. We bring agile to all we do. We encourage regular "career retros" for you to reflect on where you're at, set goals to tackle gap areas, and then review how you improved. Rinse, repeat. This process is self-driven; you get out what you put in. We give you the tools, time and support you need - you bring the passion and drive.

Dollars & other bits

You can expect: a salary reflective of the excellence we seek; significant paid professional development time; paid courses, conference tickets and travel; freedom to manage your own device and software with a $4k "toolkit allowance" every two years; access to awesome Telstra staff benefits; self-managing teams including our whole leadership structure; absence of the arbitrary processes and restrictions you'd be used to from big corporates.

The challenges

The role of the consultant is not an easy one. By definition, the work we do is about helping our customers succeed where they have previously struggled. We therefore deal with a variety of technical and non-technical challenges, including things such as: wrangling legacy code into the modern era; finding ways to put tests around the "untestable"; fighting the good fight to replace manual deployment with automated goodness; mentoring, coaching and teaching less-experienced developers; creatively applying improvements to turn "doing agile" into "being agile"; unravelling habits that have gotten teams into sticky situations; changing mindsets from "CABs and paperwork" to "test confidence and automation". These are difficult but not insurmountable - especially with the rest of the Readify team behind you as we tackle them together.

Because of our flat structure, we use a distributed leadership group to fulfil traditional management roles - you won't have a single line-manager. This can be confronting if you're not used to working in such an agile way. Your career growth, professional development, and success within Readify are largely yours to drive. We'll support you, but you're in the driver's seat.

This style of work won't suit everyone, and we're okay with that. But for those it does - you're going to love it!

The process

  1. Sign up via our Apply process - 1 minute
  2. Complete a code puzzle - around 2hrs
  3. Tech interview (+ debugging challenge) - 1.5hrs
  4. Cultural/management interview - 1hr

We know our process is tough and involved. It requires writing code. That's intentional.

We're looking for people who thrive on a challenge.

Is that you?