A candidate assessment tool that saves 900 hours of manual reviews
Rainbird worked pro bono with CodeYourFuture to power an automated candidate eligibility and suitability tool. The tool saves CodeYourFuture 900 hours’ worth of manual applicant reviews for each course, while preserving applicant privacy and promoting fairness and transparency.
Challenge: assessing 100s of applicants fairly and quickly
CodeYourFuture is a non-profit organisation that trains the most disadvantaged groups in society to develop skills that help them find employment and a career in the tech industry.
Founded in 2016, the organisation has seen dramatic growth in the popularity of its courses.
For its founder Germán Bencci and its community of employees and volunteers, increased popularity meant facing a new challenge: a high volume of applicant assessments. All course applicants had been processed manually, tracked in large spreadsheets and agreed as a team. Per applicants, this was taking one to three hours of the team’s time.
Experience levels were inconsistent across those judging applicants. And, despite providing all volunteers with training, Germán was worried that greater scale would mean increased inconsistency in the applicant selection process. He knew the organisation would need to find a way to keep the decisions on who to accept onto courses fair and consistent, while still being scalable.
Then, in 2021, the number of applicants applying to its London cohort exceeded 100 people. And with new courses starting in South Africa in 2022, Germán and the team knew it was time to do something.
We knew we had to simplify the process. It was getting too complicated for the community. It wasn’t really delivering the right results.Germán Bencci
Sally McGrath joined CodeYourFuture as its new Director of Education. She immediately identified a key problem with the current process: the team had been analysing every applicant with the same bar on the poverty level—particularly comparing each applicant’s household income and geographical region.
We were oversimplifying. We had to consider a more nuanced, accurate way and factor in postcode and size of household.Sally McGrath
While reviewing how each applicant’s eligibility and suitability was being assessed, Sally realised that all the decisions were fact-based: for instance, how much money an applicant has, how much time they have, where they live. “It seemed really clear that that was something we could systematise,” Sally said.
Approach: create an automated applicant eligibility and suitability decision-making tool
With the idea of a tool that would be able to assess applicants based on important factual information, Sally teamed up with Nirmeet—a graduate of the CodeYourFuture course—and Andrew, Rainbird’s Learning & Development Specialist and Rainbird Community Manager. Rainbird worked with the organisation on a pro-bono basis.
Over the course of a few sessions with Rainbird, Sally, Nirmeet and Andrew were able to build a Rainbird knowledge map in Rainbird Infer. Using further resources, documentation and support available via Rainbird Community, Sally and Nirmeet were able to create an automated decision-making tool that tells applicants if they’re eligible to apply for a CodeYourFuture course.
The tool enables applicants to answer a series of questions via a simple online agent and determine for themselves whether they are eligible and/or suitable for a given course. Each applicant is returned a yes/no answer and a level of confidence for each answer as a percentage.
The tool was made possible by Rainbird Infer, the key component of the Rainbird platform.
|Rainbird Infer is where human intelligence is brought to AI. It captures people’s expertise as knowledge maps. Building knowledge maps is how you visually encode decision logic. And knowledge maps are the foundation of how Rainbird automates complex decision-making.|
Results: 900 hours’ worth of manual reviews saved per course
We use a lot of human resources. However, we want to optimise how we use that human resource. We value people’s time. So, when we mix the community effort and knowledge with the tech that Rainbird offers, we can create a system that makes the most of that community.Germán Bencci
Germán and Sally estimate that the applicant eligibility and suitability tool is saving their community 900 hours per course. The selection process is now streamlined so that the team need only get involved in application reviews where they will truly make a positive difference to the outcome.
Thanks to the Rainbird tool, two other key objectives were met.
Because the eligibility and suitability tool must use sensitive applicant data to make its judgement, it’s important that privacy is protected. This is where Rainbird gives the process a distinct advantage: using the automated tool means that sensitive data is no longer exposed to a group of volunteers.
Individual’s application history is stored in a database that only admins have access to. If a decision is later challenged, the reason for a decision can be anonymously exposed without attaching names to personal circumstances (like health conditions, income, refugee status).
Transparency and fairness maintained
The tool has also allowed the CodeYourFuture team to embrace transparency in the application process.
Being able to explain all of our decisions to our community and to our candidates was one of the most important things about changing our interview process.Sally McGrath
A further important feature of the tool is that, since all decisions are based on facts, all decisions can be challenged and, if necessary, changed. Because Rainbird Infer makes them based on human decision logic, every automated decision is interpretable and explainable (that is, each can be explained in plain language). “It’s really important for us so that our community trusts in this process. It must be able to challenge it at any time,” said Sally.