---

Introduction to the Problem

Recently, we discussed in Slack how automation could simplify the candidate ranking process. One developer suggested using GPT to streamline this task, but the conversation quickly became tense. Some of us recalled negative experiences where automation led to errors, prompting us to consider how much we could trust an algorithm in such a critical process.

Context: Why This Matters

Ranking candidates is not just a task; it is a crucial stage in the hiring process. Mistakes at this stage can result in the loss of talent or, conversely, the hiring of unsuitable employees. Our team includes many people with recruitment experience, and we all understood that automation must be not only fast but also reliable. We recognized that errors could cost the company both financially and reputationally.

The Problem in Detail

One example we examined was a situation where GPT misinterpreted a candidate's resume. A candidate with extensive experience in IT was rated lower than a less experienced specialist simply because their resume contained more complex terms that GPT could not process correctly. This led to the loss of a potentially valuable employee, undermining trust in automated systems.

Initial Attempts

We started with a simple integration of GPT into our process, hoping that the algorithm could evaluate resumes based on set criteria. However, after several tests, we realized that despite high accuracy ratings, the model still made mistakes. We decided to revert to manual analysis, but this proved ineffective as the volume of candidates increased.

Technical Approach

Ultimately, we chose an approach that combined the best of both worlds: using GPT as a supportive tool for evaluation, but adding layers of verification and manual analysis. We developed an algorithm that first ranks candidates using GPT and then provides the results to the recruiting team for final review. Here is a code example we used for integration:

# Example code for GPT integration
results = gpt_model.rank_candidates(resumes)
validated_results = validate_results(results)

Product Changes

After implementing the new system, we noticed that ranking accuracy increased by 30% compared to the previous version. This also allowed us to reduce the time spent processing resumes, positively impacting the experience for both candidates and recruiters. We updated the sections on /jobs and /for-companies to reflect these changes.

What We Learned

• Automation can be beneficial but requires oversight.
• It is important to combine technological solutions with human experience.
• Full trust in AI can lead to the loss of talent.
• Continuous testing and improvement of algorithms are necessary.
• Manual verification remains an important step in the process.

What This Means for Candidates

Candidates can expect a fairer evaluation of their resumes due to our combined approach. We aim to consider not only keywords but also the overall context, allowing for a better understanding of their experience and skills.

What This Means for Recruiters

Recruiters can now work more efficiently by using GPT for preliminary analysis rather than as the sole source of evaluation. This enables a focus on more important aspects, such as candidate interaction and personal qualities.

Next Steps

We continue to monitor our system's performance and plan periodic updates to the GPT algorithms to enhance accuracy. In the future, we are also considering the implementation of additional metrics for result analysis. If we could change anything in the process, we would add more validation steps to increase confidence in the results. ---