AI is becoming so important in HR technology that HR professionals need to develop a more advanced understanding of what’s going on under the hood.
Over the rest of this year, I plan to share several articles that explain how HR vendors are utilizing artificial intelligence.
One of the takeaways from these articles should be that you’ll see there is a fairly straightforward way to make sense of what the vendor is doing and the role AI plays.
Today, let’s look at an AI assessment vendor, Knockri, and see how it uses AI to support decision-making.
An overview of the process
Knockri’s assessment tool is used primarily to support hiring decisions and can also be used for promotion decisions and to identify opportunities for leadership development.
Every tool for assessing people has to be assessing something such as knowledge, personality traits, cognitive ability, skills, or behaviors.
Knockri’s focus is on using the universe of work behavior to measure a range of job-related skills.
Its first step was to create a dictionary of about 30 key behaviors, with 300 or more sub-behaviors.
This dictionary was based on information about jobs gleaned from sources like O*Net.Broadly, Knockri’s dictionary of behaviors is similar to familiar tools like Lominger/Korn Ferry’s dictionary of competencies.
Once it had built an overall dictionary of behavior, its process for assessing a candidate for a particular job has the following familiar steps:
- Identifying the key behaviors needed in the job being filled
- Gathering data from each candidate through a structured interview
- Scoring the data to see which of the key behaviors the candidate has
- Using those scores as part of the decision-making process
These fundamental elements will exist in all AI and non-AI assessment tools.
The key point about Knockri’s process
There is a key point here that HR professionals may have already noticed; Knockri’s primary expertise is in industrial/organizational psychology and proven interview methods.
It uses AI to support that – and we’ll discuss that step in a moment.
But, the point is that we are not dealing with some mysterious black box; we are dealing with a methodology that HR practitioners can understand and evaluate.
If their evaluation is positive, then they can have confidence in using an AI tool because it aligns with their expertise in interviewing methods.
Where AI comes In
The AI element of Knockri’s process is automating the scoring of the interview.
It uses machine learning to review the interview transcript and to identify work behaviors demonstrated by the candidate.
It then compares the behaviors observed to those needed in the job to assess the candidate.
The automated scoring allows for scale, avoids interviewer fatigue, and reduces bias.
If you are happy with the quality of the assessments for the jobs you need to fill, then this technology is a boon to HR.
Checking the quality of the process
HR can ‘check’ on the AI by seeing what behavior is scored (for instance, perhaps it scored a candidate as high on “knowledge seeking”), and then asking the AI what it heard in the interview that led it to give that score.
Perhaps instead of saying “HR can check…”, we ought to say, “HR must check…” when it is evaluating a tool, because HR can’t go forward with a black box informing hiring decisions.
HR needs to do its due diligence and HR pros don’t need a degree in advanced mathematics to do so.
With every assessment, Knockri provides a detailed performance report. The performance report breaks down an applicant’s response, allowing hiring managers to see not only how well an applicant performed, but what important behaviors were discussed by the applicant, and what behaviors were missed.
This provides a fully transparent process which is essential for winning the confidence of everyone involved.
To be more rigorous, HR would ideally be doing a longitudinal assessment of its hiring processes.
The goal of these longitudinal assessments is to show how well applicants perform on the assessment is related to how well they perform on the job, or how long they remain employed with the organization.
This isn’t practical in many cases, however, note that we are using the same methodology to evaluate an AI-supported process as we would use to evaluate any hiring process.
In other words, there’s no great mystery involved.
General lessons
HR cannot use AI tools without understanding how they work.
To get this understanding you need to know what data the AI is using, and what process it is using to analyze that data.
The good news is that you will often find that what is going on under the hood is about industrial/organization psychology and good HR practices supported by machine learning to enable automation.
This means you can and should look under the hood so that you can adopt these tools with confidence.
Knockri provides a clear case of how an AI tool can fit into an HR process comfortably such that we can test if it is working for the organization.