I’ve always thought that to use AI effectively (especially in HR) it necessarily involves an enthusiasm for trying new things and learning along the way.
One exciting thing for HR to try is an AI chatbot that will answer employee questions.
With a chatbot, employees could potentially get friendly and accurate service instantly.
It could even be a method of improving the employee experience and saving money.
Being alert to risk
But effectively using AI in HR also involves being alert to risks.
Anyone following the debates about large language models (LLMs) will know that chatbots have various weaknesses.
The most talked about weakness is “hallucinations.”
This is where a user will ask the LLM a question and get a very reasonable-sounding answer.
Unfortunately, sometimes those answers are completely made up.
For example, when asked about soccer players, an LLM confidently explained: “Messi is a soccer player in Barcelona. He is also known for his role in the 2017 film ‘The Messi Diaries’”.
Before you race off to get a copy of the film, read this 2024 paper by Abhika Mishra et al “Fine-grained Hallucination Detection and Editing for Language Models”.
There is no such film, the LLM made it up.
Now call me old fashioned, but I don’t thing you want your HR chatbot making up answers when employees ask about benefits or harassment.
Dealing with its shortcomings
LLMs have some serious shortcomings that we need to consider in any sensitive business application.
Enter Galileo
One of the groups tackling everything wrong with LLMs is Galileo.
The company describes itself as “the leading Generative AI Evaluation & Observability Stack”.
(For me the takeaway from that phrase is that I’m falling behind on learning AI lingo.)
To HR users, what it means is that we are now seeing vendors offer tools that can put guardrails around an LLM application such as an HR chatbot.
Galileo offers the following capabilities:
- Reducing the chance of wrong answers being given (“hallucinations”)
- Ensuring private information is not disclosed
- Suppressing inappropriate language
- Protecting the chatbot from malicious employees who try to trick it into saying the wrong thing
The takeaway
Barriers to effectively deploying an AI chatbot in HR are real but they are being tackled with new tools.
If your organization is large enough that a chatbot is attractive, then it’s time to start experimenting.
Just be sure to keep an eye open for tools like Galileo that can address the common flaws found in LLMs.