“I don’t think there’s such a thing as over-reliance on AI”

Every employer today says they want candidates with AI capabilities, that much is clear. But when it comes to actual hiring, things get more complicated.

“Everyone already understands that the use of GenAI tools in recruitment is rapidly expanding. Candidates receive a take-home assignment, open ChatGPT or Copilot, and let the tool do the work,” says Lavi Ferdman, Co-Founder and SVP of Growth at cloud security company Upwind.

And this trend isn’t limited to development jobs. It’s spreading to roles in sales, marketing, and product as well. “The problem is that it’s hard to know what the candidate did themselves and what was generated by the tool,” he adds.

As a result, companies are responding in one of two ways: some add in-depth questions to probe the candidate’s thinking process; others invite candidates for live, in-person testing.

“At Upwind, we chose the second path. Except for special cases, candidates now come to the office. We give them the task on the spot and watch them think, write code, and solve the problem in real time,” Ferdman explains. “This completely neutralizes the use of AI tools, there’s no way around it. There’s a whiteboard, a task, and you have to know how to solve it on your own.”

Other companies are also considering moving assessments back to the office. In the meantime, they are deepening their evaluations. “More and more candidates are using AI to complete their take-home assignments, and sometimes you can actually detect the fingerprints of these tools,” says Maayan Biran Berkovitch, Recruitment Manager at Elad Systems. “Interviewers have seen repeated cases where two candidates submit nearly identical solutions to the same task, sometimes with the same structure, wording, or errors, raising questions about authenticity.”

The core challenge for employers is that it’s extremely difficult to know with certainty whether AI was used. Elad Systems now conducts in-depth follow-up conversations after the assignment to verify candidates’ thought processes and ensure they truly understand their submitted work. “That’s how we can tell whether the solution was done independently or is just the product of sophisticated copy-pasting,” she says.

The problem is exacerbated when candidates blindly rely on generative AI tools without checking and engaging in critical or creative thinking. “Chats like ChatGPT tend to ‘aim to please,’ and even when exact information is not available to them, they will provide an answer, even if it is incorrect or inaccurate. When it comes to preparing or answering homework assignments, for example, we have already seen that candidates blindly rely on the chat's answers and submit homework with significant errors. It is important to understand that these tools learn from public databases, and sometimes the information in them is unreliable,' says Ori Eden, Head of Operations at Lasso Security.

A study conducted by Lasso found that developers who rely on any generative AI in code development may receive a significantly incorrect answer to the question, which is, of course, also relevant to homework exercises in the selection process for development jobs.

Another result of using AI in the screening process is 'patterns' of answers, because ultimately, without editing, GenAI tools are programmed to answer in a templated way according to the questions asked. Therefore, when several candidates are given the same task and consult the same GenAI tools, the answers tend to be similar. “In such a case, we will prefer a candidate who provides original and thorough answers that demonstrate direction of thought and creativity (even if incorrect), written without GenAI,” she says.

You Can’t Stop the Tide With Your Hands

Despite concerns, most companies acknowledge that AI is here to stay and are adjusting accordingly. “Tools like Claude, ChatGPT, and Copilot are now part of every developer’s workflow. So the real question isn’t whether AI is being used, it’s how,” says Yonatan Itai, VP of R&D at Cyera.

Therefore, Cyera recognizes that candidates will likely use AI during homework assignments and emphasizes the ability of candidates to stand behind the products, deal with changes in real time, and explain how they used the tools in the best way. Moreover, the company does not consider the use of AI tools to be 'cheating.' “This is a new talent that candidates bring with them, the ability to use AI tools effectively. Instead of focusing on whether the candidate has used one tool or another, at Cyera we emphasize creativity, mental flexibility, and analytical abilities, the skills that are expected to stand out in the near future,” he says.

Since the starting point is that candidates will use AI tools, recruiters will have to adapt themselves and their screening processes to the new era. “I don't see the use of AI tools as a negative thing. Today, the metrics for successful professionals will also include the ability to orchestrate the various AI tools that the organization offers. Our role as hiring managers is to ensure the quality of the product. This will also require an update in our skill set as managers, and that, I think, is what makes it even more interesting,” says Shon Paz, Director of the Red Hat Regional Architects Group.

The hiring manager's job is to ensure that the understanding around the subject is deep enough. So, if the task was to write a strategic plan and the candidate built a good plan using various AI tools, they will need to explain the process: what prompt they used, what edits they made, and why. “Also, the candidate will need to explain how to take it forward. This is true for code, strategy, and business development ideas,” he says.

It is very difficult to limit candidates' use of the various tools, and moreover, says Paz, “I'm not sure it's right to limit it. We have enormous potential here that can practically completely change the employee experience in the organization, and therefore we must adapt to the change and not deny it.”

But more than an ability that needs to be explained during interviews, AI skills are an advantage and even a necessity in the current job market. “Today, knowledge of AI is a mandatory requirement for all positions. It's no longer a nice-to-have, but an expectation,” says Keren Goldberg, VP of Human Resources at Ready Group, which has adopted an integrated approach to the issue of AI use by job candidates. In some cases, such use is encouraged, while in others, it is required to work in a dynamic environment based on smart tools. Therefore, there is a division into tasks that encourage the use of AI and those that prohibit it.

“We are not looking for employees who are afraid of technology, but those who know how to harness it. A candidate who knows how to use the AI screening process as a tool and not as a replacement for thinking is exactly who we want to join us. In an era where GPT and LLM are the standard, engineers who do not integrate AI capabilities into development teams may lose a competitive advantage and be left behind,” she says. Therefore, she sees candidates who understand how to harness tools like ChatGPT or Copilot or Notion AI to improve performance. In tasks that prohibit it: “If the goal is to test an approach to a solution, prioritization, or strategic writing, the use of AI can actually enrich the answer. What interests us is how the candidate uses the tool, does he or she 'copy and paste' or use it as a basis and think critically about it.” In a task that tests independent thinking only, such as cracking a logical problem, they will indicate in advance whether the use of AI is allowed or not.

The company took the use of artificial intelligence in the selection process one step further and even developed an internal recruitment system based entirely on AI, from the initial stages of CV screening, through analysis and professional matching, to predicting candidate success in the long term.

Lightricks also uses an artificial intelligence-based system (Canditech) dedicated to homework tasks to improve the company's screening and evaluation of candidates. On the candidates' side, they encourage the use of AI tools during the performance of tasks but require transparency regarding where and how the technology is used.

“Our goal is not only to assess raw abilities, but also how candidates know how to harness technology to their advantage,” says Pazit Aviv, Director of Global Talent Acquisition at Lightricks.

Not every use of AI serves the candidate. In fact, a candidate who presents what is known as 'AI Slop', a superficial and unfocused product created by an AI tool, and does not know how to explain it in an interview, will suffer from it, says Avihay Kain, VP R&D at Guardio. On the other hand, “candidates who understand how to use these tools to streamline their work and move faster, are the new stars in the market,” he says. In the screening process, they distinguish between candidates who have succeeded in using technology and those who do not really understand their products.

Because basic AI capabilities are a threshold requirement for many jobs, some companies are actually looking for the use of AI tools during recruitment. “I don't think there's such a thing as over-reliance on AI. If the candidate knows how to ask the right questions, correctly formulate the instructions given to the AI to get a clear, focused, and effective answer, and delve deeper into the questions during the process, then there's no problem using AI,” says Yossi Haimov, EVP of Microsoft Solutions at Abra and an AI expert, author of the book Humanity 2.0: Artificial Intelligence - The Butterfly Effect.

He said, “There is no problem with the candidate drafting the instructions for the AI as they see fit, receiving the content, transferring it to Gamma, producing a presentation, transferring it to their avatar who will present it in their place, and even going up for a Zoom interview in their place. Why would there be a problem? On the contrary, a candidate who comes to us after going through this entire process is a candidate who has made the transformation and is in the new world, and has the relevant skills to work for us.”