Product1 publisher3 min readPublished
HackerRank's AI interviewer scores developers on how they work through a task with AI
HackerRank's Chakra AI interviewer, now open to all customers after more than 500,000 test interviews, scores how developers work alongside an AI assistant. Hiring teams that adopt it take on the job of checking what that score measures and which candidates it flags.
The Product Desk · Product desk
Drafted by a language model from the sources cited here and checked against its claim ledger before publication. How we use AISend a correction
What happened
- Ravisankar said a single Chakra interview now replaces a recruiter screen, a take-home assessment and a follow-up interview with an engineer.
- Chakra is designed to score candidates, with the final hiring decision staying with humans, according to Ravisankar.
- HackerRank says suspicious-activity flags were 70% to 80% lower than in its traditional assessments, with the figure varying by geography and seniority.
- Snowflake, Snorkel and Capgemini were among the companies that tried Chakra during a beta of about six months.
Compiled by The Product DeskSomething wrong?How this is made
Why it matters
- decision Folding the engineer follow-up into an AI session leaves each hiring team to decide whether a person who does the job still meets a candidate before an offer goes out.
- exposure HackerRank's own flag figure moves with geography and seniority, so a company that rejects candidates on flags has to check how they fall across its own applicant pool first.
- constraint Chakra applies the employer's criteria consistently, so a vague rubric for AI fluency gets applied to every candidate, and the score can be no clearer than what the team wrote down.
A candidate is partway through a task in a real code repository, with an AI assistant open in the same canvas. The interviewer asks why she chose one approach over another, then how her solution would change under a new constraint [7]. The interviewer is Chakra, HackerRank's AI agent, and it grades the route she took as well as the answer she reached [3].
HackerRank is pitching a measure of judgment and of what it calls "AI fluency": a candidate's skill at setting up a problem for an AI, assessing what the AI produces and guiding it to a solution [6]. "The previous modality of evaluation was evaluating the output," co-founder and CEO Vivek Ravisankar said. "Now, because of AI, anybody can produce an artifact." [8] What the product does, as described, is narrower. It watches a working session, asks follow-ups based on what the candidate is doing, and applies criteria the employer sets [7][2].
Hiring teams will tell themselves the engineer round survives in automated form. Candidates will meet software in the slot where an engineer used to sit [9]. Ravisankar's assurance that humans make the final call [1] covers who signs the offer. In the three-round sequence, the person signing had an engineer's interview to draw on [9].
The 70% to 80% drop in suspicious-activity flags [10] works out to 20 to 30 flags for every 100 under the old format [14]. Ravisankar's explanation is that open access to AI removes the reason to sneak in outside tools [15]. I think the rules account for part of it too: once AI use is allowed, opening an assistant stops being suspicious. Fewer flags show that the format flags less, and hidden help is a separate measurement. HackerRank's figure also moved with geography and seniority [10]. If flags feed rejections, that split is the first number a company checking fairness should ask for.
The TechCrunch account does not say how Chakra's scores were checked against later job performance or tested for bias across candidate groups. More than 500,000 test interviews [4] measure how much the tool was used. A hiring manager will be asked whether the people Chakra rated highly turned out to be good hires.
HackerRank built its business on tests that largely checked whether developers solved coding problems correctly, and Ravisankar believes AI has made that model less useful for measuring engineering ability [11]. For its more than 3,000 business customers [13], he stated the direction plainly. "Chakra is going to be the headline," he said. "It's going to be the way that we're going to move forward." [12]
I would use Chakra to replace the recruiter screen and the take-home. I would keep an engineer conversation for finalists until Chakra scores can be compared with how those hires perform. The tradeoff is that you keep one of the three rounds HackerRank says it folds into one [9], along with part of the time it saves.
The 2x2 for your own process has two axes. One is whether the hiring manager sees why a candidate scored as they did (the follow-up questions, the answers, the criteria applied) or only the number. The other is whether someone who does the job talks to the candidate before an offer. Visible reasoning plus an engineer conversation makes Chakra a structured first pass. Visible reasoning with no engineer means your rubric has replaced the engineer, and your team owns it. An engineer conversation with only a number upstream leaves a filter nobody can explain to a rejected candidate. With neither, the score is the hiring decision, whatever the policy says about humans deciding [1].
What to watch
- Whether HackerRank publishes data linking Chakra scores to how the people it scored perform after being hired.
- Whether HackerRank gives customers suspicious-activity flag rates broken out by geography and seniority.
- Whether beta customers such as Snowflake or Capgemini keep a human engineer round after moving to Chakra.