Intelligent System for Conducting Technical Interviews with Automated Candidate Analysis

Technical interviews for software engineering positions often involve multiple separate tools, and the assessment of candidates' solutions can be dependent on the subjective judgement of the interviewer. The purpose of the diploma thesis was to develop a prototype system that integrates video interviews, programming task solving and automated analysis of the candidate's submission using a large language model into a unified workflow.

The client-side part of the system is implemented using the Next.js framework, the React library and the TypeScript language. The Convex platform provides the database and backend logic, Clerk handles user authentication, and Stream provides video communication. The candidate completes the task in four steps: familiarises themselves with the requirements, prepares a plan, writes the programme code and explains the solution.

The code is edited in the Monaco editor, and to verify its output it can be executed in a separate environment of the Piston service via a server route. The plan, code, explanation and any execution results are automatically saved to a submissions record in the Convex database during the interview. When the interviewer concludes the interview, the system marks the submission as final and asynchronously triggers the analysis. The backend part compiles a structured prompt from the stored data, which in addition to the candidate's submission contains the task description, its difficulty level, constraints and assessment criteria.

The LLM response in JSON format is validated against the required structure and saved as a report in the ai Reports record. The report contains domain-specific scores, explanations, strengths, areas for improvement and recommendations, whilst the overall score is calculated by the backend from the four domain-specific scores. The candidate and interviewer can review the report in the application and download it in PDF format.

Functional, scenario-based and user testing with 15 participants demonstrated that the prototype supports the entire flow of a simulated technical interview and generates feedback tailored to the content of the candidate's submission. The results support the feasibility of the developed solution in a simulated environment, but more extensive testing with expert assessors would be required to evaluate the reliability of the automated scores.

The analysis is based on the candidate's submission and the task context, whereby successful code execution alone does not guarantee the correctness of the solution. The system is designed as support for the interviewer, who remains responsible for the final assessment of the candidate.