Evaluation and Validation of Requirements Engineering LLM applications
AI & Data
Semester programme:Master of Applied IT
Research group:Sustainable Data & AI Application
Project group members:Rik Hendrix
Project description
The main challenge is that AI is being used in Requirements Engineering, but the actual necessary application isn't clear. My project first uses Error Analysis to uncover the purpose and then implements an actionable LLM-as-a-Judge to solve the problem.
Context
The domain is Requirements Engineering done by LLMs. The context is Sioux Technologies.
Results
The most important outcome is that AI or LLM applications should be seen less as software applications and more as machine learning applications. This incentivises people to setup test datasets and use Error Analysis to uncover quality and important criteria for the requirements. To consequently uphold those quality criteria another AI application might be needed or a way to evaluate based on those criteria.
This can only happen when the right people are involved when looking at the output. Think of a Product Manager, Developer or Subject Matter Expert (SME).
About the project group
Previous Education: HBO-ICT
Time spend: 1 Semester