Quality Metrics in domain-specific Large Language Models

Project description

How we can validate the quality of text generation by AI after we finetune it? The project focuses on researching new ways to validate a fine-tuned model on a given domain; how well it was finetuned.

Context

The stakeholder requested this research so, in the upcoming years, they could use it as to advise future projects.

Results

We validated a few hypotheses but the most exciting one that we found to be true was the use of word-embedding plots to visualise the difference between the output of a base model compared to his fine-tuned counterpart.

About the project group

I am a semester 4 student in Open Learning. I have been working on this project alone since the second half of this semester due to some complications with my group mates.