GreenAI: Comparing LLM's based on energy efficiency
Master of Applied IT
Client company:Lectorate AI & Big data
Project description
The main question this tool answers is: How can we compare large language models based on energy efficiency? To answer this the project has focussed on two things: benchmarking AI, and communicating the findings with the public.
Context
The AI sector has undergone a boom in recent years. A often undercovered but exidingly big problem is the poor power efficiency that these models bring. Running a modern model requires using the latest hardware in large server farms, and there is little to no information out there about the efficiency of these models. Before developers can make climate consious decisions, they need first understand the impact that their selection of models has.
Results
The project has two main results: the research paper brings new insights on the best way to test LLM's, while earlier research focussed on small sample size tests, the new data is created with larger tests that simulate real world conditions.
The other result is the benchmark itself. While its actual TRL teders between level 3 and 4 as of now, next semester will provide a huge boost in operational readiness by integrating a usable user interface.
About the project group
After getting a bachelor in ICT through the Delta program, Kalle has worked on the master Applied IT within the greenAI project group. This semester has kicked of the creation of the comparison tool.
