Scientific paper proposal: An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation, week 2

[AlexWigren]

Assignment Proposal

Title

An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation

Names and KTH ID

• Alexander Wigren (wig@kth.se)
• Dawa Arkhang (arkhang@kth.se)

Deadline

• Week 2

Category

• Scientific paper

Description

The paper selected was published in IEEE Transactions on Software Engineering

Unit tests are a key part of ensuring that produced code for software is correct and functional. Manually creating these test have however historically been a very labor intensive task. Recent developments in the field of Large Language Models (LLM) has rapidly expanded its usefulness in various fields of software development, including the automated generation of unit tests. This paper by Schäfer et. al serves as an empirical analysis of the effectiveness of LLM technology for the aforementioned purpose. Concretely, the paper introduces what the authors call "TESTPILOT", an adaptive tool to automatically generate unit tests for JavaScript projects. The authors use several different models in their research and find results that seem to imply that the effectiveness of the approach is influenced by the size and training set of the LLM, rather than just the choice of specific model.

During our presentation we aim to explore how this technology works, briefly go over found results and compare this approach of unit test generation to other approaches that exist.

Relevance

Automated generation of unit tests is a key part of DevOps as it helps to save time (developers spend less time creating tests and more time creating deliverable code) and ensure correctness (errors are more likely to be found and corrected before reaching the end user). It is therefore highly relevant to explore this topic further and compare this new approach to other available methods.

[algomaster99]

The checks fail because of our issues. I am looking at it.

[algomaster99]

Closing and reopening this PR to retrigger CI.