https://www.mdu.se/

Nowadays software is ubiquitous in our society, making its role increasingly mission critical. Software applications need to be continuously maintained and evolved to keep up with the pace of market demands and emerging issues. Continuous Software Development (CSD) processes are an effective technological countermeasure to the mentioned evolutionary pressures: practices like DevOps leverage advanced automation mechanisms to streamline the application life-cycle. In this context, while handling the application development and implementation is adequately investigated, managing the continuous refinement of the corresponding documentation is a largely overlooked issue. Maintaining accurate technical documentation in CSD is challenging and time-consuming because the frequent software changes require continuous updates and such a task is handled manually. Therefore, this work investigates the automation of documentation updates in correspondence with code changes. In particular, we presen t CodeDocSync, an approach that uses Large Language Models (LLMs) to automate the updating of technical documentation in response to source code changes. The approach is developed to assist technical writers by summarizing code changes, retrieving updated content, and allowing follow-up questions via a chat interface. The approach has been applied to an industrial scenario and has been evaluated by using a set of well-known predefined metrics: contextual relevancy, answer relevancy, and faithfulness. These evaluations are performed for the retriever and generator components, using different LLMs, embedding models, temperature settings, and top-k values. Our solution achieves an average answer relevancy score of approximately 0.86 with Ope-nAI’s gpt-3.5-turbo and text-embedding-3-large. With an emotion prompting technique, this score increases to 0.94, testifying the viability of automation support for continuous technical documentation updates.