https://www.mdu.se/

mdu.sePublications
System disruptions
There are recurring service disruptions on some of the search portals due to high traffic. We are working to resolve the issue, you may temporarily encounter an error message.
3031323334353633 of 78
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
EVALUATING GROUNDING STRATEGIES FOR FACTUAL ACCURACY AND INFORMATION COVERAGE IN LLM-GENERATED REVIEW SUMMARIES
Mälardalen University, Faculty of Engineering and Health Sciences, Department of Computer Science & Engineering.
2026 (English)Independent thesis Advanced level (degree of Master (Two Years)), 20 credits / 30 HE creditsStudent thesis
Abstract [en]

Large Language Models (LLMs) are increasingly used to summarize business data such as customer reviews. Although these models produce fluent outputs, they can generate statements that are not fully supported by the input data, a problem referred to as hallucination. In business analytics, where summaries may directly influence operational decisions, this creates a significant reliability risk. This thesis investigates how different grounding strategies affect the quality of LLM-generated restaurant review summaries. Grounding constrains generation to rely on explicitly provided external data rather than on knowledge learned during training. The research question is: How do different grounding strategies affect the quality and reliability of LLM-generated review summaries? Four strategies are evaluated in a single-factor controlled experiment: Plain Prompting, Retrieval-Augmented Generation (RAG), Aggregated-Statistics Grounding, and Template-Constrained Generation. Each is applied to eleven evaluation scenarios built from real restaurant review data and supplemented with synthetic cases covering text-rating conflicts, ambiguous opinions, and high information density. Generated summaries are assessed using atomic-fact verification, golden-insight coverage scoring, citation-quality metrics, and operational telemetry, with an LLM-as-a-Judge pipeline as the primary evaluator. The results reveal clear trade-offs. RAG is the most computationally efficient but underperforms in information coverage. Aggregated-Statistics Grounding and Template-Constrained Generation achieve the highest mean factual accuracy, but their advantage over the Plain Prompting long-context baseline is not statistically significant. Template-Constrained Generation shows the weakest attribution quality. No single strategy is superior across factual accuracy, coverage, attribution quality, and operational cost, so the findings provide practical guidance for selecting a strategy based on deployment requirements.

Place, publisher, year, edition, pages
2026. , p. 88
Keywords [en]
Large Language Models, Text Summarization, Grounding Strategies, Factual Accuracy, Information Coverage, Attribution Faithfulness, Hallucination, Retrieval-Augmented Generation (RAG), Hybrid Data, LLM-as-a-Judge
National Category
Computer and Information Sciences
Identifiers
URN: urn:nbn:se:mdh:diva-78986OAI: oai:DiVA.org:mdh-78986DiVA, id: diva2:2100389
External cooperation
Zoined Oy
Subject / course
Computer Science
Presentation
2026-09-02, U2-158, Universitetsplan 1, 722 20 Västerås, Västerås, 11:00 (English)
Supervisors
Examiners
Available from: 2026-09-29 Created: 2026-09-10 Last updated: 2026-09-29Bibliographically approved

Open Access in DiVA

fulltext(3640 kB)32 downloads
File information
File name FULLTEXT01.pdfFile size 3640 kBChecksum SHA-512
8b4492596f08acdab8c5ed71d8c7b2276d0c7a0358ffcc918fe94834a618e98378c1f29971dc8988cb856f0ed7bf1934556ca358456e34544cd51076e4cdd8d7
Type fulltextMimetype application/pdf

Search in DiVA

By author/editor
Wahab, Abdul
By organisation
Department of Computer Science & Engineering
Computer and Information Sciences

Search outside of DiVA

GoogleGoogle Scholar
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 235 hits
3031323334353633 of 78
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf