Graph RAG vs. Vector RAG: A Performance Comparison in Response Generation

Resumo

The Natural Language Processing (NLP) area has been gaining prominence as one of the most promising fields in artificial intelligence, driven by recent advances in Large Scale Language Models (LSMs). These models have undergone significant development in a short period of time, becoming more robust and reaching levels of fluency and coherence increasingly close to that of human language. However, with the significant growth of LLMs, some relevant concerns have arisen, such as the occurrence of hallucinations, the importance of the knowledge used in training the models, and the environmental costs resulting from the high computational demand. Aiming to overcome these limitations, some alternatives have emerged, such as RAG (Retrieval-Augmented Generation) models. This approach combines text generation with external information retrieval, allowing the model to access dynamic databases during text generation, which can significantly reduce hallucinations. In conventional RAG models, this information is stored as vectors in a semantic space, allowing similarity searches. Despite its efficiency, this method may fail to represent more complex relationships between data. To overcome this limitation, an alternative to the traditional model is Graph RAG, a version that proposes organizing information in graph structures. This representation facilitates the mapping of explicit connections between entities, which tends to improve the quality and contextualization of the generated responses. To evaluate the impact on the performance of the models due to the data storage method, this work proposes the comparison between two models: a RAG with vector recovery and another based on graphs. For this, both will be trained using the same data set. After that, the models will be evaluated based on the responses provided to a standardized questionnaire, considering the quality and the time required to generate the responses.