Home RAGBOT: An AI-Enhanced Web Scraping and Retrieval-Augmented Question Answering System

Publications

RAGBOT: An AI-Enhanced Web Scraping and Retrieval-Augmented Question Answering System

Year : 2026

Publisher : Springer Science and Business Media Deutschland GmbH

Source Title : Lecture Notes in Computer Science

Document Type :

Abstract

Conventional web scraping and information retrieval systems are predominantly designed for data extraction; however, they lack the ability to provide context-aware and user-oriented responses. These systems typically retrieve unprocessed data with inadequate semantic interpretation. It leads to inefficiencies in obtaining meaningful insights. To address these limitations, we propose a Retrieval-Augmented Generation content extraction model (RAGBOT). It is an AI-based web scraping and question-answering system. It leverages Retrieval-Augmented Generation (RAG) techniques alongside lightweight Large Language Models (LLMs). RAGBOT integrates semantic search with real-time scraping to enable efficient retrieval and comprehension of online articles. Our method demonstrates superior performance in terms of accuracy (89.4%) and response time (3.8 s) compared to conventional retrieval and summarization systems. The proposed method was evaluated on standard hardware to assess its performance. The results demonstrate its practicality and scalability, confirming it as a robust solution for intelligent web-based information retrieval.