Challenges and solutions of real-time data integration techniques by ETL application

Publications

Challenges and solutions of real-time data integration techniques by ETL application

Challenges and solutions of real-time data integration techniques by ETL application

Year : 2024

Publisher : IGI Global

Source Title : Big Data Analytics Techniques for Market Intelligence

Document Type :

Abstract

Business organizations are trying to focus from the traditional extract-transform-load (ETL) system towards real-time implementation of the ETL process. Traditional ETL process upgrades new data to the data warehouse (DW) at predefined time intervals when the DW is in off-line mode. Modern organizations want to capture and respond to business events faster than ever. Accessing fresh data is not possible using traditional ETL. Real-time ETL can reflect fresh data on the warehouse immediately at the occurrence of an event in the operational data store. Therefore, the key tool for business trade lies in real-time enterprise DW enabled with Business Intelligence. This study provides an overview of ETL process and its evolution towards real-time ETL. This chapter will represent the real-time ETL characteristics, its technical challenges, and some popular real-time ETL implementation methods. Finally, some future directions towards real-time ETL technology are explored.