The majority of today’s traditional Business Intelligence (BI) solutions are based on an Extract Transform Load (ETL) approach. This means that data is extracted from a number of sources, transformed into the specific format needed and then loaded in to the business intelligence tool for further analysis.
An ETL task can involve a number of individual steps such as communication with a number of sources, transition of encoded values, calculating values, merging data from a number of sources, calculation aggregations etc. ETL processes also requires saving all raw data in each source so that each scheduled ETL run can perform the transformations needed and for example calculate aggregation, consolidate data and so on. The problem with this approach is the latency it adds before current data can be loaded, displayed and analyzed in the actual BI tools. It’s a heavy and expensive process that limits the possibilities of showing data and analyses that are close to real-time.
In today’s competitive environment with high consumer expectation, decisions that are based on the most current data available will improve customer relationships, increase revenue, and maximize operational efficiencies. The speed of today’s processing systems has moved classical data warehousing into the realm of real-time. The result is real-time business intelligence (RTBI).
To achieve RTBI one has to rethink the ETL process and find ways of taping in to the data streams and feed business transactions as they occur to a real-time business intelligence system that maintains the current state of the enterprise.
Another important part of the RTBI puzzle is Complex Event Processing tools (CEP). CEP tools are specialized to analyze big data stream in real time. The tools makes it possible to also do complex analysis at the same time as the data is read and written to the real-time business intelligence – and not as a step in a slow and heavy ETL process.
Real-time business intelligence is also known as event-driven business intelligence. In order to react in real-time, a business intelligence system must react to events as they occur – not minutes or hours later. With real-time business intelligence, an enterprise establishes long-term strategies to optimize its operations while at the same time reacting with intelligence to events as they occur.
RTBI might sound as science fiction to many but today