Klareco: An Indexing-Based Architecture For Interactive Visualization Of Heterogeneous Data Sources
Klareco: An Indexing-Based Architecture For Interactive Visualization Of Heterogeneous Data Sources |
Abstract
The ETL process (Extract, Transform, and Load) is critical to denormalize data for easy input in visual analysis tools. Unfortunately, this ETL process requires extensive human effort and computation to complete, often spanning months or years before in-depth analysis can be performed. In this paper, we introduce Klareco, a visualization architecture that foregoes the ETL process allowing quick access to multiple data sources. The architecture uses an indexing engine for accessing data with multiple schema. A series of small data analysis microservices add intelligence to the architecture. Finally, visualizations are designed to display and explore the data itself, as well as the structure of the data, facilitating discovery. This combination of features enables rapid prototyping of visualizations for a variety of data types, formats, and schema. We demonstrate an early version of the architecture using a case study in the domain of oil and gas exploration and to optimize production.
Downloads
Citation
Paul Rosen, Alan Morris, Gene Payne, Bill Keach, Ian Walton, Bryony Richards-McClung, John McLennan, Randy Polson, Raymond Levey, Terry Ring, and others. Klareco: An Indexing-Based Architecture For Interactive Visualization Of Heterogeneous Data Sources. Workshop on Data Systems for Interactive Analysis (DSIA) at IEEE VIS, 2015.
Bibtex
@inproceedings{rosen2015klareco, title = {Klareco: An Indexing-based Architecture for Interactive Visualization of Heterogeneous Data Sources}, author = {Rosen, Paul and Morris, Alan and Payne, Gene and Keach, Bill and Walton, Ian and Richards-McClung, Bryony and McLennan, John and Polson, Randy and Levey, Raymond and Ring, Terry and others, }, booktitle = {Workshop on Data Systems for Interactive Analysis (DSIA) at IEEE VIS}, year = {2015}, abstract = {The ETL process (Extract, Transform, and Load) is critical to denormalize data for easy input in visual analysis tools. Unfortunately, this ETL process requires extensive human effort and computation to complete, often spanning months or years before in-depth analysis can be performed. In this paper, we introduce Klareco, a visualization architecture that foregoes the ETL process allowing quick access to multiple data sources. The architecture uses an indexing engine for accessing data with multiple schema. A series of small data analysis microservices add intelligence to the architecture. Finally, visualizations are designed to display and explore the data itself, as well as the structure of the data, facilitating discovery. This combination of features enables rapid prototyping of visualizations for a variety of data types, formats, and schema. We demonstrate an early version of the architecture using a case study in the domain of oil and gas exploration and to optimize production.} }