Rancang Bangun Engine ETL Data Warehouse dengan Menggunakan Bahasa Python
Abstract
Big companies that have many branches in different locations often have difficulty with analyzing transaction processes from each branch. The problem experienced by the company management is the rapid delivery of massive data provided by the branch to the head office so that the analysis process of the company's performance becomes slow and inaccurate. The results of this process used as a consideration in decision making which produce the right information if the data is complete and relevant. The right method of massive data collection is using the data warehouse approach. Data warehouse is a relational database designed to optimize queries in Online Analytical Processing (OLAP) from the transaction process of various data sources that can record any changes in data that occur so that the data becomes more structured. In applying the data collection, data warehouse has extracted, transform, and load (ETL) steps to read data from the Online Transaction Processing (OLTP) system, change the form of data through uniform data structures, and save to the final location in the data warehouse. This study provides an overview of the solution for implementing ETL that can work automatically or manually according to needs using the Python programming language so that it can facilitate the ETL process and can adjust to the conditions of the database in the company system.
Downloads
References
[2] S. Lim, “Data warehouse Untuk Pengelolaan Penjualan Pada Pt. Lippo Karawaci, Tbk.,” J. Ilm. SISFOTENIKA, 2012.
[3] F. Y. Al Irsyadi, “Implementasi Data warehouse dan Data Mining untuk Penentuan Rencana Strategis Penjualan Batik (Studi Kasus Batik Mahkota Laweyan),” KomuniTi, 2014.
[4] T. Connolly and C. Begg, Pearson Database Systems A Practical Approach to Design Implementation and Management 6th Global Edition. 2014.
[5] A. Simitsis, P. Vassiliadis, T. Sellis, C. Of, and I. Of, “Modeling and Optimization of Extraction-Transformation-Loading (ETL) Processes in Data warehouse Environments,” 2004.
[6] K. Haryono, “Penerapan data warehouse dalam pengelolaan sistem keuangan daerah,” J. Warehouse., vol. 1, pp. 1–9, 2005.
[7] L. Muñoz, J.-N. Mazón, and J. Trujillo, “Automatic generation of ETL processes from conceptual models,” in Proceeding of the ACM twelfth international workshop on Data warehousing and OLAP - DOLAP ’09, 2009.
[8] R. Syah, “Rancang Bangun Data warehouse untuk Analisis Strategi Produksi Penjualan Usulan : PT.XYZ,” TECHSI - J. Penelit. Tek. Inform., 2014.
[9] C. Thomsen and T. B. Pedersen, “pygrametl: a powerful programming framework for extract-transform-load programmers,” in DOLAP 09 Proceeding of the ACM twelfth international workshop on Data warehousing and OLAP, 2009, pp. 49–56.
[10] R. Kimball and J. Caserta, The Data warehouse ETL Toolkit. 2014.
[11] D. Schuff, K. Corral, and O. Turetken, “Comparing the understandability of alternative data warehouse schemas: An empirical study,” Decis. Support Syst., vol. 52, no. 1, pp. 9–20, Dec. 2011.
Copyright (c) 2019 Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi)
This work is licensed under a Creative Commons Attribution 4.0 International License.
Copyright in each article belongs to the author
- The author acknowledges that the RESTI Journal (System Engineering and Information Technology) is the first publisher to publish with a license Creative Commons Attribution 4.0 International License.
- Authors can enter writing separately, arrange the non-exclusive distribution of manuscripts that have been published in this journal into other versions (eg sent to the author's institutional repository, publication in a book, etc.), by acknowledging that the manuscript has been published for the first time in the RESTI (Rekayasa Sistem dan Teknologi Informasi) journal ;
