Rancang Bangun Engine ETL Data Warehouse dengan Menggunakan Bahasa Python

Keywords: Manually, Automation, ETL, Python, Data Warehouse

Abstract

Big companies that have many branches in different locations often have difficulty with analyzing transaction processes from each branch. The problem experienced by the company management is the rapid delivery of massive data provided by the branch to the head office so that the analysis process of the company's performance becomes slow and inaccurate. The results of this process used as a consideration in decision making which produce the right information if the data is complete and relevant. The right method of massive data collection is using the data warehouse approach. Data warehouse is a relational database designed to optimize queries in Online Analytical Processing (OLAP) from the transaction process of various data sources that can record any changes in data that occur so that the data becomes more structured. In applying the data collection, data warehouse has extracted, transform, and load (ETL) steps to read data from the Online Transaction Processing (OLTP) system, change the form of data through uniform data structures, and save to the final location in the data warehouse. This study provides an overview of the solution for implementing ETL that can work automatically or manually according to needs using the Python programming language so that it can facilitate the ETL process and can adjust to the conditions of the database in the company system.

Downloads

Download data is not yet available.

References

[1] J. O’Brien and G. Marakas, Management Information Systems, Tenth Edit. 2011.
[2] S. Lim, “Data warehouse Untuk Pengelolaan Penjualan Pada Pt. Lippo Karawaci, Tbk.,” J. Ilm. SISFOTENIKA, 2012.
[3] F. Y. Al Irsyadi, “Implementasi Data warehouse dan Data Mining untuk Penentuan Rencana Strategis Penjualan Batik (Studi Kasus Batik Mahkota Laweyan),” KomuniTi, 2014.
[4] T. Connolly and C. Begg, Pearson Database Systems A Practical Approach to Design Implementation and Management 6th Global Edition. 2014.
[5] A. Simitsis, P. Vassiliadis, T. Sellis, C. Of, and I. Of, “Modeling and Optimization of Extraction-Transformation-Loading (ETL) Processes in Data warehouse Environments,” 2004.
[6] K. Haryono, “Penerapan data warehouse dalam pengelolaan sistem keuangan daerah,” J. Warehouse., vol. 1, pp. 1–9, 2005.
[7] L. Muñoz, J.-N. Mazón, and J. Trujillo, “Automatic generation of ETL processes from conceptual models,” in Proceeding of the ACM twelfth international workshop on Data warehousing and OLAP - DOLAP ’09, 2009.
[8] R. Syah, “Rancang Bangun Data warehouse untuk Analisis Strategi Produksi Penjualan Usulan : PT.XYZ,” TECHSI - J. Penelit. Tek. Inform., 2014.
[9] C. Thomsen and T. B. Pedersen, “pygrametl: a powerful programming framework for extract-transform-load programmers,” in DOLAP 09 Proceeding of the ACM twelfth international workshop on Data warehousing and OLAP, 2009, pp. 49–56.
[10] R. Kimball and J. Caserta, The Data warehouse ETL Toolkit. 2014.
[11] D. Schuff, K. Corral, and O. Turetken, “Comparing the understandability of alternative data warehouse schemas: An empirical study,” Decis. Support Syst., vol. 52, no. 1, pp. 9–20, Dec. 2011.
Published
2019-08-01
Section
Artikel Teknologi Informasi