An Optimized ETL Fault-Tolerant Algorithm In Data Warehouses

作者：Tu, Shitao^*; Zhu, Lanjuan

来源：International Conference on Information Science and Technology (ICIST), China,Jiangsu,Yangzhou, 2013-03-23 to 2013-03-25.

DOI：10.1109/ICIST.2013.6747594

• Business logic
• Data migration
• Data validation
• Extraction transformation loadings
• Fault-tolerant algorithms
• Time-periods
• Transformation rules

Extraction-Transformation-Loading (ETL) plays an important role in data warehouse. Typically, performance is considered the main factor in ETL projects. Actually, fault-tolerance and many other aspects influence the results of ETL greatly especially when the time period of projects are long and transformation rules cannot be determined from beginning, such as the situation of changing business logic. To satisfy the fault-tolerance and data validation in such kinds of situation, in this paper, we introduce a fault-tolerant algorithm which gives Redo strategy for different process interrupt scenarios. Moreover, we present a compound refresh mode consisting of full and incremental refresh to guarantee data correctness in changing business logic as well as timely data migration.

• 出版日期2013
• 单位上海交通大学