توضیحات
Abstract
Hadoop is a popular open-source implementation of the MapReduce programming model for cloud computing. However, it faces a number of issues to achieve the best performance from the underlying system. These include a serialization barrier that delays the reduce phase, repetitive merges and disk access, and lack of capability to leverage latest high speed interconnects. We describe Hadoop-A, an acceleration framework that optimizes Hadoop with plugin components implemented in C++ for fast data movement, overcoming its existing limitations. A novel network-levitated merge algorithm is introduced to merge data without repetition and disk access. In addition, a full pipeline is designed to overlap the shuffle, merge and reduce phases. Our experimental results show that
Hadoop-A doubles the data processing throughput of Hadoop, and reduces CPU utilization by more than 36%
MapReduce [6] has emerged as a popular and easy-to-use programming model for numerous organizations to process explosive amounts of data, perform massive computation, and extract critical knowledge for business intelligence. Hadoop [1] is an open-source implementation of MapReduce, currently maintained by the Apache Foundation, and supported by leading IT companies such as Google and Yahoo!. Hadoop implements MapReduce
framework with two categories of components: a JobTracker and many TaskTrackers. The JobTracker commands TaskTrackers (a.k.a. slaves) to process data in parallel through two main functions: map and reduce. In this process, the JobTracker is in Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee
Year:2011
By:Yandong Wang, Xinyu Que ,Weikuan Yu ,Dhiraj Sehgal ,Dror Goldenberg
File Information:English Language/10 Page/Size:558 K
Download:click
سال :2011
کاری از:Yandong Wang, Xinyu Que ,Weikuan Yu ,Dhiraj Sehgal ,Dror Goldenberg
اطلاعات فایل:زبان انگلیسی/10 صفحه/حجم:558 K
لینک دانلود:روی همین لینک کلیک کنید
نقد و بررسیها
هیچ دیدگاهی برای این محصول نوشته نشده است.