دانلود رایگان مقاله MaMR: مدل برنامه نویسی نگاشت کاهش برای برنامه کاربردی ابری مواد

Abstract

With the increasing data size in materials science, existing programming models no longer satisfy the application requirements. MapReduce is a programming model that enables the easy development of scalable parallel applications to process big data on cloud computing systems. However, this model does not directly support the processing of multiple related data, and the processing performance does not reflect the advantages of cloud computing. To enhance the capability of workflow applications in material data processing, we defined a programming model for material cloud applications that supports multiple different Map and Reduce functions running concurrently based on hybrid share-memory BSP called MaMR. An optimized data sharing strategy to supply the shared data to the different Map and Reduce stages was also designed. We added a new merge phase to MapReduce that can efficiently merge data from the map and reduce modules. Experiments showed that the model and framework present effective performance improvements compared to previous work.

7. Conclusions and futurework

In this paper, we have defined a programming model for material cloud applications that supports multiple different Map and Reduce functions running in parallel. MaMR uses a hybrid shared-memory BSP model that can make full use of the data nodes in a cloud computing system. We have designed an optimized data-sharing strategy using the BSP model to support the shared data for Map and Reduce. Meanwhile, we further provide multicopies of the output to reduce the shuffle overhead. We add a new Merge phase to Map-Reduce that can efficiently merge data already partitioned and sorted (or hashed) by the map and reduce modules. In future work, we will explore this new method to improve the parallel efficiency. Currently, more large cloud computing systems should be used to test and verify the MaMR model. The advantages of the programming model should be further amplified by more material data.