دانلود رایگان مقاله انگلیسی افزایش مدل های سیستم آب با یکپارچه سازی داده های بزرگ - الزویر 2018

Abstract

 The past quarter century has witnessed development of advanced modeling  approaches, such as stochastic and agent-based modeling, to sustainably manage water systems in the presence of deep uncertainty and complexity. However, all too often data inputs for these powerful models are sparse and outdated, yield ing unreliable results. Advancements in sensor and communication technologies have allowed for the ubiquitous deployment of sensors in water resources systems and beyond, providing high-frequency data. Processing the large amount of het erogeneous data collected is non-trivial and exceeds the capacity of traditional  data warehousing and processing approaches. In the past decade, significant advances have been made in the storage, distribution, querying, and analysis of big data. Many tools have been developed by computer and data scientists to facilitate the manipulation of large datasets and create pipelines to transmit the data from data warehouses to computational analytic tools. A generic frame work is presented to complete the data cycle for a water system. The data cycle presents an approach for integrating high-frequency data into existing water related models and analyses, while highlighting some of the more helpful data management tools. The data tools are helpful to make sustainable decisions, which satisfy the objectives of a society. Data analytics distribution tool Spark is introduced through the illustrative application of coupling high-frequency de mand metering data with a water distribution model. By updating the model in near real-time, the analysis is more accurate and can expose serious misin terpretations.

7. Discussions and Conclusions

The aim of this manuscript is to encourage development and enhancement of water computer models by integrating big data. High-frequency data is col lected from heterogeneous sources across environmental systems. However, the collected data is processed and analyzed at discrete actions. Each action can be thought of as collecting a hunk of data to process and analyzing it to make engineering and scientific discoveries. Despite significant challenges, the data should be integrated with water models in an automated fashion to create real time models and complete the data cycle for a water system. A broad framework is proposed to enhance the current water computer mod els with a new API that enables near real-time dynamic modeling and completes the data cycle. In this way, the model is able to characterize some parameters using data that becomes available in the water data lake. The results of a sim ulation are also stored in a water data lake for further analysis. The ultimate outcome of this modeling is to enable a stakeholder to gain better understand ing on the status quo of a water system and manage this system with more confidence. This type of model enhancement provides ways to encounter water systems as a whole rather than a set of technical, economical, and social sys tems that are studied separately and in isolation. The outcome of this holistic approach is useful to assess the performance of all aspects of a system. Most importantly this manuscript emphasizes the increasing importance of computing and analytics in water systems modeling. While many of the chal lenges are being addressed by the computer science field, future water profes sionals will need the basic skills to interface with complex database structures and ever evolving API’s.