On the Data Stream Processing Frameworks: A Case Study

Abstract

We are living in a technological era where everything is computerized and processed with high precision. The key feature of such a revolution is data. Data is everywhere, it brings crucial information which can be used for the prosperity of our society. In response to our needs, data is growing exponentially and the speed, at which data is generated, is increasing at an unbelievably rapid pace. Dealing with large-scale data is challenging since it requires new approaches and new architectures to the way data is processed. Streaming data came up with a bright concept: processing and analyzing different types of data from various sources in a real-time manner. It creates new dimensions for the way we interact with data. In this study, we investigate a data stream processing approach that can be utilized to process click stream data for targeted advertisements. We collect click stream events from e-commerce websites and process these events to identify predefined patterns. We introduce a software architecture that can be used one layer above the open source data stream processing frameworks. We discuss the details of the prototype of the proposed architecture. We evaluate the prototype with an experimental study. The results are promising.