Data Analysis in Blockchain

Miguel Afonso Rodrigues Hipólito Baptista

Key information

Authors:

Miguel Afonso Rodrigues Hipólito Baptista (Miguel Afonso Rodrigues Hipólito Baptista)

Supervisors:

Miguel Leitão Bignolas Mira da Silva (Miguel Mira da Silva); Paulo José Osório Rupino Cunha

Published in

November 9, 2022

Abstract

Blockchain and Data Analysis are two topics with increasing studies, and both are being integrated for multiple applications. However, accessing data on a blockchain is not a process as straightforward as on a regular centralized data repository, like a database. In this work, two systematic literature reviews (SLR) are performed and a novel architecture is proposed. The main conclusions of these reviews are, (1) blockchain's main benefit is creating trust, security and privacy in a digital environment to gather data from different sources; (2) blockchain's main challenge is the time necessary to access and analyze stored data, and lack of tools to do so; (3) using distributed file systems (DFS) can avoid high storage and computation costs; (4) the most common way of accessing data in blockchain, although sub-optimal, is smart contracts. With the gathered knowledge, a novel architecture is developed and presented. Besides blockchain's and DFS joint inherent capabilities, the architecture main benefit is the ability to make fast and up-to-date predictions using incremental machine learning. A proof-of-concept demonstrating its use was also implemented using Hyperledger Fabric (Blockchain), the InterPlanetary File System (DFS), Kafka (Distributed Data Streaming Event Platform) and RiverML (Incremental Machine Learning). The system is evaluated, showcasing its scalability and potential applications. Laslty, we present the main contributions, research limitations and future work.