Big Data Engine

Scalable, plug’n’play softeware infrastructure enabling computational models and real-time predictions, offline non real time model processing, with a single window to access all insightful information

Teaser
Teaser:
The Big Data Engine (BDE) is the underlying software infrastructure of PIXEL fitting the most demanding data needs of all stakeholders in the maritime industry, modular and flexible by plug’n’play FIWARE data collection agents, and ready for 3rd party integration through REST API and integration with Port Community Systems. It was built in a lean product development process with decision-makers of small, medium and large ports to fit the data challenges of most European ports.
Technology
Technology:

FIWARE data collection agents to plug in heterogeneous real-time stream data sources into the Big Data engine, based on ElasticSearch and Apache Kafka, sourced over REST API, and served with an advanced model manager that can describe and manage different models and predictive algorithms with different inputs and implementation approaches.

Schema of the product:

Benefits

Benefits

Impacts

Impact

1. Easy to connect new data sources, with support for FIWARE data models conversion, data sources discovery and data sources integration.

1. Improvement of context awareness. Better decision, less time wasted looking for data.

2. Will support high-volume, and high-quality time series data acquisition and easy integration with several PCS systems.

2. Enabling graphical representation of data for better understanding operations, impact of environmental actions, etc.

3. Having all data from multiple resources in a single view to the best context and improve drastically decision making.

3. Better evaluation of environmental-related actions, helping prioritise and select the most impactful activities.

Keywords: Big data management, Model Orchestrator, Data Stream Processing, Data collection agents, FIWARE, data types, plug’n’play, process models, model access, modularity.