Our client is a startup that aims to build Prism, a Big Data platform to manage, monitor and access data sources, processes and industry-specific rules adapters to provide business users and administrators a single tool for all their data needs.

The client wanted to build a data pipeline platform that is capable of near-real time data processing. They needed a high-quality, scalable, single sign-on, and secure solution that uses the following technology stack: Elasticsearch, Shibboleth, Docker, Ansible, Apache Spark, Spring Boot, Hibernate, Apache CXF, Java, Python and Angular 4.

Infonius Solutions, with its technical expertise in the aforementioned technologies, analyzed the proposed architecture and provided solutions with each of the product’s components. The product is essentially a suite consisting of analytical applications that assist enterprises to visualize their data and generate reports and alerts. It consists of the following applications:

  • Reporting Portal: A microservice that generates reports for data visualization. We have developed a feature called as ‘Query Builder’ that allows a user to create and manage data queries. The pluggable APIs that consumes these queries are being used to get data from any datasource. Currently, Elasticsearch, Oracle 11g, Oracle 12c, and Postgresql are the datasources. To visualize the data, we have implemented numerous use-case specific visualizations in canvasJs and D3.js.

  • Alerts Portal: a microservice that consists of a rule-based engine to send alerts. We have implemented an alerting engine that extends Reporting Portal’s Query Builder to create and manage alerts. To alert the end customers, we have also built a framework that generates customized emails. If a customer wants to have customized alerts, it can be achieved because of the pluggability of Prism.

  • User Management Portal: A user management microservice for defining user roles, providing access control to access the applications within the suite. We have implemented a portal that enables an administrator to create and manage roles, other users, resources, and companies. This service also has user authentication APIs that can be easily plugged with a datasource. Currently, LDAP and relational databases are supported out of the box for user authentication.

  • Administration Portal: A microservice to manage the microservices and data sources in the platform. We have used Ansible to start, stop or restart a service. We have also designed a dashboard that displays the health status of each service.

All these applications are single sign-on enabled. We have configured Shibboleth to act as our Identity Provider that generates tokens, facilitating single sign-on and single logout features.

To maintain high-quality coding standards, we have decided to use Behaviour-Driven Development framework – Cucumber. With Cucumber, we have implemented automation testing and integration testing with good code coverage.

For deployment and scalability and to maintain continuous integration paradigm, we have employed a combination of Docker, Ansible, and Jenkins. Docker provides lightweight containers for all our microservices. We have configured Jenkins to ensure that all the automated test cases work fine. After running the test cases, Jenkins builds the Docker container of the microservice and enlists it in Docker Registry, which acts like a repository for all the Docker images. The microservices can then be pulled from Docker Registry and used for end-customer’s deployment. Ansible is used for customization in the Docker configuration so that the microservices (particularly clustered service such as Elasticsearch) can be scaled. This sophisticated configuration has enabled us to deploy Prism for any new customer within an hour.

Infonius Solutions has also provided customizations to our client’s customers:

  • For a communications service provider, we have provided highly customized solutions for data processing and visualization akin to the market-popular product – QlikView.

  • For a mobile tower manufacturing company, we have designed the geo-location based dashboard that provides data visualization on the map of the United States. This dashboard at its core uses Prism’s data processing.

  • For an asset management firm, we have built a customized interface that tweaks Prism to act as an intuitive Inventory Management system.

Below are the success stories of clients’ several B2B customers spanning different industries:

  • A communications service provider is helping its customers – major US telecom companies – in analyzing the quality of calls. The company’s Data Analytics team uses Prism to generate reports for their massive telecom data (of the order of 150 TB).

  • A mobile tower manufacturing company is using the product to find the optimal call quality at its potential installation locations. They also use the geo-location based alerting functionality that narrows down alerts to the equipment level.

  • An Asset Management firm is using Prism to handle their clients’ fund schemes.