Estafet MDM, Data Integration and BI Capabilities

We are well versed in delivering end-to-end BI solutions, integrating data processing, ETL, and advanced analytics. Capabilities demonstrated in past projects include:


  • Real-Time Analytics: Building and maintaining BI dashboards and reporting tools (e.g., OBIEE, Apache Superset) to provide actionable insights and real-time visibility into business operations.

  • Comprehensive Data Management: Expertise in master data management, ensuring clean, consistent data through batch and real-time integrations using tools like Talend and Java microservices.

  • Scalable Data Pipelines: Development of flexible cloud-based data pipelines for managing large volumes of diverse data, supporting commercial, AI, NLP, and machine learning applications.

  • Performance Optimisation: Migrating and optimising data warehouses on platforms like Oracle Cloud and Snowflake to enhance data load performance and business intelligence delivery.

  • System Integration: Creating middleware layers for seamless integration of disparate systems, improving operational efficiency and customer service.

  • Collaborative Solutions: Facilitating collaboration across multiple teams and stakeholders to achieve business goals, streamline operations, and drive quality improvements.

Select Examples

Enhancing Operational Efficiency in Smart Metering

Estafet supplied multiple teams to deliver two large smart metering projects for Arqiva, who manage critical parts of Thames Water’s smart metering and DCC’s Northern region energy smart metering communications infrastructure. Both projects included delivery of a BI layer as part of the overall solution.

At Thames we build the middleware layer which aggregates and communicates smart meter readings and alerts to Arqiva, Thames and Sensus. We also built the BI layer using OBIEE, developing comprehensive business intelligence, service, and management Information reports. We designed and built all layers of the OBIEE solution, including the data extraction, abstraction, and query layers, and created a new OBIEE RPD data model. We developed OBIEE dashboards, widgets, and email alerts, and performed performance tuning and database updates. The reports included billing, invoice, connection mode, status, and task-related insights, enabling stakeholders to make informed decisions based on real-time and historical data.

For the DCC, we built the Communications Hub Manager, a key application that processes and manages all the messaging, alerts and firmware updates for millions of communications hubs, the networking devices attached to smart meters. As critical energy infrastructure, this involved complex encryption and security requirements.

The Estafet team additionally built the BI layer using JSP-built dashboards which enabled visibility of the application’s operations, providing comprehensive reporting on job statuses related to the JBoss Fuse integration and Informatica ETL processes. Sourcing data from operational databases and control structures, this layer offered both summary and detailed reports, with multiple filtering options to view job statuses and associated information as well as ETL error-related data, with options for exporting and printing. We also exported certain data to Splunk and created dashboards tracking security events among others. This solution ensured real-time tracking, filtering, and detailed analysis of jobs and tasks, enhancing operational oversight and decision-making.



Snowflake and Oracle Data Warehouse at Elsevier

Elsevier, a global leader in information and analytics, was migrating its data warehouse to a new platform: Oracle Cloud with a Snowflake instance for reporting. The business goal was to drive significant performance improvements and to help business users to analyse the data and provide business intelligence to management.

Our role was to improve the performance of the data load, developing a process to populate (and refresh) the data warehouse, as well as analysing and improving the performance of the database with respect to CPU, memory and I/O. This included shaping the ETL processes through iterative benchmarking and testing, as well as solving performance bottlenecks using database sizing, configuration changes, parallelism, compression, partitioning, and a holistic indexing strategy.

The impact was that source data could be loaded and processed much faster, leading to more timely and accurate business intelligence.



Transaction Monitoring and Interpretation for SMEs


Pollinate run an innovative and highly configurable SaaS merchant acquiring solution which they offer banks focussing on their SME customers. The platform automates and simplifies the acquiring process and enables SMEs to have near-live visibility and control of transactions, statuses and outlets at various levels of granularity, as well as supplying analytics related to this data.

Estafet worked on various aspects of the solution, including the data pipelines feeding transaction and associated data to the platform, the developer portal, the application front-end,  the architecture of the ACE rewards platform and BI. As part of the BI team, an Estafet BI consultant engaged in the development, configuration, support and maintenance of, in particular Apache Superset, including devising test approaches to validate charts and dashboards.  Technologies used were  Python, Superset, PostgreSQL, and Databricks.


Master Data Management


We have delivered architecture, development and test automation on two MDM solutions at Elsevier: one holds customer data and the other holds products. They are consumed and fed by many other core business systems to ensure there is a single golden record for each class of data. These interfaces fall into two categories: batch and individual records. The batch interfaces are implemented using Talend to transform and load data into the MDM solution and also to extract and transform data for consumption by other systems. Individual records are synchronised via Java microservices, exposed through an API gateway.

The impact across the business is clean, consistent data,


Data Pipelines for new AI insights


Elsevier, a global leader in information and analytics, wanted to combine interdisciplinary sources such as research data, public data, and free-form text from publications to solve new problems. This required flexible cloud services that could collaborate on many different types of problems.

Our team of data scientists, analysts, architects, developers, and testers created data pipelines to manage the ingestion of millions of documents and structured data into a new AWS cloud platform. By adopting a microservice architecture, it was possible to add new data sources and features to support data enrichment, harmonisation, Natural Language Processing, Machine Learning and Knowledge Graphs.

Some of the use cases, such as re-purposing of therapeutic drugs, were anticipated at the start of the project; however, others (such as Corpora-as-a-Service) evolved during development and were assembled primarily from pre-existing services. The impact for customers was that new and unexpected insights were revealed within existing data sources which could be used to quickly develop and release new products.


Cutting Call-Centre costs at Capita Life & Pensions

Capita Life & Pensions is a major supplier of business process outsourcing (BPO) services to the UK Life and pension industry. They run multiple policy systems involving different technologies from legacy providers. This meant that the all centre staff had to enter different systems for each policy, increasing call times and impacting on customer service.

The challenge was not only technical but cultural: we had to get all the suppliers working towards a business goal, rather than their existing contract.

Our solution was to form a layer between the call centre user interface and the back-end systems. This needed to be flexible (to allow for evolving requirements), whilst enabling rapid test cycles to isolate and resolve problems quickly. We delivered the software, testing and also collaboration across teams and suppliers.

The impact on the business was a streamlined call-centre operation with shorter call-times and higher customer satisfaction. In addition to cutting costs, we were also able to drive up quality across the end-to-end solution, meaning services could be re-used for future business initiatives.


BAM and KPIs on a new Merchant Acquiring Platform

Estafet provided and managed multiple teams delivering a Customer Management Solution at WorldPay. The teams worked on a major platform transformation programme in relation to client onboarding and new outlets and terminals. As part of this engagement, we delivered a pilot project to implement Business Activity Monitoring (BAM) and Key Performance Indicators (KPIs). Our team reviewed and validated appropriate KPIs, providing best practice guidelines for their development and governance. We then implemented these KPIs within existing processes, developed BAM dashboards, and integrated them with the WebCenter UI. The project included configuring alerts and SLAs, resulting in real-time visibility into critical business processes. This pilot provided a proof of concept, a framework for future KPI development, and enhanced the overall efficiency and decision-making capabilities at WorldPay.

Contact us

If you would like to have a more detailed discussion about any of these areas and/or how Estafet can help your organisation and teams with best practice in improving your SDLC, you are very welcome to contact us at
Stay Informed with Our Newsletter!

Get the latest news, exclusive articles, and updates delivered to your inbox.