Elevating Customer Experience with Microland's Unified Data & Visualization Platform for a Leading UK Energy & Utility Partner

Overview
Our client, a UK-based technology and data services company committed to simplifying the energy market and accelerating the transition to a net-zero future, specializes in developing innovative digital platforms and services for utility companies. The company faced challenges in managing complex utility data and improving customer experience through advanced data visualization. Their existing legacy platform, built on Infor Birst with an MS-SQL Server backend, suffered from scalability limitations, which hindered efficient data handling and visualization. Seeking a solution to improve performance and user experience, they turned to Microland.

Through a detailed assessment, Microland implemented its Unified Data Platform Solution, leveraging Microsoft Fabric, Databricks, Power BI, and Power Platform. This modernized platform introduced federated development, multitenancy, and seamless integration with diverse data sources, enhancing data visualization, security, and operational efficiency. Additionally, it enabled the seamless extraction of large datasets exceeding 30 million records, overcoming scalability bottlenecks and ensuring timely data delivery. With this transformation, our client now delivers a superior digital experience, empowering utility companies with actionable insights and improved regulatory compliance.

Challenges
Our client’s data platform team faced significant challenges in managing and securing complex utility data while provisioning it to more than 100 customers. With a growing number of contracted customers across the UK, the company struggled with rapidly scaling to keep up with the increasing data volume and customer base. The lack of a scalable, secure data and visualization platform resulted in security risks, administrative inefficiencies, and poor customer experience.

The data platform team was looking for a solution that addressed the requirements below:

1. Multitenancy & Federated Development: The client needed a comprehensive solution to isolate data and resources for each client and/or contract they signed with their group of end customers. Additionally, they required a federated development model to support multiple contracts effectively.
2. Integration with Diverse Data Sources: Handling very large datasets with varied data types and formats required seamless connectivity across diverse data sources to facilitate effective data analysis for both the client and their end customers.
3. Security & Compliance: Given the highly confidential nature of the data, including sensitive personal and financial information, it was crucial to restrict access to corporate resources only to authorized users and groups with the right role-based access privileges. The client needed a solution to prevent over-privileged access, adhere to stringent regulatory requirements, and mitigate the risk of data breaches.
4. Access Management: Managing access to workspaces, data, semantic models, and reports for both internal and external users was complex. The client needed a robust access control mechanism to ensure secure data sharing.
5. Deployment to Multi-Contract Environments: The client had to provision data and reports for multiple contracts, each with varying data and reporting requirements. This required an efficient mechanism for deploying resources across multiple environments.
6. Large Data Extraction: End customers frequently request extensive data to build reports. However, there was no mechanism to provision such large datasets, creating an operational burden. A scalable, self-service capability for large data extracts was necessary to enhance efficiency.

Microland’s Comprehensive Solution
To achieve the desired outcome, the client partnered with Microland to design and implement a Unified Data & Visualization Platform. Leveraging Microsoft Fabric, Databricks, Power BI, Power Platform, and Terraform, the solution empowered the client with a scalable, secure, and automated mechanism to manage data provisioning, governance, and analytics.

With Microsoft Fabric at its core, the platform provided a seamless approach to securely share data through semantic models and Power BI visualizations while integrating with diverse data sources. The solution enabled federated development, streamlined access management, and ensured real-time or near real-time monitoring of users, capacity, and data lineage. It also supported data sensitivity labeling, encryption, lineage tracking, and the seamless download of large data files.

By integrating Databricks, Microland facilitated the extraction of large datasets (>30 million records) and a large user base (1000+), allowing end customers to remotely invoke jobs via Power Apps embedded within Power BI reports. Additionally, Power Platform (Power Apps & Power Automate) automated large data extracts by capturing user-selected filters and invoking Databricks jobs, offering a self-service mechanism for downloading curated datasets.

The partnership with Microland also modernized the client’s reporting ecosystem by migrating legacy reports to Power BI, while Terraform automated the provisioning of Microsoft Fabric items, security groups, and access controls. This transformation provided the client with enhanced security, multi-tenancy, large-scale data handling, and AI-driven insights, ensuring a future-ready data platform for their end customers.

Key features of the solution include:

1. Automated Platform Provisioning Personalizable to Contract: Microland developed an automated platform provisioning module utilizing JSON-based configurations, enabling flexible customization of platform elements and services like Workspaces, Sub Domain/s, MS Fabric Items (Lakehouse, Warehouse, Data Pipelines, Spark Notebook, etc.), Security Groups, Azure Git Repos, and Branches, tailored to contract requirements. The module also streamlined user management, security group creation in Microsoft Entra, and workspace access management by associating security groups and managing user assignments.
2. Robust Security & Access Management: Microland's integrated solution, featuring layered workspace architecture, tenant-level access controls, row-level security at the semantic layer, role-based access, and authentication via Service Principal and Workspace Identity, ensures robust security and safeguards the data and visualization platform for both internal and external users.
3. Comprehensive Auditing and Monitoring: Leveraging Microland recommended Capacity Monitoring App, Microsoft Fabric's Administrative Monitoring Report and Monitoring Hub, along with integration capabilities with Azure Monitor, this solution serves as a reliable and robust platform monitoring system.
4. Seamless Federated Development: The integration of Azure Git Repos/Branches with Workspace, combined with Microsoft Fabric's version control support for Power BI reports and other Fabric items, simplified collaborative development across geographically distributed teams, streamlining new feature releases for the data and visualization platform.
5. Automated DevOps (CI/CD) for Faster Deployment: Microland’s application deployment solution, built using Azure DevOps pipelines and integrated with Microsoft Fabric’s deployment pipeline via its deployment API, streamlines the experience of deployment and promotion of data artifacts and reports to higher environments.
6. Self-Service Large Data Extracts: Microland’s distinctive solution, a potential Intellectual Property (IP) of a customer, powered by Microsoft Fabric, Databricks, and Power Platform, allows the direct downloading of large data files through Power BI reports.
7. AI-Powered Insights for Smarter Decision-Making: The platform's AI capabilities, enhanced by Copilot, enabled users to generate insights rapidly through automated data analysis, report creation, and visualization suggestions. Its natural language interface allowed users of all skill levels to engage with data effortlessly through natural language queries.
8. Custom Branding for Enhanced Customer Experience: Microland's recommended Power BI App solution proved to be highly effective in enabling the client to implement and manage custom branding tailored to each of their end customers. This solution allowed the client to seamlessly integrate unique branding elements, such as logos and color schemes. By leveraging the flexibility and customization capabilities of Power BI App, the client was able to deliver a more personalized and cohesive user experience to their diverse customer base. This not only enhanced the visual appeal of the app but also reinforced brand identity and consistency across different customer interactions. As a result, the client achieved greater customer satisfaction and strengthened their market presence by offering a tailored and professional analytics experience.

Business Outcomes:
The client and its end customers have experienced substantial benefits across multiple dimensions through the implementation of the Unified Data & Visualization solution. Cost efficiency, operational efficiency, enhanced security, and improved compliance were the key benefits. These advancements have delivered measurable value, enhancing efficiency, security, and compliance for both the client and its customers.

• Cost Savings: By reducing annual man-hours by 10,500 for IT and support staff and implementing elastic VMs for autoscaling, the Unified Data & Visualization solutions have collectively generated savings of approximately £1.8 million per year.
• Improved UX: Automated platform provisioning tailored to contracts, user and security group management in Entra, comprehensive auditing and monitoring, federated development, DevOps (CI/CD), large data extract, AI-powered insights, and custom branding have improved the overall user experience by 40%.
• Enhanced Security: Microland's integrated security solution reduced the risk of unauthorized access to sensitive personal and utility data, safeguarding confidential information and ultimately improving the security posture by 30%.
• Improved Compliance: Microland's integrated and robust security and access management solution helped the client comply with data security and regulatory requirements such as GDPR and its security guidelines by ensuring that access controls were in place and up to date.