In a modern marketplace characterized by a surfeit of data, business leaders are often frustrated as they try to piece disparate data sources together to derive crucial insights into industry trends and customer behaviors. Data sources only continue to multiply, meanwhile, and partnerships between companies and applications continue to explode, guaranteeing that the problem will only get worse. More than ever, businesses need a scalable data governance framework to let them stitch all of that data together and align it with their business outcomes.
The bad news? According to forthcoming research by Hakkoda, 46% of companies say that ensuring data quality and governance is a major operational challenge their organizations face.
In this blog, we will explore the challenges faced by a large financial services company striving to accelerate the adoption of its data governance program during development of a Snowflake Data Vault. We will also walk through how, together with Hakkoda’s data teams, they were able to overcome these challenges to establish new data ownership, rapidly expand their data quality rules, and gain a deeper understanding of master data management.
Background: How Data Governance Empowers Data-Driven Decisions
Data governance enables holistic views of critical data in several ways.
First, data ownership and accountability ensures strong advocacy for the investment and development of data on an enterprise level, ensuring it is managed as a valuable company asset.
Second, measuring and monitoring the completeness and accuracy of the company’s most critical data enables leaders to really understand the quality of their data and evaluate the implications of making business decisions from their data sources.
Third, data governance can ensure key performance metrics, executive dashboards, and external reports contain well-defined and consistent information which is critical for managing reputational risk with investors, the board, stockholders and customers.
Lastly, implementing master data management ensures information across all data sources can be consolidated into a golden record that enables 360-degree views of information, fast tracks mergers and acquisitions, and enables data sharing internally and externally.
When employees have a holistic view of their business’s information, those interacting with customers will be able to serve and sell, empowered by data-driven insights. This also establishes a strong foundation for future innovation, including AI integration.
The Challenge: How to Accelerate an Existing Data Governance Framework
After building and implementing data governance programs from the ground up in three financial services organizations, I looked forward to working with a client that wanted to accelerate the growth of its already established data governance framework.
This new client had already done all the right things to lay a strong foundation for its data governance Program: establishing a data governance team, implementing a tool, establishing a Council, approving a policy, and meeting regularly with Data Owners and Stewards.
With all of the pillars of success apparently in place, they nonetheless found themselves asking: why were data governance practices still so slow to grow?
The client’s technology leadership identified the importance of including a data governance workstream in developing a Snowflake Data Vault—a modern data modeling architecture that enables flexible and scalable development of data warehouses. Hakkoda partnered with the client’s data engineering team to develop the Data Vault and upskill the team for continued rapid development. We also worked with the client’s data governance team to accelerate the adoption of existing data governance frameworks throughout the data lifecycle.
Closing the Loop Between Business Stakeholders and Technology Development Teams
It’s a well-established project management principle that successful projects require a tight partnership between business stakeholders and technology development partners.
Unfortunately, that’s often harder than it sounds.
Although both business and technology teams participated in a mutual kick-off meeting, it wasn’t until the data governance team started gathering business and technical metadata that the misalignments started to surface between the two groups.
Business stakeholders were frustrated with the technology team for prioritizing the ‘wrong’ data sources and introducing yet another data repository when the existing one was ‘fine.’ The technology development team, meanwhile, was frustrated with the business stakeholders for dismissing the responsibility to document business requirements and gather examples of data pain points within the data domain.
Through a series of Domain Working Group meetings with both groups at the table, several communication issues were identified which resulted in a new engaged Domain Owner stepping forward to take ownership and accountability for defining the highest priority Key Business Terms, reference data, and data standards. The technology team spent more time answering questions about the benefits of replacing the existing data repository with a Snowflake Data Vault.
The business stakeholders were intrigued by the faster development cycles and scalability of the new Data Vault architecture methodology. With these benefits articulated, additional Data Owners joined the Working Group and advocated for the prioritization of their own use cases and data sources.
Fast-Tracking the Success of a Data Governance Framework
Now that the technology team had the attention of an engaged Domain Owner and several Data Owners, and now that the business stakeholders understood the condensed timeline for moving new data sources to production, attention was turned to developing and governing an achievable set of 25 Key Business Terms, 10 Reference Tables, and 3 large data sources.
Because the objective was end-to-end data governance practices built into the Data Vault solution within the 5 month project timeline, the scope needed to be limited. Data governance development included business glossary definitions, data catalog expansion, data classification, data quality rule development, data issue remediation, service-level agreements (SLAs), role-based access controls (RBAC), data observability, data lineage, and data quality dashboards.
The timeline for the initiative was an ambitious one, but with a knowledgeable data governance team and established procedures for most of these deliverables, the 8 part-time person team was able to deliver on its objectives. In addition to the items above, the team was also able to compile a data governance playbook that could serve as a roadmap for repeating this work for other Key Business Terms and data sources going forward.
Saving Time and Resources with a Data Governance Framework and Master Data Management
Master Data Management (MDM) was another component of this initiative and required developing data standards, data survivorship business rules that systematically retained one data source’s information over another’s in the golden record, and an entity resolution process that generated and managed unique keys for each single golden record established from the merging of 3 data sources.
The Domain Working Group meetings were instrumental in helping both business stakeholders and technology developers walk through examples and requirements for merging sometimes incomplete, inaccurate, and inconsistent data from 3 sources into a single complete, accurate, and consistent golden record.
As business stakeholders started to understand the savings in time spent querying 3 data sources, reconciling and explaining differences between sources, and deciding which data is most trusted, and also started to see the benefits of having a single authoritative view of their domain data, enthusiasm for the Data Vault initiative increased.
Embedding data governance practices and tools by creating a data governance workstream within a business or technology project is one of many approaches an organization can take to expand or accelerate engagement, adoption, and implementation of a data governance program.
The success of this Data Vault project was partially attributed to the established data governance framework and team, but the biggest benefit was the adoption of data governance by dozens of previously unaware employees through exposure to the data governance program and witnessing real-life benefits of active end-to-end data governance made part of their everyday job responsibilities.
Accelerating Your Data Governance Framework with Hakkōda
In the end, the collaborative effort of the client’s many stakeholders and Hakkoda’s data experts proved to be instrumental to the success of the Data Vault project. As a bonus, their cooperation was highly successful in establishing new data ownership, the rapid expansion of data quality rules throughout the organization, and a deeper understanding of master data management. This acceleration of the client’s data governance efforts set the firm up for scalable, org-wide data management success, to the direct benefit of business leaders looking to synthesize actionable insights from a trusted source of truth.
For those business leaders who still count their organizations among the 46% struggling to ensure the quality of their data with a scalable governance framework, this client’s journey is a great example of how common pain points can be resolved with buy-in from key stakeholders and the guidance of a trusted data partner rich in both technical expertise and industry experience.
Ready to accelerate the adoption of your existing data governance framework, or looking to build one from the ground up? Let’s talk today.
The post How Accelerated Adoption of a Data Governance Framework Helped a Large Financial Services Organization Build a Snowflake Data Vault appeared first on Hakkoda.