As someone who focuses on the use and application of big data and advanced analytics, I understand only too well the challenges of data security and governance. However, I also see first-hand the huge advantages it brings when executed properly; by correctly implementing data security and data governance, you can improve data quality, ensure compliance, and drive decision-making across the business.
In this article, I’ll explain the vast possibilities that data can offer, but only when managed securely. I’ll then go on to explain some of the challenges of data security and governance and how having the right tools, systems, people, and processes in place is fundamental.
Data and the Art of the Possible
While most of the large companies I work with fully understand the value of data, some of the smaller ones don’t quite see it. I help them to realize that data can deliver improved insights and drive smarter business decisions. What’s more, with the volume of data that businesses collect, there’s now the opportunity to be predictive as well as reactive. For example, companies are using sensors to trigger problems and fix them before they happen. I work with my customers to help them truly visualize what they can achieve with data and to compose a list of priorities that we can accomplish to improve business operations. Businesses can combine historical trends and patterns with predictive analysis to be a step ahead of the game and gain a competitive advantage. Of course, to realize those advantages, data security and governance are paramount.
The Challenges of Achieving Data Security and Governance
Data security is fundamental to protecting our businesses and building trust with our customers. However, the huge expanse of data we collect, store, and analyze makes that harder to achieve than ever. One of the biggest challenges that organizations face is cleaning their data. While you can ensure that, moving forward, data is collected and stored consistently and in adherence with regulatory compliance, the same can’t be said for historical data. With so many different systems in place, standardizing data is a fundamental first step toward data governance.
What’s more, rules and regulations regarding data are vastly different across industry verticals, and they’re changing all the time. For example, there might be a rule to say data can’t leave the country, but you could question whether that’s still the case if it’s anonymized or aggregated. Data compliance—e.g. General Data Protection Regulation (GDPR), California Consumer Privacy Act (CCPA), General Data Protection Law (LGPD), Health Insurance Portability and Accountability Act of 1996 (HIPAA), Right to Be Forgotten (RTBF)— is incredibly complicated, and it’s vital to try to interpret those laws to avoid financial and reputational damage. After all, no company wants to be on the front page of the Wall Street Journal because they’ve made a huge compliance error.
Data Security Across Different Industry Verticals
As an architect, I work with customers to ensure they make the right architectural decisions when it comes to data. It involves everything from interviewing end users, researching potential solutions, and overseeing projects to ensure they retain momentum. However, the data environments are all vastly different for these customers. Architectural decisions have to be made within the privacy and compliance guardrails of the company in question and the industry in which it operates. You must think about the HIPAA laws that are in place in the healthcare industry to ensure the confidentiality, integrity, and availability of protected health information. Data has to be treated in a certain way, and this is a big thing when it comes to implementing an architectural solution. Unfortunately, a lot of customers we work with don’t set enough time aside for data governance. We try to educate our customers about regulations if they’re not familiar with them. In fact, Microsoft often hires people with specific industry experience in areas such as retail, manufacturing, and finance. Having ex-doctors on hand who know the regulatory pieces very well, in the case of healthcare, can make all the difference.
The Importance of Using the Right Solutions
Trying to achieve data security manually is impossible; it would be far too time-consuming to locate sensitive data in that way. Instead, automation is key. A lot of the data governance world is set up to automate data discovery, classify data, and track the lineage of data. With many different types of data coming from many different sources, these tools are fundamental to manage, secure, and understand it.
Any tools that you use must be able to handle all data, regardless of size, type, or speed. With today’s technologies, data can come in at millions of events per second, and you need to know that any solution you use can cope with that. The alternative is looking back in a year’s time and having to rebuild. Moreover, you have to find a way to enable analytics while retaining data privacy. While data may be valuable, if it contains sensitive information, you may need to aggregate it before you can present it. Sometimes, even the landing of sensitive data is unnecessary; if the business doesn’t require the lowest grain of information, you can store just the top-level information to ensure regulatory needs are met. If you do need the lowest grain of data, then data governance is fundamental as part of any solution. You should always consider how you’ll store data and who’s going to have corresponding data access.
Building a Data Security and Governance Centre of Excellence
Some of the customers I work with have a data governance center of excellence. People within the center of excellence will be responsible for different aspects of data security, understanding the regulations and laws, and putting best practices in place. Once rules are locked, tools and procedures become far cleaner and more secure. In addition, the data governance piece manages people, too. After all, tools can only do so much; you need processes in place for the actions that will be taken. We often use ourselves as an example, helping our customers understand the roles and responsibilities that should sit within a center of excellence. Tools have really come a long way in handling all the data our business collects. A megabyte used to be a lot, but now we’re often working with multiple terabytes of data; tools have had to evolve to handle that huge increase. However, data is only valuable if it’s secured and managed properly. Achieve that, and your business can really set itself apart from the competition.
About the author: James Serra is a big data and data warehousing solution architect at Microsoft. He is an expert in data warehousing and data management and has vast experience in data modeling, data governance, and development methodologies. As a thought leader in the use of big data and advanced analytics, he understands only too well the need for data security.
From the Privacera team (Please note, James Serra is not employed by Privacera, and his piece does not imply an endorsement).
Simplify across your ever-complicated hybrid data estate and build a competitive advantage by implementing enterprise-wide data security and data governance. Request your demo today to see how.