10 Principles of Modern Data Architecture and How to Implement Them
-
Loading... - Elizabeth Martinez
- 24 Jun 2022
- 160 Views
- 0 Like
- 0 Comment
Data architecture’s just like regular architecture. In both spheres, principles underlying good architecture should be observed. Sure, there’ll be certain designs that work well for a broad swathe of applications and other designs that are a little more niche, but no matter the exact nature of the structure, you can bet that if it’s a successful one, the architect bore in mind the essentials.
What is Data Architecture?
Data architecture can get complicated.
Flow chart of data architecture as it relates to data scientists
But there’s no need to get this complicated straightaway. Most approaches to architecture start with a foundation, and that’s what we’re about to lay down here.
Data architecture can be described as how an entity organizes its data.
There are three aspects to this:
How is the data stored?
How is the data processed?
How is the data used?
We will see these questions crop up all over data architecture concerns, sometimes two or all three at once.
But, to deal with each in turn, storage includes factors such as accuracy, access, control, and scalability. This is the ‘data lake’ of raw data.
Processing covers security, data transmission to and from peripheral sources, and flexibility. The processed data forms the ‘data warehouse.’
Usage covers interfaces, data sharing, and application.
Some companies have very formal approaches to these three aspects of data architecture, some less so. But all companies should cover them in some manner. This way, they can ensure that data management is given the priority it deserves.
Such are the penalties for being careless with data (the average fine for US companies found guilty of a data breach was $4.24 million in 2021) that organizations owe it to themselves, their clients, and any of their contacts to apply a little conscientiousness to their data. Data is precious, so businesses need to view it with the same if not higher regard as capital.
It is to this necessary veneration of data that we’ll turn first.
1. Data Culture
With any paradigm shift, it’s no good just attending to one aspect of a company in isolation if you want major change. For instance, sexism in the workplace is being challenged (albeit slowly) but not with an exclusive concentration on recruitment or any other single area. To ensure the root and branch change required, it has been necessary to tackle the entire environment and psychology of the workplace. In other words, its culture.
Exactly the same with data. There must be a prioritization of data concerns, which is imparted by getting everyone to adhere to the data creed. Data is no longer just the preserve of data scientists.
Here’s one way of depicting this:
One of the biggest mistakes companies make is to recruit a team of data staff, give them a fancy office with all the latest gear, and then sit back, thinking the data job’s done. The trouble is, that the data that your new department is looking after for you will be accessed by lots of others, both internal teams and those beyond the company. If those others aren’t so mindful of data matters, you may have trouble.
These others might end up spreading data to those without the right to access it. We’ve already mentioned the importance of data security and access governance’s value. Almost as bad, they might not provide it to those who need it and workflows may suffer.
All staff have a responsibility to ensure data reaches absolutely everyone who needs it, and absolutely nobody else. Your job is to inculcate this into them so that they begin to see data for the valuable commodity it is, and not just something that might or might not be up for grabs to who-knows-who.
The need to share leads us to our next principle.
2. Dish the Data
So, staff should supply data to one another where tasks require it. But it goes further than this. There should be attention given to making the data work for everyone in the same way. A very salient aspect of this is metrics. A particular metric should mean the same in marketing as it does to the sales team. There has to be a common vocabulary, with no obscure within-office dialects.
Let’s say two parts of the business are working with similar figures, but one works exclusively with monthly data, while the other works only with weekly data. If at all possible, there should be an effort made to unify their data, so that meaningful comparisons and relational appraisals can be made with greater ease and speed.
The more cross-office consensus on what specific data represents and where it directs the organization, the more your business will benefit from joined-up thinking from joined-up departments.
Your excellent data professionals might need a bit of encouragement when it comes to sharing in the first place. It’s often the case that data staff can think of themselves as guardians when they should really think of themselves as facilitators. And part of this facilitation boils down to cutting the jargon. In this regard, there should, in a very real sense, be an effort to get everyone to speak a shared language.
One final point: make sure that your company’s data is organized in such a way that its accessibility is safeguarded. For example, try to make it secure against power outages so that uptime can be optimized and a protected ability for customers to use your services. Read More...
