How to use data governance for AI/ML systems
-
Loading... - Kofi Eshun
- 30 Aug 2022
- 59 Views
- 0 Like
- 0 Comment
Data governance assures that data is available, consistent, usable, trusted and secure. It is a concept that organizations struggle with, and the ante is upped when big data and systems like artificial intelligence and machine language enter the picture. Organizations quickly realize that AI/ML systems function differently from traditional, fixed record systems.
With AI/ML, the objective isn’t to return a value or a status for a single transaction. Rather, an AI/ML system sifts through petabytes of data seeking answers to a query or an algorithm that might even seem to be a little open ended. Data is parallel-processed with threads of data being simultaneously fed into the processor. The vast amounts of data being simultaneously and asynchronously processed can be weeded out by IT in advance to speed processing.
This data can come from many different internal and external sources. Each source has its own way of collecting, curating and storing data — and it may or may not conform to your own organization’s governance standards. Then there are the recommendations of the AI itself. Do you trust them? These are just some of the questions that companies and their auditors face as they focus on data governance for AI/ML and look for tools that can help them.
How to use data governance for AI/ML systems
Ensure your data is consistent and accurate
If you’re integrating data from internal and external transactional systems, the data should be standardized so that it can communicate and blend with data from other sources. Application programming interfaces that are prebuilt in many systems so they can exchange data with other systems facilitate this. If there aren’t available APIs, you can use ETL tools, which transfer data from one system into a format that another system can read. Read More...
