How do I manage my data without destroying what I already have?
Many companies already have data governance programs. In financial institutions, for example, it's an agenda that's been in place for several years, driven by the requirements of the Central Bank of Argentina (BCRA).
What has changed is the demand. AI and its models require clean, governed, secure, and high-quality data, which is why governance departments have gained importance within companies. Historically, governance had two pillars: data quality (format, structure) and regulations, which include privacy and are usually handled by specialized lawyers.
In this video we explore these key points:
- Data governance gained importance because AI demands clean, secure, and high-quality data.
- You have to govern the input data, the model, and the data that the model generates.
- Regulations require controls (for example, for bias) and end-to-end traceability.
- It is advisable to choose broad and agnostic tools, capable of also governing conversations and documents generated by AI.
- Synthetic data allows training and validation without using production data.
The data that comes in, the model, and the data that comes out
If a model is fed with poor-quality information, the result will be bad, no matter how good the team building it is. That first step, prepare the data for the model, It is the best known and easiest to plan with any industry; but the gray area appears later.
The model needs to be governed to ensure it complies with regulations and avoids bias, as does the information it produces, because many use cases generate new, self-reinforcing data. Previously, a data project ended with a dashboard for a single user; now the data supply chain expands, and each link must be governed.
Synthetic data: training and validation without touching production data
One technique that helps is the synthetic data generationThis information, while not real, closely resembles real data and is created using rules that replicate business scenarios, including failures. It allows for training or validating a model without using production data, bringing testing closer to reality, which rarely has the ideal conditions of a laboratory. This information, too, must be governed.
A double challenge: technology and regulations
Today, organizations not only face new technology, but also regulatory changes. The European GDPR pushed the agenda in the region, Brazil has its own regulations, and in Argentina, new data quality and use policies have begun to be implemented.
When a model reaches the end customer, regulations require controls, such as the absence of bias, and end-to-end traceability throughout the entire data lifecycle. One example is platforms that assist in the selection of personnel, a process highly prone to bias. Something similar occurs in banking: obtaining metrics to demonstrate to the Central Bank of Argentina (BCRA) that Their model does not contain gender bias.. To achieve a productive model, it must be governed.
The obstacle isn't a single thing: it's a mix of tools, knowledge, and processes, in a context where technology is advancing faster than the tools that allow us to control and audit it. Nor is there yet a five-step guide that suffices.
How to approach it: govern conversations and documents, not just data
The first recommendation is to choose a tool, or suite, that is as comprehensive and agnostic as possible. Solutions specializing in data quality are very good, but they fall short today. Traditional tools govern data, traceability, and lineage. Now, new objects must also be governed, such as the document generated by a chatbot when a customer requests their account or credit card statement.
That document is a company asset and must be justifiable and auditable. The same applies to conversations: it's essential to know what a user asked the agent and what the agent based their response on. In banking, the challenge is to govern every element that comes into contact with a customer, from beginning to end.
