The probabilistic frontier: how to orchestrate agents in the modernization of core systems with AI

Artificial intelligence is transforming the possibilities for modernizing core systems. Its ability to interpret legacy code, analyze functionalities, and propose transformations This opens up new opportunities for organizations that need to evolve critical platforms without disrupting their operations..

However, this evolution presents the challenge of how to incorporate technologies that generate probabilistic responses into systems where operations require predictable behavior, precise business rules, and verifiable results. This challenge is particularly relevant in a market that is accelerating the adoption of AI on existing infrastructures.

In this article we analyze how to address the boundary between the probabilistic capabilities of AI and the certainties required by core systems, what criteria to consider when orchestrating intelligent agents, and how to combine automation, validation, and business knowledge to advance in a controlled modernization.

Modernizing core systems with AI: opportunities and challenges for organizations

According to the report 2025 State of Mainframe Modernization by Kyndryl, Based on a survey of 500 technology and business leaders, 88% of the organizations surveyed have already implemented or plan to implement generative artificial intelligence in their mainframe environments. At the same time, 94% of the organizations surveyed indicated that regulations significantly influence their modernization decisions.

These figures reflect a tension that companies must manage, which consists of taking advantage of new technological capabilities while maintaining the stability, security, and compliance of the systems that support their business.

In sectors such as banking and insurance, where a modification can affect financial operations, calculations, transactions or sensitive information, modernization involves much more than transforming code.

Behind every application lie years of decisions, dependencies, exceptions, and business rules that aren't always documented. Some remain hidden in COBOL programs, assembler routines, or batch processes that continue to run for years or even decades.

Generative AI can help understand and transform these environments, but its ability to produce code does not guarantee that it correctly interprets all the conditions that determine its operation.

The challenge lies in establishing a clear boundary between what AI can interpret and propose, what can be verified through reliable mechanisms, and the decisions that need to remain under human control.

Understanding the role of code in business

One of the biggest mistakes when incorporating AI into the modernization of legacy systems is assuming that generating technically correct code is equivalent to preserving the behavior of the original system.

A program can compile correctly, pass certain tests, and produce results with apparent consistency, without considering all the rules that the system has incorporated throughout its history.

Let's consider a banking application developed in COBOL that calculates interest, applies commercial terms, and processes certain exceptions depending on the type of client or product. An AI agent could analyze that program, explain its logic, and propose an implementation in another language.

It could even generate tests that allow comparing both developments, but what happens if there is an exceptional condition incorporated 15 years ago, which only runs under a rare combination of variables?

If that rule is not documented, does not appear in the test cases, and was not correctly identified during analysis, the new application could work fine in most scenarios and produce incorrect results when that exception occurs.

Therefore, the problem would not necessarily be in the quality of the generated code, but in the incomplete understanding of the behavior that should be preserved.

It's important to keep in mind that before deciding how to transform an application, it's necessary to reconstruct what it does, what it depends on, what rules it implements, and what the consequences of modifying it would be. This analysis is especially important in organizations where functional knowledge is distributed among historical documentation, technical teams, and specialists who are familiar with specific operational details.

Generative AI can help recover some of that knowledge, but its interpretations should be treated as hypotheses that need to be tested against information and evidence from the real system.

What seems to have fallen out of use may still be essential

Many systems have functionalities that seem to have become obsolete, although they are still linked to processes necessary for the operation of the business.

Imagine a batch process that runs once a year to perform an accounting adjustment or generate information related to a regulatory close. An analysis limited to recent runs might suggest that this program is practically unused. If no active dependencies appear in the monitoring tools either, the recommendation might be to eliminate it or exclude it from the scope of modernization.

Nevertheless, The absence of visible dependencies does not constitute sufficient evidence to conclude that a feature is obsolete.. A sporadic execution can be as critical as an operation that processes thousands of transactions daily.

To make a reliable decision, it is critical to combine static analysis, dependency review, execution logs, functional knowledge, and validations with business stakeholders.

The difference is significant, since it's not just about identifying code that seems unnecessary, but about determining what impact stopping its execution would have.

At IT Patagonia, this approach relates to our experience in the analysis, maintenance, and evolution of critical applications, from which we address the technical diagnosis of programs, compilations, and versions, as part of defining modernization strategies adapted to each environment.

The goal is to build sufficient knowledge to making decisions about the evolution of systems, instead of starting with a technological transformation whose scope is not yet fully understood.

Generative AI and deterministic processes: how to distribute responsibilities

Once the existing system is understood, a second challenge arises, which lies in determining which tasks should be delegated to AI and which require deterministic execution and validation mechanisms.

The difference between the two approaches is not only technological:

• Generative AI works with probabilistic models capable of interpreting information, recognizing patterns, and producing responses. This feature is valuable for analyzing complex code, explaining functionalities, generating documentation, or proposing transformation alternatives.
• Deterministic processes, on the other hand, allow the execution of explicit rules and the obtaining of reproducible results under equivalent conditions.

In a modernization initiative, both capabilities can complement each other. 

For example, one agent could analyze a set of COBOL programs and identify potential relationships between components. Another could propose a transformation of certain functionalities. Subsequently, analysis and testing tools could compare the behavior of the original version with that of the proposed version.

This separation allows the process to be organized around four responsibilities:

• Grasp: This involves interpreting the existing code, its relationships, and its possible business rules.
• Propose: It seeks to generate transformation alternatives and document the suggested changes.
• Find out: It consists of performing checks, comparing results, and detecting differences.
• Authorize: It aims to determine whether the transformation can proceed based on the risk and available evidence.

The principle is simple: just because an agent has the technical ability to perform an action does not mean that they should be authorized to carry it out.

To apply this to a specific situation, a tool could automatically generate a modification to a program involved in the settlement of financial transactions. However, this does not mean it needs permission to incorporate that change directly into the production environment.

Autonomy needs to be defined according to the criticality of the task, the potential impact of an error, and the possibility of verifying its results before affecting the operation.

A repeatable process can also make mistakes.

Here a distinction appears that is often left out of conversations about automation, considering that always getting the same result does not mean getting the right result.

Let's consider the migration of a process that calculates bank fees. A deterministic mechanism could compare thousands of transactions between the original and modernized applications, verifying that both produce exactly the same amounts.

This comparison provides important evidence of functional equivalence. However, if the original application contains an outdated rule that no longer reflects current business conditions, the new solution could perfectly replicate a behavior that the business needs to correct.

In that case, the automation would have demonstrated technical consistency, but not functional validity. Validating a transformation should consider at least two dimensions. On the one hand, verifying that the expected behavior is preserved and, simultaneously, ensuring that this behavior remains appropriate for the current business needs.

This difference necessitates the inclusion of functional specialists and those responsible for critical processes within the evaluation process. It's not enough to simply ask if the code works as it did before; it's also necessary to determine if it should continue to function that way.

Orchestrating agents means designing permissions, controls, and conditions of autonomy

The incorporation of intelligent agents allows tasks to be distributed among different specialized capabilities, but it also introduces new risks if that distribution does not include explicit responsibilities and limits.

An agent dedicated to code analysis does not require the same permissions as one tasked with running tests. Similarly, a tool that suggests modifications should not automatically have access to implement them in production.

Designing an agent architecture involves defining what information each agent can access, what tools it can use, what actions it is allowed to perform, and under what conditions it needs additional authorization. This is especially important when agents are working with repositories, internal documentation, development tools, or environments containing sensitive information.

In core systems, one way to organize this architecture is to establish progressive levels of autonomy:

• Autonomy to analyze and propose. Agents can interpret code, identify relationships, generate preliminary documentation, and suggest transformations within authorized environments.
• Execution subject to approval. Certain actions, such as modifying code, accepting functional rules, or implementing changes in controlled environments, require additional authorizations and verifications.
• Actions that should not be performed independently. Modifications to production data, the removal of controls, or the implementation of critical changes must remain subject to the authorization mechanisms defined by the organization.

This approach allows AI to be leveraged in exploration, analysis, and generation activities, while maintaining stricter controls over actions that could compromise system integrity. Furthermore, Avoid confusing autonomy with lack of supervision.

A process can utilize multiple agents while retaining human approval points, automated checks, and rollback mechanisms. The key is not to maximize the number of automated decisions, but to determine which ones can be responsibly delegated.

Evidence and traceability in core modernization

In an AI-assisted modernization, preserving evidence should not be limited to simply recording that certain tests were run. The organization needs to be able to reconstruct what code was analyzed, what transformation was proposed, which versions were involved, what checks were performed, and who authorized the process.

This is particularly important in regulated sectors, where the traceability of technological decisions is part of risk management.

In banking, for example, the demands on technological risk management, software development and maintenance make it necessary to establish controls over changes that affect critical systems.

The discussion also connects to the governance of artificial intelligence, including traceability, documentation, and evidence preservation. They allow us to respond to regulatory requirements.

Although the focus there is on AI models and their results, the common principle is that an organization needs to demonstrate how it arrived at a decision and what mechanisms it used to evaluate and verify its impact.

In core modernization, this need extends to the transformation process. For example, when modifying a routine involved in transaction processing, it should be possible to identify:

• The original version.
• The proposed change.
• The results of the comparative tests.
• The differences detected and the corresponding approvals.

This information allows for the evaluation of the change before its implementation and facilitates its subsequent review. Therefore, evidence is not merely a documentary requirement; it is a tool for reducing uncertainty and improving the quality of decisions.

When is it appropriate to preserve a functionality?

The incorporation of AI can help answer a question that should be asked before any transformation, and that focuses on whether it really makes sense to modernize a specific functionality.

Sometimes, the availability of new technologies leads to modernization being considered an end in itself. However, replacing an application is not always the most convenient alternative from an economic, operational, or strategic point of view.

Imagine an assembler routine that performs a specific operation with high performance, stable behavior, and strong integration with other core components. Its age might make it a candidate for replacement. However, if it presents no maintenance issues, doesn't limit new functionalities, and its replacement introduces costs or risks that outweigh the expected benefits, preserving it could be a reasonable decision.

In this scenario, AI can help document its operation, identify dependencies, and analyze potential alternatives. But the result of the analysis shouldn't necessarily be a migration. It might be recommended to maintain the current routine, improve its documentation, strengthen testing, or encapsulate it to facilitate integration with new applications.

In other words, the value of a modernization strategy should not be measured by how much code is transformed, but by how much it improves the organization's ability to operate and evolve its business.

The decision may involve transforming, optimizing, integrating, replacing, or preserving certain components. To choose among these alternatives, the technical analysis needs to be complemented by business criteria: maintenance costs, operational criticality, knowledge availability, integration capabilities, and expected risks and benefits.

From experimentation to assessment: how to incorporate AI into a modernization strategy

One of the main opportunities of artificial intelligence is in expanding analytical capacity during the initial stages of a modernization.

Instead of starting directly with code conversion, organizations can use AI tools to accelerate understanding of their applications, identify relationships, generate preliminary documentation, and explore alternatives.

But these capabilities need to be integrated into an evaluation process that allows technical findings to be transformed into decisions.

From our experience, The initial assessment is a crucial stage To understand the state of the system, establish priorities, and build a modernization roadmap. The knowledge generated during the early stages of a project is a fundamental part of its value, even if it hasn't yet translated into significant, visible changes to the applications.

Incorporating AI into this process allows for exploring new ways to accelerate certain analysis tasks, provided that the results are verified through technical and functional checks. A technology's ability to perform a task does not guarantee that it is advisable to delegate all responsibility to it..

The incorporation of agents requires distinguishing between interpreting information, proposing transformations, verifying results, and implementing modifications. Each responsibility presents different risks and requires authorization conditions commensurate with its impact.

Furthermore, it should be considered that the degree of autonomy should not remain fixed throughout the project. It can evolve as more knowledge about the applications is gained, verifications are strengthened, and it is demonstrated that certain tasks can be performed reliably.

This allows progress to be made gradually, without making the autonomy of the agents an objective independent of the results of the business.

What should an AI-assisted assessment produce?

A modernization diagnosis that incorporates AI capabilities could be organized around four main outcomes:

1. A map of the system and its dependencies, which allows Identify applications, components, integrations, batch processes, known rules, and relationships that require further investigation.

2. A criticality and risk assessment, for determine which functionalities support essential processes, which ones present greater uncertainties, and what consequences their transformation could have.

3. A proposal of evolutionary alternatives, with the aim of comparing modernization, integration, optimization or preservation options according to their costs, benefits and risks.

4. An evidence-based roadmap, oriented to prioritize interventions, define verifications, and establish what conditions must be met before moving on to new stages.

These results allow us to link technological analysis with investment decisions, operational continuity, and scalability. Furthermore, they provide a basis for determining which activities can be accelerated through AI and which require intervention from technical specialists, functional managers, or risk teams.

Checklist: 8 criteria to evaluate the autonomy of AI agents in core systems

Before incorporating intelligent agents into a modernization process, it is advisable to assess what conditions exist for delegating responsibilities.

The following checklist can be used as a starting point for a conversation between technology, architecture, business, security, and compliance teams.

• Is it defined which tasks the agent can perform autonomously, which require approval, and which are prohibited?
• Are the applications, data, tools, and environments that you need to access known?
• Were the business rules and dependencies that could be affected by their actions identified?
• Are there independent mechanisms to verify the results it proposes?
• Are there test cases available that cover exceptions and rare scenarios?
• Is it possible to reconstruct what was analyzed, what was proposed, what was implemented, and who authorized each change?
• Were criteria defined for interruption, reversal, and escalation in the event of unexpected results?
• Is it proven that incorporating the agent adds value compared to a conventional alternative?

This assessment is not intended to establish a universal permissions scheme. Each organization should adapt its criteria according to its technological architecture, the sensitivity of the information, regulatory requirements, and the impact of the processes involved.

Its main purpose is to transform a general discussion about autonomy into concrete decisions linked to responsibilities and controls.

Evolve the core without compromising its functionality

Generative artificial intelligence opens opportunities to understand legacy systems, recover technical knowledge, and accelerate certain modernization activities. But its adoption also requires recognizing that correctly interpreting one part of the system does not equate to understanding all the conditions that determine its behavior.

AI-assisted modernization needs to combine probabilistic capabilities with deterministic mechanisms, business knowledge, and evidence-based decisions.

The challenge is not choosing between automation and control, but rather design an architecture where both capabilities can complement each other. This involves identifying what AI can propose, what results can be verified, and what decisions must remain subject to specific authorizations.

It also requires recognizing that not all functionalities need to be transformed and that preserving a component can be as valuable as modernizing it, when that decision is based on a rigorous evaluation.

From our perspective, core modernization is a process that begins long before modifying a single line of code. 

How to incorporate AI into core modernization

As we explained previously, modernization involves building the conditions so that the systems that support a company today can continue to evolve reliably.

Is your organization evaluating modernizing critical applications, identifying dependencies, or incorporating AI agents into its transformation processes?

We invite you to Schedule an assessment with our team To assess AI-assisted modernization opportunities and the validation mechanisms your core system needs. The first step is understanding what should be transformed, what should be preserved, and what evidence supports moving forward.