Why is it important to optimize batch processes?

As daily transaction volumes increase and user habits change in the way they operate with financial institutions, one of the main concerns of the sector is to reduce batch process times, especially the batch window. 

The key today is to maximize the availability of services, Aiming to have online channels active 24×7. Also due to the ability to react quickly to eventualities or unforeseen events, and to solve problems in a timely manner.

In this article, in addition to analyzing this problem, we share the value contribution of the tool OPTI of IT Patagonia, which ensures that processes are optimized, eliminated or debugged, as necessary, and have a positive impact on batch chain times, with automated generation of reports, analysis and monitoring, which facilitates control and the possibility of resolving incidents in an early and agile manner. 

What is a batch process, batch chain and batch window?

Before beginning to analyze the problem described, it is important to define three essential elements for the normal operation of a financial institution.

A batch process It is a set of critical tasks for the daily operation of a financial institution, which must be completed so that the next day each of its branches and its virtual media - such as mobile applications or the website -, function correctly, in a timely manner and in a timely manner.  

Meanwhile, the batch chain is a sequence of tasks or processes that are executed in batches, one after another, as part of an automated process. These micro processes are usually designed to perform repetitive or intensive operations, in resources that do not require real-time interaction. 

The batch chain can include activities such as data collection, processing, reporting, and distribution of results.

Its efficient management is crucial to optimize system performance and ensure efficiency in processing large volumes of data. 

In a financial system, the batch chain may include processing current accounts, and within those current accounts, payroll accounts, processing daily transactions, generating account statements, calculating interest, and updating records.

For its part, the batch window or critical batch window, refers to the period of time during which batch processes are executed, generally outside of a branch's normal operating hours.

It is the moment when the necessary and essential processes for the operation are running, taking into account that in every Bach process there are essential parts and others that are not critical.

As he points out Maximiliano Casalaspro, Data Center Services Manager IT Patagonia, it is important to note that batch processes do not require real-time interaction and can be executed in a deferred manner, since they are not decisive for the immediate operations of the branch. 

Some examples of these are the generation of current account reports and the maintenance of savings accounts, which do not need to be done during customer service hours.

Common practice is to schedule them to run at night, when the workload on the system is lower and there are fewer users interacting. This minimizes the impact on system performance during daylight hours, when higher user traffic and real-time transactions are expected. 

This separation between real-time and batch processes allows for optimised resource utilisation and ensures efficient, uninterrupted system operation during customer service hours. 

Additionally, by running batch processes in a specific time window, you can set schedules and priorities to ensure that critical tasks are completed within specific timeframes and are avoid conflicts with other system operations.

The Optimization Challenge: Why Do It?

Optimizing the batch process is crucial to maximizing operational efficiency.

This results in reduced processing time, minimizing resource usage, and improving system scalability. 

And it leads to increased productivity, lower operating costs and a better experience for end users by ensuring fast, reliable and high-quality processing of large volumes of data or batch tasks.

For financial organizations, it is essential to operate in a timely manner. Based on this objective, The duration of the batch process should be as short as possible., since in general, while it is running, many operations do not work and service availability is reduced. 

A clear example of the latter is when a person logs into home banking and wants to make a credit card payment at two in the morning. It may happen that the system indicates that the transaction cannot be made at that time. 

In certain cases, some banks allow the transaction to be scheduled to take place after six in the morning. But sometimes it happens that the person cannot complete the operation at the time he or she needs to. 

This situation, which often generates frustration and compromises user satisfaction, occurs because during that period, the batch process is in progress and no operation can be performed. 

So, what we are looking for is to reduce that batch window so that the operation is as short as possible, and so that the user can operate normally. 

The importance of being able to do so lies in the need to adapt to people's new habitsUnlike in previous years, today we are constantly using our cell phones to make transfers, check our bank accounts, or make purchases with a credit or debit card, even at night. 

Behaviors like these undoubtedly significantly increased the interaction time with entities, as well as placing much greater demands on the batch process. 

How to optimize a batch process?

Optimizing the batch process involves a series of key steps, aimed at improve overall system efficiency and performance. 

The important thing to keep in mind is that batch process optimization is not a one-time event, but rather an ongoing effort that requires monitoring, iterative adjustments, and constant adaptation as system requirements and operating conditions evolve.

First, it requires a thorough evaluation of the existing workflow, to Identify bottlenecks, inefficiencies and areas for potential improvement. 

This may involve the Software architecture analysis, algorithm design and database query optimization to reduce execution time. 

Once the points for improvement have been identified, optimization techniques can be applied, such as task parallelization, parameter tuning, query optimization, and efficient use of available memory and hardware resources. 

Additionally, implementing continuous monitoring and analysis tools is crucial to monitor batch process performance and make adjustments as needed to ensure optimal performance over time.

On the other hand, batch process optimization can also benefit from the automation and task scheduling to minimize manual intervention and reduce the risk of human error. 

Implementing automated workflows and scheduling recurring tasks can help streamline the process and ensure consistent, reliable execution. 

In addition, thorough testing is critical to validate the changes made during the optimization process, and to ensure that no new problems or performance degradations are introduced. 

How long does a batch window last?

The duration of a batch window, during which services are unavailable, varies. It depends greatly on each bank and the timetables they have for this purpose. 

Although an average of 4 to 6 hours can be mentioned, there are entities with a smaller number of transactions that only require three hours to complete their entire process. In the case of banks with a large volume to process, the window can last between 5 and 6 hours.

There are even days with a high transaction load, such as the end of the month, when there are many salary payments, or the day after a long weekend, especially Carnival or Easter, when the times tend to extend to 8 hours. 

Therefore, when faced with a problem in the batch process, The higher the operational level of the entity, the less time it will have to solve it and prevent it from impacting the opening of branches.. 

Hence, the ability to organize, cleanse, cure and optimize batch chains is very important. 

The aim is to ensure that on complex days the duration of the window is not affected or extended more than necessary, in order to avoid affecting the operations of the following day. This is especially true during branch opening hours or when people operate more frequently.

Impact of batch chain on transaction processing

The batch process executes all non-online operations during the day. 

For example, payroll processing, account maintenance, statement generation or card interest payments are processed overnight. 

Also the generation or renewal of fixed terms, which although they are done at the time requested by the client, remain in a "frozen" state and are processed at night, when the banking entity begins to impact all the operations of the day. 

During batch processing, entities are generally not 100% available, as there are many operations that cannot be done at that time.

For this reason, work is being done to optimize the batch with the aim of reducing these times and ensuring that the system be unavailable for as little time as possible, remaining enabled online.

This is how Maximiliano analyzes it, highlighting that the objective of all banking entities is always be available 7×24 at the operational level, that everything is online and we stop having batch, or that it is very limited. “We hope that applications are not affected by the batch process. That is why the work we do in our Mainframe Center of Excellence in terms of optimization,” he notes. 

Although 100% is not available in home banking during the early hours of the morning, it is important to optimize and reduce the batch process as much as possible. In addition, it is necessary to ensure that any inconvenience that may occur in a batch process does not impact the operations of the following day. 

Can a bank be 100% available 7×24? 

Breaking away from the batch model is very complex, because batch processes have to run and many are not prepared to do it online. 

This situation requires a change at the programming level. It is a long road to travel and in the meantime, The focus is on reducing batch process time.

On the other hand, there are always unavoidable factors that can cause disruptions, in addition to scheduled maintenance and system upgrades. These include unforeseen hardware failures, cyber attacks, and natural disasters, among other inconveniences.

However, the aspiration to achieve the closest approximation to that ideal objective is always present and implies a strong driver for continuous improvement.

How to optimize batch processes? 

In relation to those systems, programs or tasks that are within the batch window, which are critical for the normal operation of the following day and which take a lot of time, what should be done is optimize them with new and better practices or those that are used today. 

The goal is to make them work more efficiently and Their execution times are reduced by a 40% or 60%.

As for the Bach chains that have been in operation for many years, Maximiliano explains that it must be verified whether it is actually necessary for them to be there and, in the case that they are not necessary, purge and eliminate them.

For example, an entity that used the Red Link interbank ATM network and switches to Banelco must analyze whether all Red Link processes have been eliminated or not, to avoid problems. “Taking this example into account, what we do is verify that there are no Link processes running and only those from Banelco remain,” says our expert.

In this way, the entity ensures that what is running within the critical window is what is required for its branches to function properly the following day. In addition, it also ensures that the critical window takes as little time as possible. 

Benefits associated with batch process optimization

In addition to achieving execution times reduced between a 40% and a 60%, the optimization brings other advantages.

One of them is to speed up the detection of batch processes. A benefit provided by our exclusive OPTI tool, which has been in the market for 15 years and is installed in all our clients.

To visualize the value contribution, you have to imagine that large banks run between 50,000 and 100,000 processes daily. 

Given this volume, it would be very difficult for teams to identify which processes to optimize if they did not have a solution with the characteristics that we have.

OPTI ensures that the process that is optimized, eliminated or debugged is the one that really has an impact on chain times.. Automating the generation of reports, analysis and monitoring, and facilitating batch controls. 

The tool takes a “snapshot” of the batch process and, in approximately two to three weeks, allows you to know where you need to optimize and reduce times. 

“We then work with a team specialised in the different platforms, who are responsible for optimising these programmes or cleaning up and debugging the chains, so that the times are reduced,” says Maximiliano. 

Although there is a scheduler that runs the batch chain processes, it is needed qualified people to operate it and ensure that the processes are completed in a timely manner. 

“It is a joint effort that marks the differential value for which financial institutions continue to choose us, since we allow them to continue improving batch process times,” adds our Data Center Services manager. 

What are the challenges or problems of batch processes?

Among the main challenges that arise is the existence of Mainframe platforms with programs developed in Cobol II - or other older versions - when the ideal would be to have Cobol 6. 

It also happens that files are taken in a disordered manner or with a program logic that is set up for a certain volume, which has changed over the years," warns Maximiliano. 

In these types of situations, it is necessary to readjust the programs and, at the database level, adjust the indexes. 

As technology changes, the way we access data changes and it is necessary to update. The way we operate also changes, and as a result, the logic of the programs. 

For example, before you had to go to a bank branch to renew a fixed-term deposit and now you can do it from a cell phone. 

Highly disorganized batch chains may also occur.

OPTI: an X-ray of batch processes in a few weeks

After many years of working with clients in the financial ecosystem, in which as a company we acquired a deep knowledge of the batch process and the needs of the sector, we developed the OPTI tool. 

One of the triggers was the need for power early detection of deviations and develop the ability to project what time batch processes would end.

It is a tool with its own intelligence that, based on a series of indicators and statistics that it receives through the scheduler, detects the times at which the batch will end at any time during the week, and projects the following 6 months. 

Constantly evolving, based on the needs of our clients, OPTI is now implemented in the 80% of the main banks in Argentina. 

How is batch process monitoring performed? OPTI application example

Shedulers are in charge of manage the tasks that are executed under a given premiseFor example, they may be executed at a specific time, once a task is completed or in response to a specific event. As this happens, different tasks and events are triggered. 

OPTI synchronizes with any of the shedulers currently available on the market, such as Control-M, Probatch and Shopdrack. 

Performs additional tracking and allows detect if there is a task that has many cancellations or suffers from different inconveniences, which can affect batch process times.

OPTI therefore constantly monitors all processes in an efficient manner, which would be very difficult to do otherwise, as there are sometimes up to 50,000 processes to monitor.

The tool warns if, for example, there is a process that will start to take longer and longer as the days go by. In this way, it prevents the problem from exploding suddenly and allows preventive and early actions to be taken to solve the problem. 

OPTI's artificial intelligence makes it possible to understand when there is a deviation, with a logical level of interpretation: an increase of 50% in a 1-minute process is not the same as one in a 40-minute process. 

The tool knows by itself when these processes are of short or long duration, if they are within the bump window and if they can affect the schedule.  

Conclusions

One of the main challenges for financial institutions is to optimize their batch processes and reduce their critical window, in order to Improve service availability and reduce execution times. 

In fact, batch process optimization is positioned as a key element to ensure continuous operation and customer satisfaction, in an increasingly digitalized and demanding environment.

The role played by IT Patagonia's OPTI tool in this process is fundamental, as it allows batch processes to be identified, monitored and improved in efficiency. 

Also know the Keys to modernizing the Mainframe that allow core ecosystems to be transformed to meet the needs of today's market.