Blog - Databorough Services Inc.

Field Resizing Conundrum IBM i

With a legacy database you will need to change every program that uses the file at the same time you change the file definition. You simply have no choice. Level checking forces you to do so.

With a modern database using I/O servers, you can minimize the effort required. (Terms & Conditions apply)

Conditions:

1. Add an unused field to the end of the existing file.
2. Replace all the file IO with an IO server that does exactly what the READ WRITE SETLL CHAIN functionality does. You still need to recompile every program that uses the file, but life gets better after introducing the I/O server. This can be accomplished with minimal programming changes.

The Secret Sauce: (parameters used to call the I/O server)

FUNCTION I-Insert U-Update D- Delete…
POINTER Pointer is provided by caller of I/O server
KEY. Key field or fields (optional depending on function)

Why locate the record format with a pointer?

1. The parameter list will not have to change every time you add a field to the file.
2. Programs using the I/O server will only have to be recompiled when an existing field or the record length changes.

Why add a 20-byte field of unused space to the record?

To reserve space in I/O server to accommodate adding fields to the file/table in the future without disturbing the running of existing programs. The only programs that need to be recompiled are the ones using the newly added fields. Programs not using the new field are not affected by the change and will continue to work without recompilation.

This investment in IO server will allow you to handle DB Changes with minimal disruption because only one program accessing the file. This also simplifies the administration of the file.

Adding a Field

1. Add the field/column to the file/table.
2. Be careful not to change the length of the row/ record length
3. Compile the I/O server and the programs requiring the new field
4. You don’t need to compile anything that is not using the new field.
5. Copy data.

Resizing a field:

Make Changes to the file/table.
Copy data from the old format to the new format.
Recompile all programs using I/O server & the I/O server

I/O servers go a long way to simplify the maintenance and administration of a file.

See our open-source page for examples of a modern DB. https://databoroughservices.com/rpg-open-source-project/

Agility Deficit Disorder (ADD) on IBM i

Do you get the chills at the thought of making database (DB) changes?

ADD symptoms include poor data quality, monolithic programs, poor response times(agility), technical debt, integration issues, and being forced to make massive changes instead of incremental improvements.

Data Quality: It’s not uncommon to find junk records in the database. You may or may not know how they got there. Improve data quality by adding constraints. Constraints are part of the operating system, that manages relationships between files. Bonus: You don’t have to write/maintain any code to do it. Just monitor for the error.

Monolithic Programs: The monster programs critical to your business that contain the business logic. Because of the complexity, this program usually requires a highly skilled resource to work on it. Extensive testing is also needed to avoid any unforeseen problems. Moving the business logic to the database via event trigger programs removes a lot of complexity from a scary program. Event Trigger programs facilitate a large part of data validation and business logic. When moved into the database engine, it will standardize where it can be found, making maintaining and testing business logic changes much safer. Changing a trigger program is much simpler, faster, and safer than changing a monolithic program.

Response times: When your users demand information currently unavailable in the database, this becomes a major issue. Adding a field to a file causes a level check and the recompile of every program referencing the file. You can dramatically minimize this pain and improve agility by adopting data-centricity. The complexity and risk can be significantly reduced, between the design and normalization of the DB, I/O servers, and the event trigger programs.

Technical debt: the implied cost of additional work that arises when software developers choose a quick and easy solution over a more comprehensive one that would take longer to implement. The question becomes, how long can you continue with a program-centric strategy?

Integration: Without moving the business logic to the database, integration with front-end GUI programs is complicated because the business logic has to be duplicated into the API. This complicates things in two ways: the back-end programmer needs to communicate the business rules for the API’s development to the API development team and then maintain it. Using event trigger programs to validate the data, minimizes API complexities and avoids maintaining the logic in multiple places.

Massive Changes: When time and pressure finally demand a DB change, it is scary, especially if it is a commonly used file such as the customer master. These changes could involve so many programs that you must freeze other developments until completion. Agility and cognitive computing (AI) demand that you transition to a data-centric application.

ADD is caused by not keeping up with technological advancements. The cure requires some education, planning, and some pain, but the benefits are substantial.

To see what a modern data-centric application looks like, visit our open-source page.

https://databoroughservices.com/rpg-open-source-project/

To better understand the solution you are welcome to book a no-strings-attached discussion.

https://calendly.com/neil-woodhams-dbs/data-centric-application-example

At Databorough Services, we understand the critical role that a well-maintained database plays in ensuring an organization’s smooth operation and efficiency. As the technology landscape continues to evolve rapidly, it becomes essential for businesses to adapt and optimize their databases to meet the demands of modern-day computing.

API’s vs Database Renovation

Functionality:

- API: An API provides a set of protocols, tools, and definitions for building and integrating software applications. It defines how software components should interact, enabling different systems to communicate with each other. (The bridge between frontend and the application code)
- Agile Data Centric Database: An agile database, on the other hand, refers to a database design approach that emphasizes flexibility and adaptability. It allows for iterative development, making it easier to achieve incremental improvements over time. By implementing event processing, constraints and the steps necessary to achieve a 3rd normal form, you are adding agility to the database.

They both perform important and separate functions in a modern application. It’s the functionality that they could share which will be the focus of this post. They both can be used to enforce the business rules. An API In a program centric application likely encapsulates business logic from pre GUI (Graphical User Interface) into the API. In a data centric world a significant amount of the business logic can be built into the database and not the API.

Flexibility:

- API: APIs provide flexibility in terms of integrating different systems and services. They allow developers to access specific functionalities or data from one system to another..
- Agile Data Centric Database: Agile databases provide flexibility by adapting to changing business needs and requirements. They can, depending on the level of agility achieved, accommodate changes in data structures, relationships, and business rules without requiring significant redesign or disruption.

The API provides the connection between the database and the GUI. The database contains and maintains the companies most valuable asset. (DATA)

Development Process:

- API: Developing an API involves defining endpoints, request-response formats, authentication mechanisms, and documentation. It typically follows a structured process and involves collaboration between teams responsible for building different components of the system.
- Agile Data Centric Database: Agile database development follows agile principles, emphasizing iterative development, continuous feedback, and collaboration between developers, business stakeholders, and database administrators. Changes are made incrementally based on evolving requirements and user feedback.

It is very likely that it is faster to develop an API than it is to renovate your database. However speed is not the only consideration. The ongoing maintenance of the application should be considered along with the mounting technical debt. When you create an API with business logic included you create a need to maintain those business rules in 2 or more places. When you associate the business rules with a file, through an event trigger program you consolidate the business logic in one place and reduce the maintenance effort in the future.

Long term, the database of an IBM i application will need to evolve to meet these future demands, focusing on scalability, performance, security, integration, flexibility, analytics, cloud compatibility, and high availability.

In summary, APIs and agile databases potentially have overlapping purposes, they both play important roles in building modern software systems. APIs should facilitate communication and integration between software components, Agile databases support flexible and adaptive data management, including business logic while accommodating changing business needs. Together, they enable the development of dynamic and responsive applications that can evolve and scale over time.

For the sake of agility it is better to make a strategic decision to enforce business rules at the database level rather than in API’s, to avoid the programming redundancy and expense of doing it more than once.

Because it can be intimidating making database changes, such as adding a field or changing a field size, and especially so during normalization. I’m going to include an example created specifically for one of our customers to safely test and implement file normalization changes.

https://databoroughservices.com/2023/04/27/low-risk-file-normalization-modernization/

You asked for Modernization

You got a new user interface. Be careful how you throw around the term modernization. You thought a new user interface was all you needed. You forgot about the DB (database). The demands of a modern application may exceed the capabilities of your DB

To make what I’m about to suggest relevant let me share some coding practices from the past

There was a time when we described files in programs. It was possible to describe a field as numeric in one program and character in another. Then we started using externally described files and it improved the quality of the data a lot.

Since the introduction of externally described files, there have been many improvements. They focus on data quality and the programming effort required to achieve optimum results.

Before event trigger programs: An ironclad method of vetting every record/row added or updated in a file/table. (No more wondering about how those orphaned records got there) Business logic is associated with the add/ change/ delete function. Data quality is checked at the point of entry. Not even DFU or SQL can get around this.

After event triggers: can keep data up to date in real-time. Traditional end-of-day, week, month, and year processing can be performed as a result of successfully adding, updating, or deleting a record/row.

Constraints: save a lot of programming because they enforce the data relationships and prevent orphaned records. You let the DB management system check the constraints you define. You could continue to perform this check in the programs or simply let the operating system do it.

I/O (input/output) servers: are border control points for Security & rollback considerations. They simplify the administration of security and data usage.

AO Foundation (my solution) can assist you by generating programs that support these modern concepts

By doing the UI first, you are increasing the technical debt because you will be forced to convert the business logic and the constraints to the language of the new UI and or use an API. An API is a repackaging of the existing business logic, which means both versions of the business rules will need to be maintained when requirements change.

Side effects of renovating the database first.: Easier to replace the user interface.

Consolidation of the business rules in trigger programs improves agility, thus making it a more strategic solution while minimizing API requirements. Not to mention the cost associated with maintaining several versions of the business rules.

If you’re ready to modernize your application, take advantage of this special offer. Don’t let outdated coding practices hold you back. Upgrade your application today.

https://databoroughservices.com/special-offer/

IT Managers: Why are you spending resources on something the operating system does?

Specifically the database: If you’re not taking advantage of current technology I mean trigger programs, constraints & I/O servers, you’re building up technical debt and it’s going to cost you big time in the future.

The sooner you accept that a modern database will serve you better than using program centric techniques to do things that are handled by the operating system, the sooner you will realize the benefits.

The problem with enforcing data integrity in monolithic programs is it requires programming effort for something that can and should be done by the operating system.

One problem with keeping the business logic in a monolithic program is a file may be added to and maintained by multiple programs. Which creates a need to keep the business logic in sync in multiple programs. By consolidating the business logic in one place (trigger program/ service programs) you negate the need to do so in the application programs. It is simply the best place to do the job

Time for a new User Interface: It doesn’t matter what language you’re going to use for the new UI. If you’re not using trigger programs and constraints you will be forced to convert the referential integrity logic and the business logic to the new language. This means new programming skills will be needed and old programming skills will not.

You also need to consider what future demands are going to be put on your data, in the form of quality and security. I’m saying it’s better to deal with the database before the user interface. You may find yourself in a situation where you are forced to redo the user interface to meet your data requirements.

To see an example of a modern database using advanced RPGLE programming techniques you can download a savf with a small 5-file application from this open source page.

Kicking the can down the road

The legacy application dilemma.

The most common way to kick the can down the road is by modernizing the user interface without modernizing the database.

Another way to kick the can down the road is to modernize your database using surrogate logicals.

Reasons why the UI is chosen to be modernized before the database?

The boss says I want a modernized application. Most of all I want a new user interface.

It’s easier to just address specific issues as opposed to taking a larger more strategic approach.

Failure to recognize the implications of doing the UI first or the benefits of the MVC architecture.

Fear of risk

Strategic Argument

Doing the user interface first is counterproductive because you will be forced into converting the business logic from the legacy user interface to the modern browser language. That means new programming skills will be required and old programming skills will not. Secondly, it doesn’t get you any closer to a modern architecture MVC (Model View Controller).

In contrast, by moving the business logic from the legacy UI to trigger programs and taking advantage of DB2 constraints you will be making good use of existing RPG skills, as well as making the task of modernizing the UI simpler.

Surrogate logicals do not give you the full functionality of SQL.

Risk can be managed by running parallel databases. Simply add an after trigger to the original file, which will execute the new trigger program version of the same file (test library). Most importantly this allows for testing until satisfied.

Download a sample of a modernized 5 file application.

Check out a special offer to have one of your files modernized.

Low risk file normalization/ modernization

This video will guide you through the necessary steps to modify your files and simultaneously maintain both versions of the database until you are satisfied the new version is accurate.

https://lnkd.in/g4gyFFnq

Sample Modern Database

This example contains several test programs used to test the modernized database. The test programs can be found in the source file SRCIOS they can be modified to add records or update records. Remember that referential constraints are in place and validations are performed in the trigger programs. You need to be ILE literate to understand and work on these programs. Don’t be afraid to contact us for assistance.

Download Sample

DB2

DB2 is a relational database management system (RDBMS) developed by IBM. It was first introduced in the 1980s and has since gone through many iterations and updates.

DB2 is designed to manage large amounts of structured data and provides a wide range of tools and features to ensure data integrity, reliability, and security. It supports SQL (Structured Query Language), which is a standard language for accessing and manipulating data in a relational database.

Some of the key features of DB2 include:

Scalability: DB2 is designed to handle large amounts of data and can scale up or down depending on your needs.
Security: DB2 provides a range of security features, including encryption, access controls, and auditing, to ensure that your data is protected.
Availability: DB2 is designed to be highly available, with features like automatic failover and backup and recovery options.
Compatibility: DB2 is compatible with a wide range of operating systems and platforms, including Windows, Linux, and UNIX.
Performance: DB2 is optimized for performance, with features like data compression and indexing to ensure that queries and transactions run quickly and efficiently.

Overall, DB2 is a powerful and versatile RDBMS that is widely used in enterprise environments for managing large amounts of structured data.

Data Scientist

A data scientist is a professional who applies scientific methods, statistical algorithms, and machine learning techniques to extract insights and knowledge from structured and unstructured data. In simpler terms, data scientists analyze and interpret complex data sets to uncover patterns, trends, and insights that can be used to inform business decisions, product development, and other applications.

Data scientists are skilled in data manipulation and have a deep understanding of programming languages, statistical modeling, and data visualization tools. They work with large amounts of data from various sources, including databases, social media platforms, and sensor networks, to extract meaningful insights.

Data scientists are essential in today’s data-driven world, where businesses and organizations require accurate insights to make informed decisions. They can work in various industries, including finance, healthcare, e-commerce, and marketing, among others.

To become a data scientist, one typically needs a strong foundation in mathematics, statistics, and computer science, along with experience in programming languages such as Python or R. Additionally, data scientists must possess critical thinking skills, attention to detail, and the ability to communicate complex data analysis in a clear and concise manner.

In summary, data scientists are highly skilled professionals who play a crucial role in interpreting and analyzing complex data sets to inform business decisions and drive innovation.