Batch Processing

EMMA MOORE

Batch Processing

Using batch processing input data is grouped together into batches. No data is processed until a complete batch has been put together. Then all the data in a batch is processed in one go. There may be a long time delay between data being input into the system and the data being processed.

In this type of system the important data that the computer stores all the time is kept in a file called the master file. The data in the master file is sorted into order using one of the fields in the records in the file, known as the primary key field. The primary key field must uniquely identify each record in the file.

Each piece of input data (which will update the contents of the master file) is known as a transaction. All the input data is put together into a batch in a file called the transaction file. There are three different types of transaction that any processing system will have to cope with. They are:

Add a new record to the master file.
Delete a record from the master file.
Amend / update an existing record in the master file.

At some predetermined time (e.g. the end of the day or week) the computer system will process the data stored in the transaction file and make any changes that are necessary to the master file as a result of the transactions. This will produce an updated master file and an error report detailing any transactions that could not be processed for some reason.

Before this processing can take place the transactions must be prepared for processing. To do these two operations must be carried out:

Validating: Each transaction that is entered is validated to check that it is unlikely to halt processing by causing an error. It is very important that processing is not halted in a batch processing system because processing often takes place overnight without a human operator being present. If an invalid transaction caused the processing to stop then a whole night's processing time could be lost. Transactions are usually validated as the operator keys them in. Any errors that are found can be reported and corrected straight away.

Sorting: The data in the transaction file is sorted into order using the same primary key field as the master file. Sorting cannot take place until the whole batch of transactions entered and stored in a file. The transaction and master files need to be put into the same order before processing occurs because the two files are processed sequentially. The files are usually stored on a serial access medium such as magnetic tapes. If the records on the tapes were not sorted into order then whenever a transaction record was read from the transaction file the computer system would have to waste a lot of time searching for the matching record in the master file.

How often batches of data are processed will depend upon the application (i.e. the job that is being done). Batch processing is usually only carried out on mainframe computers and when very large volumes of data have to be processed in a database.

Above is an example this of a data flow diagram, which shows what happens in a batch processing system.

BATCH PROCESSING ALGORITHM

During the processing phase (represented in the bottom half of the flowchart above) all of the records are copied from the master file to the new master file. Any changes that must ...

This is a preview of the whole essay

Above is an example this of a data flow diagram, which shows what happens in a batch processing system.

BATCH PROCESSING ALGORITHM

During the processing phase (represented in the bottom half of the flowchart above) all of the records are copied from the master file to the new master file. Any changes that must be made to the master file are detailed in the transaction file and carried out when the records are copied to the new master file. The processing that takes place for each transaction in the transaction file will depend upon the type of transaction:

There is nothing to stop a batch processing system being implemented using disks for storage rather than magnetic tapes. Tapes are usually used because they are cheaper to purchase, can store more data and can read and write data faster than disks.

BATCH PROCESSING EXAMPLE: PAYROLL SYSTEM

A payroll system might be implemented using batch processing. The master file could contain the records for all of company’s employees, including their employee number, rates of pay and how much they have been paid so far this year. The records in the master file are sorted using the employee number as the primary key field. The input data put into the transaction file would consist of records showing how many hours each employee had worked in the current week. Sometimes transactions would be used to add a new record if a new employee started or delete a record when an employee left the company.

At the end of the week the transaction file would be processed. Before processing it would have to be sorted into the same order as the master file, i.e. records in order of employee number.

The computer would then process the transactions, using the information about how many hours each employee has worked this week (from the transaction file) and their rates of pay (from the master file) to calculate the employee's wages for the week. Payslips can then be printed and the master file can be updated to increase the amount paid so far this year by the wages paid this week. An error report will also be produced.

Transaction Processing (or Interactive or On-Line or Pseudo Real Time Processing)

For some applications the master file needs to be kept up to date all of the time. For example in a travel agents whenever a seat is booked on a flight the number of seats that remain available on the flight must be reduced by one immediately. If this update was not done until the end of a day (as might happen in a batch processing system) then the flight could become overbooked.

Transaction processing systems are used whenever the master file must be kept up to date. A transaction processing system operates like this:

When a transaction is entered it is placed in a queue of transactions waiting to be made. The transactions are processed in the order that they are placed into the queue. If there are many people using the system at the same time then there could be lots of transactions being made from different computer terminals.

The computer system will process one transaction at a time. Once the system starts processing a transaction it will not process any other transactions until the current transaction is finished. When a transaction is processed the master file is updated immediately. Therefore the master file is always kept up to date.

Well-designed transaction processing systems will process a transaction within a few seconds of the transaction being inputted.

Transaction processing systems need to use direct access files. In a direct access file any record in the file can be updated directly, without having to read through all of the records that come before it in the file. Serial access media such as magnetic tape cannot be used. The time required to find the record in the master file that a transaction related to would make processing incredibly slow.

Transaction processing is interactive, i.e. processing takes place as a "conversation" between the user and the computer: The computer responds to the user’s input by outputting some data before the user can input any more data.

Transaction Processing Example: Point of Sale Stock Control System

A supermarket uses a Point of Sale (POS) terminal to keep track of the goods that it has in stock and to produce bills for customers. Whenever an item is sold the product code of the item is read into a till by a barcode scanner. Sometimes the barcode is not scanned properly. If this happens the till makes a beeping sound and the product code must be rescanned or entered using a keyboard.

Once the product code has been entered the till looks up the price and name of the product in the supermarket's stock database. This information is printed on the customer's receipt. It also updates the product's record in the stock file to indicate that the product has been sold.

This system can be depicted by this system flowchart:

The system has to be a transaction processing system because a customer at a till would not be prepared to wait for a batch processing system to look up the price and name of the product

REAL-TIME PROCESSING

For some tasks a computer must process information to meet some real world time deadline. The time deadline must be met regardless of how much work the computer has to do. Usually real time computer systems are required to process information very quickly. Most real time systems appear to process data instantly. A computer system which must process data extremely quickly to meet a real world deadline is known as a real time system.

For example, consider a control system that is used to fly a guided missile to hit a target. Once the missile is launched the control system must guide it to its target. The missile's guidance system identifies its position by examining the contours / features of the ground it is flying over. This input data is used to ensure that the missile follows the path it has been programmed with. Using the inputs the system will decide how to move ailerons/fins to change direction and control the speed of the rocket motor.

Because a missile flies very fast the inputs must be processes very quickly to affect the outputs. If it took two seconds for the missile to change its direction to avoid an unexpected object then the missile would probably hit the object before the change of direction could take place. The data must be processed in thousandths of a second to avoid a disaster. There is a real world time limit on how long the computer system can take to process data without having terrible consequences. Therefore a missile control system has to be a real time system.

Most real time systems are . Other examples include the cooling system in a nuclear reactor, which prevents the reactor overheating, or any system, which must control a dangerous chemical reaction.

Real time systems are very fast but they do not process data instantly. Any processing will take some time. However real time systems will process data very quickly. To ensure that any input data can be processed sufficiently quickly real time systems must incorporate a lot of redundancy. This means having much more hardware than is required to process the expected amount of input data. The extra hardware is known as redundant hardware because it is not normally used. Under normal circumstances only a small fraction of the hardware is required. In extreme circumstances when there is an unusually large amount of input data to process the redundant hardware can be used. Therefore the system will not slow down and will still be capable of meeting deadlines.

Having to incorporate lots of redundant hardware makes real time systems very expensive. They are usually only used in safety critical situations.