What do you mean by data processing?
Data processing is the conversion of data into usable and desired form. This conversion or “processing” is carried out using a predefined sequence of operations either manually or automatically. Most of the data processing is done by using computers and thus done automatically. The output or “processed” data can be obtained in different forms like image, graph, table, vector file, audio, charts or any other desired format depending on the software or method of data processing used.
Fundamentals of data processing & how data is processed
Data processing is undertaken by any activity which requires a collection of data. This data collected needs to be stored, sorted, processed, analyzed and presented. This complete process can be divided into 6 simple primary stages which are:
- Data collection
- Storage of data
- Sorting of data
- Processing of data
- Data analysis
- Data presentation and conclusions
Once the data is collected the need for data entry emerges for storage of data. Storage can be done in physical form by use of papers, in notebooks or in any other physical form. With the emergence and growing emphasis on Big Data & Data Mining the data collection is large and a number of operations need to be performed for meaningful analysis and presentation, the data is stored in digital form. Having the data into digital form enables the user to perform a large number of operations in small time and allows conversion into different types. The user can thus select the output which best suits the requirement.
This continuous use and processing of data follow cycle called as data processing cycle and information processing cycle which might provide instant results or take time depending upon the need of processing data. The complexity in the field of data processing is increasing which is creating a need for advanced techniques.
Storage of data is followed by sorting and filtering. This stage is profoundly affected by the format in which data is stored and further depends on the software used. General daily day and noncomplex data can be stored as text files, tables or a combination of both in Microsoft Excel or similar software. As the task becomes complex which requires performing specific and specialized operations they require different data processing tools and software which is meant to cater to the peculiar needs.
Related: Data presentation and analysis
Storing, sorting, filtering and processing of data can be done by single software or a combination of software whichever feasible and required. Data processing thus carried out by software is done as per the predefined set of operations. Most of the modern-day software allows users to perform different actions based on the analysis or study to be carried out. Data processing provides the output file in various formats.
Different types of output files obtained as “processed” data
- Plain text file – These constitute the simplest form or processed data. Most of these files are user readable and easy to comprehend. Very negligible or no further processing is these type of files. These are exported as notepad or WordPad files.
- Table/ spreadsheet – This file format is most suitable for numeric data. Having digits in rows and columns allows the user to perform various operations like filtering & sorting in ascending/descending order to make it easy to understand and use. Various mathematical operations can be applied when using this file output.
- Charts & Graphs – Option to get the output in the form of charts and graphs is handy and now forms standard features in most of the software. This option is beneficial when dealing with numerical values reflecting trends and growth/decline. Though there are ample charts and graphs are available to match diverse requirements there exists situation when there is a need to have a user-defined option. In case no inbuilt chart or graph is available then the option to create own charts, i.e., custom charts/graphs come handy.
- Maps/Vector or image file – When dealing with spatial data the option to export the processed data into maps, vector and image files is of great use. Having the information on maps is of particular use for urban planners who work on different types of maps. Image files are obtained when dealing with graphics and do not constitute any human readable input.
- Other formats/ raw files – These are the software specific file formats which can be used and processed by specialized software. These output files may not be a complete product and require further processing. Thus there will need to perform multiple data processing.
Methods of data processing
- Manual data processing: In this method data is processed manually without the use of a machine, tool or electronic device. Data is processed manually, and all the calculations and logical operations are performed manually on the data.
- Mechanical data processing – Data processing is done by use of a mechanical device or very simple electronic devices like calculator and typewriters. When the need for processing is simple, this method can be adopted.
- Electronic data processing – This is the modern technique to process data. The fastest and best available method with the highest reliability and accuracy. The technology used is latest as this method used computers and employed in most of the agencies. The use of software forms the part of this type of data processing. The data is processed through a computer; Data and set of instructions are given to the computer as input, and the computer automatically processes the data according to the given set of instructions. The computer is also known as electronic data processing machine.
Types of data processing on the basis of process/steps performed
- Batch Processing
- Real-time processing
- Online Processing
What makes processing of data important
Nowadays more and more data is collected for academic, scientific research, private & personal use, institutional use, commercial use. This collected data needs to be stored, sorted, filtered, analyzed and presented for it to be of any use. This process can be simple or complex depending on the scale at which data collection is done and the complexity of the results which are required to be obtained. The time consumed in obtaining the desired result depends on the operations which need to be performed on the collected data and on the nature of the output file required to be obtained. This problem becomes starker when dealing with the very large volume of data such as those collected by multinational companies about their users, sales, manufacturing, etc.
The need for data processing becomes more and more critical in such cases. In such cases, data mining and data management come into play without which optimal results cannot be obtained. Each stage starting from data collection to presentation has a direct effect on the output and usefulness of the processed data.