In the era of Information and communication technology, the concept of Big Data represents the massive amount of fast moving data in different formats generated from various sources. To be precise, the world is ‘digitalized’ to the point that we have to interact with machines, instruments, and the internet in almost everything we do. While doing so, we leave digital footprints or a trail of evidence of their activities. These digital footprints or evidence are the very things that produce Big Data. Facebook data feeds, survey records, click streams on a mobile app or a web site, business transactions in a financial institute, data from IoT devices all fall into Big Data streams. When analyzed, these big data streams produce insights that were previously inaccessible. These insights or knowledge becomes immeasurably valuable when making decisions.

Big Data comes from the following three types of sources.

  • Streaming Data – Data generated in connected devices (eg. IoT devices)
  • Social Media Data – Data generated in Social Media. (These type of Big Data are especially useful in Marketing, Sales and Support.)
  • Publicly available Data – Open Data sources, very large in size. (eg. Data.gov, World Factbook, Open Data Portal)

Characteristics of Big Data.

 

Introduction: what is Big Data?

Big Data itself reveals the meaning of its concept. The term Big Data came to attention in rather recent times like at the beginning of the year 2001, but “data ” has been the root for generating knowledge and wisdom since the beginning of the human civilization. Information derived from data is used to generate knowledge which then becomes the wisdom in the human mind.

In the era of Information and communication technology, the concept of Big Data represents the massive amount of fast moving data in different formats generated from various sources. To be precise, the world is ‘digitalized’ to the point that we have to interact with machines, instruments, and the internet in almost everything we do. While doing so, we leave digital footprints or a trail of evidence of their activities. These digital footprints or evidence are the very things that produce Big Data. Facebook data feeds, survey records, click streams on a mobile app or a web site, business transactions in a financial institute, data from IoT devices all fall into Big Data streams. When analyzed, these big data streams produce insights that were previously inaccessible. These insights or knowledge becomes immeasurably valuable when making decisions.

Big Data comes from the following three types of sources.

  • Streaming Data – Data generated in connected devices (eg. IoT devices)
  • Social Media Data – Data generated in Social Media. (These type of Big Data are especially useful in Marketing, Sales and Support.)
  • Publicly available Data – Open Data sources, very large in size. (eg. Data.gov, World Factbook, Open Data Portal)

Characteristics of Big Data.

Doug Laney, an industry analyst owns the credits for the first formal definition of Big Data. His definition is known as “the 3Vs”.It describes the three fundamental characteristics of Big Data – Variety, Velocity and Volume.

Variety
Big data is generated in a variety of sources thus they are in both structured and unstructured formats. Structured Big Data generally referrers to data stored in traditional databases. Unstructured Big Data can be text, financial transactions, emails, audio or video data.

Velocity
Big Data is generated and received at such an incredible speed (because of the huge number of interactions previously mentioned) that represent the changing world. Thus they must be processed at a matching speed (almost real time) to get their real advantage.

Volume
The most important characteristic of Big Data is its’ massive volume. Their volume can differ from tens of terabytes to hundreds of petabytes. The higher volume means more knowledge but it also makes Big Data impossible to process and store using normal, traditional data processing software and data warehouses. Big Data Analysis is an expensive process done with specialized, high powered computer systems and applications but the result is of very high value.

Big Data Analysis

Big Data analysis is meant to uncover hidden patterns, unknown co-relations which provides insights about market trends, customer preferences, and buying habits etc.

Big Data Analysis Tools.

Professions like Data Scientists, Data Analysts, Predictive Modelers, and Statisticians use various kinds of tools in Big Data Analysis.

 Predictive Analysis tools – predicts future events, activities, results, trends.
 Machine Learning tools – Helps computer programs to learn for themselves.
 Deep Learning tools – Allows automation of predictive analytics.

Education, health, business, banking and all other industries use results of Big Data Analysis to make changes aimed at progress.

Education
Universities and schools produce a large ocean of data of their students, teachers, courses and extracurricular activities. Analyzing Students’ grades, their preferred courses and extracurricular activities help to measure the students’ progress, their strengths and weaknesses, the effectiveness of teaching and evaluation methods. More importantly, it helps them choose a carrier.

Business
Business organizations gather data about their customers’ needs and wants, preferences, buying habits. Then they analyze that data to understand their customer better to increase customer satisfaction. In return, more profits and revenue opportunities are generated.

Health
Health industry also generates a large amount of data of their patients’ medical records, results from various medical tests and treatments etc. Analyzing this data helps to evaluate the effectiveness of medical treatments and tests, predicting outbreaks of epidemics, better diagnosis of illnesses etc.

In general, organizations having insights/knowledge are benefited from the following advantages.

 Cost and time reduction.
 Development of new products.
 Smart decision making.
 New revenue opportunities.
 Effective marketing.
 Better customer service.
 Higher operational efficiency.

Organizations that handle Big Data effectively, obviously have a better chance at survival and profitability than the organizations that don’t.Because, Knowledge is power. Big data is power.