Overview and characteristics of big data

Today, we are going to explore the characteristics, size, and analyst capabilities of big data. This content is also
part of Big Data Analysis Planning, the first course among big data analysis companies.

Big data features
Big data is the result of extracting value from data, including large amounts of structured and unstructured data, and analyzing the results. Big data can be understood as a concept including existing database management tools and big data analysis and visualization technology.

Big data means big data and has several characteristics along with a quantitative aspect.

- 3V : Vloume, Variety, Velocity
- 5V : 3V + Value, Veracity
- 7V : 5V + Validity, Volatility

 

Korea Big Data News

 

 

division	Main Content
Volume	- Increasing amount of digital information due to technological advancement and dailyization of IT services - Data set size increased from tens of TB to several PB - Beyond the limit of acceptance of data collection/management/processing software
Variety	- Increasing data types such as logs, SNS, location, purchase, and IOT - Increasing unstructured data types such as text and multimedia - Includes structured data (address, name, age, etc.) stored in fixed fields
Velocity	- The amount and content of data is constantly changing - Increasing real-time information such as objects (sensors, monitoring, etc.) and streaming information - Data generation, movement (distribution) speed increase
Value	- It's important to find the value implied inside large amounts of data - Difficult to work with existing DBMS or data analysis system
Veracity	- Data Accuracy, Validity, and Reliability (Data Confidence Level) - The attribute that determines whether the data is valid for making a decision.

Among them, finding the implied value and making decisions appropriately seems to be thekey. What is the standard for the size of big data?

Big data size and data unit
It ranges from terabytes (TB) to several petabytes (PB), and is characterized by constantly changing size.
Byte <KB <MB <GB <TB <PB <EB <ZB <YB <BB <GeB

There is a domestic public data portal site that provides such big data. I thought that it would be good to do training to analyze and utilize the big data provided by the site. I decided to visit the site next time.


Korean public data site
- http://data.go.kr

 

Korean public data portal site

 

I think that in order to become a person who can use this data, professional knowledge in the field of analysis is essential among mathematical knowledge and programming skills. Because technology is, after all, a means of insight.


Data analyst competency
Mathematical knowledge (probability, statistics), programming ability, professional knowledge in the field of analysis, creative and logical thinking, collaboration with people in other fields, composure and communication skills are required.

The data analyst seems to be a job that requires a variety of competencies, so it is difficult, but it seems to be more attractive. Next, we will look at the types of big data.

Then have a nice day today too!