Big Data: A New Era
With the cumulative growth of data sets increasing by the day, the classical data concept has been replaced by big data. Today, big data is an integral part of many areas of our lives. So, what is the difference between data and big data?
By Sibel Dinç
June 6, 2021
With the concepts of computers and the Internet that came into our lives in the last quarter of the 20th century, data sets were transferred to the digital environment. This process has gained momentum since the beginning of the 21st century, and the new datasets we have increasing by the day have become impossible to analyse with classical analysis tools. In this respect, what differentiates big data from data is its enormous data set which exceeds computability limits.
From Data to Knowledge or Vice Versa?
Well, have you ever thought about why we give so much importance to data? Why should data, which is usually made up of numbers but mostly from texts, pictures, videos and so many different sources, make sense to us? Because these data sets lead us to the concept of "Knowledge", and as Thomas Hobbes said already in the 17th century: "Knowledge is Power". Based on this concept, having power in an area means domination in that area, and power is derived from knowledge and vice versa.
It is a fact that we acquire information from the data. What about the opposite? In order to test our competence in a subject we think we know, we need to go back to data, and as we will understand here, this is a vicious cycle. Data is hungry for knowledge, and knowledge is hungry for data. This whole philosophy gives us the subtext of the meaning attributed to the data. Because each data obtained brings us one step closer to the big picture. This means getting one step closer to seeing the original.
Let's Get to Know Big Data a Little
Big Data Types are divided into three categories: structured, semi-structured and unstructured. When it comes to data, most people think of a quantitative data set; in other words, numeric data sets consisting of numbers. However, when we say big data, most of them consist of unstructured data sets. There are many sources of unstructured datasets ranging from text files to image files. The analysis of these data sets cannot be done with classical data analysis methods. First of all, powerful analysis tools should be used in order to introduce these data sets to big data analysis tools, and often even to be subjected to coding and analysis. As of 2021, 15 of the big data analysis tools are listed
here. The most famous of these are undoubtedly Hadoop, Knime, RapidMiner, Tableau and R. Especially in recent years, it has been seen that the Tableau program is frequently preferred for data visualization.
Mini TIP! If you need a data set for your study or if you want to contribute to the dissemination of information by sharing your data set with other researchers, you can use the
Kaggle website, which many researchers frequently use.
Big Data is Killing Statistics Science!
In fact, it would not be wrong to say that big data surpasses statistics in a way. Data analysis has been conducted through the methods employed in statistics science for years. However, as it is known, statistics do not work with the whole data set and analyses are made with the fittest sample obtained from the population. Of course, since statistics does not work with the entire data set, a margin of error should always be included. At this point, instead of choosing a sampling from the big data population, use the population directly and actually resets the concept of margin of error if there is no
systematic error.
Areas of Use
We know that big data is used in many areas. However, we can say that the greatest momentum in terms of digital society has been achieved with the increase in the use of social media platforms. Recognizing the importance of people's behaviour and preferences is essential for many companies that try to increase their consumer base and gain loyal consumers through social media tools.
The most striking example in this regard is Facebook. As a result of the research conducted by Cambridge and Stanford University's researchers in the news published by
The New York Times, it has been concluded that Facebook knows us better than ourselves because of the data it has. Here's a spooky part of big data! On the bright side, while social platforms such as Spotify and Netflix also gather user data, the use of such data is intended exclusively for recommendation purposes. This is the good news for us!
What About the Data Privacy?
If an application knows us better than ourselves, it means that there is a lot of data about us. So how much of this data was taken with our permission? Also, are we sure that this data is not shared with other companies or third parties? Lots of problems are pending in this regard. Data privacy is a very important concept. Although the
General Data Protection Regulation (GDPR) regulation of the European Union aims at tackling certain issues, this data protection framework has validity for limited countries and still has deficits in many areas.
The concept of data privacy in the digital medium is an issue that should be given utmost importance and still maintains its warmth. On the other hand, if we do not go far enough in data protection, it will lead to the exploitation of the data of each individual using the internet, not only in a specific area but also in many areas to be used for malicious purposes. For an up-to-date article on this topic, see
here.
Finally, to take a broad perspective on the concept of Big Data, I recommend you take a look at the following watch list:
Sibel Dinç is ongoing at King's College London in Big Data in Culture & Society MA program. She worked as a research assistant in Turkey. She is interested in social media studies. She takes an active role in The Institute for Internet and Just Society as a researcher.
Read More
Watch Our Episodes