Data science is the study of data analysis by advanced technology (Machine Learning, Artificial Intelligence, Big data). It processes a huge amount of structured, semi-structured, and unstructured data to extract insight meaning, from which one pattern can be designed that will be useful to take a decision for grabbing the new business opportunity, the betterment of product/service, and ultimately business growth. Data science process to make sense of Big data/huge amount of data that is used in business. The workflow of Data science is as below:
- Objective and the issue of business determining – What is the organization’s objective, what level the organization wants to achieve, and what issue the company is facing -these are the factors under consideration. Based on such factors which type of data are relevant is considered.
- Collection of relevant data- relevant data are collected from various sources.
- Cleaning and filtering collected data – non-relevant data are removed.
- Explore the filtered, cleaned data – Finding any hidden pattern, synchronization in data, plotting them in the graph, chart, etc. form that is understandable to a non-technical person.
- Creating a model by analyzing data – creating a model, validate it.
- Visualization of finding by interpreting data or creating a model for a business person.
- Help businesspeople in making the decision and taking the step for the sack of business growth.
Data Mining: It is a process of extracting insight meaning, hidden patterns from collected data that is useful to take a business decision for the purpose of decreasing expenditure and increasing revenue. Big Data: This is a term related to extracting meaningful data by analyzing the huge amount of complex, variously formatted data generated at high speed, that cannot be handled, or processed by the traditional system. Data Expansion Day by Day: Day by day amount of data increasing exponentially because of today’s various data production sources like a smart electronic devices. As per IDC (International Data Corporation) report, new data created per each person in the world per second by 2020 will be 1.7 MB. The amount of total data in the world by 2020 will reach around 44 ZettaBytes (44 trillion GigaByte) and 175 ZettaBytes by 2025. It is being seen that the total volume of data is double every two years. The total size growth of data worldwide, year to year as per the IDC report is shown below:
Source of Big Data:
- Social Media: Today’s world a good percent of the total world population is engaged with social media like Facebook, WhatsApp, Twitter, YouTube, Instagram, etc. Each activity on such media like uploading a photo, or video, sending a message, making comment, putting like, etc create data.
- A sensor placed in various places: Sensor placed in various places of the city that gathers data on temperature, humidity, etc. A camera placed beside the road gather information about traffic condition and creates data. Security cameras placed in sensitive areas like airports, railway stations, and shopping malls create a lot of data.
- Customer Satisfaction Feedback: Customer feedback on the product or service of the various company on their website creates data. For Example, retail commercial sites like Amazon, Walmart, Flipkart, and Myntra gather customer feedback on the quality of their product and delivery time. Telecom companies, and other service provider organizations seek customer experience with their service. These create a lot of data.
- IoT Appliance: Electronic devices that are connected to the internet create data for their smart functionality, examples are a smart TV, smart washing machine, smart coffee machine, smart AC, etc. It is machine-generated data that are created by sensors kept in various devices. For Example, a Smart printing machine – is connected to the internet. A number of such printing machines connected to a network can transfer data within each other. So, if anyone loads a file copy in one printing machine, the system stores that file content, and another printing machine kept in another building or another floor can print out that file hard copy. Such data transfer between various printing machines generates data.
- E-commerce: In e-commerce transactions, business transactions, banking, and the stock market, lots of records stored are considered one of the sources of big data. Payments through credit cards, debit cards, or other electronic ways, all are kept recorded as data.
- Global Positioning System (GPS): GPS in the vehicle helps in monitoring the movement of the vehicle to shorten the path to a destination to cut fuel, and time consumption. This system creates huge data on vehicle position and movement.
- Transactional Data: Transactional data, as the name implies, is information obtained through online and offline transactions at various points of sale. The data contains important information about transactions, such as the date and time of the transaction, the location where it took place, the items bought, their prices, the methods of payment, the discounts or coupons that were applied, and other pertinent quantitative data. These are some of the sources of transactional data: orders for payment, Invoices, E-receipts and recordkeeping etc.
- Machine Data: Automatically generated machine data is produced in reaction to an event or according to a set timetable. This indicates that all of the data was compiled from a variety of sources, including satellites, desktop computers, mobile phones, industrial machines, smart sensors, SIEM logs, medical and wearable devices, road cameras, IoT devices, and more. Businesses can monitor consumer behaviour thanks to these sources. Data derived from automated sources expands exponentially in tandem with the market’s shifting external environment. These sensors are used to capture this kind of information: In a broader sense, machine data includes data that is generated by servers, user applications, websites, cloud programmes, and other sources.