Uber Rides Data Analysis using Python

In this article, we will use Python and its different libraries to analyze the Uber Rides Data.

Importing Libraries

The analysis will be done using the following libraries : 

• Pandas:  This library helps to load the data frame in a 2D array format and has multiple functions to perform analysis tasks in one go.
• Numpy: Numpy arrays are very fast and can perform large computations in a very short time.
• Matplotlib / Seaborn: This library is used to draw visualizations.

To importing all these libraries, we can use the  below code :

Python3

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns

Importing Dataset

After importing all the libraries,  download the data using the link.

Once downloaded, you can import the dataset using the pandas library.

Python3

dataset = pd.read_csv("UberDataset.csv")
dataset.head()

Output : 

To find the shape of the dataset, we can use dataset.shape

Python3

dataset.shape

Output : 

(1156, 7)

To understand the data more deeply, we need to know about the null values count, datatype, etc. So for that we will use the below code.

Python3

dataset.info()

Output : 

Data Preprocessing

As we understood that there are a lot of null values in PURPOSE column, so for that we will me filling the null values with a NOT keyword. You can try something else too.

Python3

dataset['PURPOSE'].fillna("NOT", inplace=True)

Changing the START_DATE and END_DATE to the date_time format so that further it can be use to do analysis.

Python3

dataset['START_DATE'] = pd.to_datetime(dataset['START_DATE'],
                                       errors='coerce')
dataset['END_DATE'] = pd.to_datetime(dataset['END_DATE'],
                                     errors='coerce')

Splitting the START_DATE to date and time column and then converting the time into four different categories i.e. Morning, Afternoon, Evening, Night

Python3

from datetime import datetime

dataset['date'] = pd.DatetimeIndex(dataset['START_DATE']).date
dataset['time'] = pd.DatetimeIndex(dataset['START_DATE']).hour

#changing into categories of day and night
dataset['day-night'] = pd.cut(x=dataset['time'],
                              bins = [0,10,15,19,24],
                              labels = ['Morning','Afternoon','Evening','Night'])

Once we are done with creating new columns, we can now drop rows with null values.

Python3

dataset.dropna(inplace=True)

It is also important to drop the duplicates rows from the dataset. To do that, refer the code below.

Python3

dataset.drop_duplicates(inplace=True)

Data Visualization

In this section, we will try to understand and compare all columns.

Let’s start with checking the unique values in dataset of the columns with object datatype.

Python3

obj = (dataset.dtypes == 'object')
object_cols = list(obj[obj].index)

unique_values = {}
for col in object_cols:
  unique_values[col] = dataset[col].unique().size
unique_values

Output : 

{'CATEGORY': 2, 'START': 177, 'STOP': 188, 'PURPOSE': 11, 'date': 294}

Now, we will be using matplotlib and seaborn library for countplot the CATEGORY and PURPOSE columns.

Python3

plt.figure(figsize=(10,5))

plt.subplot(1,2,1)
sns.countplot(dataset['CATEGORY'])
plt.xticks(rotation=90)

plt.subplot(1,2,2)
sns.countplot(dataset['PURPOSE'])
plt.xticks(rotation=90)

Output : 

Let’s do the same for time column, here we will be using the time column which we have extracted above.

Python3

sns.countplot(dataset['day-night'])
plt.xticks(rotation=90)

Output : 

Now, we will be comparing the two different categories along with the PURPOSE of the user.

Python3

plt.figure(figsize=(15, 5))
sns.countplot(data=dataset, x='PURPOSE', hue='CATEGORY')
plt.xticks(rotation=90)
plt.show()

Output : 

Insights from the above count-plots : 

• Most of the rides are booked for business purpose.
• Most of the people book cabs for Meetings and Meal / Entertain purpose.
• Most of the cabs are booked in the time duration of 10am-5pm (Afternoon).

As we have seen that CATEGORY and PURPOSE columns are two very important columns. So now we will be using OneHotEncoder to categories them.

Python3

from sklearn.preprocessing import OneHotEncoder
object_cols = ['CATEGORY', 'PURPOSE']
OH_encoder = OneHotEncoder(sparse=False)
OH_cols = pd.DataFrame(OH_encoder.fit_transform(dataset[object_cols]))
OH_cols.index = dataset.index
OH_cols.columns = OH_encoder.get_feature_names()
df_final = dataset.drop(object_cols, axis=1)
dataset = pd.concat([df_final, OH_cols], axis=1)

After that, we can now find the correlation between the columns using heatmap.

Python3

plt.figure(figsize=(12, 6))
sns.heatmap(dataset.corr(),
            cmap='BrBG',
            fmt='.2f',
            linewidths=2,
            annot=True)

Output : 

Insights from the heatmap:

• Business and Personal Category are highly negatively correlated, this have already proven earlier. So this plot, justifies the above conclusions.
• There is not much correlation between the features.

Now, as we need to visualize the month data. This can we same as done before (for hours). 

Python3

dataset['MONTH'] = pd.DatetimeIndex(dataset['START_DATE']).month
month_label = {1.0: 'Jan', 2.0: 'Feb', 3.0: 'Mar', 4.0: 'April',
               5.0: 'May', 6.0: 'June', 7.0: 'July', 8.0: 'Aug',
               9.0: 'Sep', 10.0: 'Oct', 11.0: 'Nov', 12.0: 'Dec'}
dataset["MONTH"] = dataset.MONTH.map(month_label)

mon = dataset.MONTH.value_counts(sort=False)

# Month total rides count vs Month ride max count
df = pd.DataFrame({"MONTHS": mon.values,
                   "VALUE COUNT": dataset.groupby('MONTH',
                                                  sort=False)['MILES'].max()})

p = sns.lineplot(data=df)
p.set(xlabel="MONTHS", ylabel="VALUE COUNT")

Output :

Insights from the above plot : 

• The counts are very irregular.
• Still its very clear that the counts are very less during Nov, Dec, Jan, which justifies the fact that  time winters are there in Florida, US.

Visualization for days data.

Python3

dataset['DAY'] = dataset.START_DATE.dt.weekday
day_label = {
    0: 'Mon', 1: 'Tues', 2: 'Wed', 3: 'Thus', 4: 'Fri', 5: 'Sat', 6: 'Sun'
}
dataset['DAY'] = dataset['DAY'].map(day_label)

Python3

day_label = dataset.DAY.value_counts()
sns.barplot(x=day_label.index, y=day_label);
plt.xlabel('DAY')
plt.ylabel('COUNT')

Output :

Now, let’s explore the MILES Column .

We can use boxplot to check the distribution of the column.

Python3

sns.boxplot(dataset['MILES'])

Output :

As the graph is not clearly understandable. Let’s zoom in it for values lees than 100.

Python3

sns.boxplot(dataset[dataset['MILES']<100]['MILES'])

Output :

It’s bit visible. But to get more clarity we can use distplot for values less than 40.

Python3

sns.distplot(dataset[dataset['MILES']<40]['MILES'])

Output :

Insights from the above plots :

• Most of the cabs booked for the distance of 4-5 miles.
• Majorly people chooses cabs for the distance of 0-20 miles.
• For distance more than 20 miles cab counts is nearly negligible.