Analyzing Titanic Dataset¶
Prepared by Mahsa Sadi on 2020 - 06 - 23
In this notbook, we perform five steps on the Titanic data set:
- Reading Data
- Visualizing Data
- Analyzing Data
- Cleaning Data
- Modeling Data: To model the dataset, we apply logistic regression.
In [1]:
import pandas
In [2]:
import sklearn
In [3]:
from sklearn import model_selection
In [4]:
from sklearn.linear_model import LogisticRegression
In [5]:
from sklearn.metrics import classification_report
In [6]:
from sklearn.metrics import confusion_matrix
In [7]:
from sklearn.metrics import accuracy_score
In [8]:
import numpy
In [9]:
import seaborn
In [10]:
import matplotlib.pyplot
In [11]:
%matplotlib inline
In [12]:
import math
Reading the Data Set¶
The data set is in a csv file in the same directory as the code.
In [13]:
data_set = pandas.read_csv ('titanic.csv')
In [14]:
data_set.head (20)
Out[14]:
In [15]:
data_set.info()
Summerizing the Data Set¶
In [16]:
# how many rows and columns?
data_set.shape
Out[16]:
In [17]:
#how many survived and how many died?
data_set.groupby ('Survived').size()
Out[17]:
Visualizing Data¶
In [18]:
seaborn.countplot (x ="Survived", data = data_set)
Out[18]:
=> More people died than survived
In [19]:
seaborn.countplot (x ="Survived", hue = "Sex", data = data_set)
Out[19]:
=> More men died than women
In [20]:
seaborn.countplot (x ="Survived", hue = "Pclass", data = data_set)
Out[20]:
=> More 3rd class passengers died than others.
In [21]:
data_set ["Age"]. hist ()
Out[21]:
=> The majority of the passengers were 18 to 40 years old.
In [22]:
data_set ["Fare"].hist (bins = 20, figsize = (10,5))
Out[22]:
In [23]:
seaborn.countplot (x ="Siblings/Spouses Aboard", data = data_set)
Out[23]:
In [24]:
seaborn.countplot (x ="Parents/Children Aboard", data = data_set)
Out[24]:
=> The majority of passengers were alone or with one family member.
In [25]:
seaborn.boxplot( x = 'Pclass', y = 'Age', data = data_set)
Out[25]:
=> First and second class passengers were older.
In [26]:
seaborn.heatmap (data_set.isnull(),yticklabels = False, cmap = 'YlGnBu')
Out[26]:
Cleaning the Dataset¶
In [27]:
cleaned_data_set = data_set.drop (columns = ['Name'], axis = 1)
In [28]:
cleaned_data_set.head (5)
Out[28]:
In [29]:
#Idnetify the cells with missing dada
cleaned_data_set.isnull().head (5)
Out[29]:
In [30]:
cleaned_data_set.isnull().sum ()
Out[30]:
To apply logistic regression all the columns should have categorial or numerical values.
In [31]:
pandas.get_dummies (cleaned_data_set ['Sex']).head (5)
Out[31]:
In [32]:
binary_sex = pandas.get_dummies (cleaned_data_set ['Sex'],drop_first = True)
In [33]:
binary_sex.head (5)
Out[33]:
In [34]:
pandas.get_dummies (cleaned_data_set ['Pclass']).head (5)
Out[34]:
In [35]:
binary_Pclass = pandas.get_dummies (cleaned_data_set ['Pclass'],drop_first = True)
In [36]:
binary_Pclass.head (5)
Out[36]:
In [37]:
modified_data_set = pandas.concat ([cleaned_data_set, binary_sex, binary_Pclass], axis = 1)
In [38]:
modified_data_set.head (5)
Out[38]:
In [39]:
final_data_set = modified_data_set.drop (columns = ['Sex', 'Pclass'])
final_data_set.head (5)
Out[39]:
In [40]:
final_data_set.info()
Modeling the DataSet¶
In [41]:
Y= final_data_set ['Survived']
X= final_data_set.drop (['Survived'], axis = 1 )
In [42]:
X.head (5)
Out[42]:
In [43]:
Y.head (5)
Out[43]:
In [44]:
test_set_size = 0.2
seed = 1
In [45]:
X_train, X_test, Y_train, Y_test = model_selection.train_test_split (X,Y, test_size = test_set_size, random_state = seed)
In [46]:
model = LogisticRegression (solver = 'liblinear')
In [47]:
model.fit (X_train, Y_train)
Out[47]:
In [48]:
predictions = model.predict (X_test)
In [49]:
report = classification_report (Y_test, predictions)
In [50]:
print (report)
In [51]:
print (confusion_matrix (Y_test, predictions))
In [52]:
accuracy_score (Y_test, predictions)
Out[52]: