![\"Photo](\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2023\/08\/02mdodL6yNGWhe1jK-scaled.jpg\")
In a nutshell, this machine learning tool’s primary objective is to classify collisions that will most likely require a callout (medical, tow, or both) based on other known factors. By leveraging this tool, responders would be able to efficiently allocate their resources across different parts of the city, based on various conditions such as weather and time of day.<\/p>\n
For such a tool to be accurate and effective, a large data source would be needed to make predictions from the historical data – thankfully the city of Chicago already has such a resource (the Chicago Data Portal<\/a>),<\/strong> so this data will be used as the test case.<\/p>\n Implementing these kinds of predictive models would certainly improve preparedness and response time efficiency when dealing with collisions on city streets. By gaining insights into the underlying patterns and trends within the collision data, we can work towards fostering safer road environments and optimising emergency services.<\/p>\n I go into the details of data cleaning, model building, fine tuning and evaluation below, before analysing the model’s results and drawing conclusions. A link to the github folder for this project, which includes a jupyter notebook and a more comprehensive report on the project, can be found here<\/a><\/strong>.<\/p>\n I’ve listed the basic data analysis libraries used in the project below; standard libraries such as pandas and numpy were used throughout the project, along with matplotlib’s pyplot and seaborn for visualisation. Additionally, I used the missingno library to identify gaps in the data – I find this library incredibly useful for visualising missing data in a dataset, and would recommend it for any data science project involving dataframes:<\/p>\n Functions from the machine learning module SciKit learn (sklearn) were imported to build the machine learning engine. these functions are shown here – I will describe the purpose of each of these functions in the Classification Model section later:<\/p>\n The data for this project were all imported from the Chicago Data Portal, from two sources:<\/p>\n With both datasets imported, they can now be merged to add district data to the final analysis. This is done using the .merge() function in pandas – I used an inner join on both dataframes to capture all information in both, using the beat data in both as the join key (listed as beat_of_occurrence in the traffic crashes dataset, and BEAT_NUM in the police beats dataset):<\/p>\n A quick look at the information provided from the .info() function shows a number of columns with sparse data. This can be visualized using the missingno matrix function:<\/p>\n This displays a matrix of missing data in all columns, as can be seen here:<\/p>\n By dropping columns with sparse data, a much cleaner dataset can be extracted; the columns to drop are defined in a list and then removed from the dataset using the .drop() function:<\/p>\n This leads to a much cleaner msno matrix:<\/p>\n Looking at the data for latitude and longitude, a small handful of rows had null values, and others mistakenly had zero values (most likely a reporting error):<\/p>\n These would cause errors in training the model, so I removed them:<\/p>\n With the data adequately cleaned, I was able to progress with developing the classification model.<\/p>\n Before proceeding with the machine learning model some exploratory data analysis (EDA) needs to be performed – each of the columns of the data frame are plotted on a histogram, with bins of 50 to show the distribution of the data. Histograms are useful in the EDA step for a number of reasons, namely that they give an overview of the data distribution, help to identify outliers, and ultimately assist in making decisions on feature engineering:<\/p>\n A cursory look at the column histograms indicates that the latitude data is bimodal, while the longitude data is rightly skewed. This will need to be standardized so that it can be better applied for machine learning purposes.<\/p>\n Additionally, it appears the crash hour column is cyclic in nature – this can be transformed using a trigonometric function (for example sine).<\/p>\n Scaling is a technique used in data preprocessing to standardise features so they have similar magnitudes. This is particularly important for machine learning models, since models are generally sensitive to the scale of input features. I’ve defined the StandardScaler() function to act as the scaler in this model – this scaling function transforms the data so that it has a mean of 0 and standard deviation of 1.<\/p>\n For data with a skewed or bimodal distribution, scaling can be done using logarithmic functions. Log functions make skewed data more symmetrical and reduce the tail on the data – this is useful when dealing with outlier values. I scaled the latitude and longitude data in this way; as the longitude data is all negative, the negative log was calculated and then scaled.<\/p>\n This produces the desired effect, as can be seen below:<\/p>\n In comparison, cyclic data is usually scaled using trigonometric functions such as sine and cos. The crash hour data looks roughly cyclic based on earlier observations, so I applied a sine function to the data as below – since numpy’s sin() function is in radians, I first converted the input to radians before calculating the sine of the input:<\/p>\n A histogram of the transformed data can be seen below:<\/p>\n Finally I removed the unscaled data from the model to avoid this interfering with model predictions:<\/p>\n Another important step in data preprocessing is data encoding – this is where non-numerical data (for example categories) are represented in numerical format, to make it compatible with machine learning algorithms. For categorical data in this model, I used a method called label encoding – each category in a column is given a numerical value before it’s inputted to the model. A diagram of this process is shown below:<\/p>\n I encoded the columns in the dataset, first segmenting out the columns I wanted to keep from the original dataset and making a copy of the dataframe (collisions_ml). I then defined the categorical columns in a list, and used the LabelEncoder() function from sklearn to fit and transform the categorical columns:<\/p>\n With the data now sufficiently preprocessed, the data can now be split into train and test data, and a classification model can be fitted to the data.<\/p>\n It’s important to separate data into a training and test sets when building a machine learning model; the training set is a fraction of the initial data which is used to train the model on the right responses, whereas the test set is used to evaluate model performance. Keeping these separate is necessary to reduce the risk of overfitting and model bias.<\/p>\n I separated out the crash_type column using the drop() function (the remaining features will be used as the variables to predict crash_type), and defined crash_type as the y result to be predicted using the model. The train_test_split function from sklearn was used to take 20% of the initial dataset as training data, with the rest to be used for model testing.<\/p>\n For this project, a K-Nearest Neighbors (KNN) classification model is used to predict results from the features. KNN models work by checking the value of the K nearest known values around an unknown data point, then classifying the data point based on the values of those "neighbor" points. It’s a non-parametric classifier, meaning it doesn’t make any assumptions about the underlying data distribution; however it is computationally expensive, and can be sensitive to outliers in the data.<\/p>\n I instantiated the KNN classifier with an initial n_neighbors equal to 3, using Euclidean metrics, before fitting the model to the training data:<\/p>\n Once the model was fitted to the training data, I made predictions on the test data as below:<\/p>\n Evaluation of a machine learning model is typically done using four metrics; accuracy, precision, recall, and F1 score. The differences between these metrics are very subtle, but in plain English these terms can be defined as follows:<\/p>\n I computed the metrics of the KNN model using the below code snippet – I also calculated the difference between the model’s accuracy on the train and test set, to assess fit:<\/p>\n The initial metrics of the KNN model are given below:<\/p>\n The model scored well on test accuracy (79.6%), precision (82.1%), recall (91.1%), and F1 score (86.3%) – however the test accuracy was much higher than the train accuracy at 93.1%, a 13.5% difference. This indicates the model is overfitting the data, which means it would struggle to accurately make predictions on unseen data. Therefore the model needs to be adjusted for a better fit – this can be done using a process called hyperparameter tuning.<\/p>\n Hyperparameter tuning is the process of selecting the best set of hyperparameters for a machine learning model. I fine-tuned the model using k-fold cross-validation<\/em> – this is a resampling technique where the data is split into k<\/em> subsets (or folds<\/em>), then each fold in turn is used as the validation set while the remaining data is used as the training set. This method is effective at reducing the risk of bias being introduced to the model by a particular choice of training\/test set.<\/p>\n The hyperparameters for the KNN model are number of neighbors (_nneighbors<\/em>) and the distance metric. There are a number of different ways to measure distance in a KNN classifier, but here I focused on two options:<\/p>\n Note that I could have also fine-tuned the weight parameter (which determines whether all neighbors vote equally or if the closer neighbors are given more importance), but I decided to keep the voting weight uniform.<\/p>\n I defined a parameter grid with n_neighbors of 3, 7, and 10, as well as metrics of Euclidean or Manhattan. I then instantiated a RandomizedSearchCV<\/em> algorithm, passing in the KNN classifier as the estimator, along with the parameter grid. I set the algorithm to split the data into 5 folds by setting the cv<\/em> parameter to 5; this was then fit to the training set. A snippet of the code for this can be seen below:<\/p>\n The best accuracy and classifier were retrieved from the algorithm, indicating the classifier performs best with n_neighbors set to 10 while using the Manhattan distance metric, and that this would lead to an accuracy score of 74.0%:<\/p>\n as such these parameters were inputted to the classifier, and the model was retrained:<\/p>\n Performance metrics were again extracted from the classifier, in the same manner as before – a screengrab of the metrics for this iteration can be seen below:<\/p>\n Cross-validation led to slightly poorer results for all metrics; test accuracy dropped by 2.6%, precision by 1.5%, recall by 0.5%, and F1 score by 1%. however the training-test accuracy difference dropped to 3.8%, where it was initially 13.5%. This indicates the model is no longer overfitting the data, and is therefore more suitable for predicting unseen data.<\/p>\n In summary, the KNN classifier performed well in predicting whether a collision would require a tow or ambulance. Initial metrics from the model’s first iteration were impressive, however the disparity between test and training accuracy indicated overfitting. Hyperparameter tuning allowed for the model to be optimised, which significantly reduced the gap in accuracies between the two datasets. While performance metrics did take a small hit during this process, the benefit of a model with better fit outweighs these concerns.<\/p>\n Using supervised learning to predict service callouts to Chicago car collisions<\/p>\n","protected":false},"author":18,"featured_media":12335,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"is_member_only":false,"sub_heading":"Using supervised learning to predict service callouts to Chicago car collisions","footnotes":""},"categories":[44,47,22],"tags":[749,448,508,446,467],"sponsor":[],"coauthors":[29088],"class_list":["post-12334","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-data-science","category-data-visualization","category-machine-learning","tag-classification","tag-data-science","tag-data-visualization","tag-machine-learning","tag-python"],"yoast_head":"\n
\nData Collection and Preparation<\/h2>\n
Initial Setup<\/h3>\n
#generic data analysis \nimport os\nimport pandas as pd\nfrom datetime import date\nimport matplotlib.pyplot as plt\nimport numpy as np\nimport seaborn as sns\nimport missingno as msno<\/code><\/pre>\n#Preprocessing\nfrom sklearn.preprocessing import LabelEncoder\nfrom sklearn.preprocessing import StandardScaler\n\n# Models\nfrom sklearn.neighbors import KNeighborsClassifier\n\n# Reporting\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.model_selection import RandomizedSearchCV\n\n#metrics\nfrom sklearn.metrics import accuracy_score\nfrom sklearn.metrics import f1_score\nfrom sklearn.metrics import precision_score\nfrom sklearn.metrics import recall_score<\/code><\/pre>\n\n
Data Cleaning<\/h3>\n
#joining collision data to beat data - inner join\ncollisions = collision_raw.merge(beat_data, how='inner',\n left_on='beat_of_occurrence',\n right_on='BEAT_NUM'\n )<\/code><\/pre>\n#visualising missing data\n#sorting values by report received date\n\ncollisions = collisions.sort_values(by='crash_date', ascending=True)\n\n#plotting matrix of missing data\nmsno.matrix(collisions)\nplt.show()\n\n#info of sorted data\nprint(collisions.info())<\/code><\/pre>\n![\"Unrefined](\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2023\/08\/1oLxFaUyicszWIidepbsBNw.png\")
#defining unnecessary columns\ndrop_cols = ['location', 'crash_date_est_i','report_type', 'intersection_related_i',\n 'hit_and_run_i', 'photos_taken_i', 'crash_date_est_i', 'injuries_unknown',\n 'private_property_i', 'statements_taken_i', 'dooring_i', 'work_zone_i',\n 'work_zone_type', 'workers_present_i','lane_cnt','the_geom','rd_no',\n 'SECTOR','BEAT','BEAT_NUM']\n\n#dropping columns\ncollisions=collisions.drop(columns=drop_cols)\n\n#plotting matrix of missing data\nmsno.matrix(collisions)\nplt.show()\n\n#info of sorted data\nprint(collisions.info())<\/code><\/pre>\n![\"msno](\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2023\/08\/1SNeLUh-Guxu19MRypXuZyg.png\")
![\"Both](\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2023\/08\/1-VTY_JeXIdXjZ9V74i6uWA.png\")
#Some incorrect lat\/long data - need to remove these rows\ncollisions = collisions[collisions['longitude']<-80]\ncollisions = collisions[collisions['latitude']>40]<\/code><\/pre>\n
\nClassification Model<\/h2>\n
Exploratory Data Analysis<\/h3>\n
#plotting histograms of numerical values\ncollisions.hist(bins=50,figsize=(16,12))\nplt.show()<\/code><\/pre>\n![\"Histograms](\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2023\/08\/1QIwJkCWGNyruz63AuWwmUg.png\")
![\"Latitude-longitude](\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2023\/08\/1yrI0ge5Rru-Rg_rUSXINLQ.png\")
![\"Unscaled](\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2023\/08\/1vpXsn_Ei2bLslIKvBU7ptQ.png\")
Scaling and Transformation<\/h3>\n
#scaling latitude and longitude data\nscaler = StandardScaler()\n\n# Logarithmic transformation on longitude\ncollisions_ml['neg_log_longitude'] = scaler.fit_transform(np.log1p(-collisions_ml['longitude']).\n values.reshape(-1,1))\n\n# Normalisation on latitude\ncollisions_ml['norm_latitude'] = scaler.fit_transform(np.log1p(collisions['latitude']).\n values.reshape(-1, 1))<\/code><\/pre>\n![\"Scaled](\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2023\/08\/1uEIRc5hzYyDcqFgXih4VoA.png\")
#transforming crash_hour \n#data is cyclic, can be encoded using trig transforms\n\n#trig transformation - sin(crash_hr)\ncollisions_ml['sin_hr'] = np.sin(2*np.pi*collisions_ml['crash_hour']\/24)<\/code><\/pre>\n![\"Scaled](\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2023\/08\/1cJdezFaVwqTcpwV8vsX_IA.png\")
#drop previous latitude\/longitude columns\nlat_long_drop_cols = ['longitude','latitude']\ncollisions_ml.drop(lat_long_drop_cols,axis=1,inplace=True)\n\n#drop crash_hour column\ncollisions_ml.drop('crash_hour',axis=1,inplace=True)<\/code><\/pre>\nData Encoding<\/h3>\n
![\"An](\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2023\/08\/1VANp9LwCoxoSEK1IA705PQ.png\")
#segmenting columns into lists\nml_cols = ['posted_speed_limit','traffic_control_device', 'device_condition', 'weather_condition',\n 'lighting_condition', 'first_crash_type', 'trafficway_type','alignment', \n 'roadway_surface_cond', 'road_defect', 'crash_type','damage','prim_contributory_cause',\n 'sec_contributory_cause','street_direction','num_units', 'DISTRICT',\n 'crash_hour','crash_day_of_week','latitude', 'longitude']\ncat_cols = ['traffic_control_device', 'device_condition', 'weather_condition', 'DISTRICT',\n 'lighting_condition', 'first_crash_type', 'trafficway_type','alignment',\n 'roadway_surface_cond', 'road_defect', 'crash_type','damage','prim_contributory_cause',\n 'sec_contributory_cause','street_direction','num_units']\n\n#making a copy of the dataset\ncollisions_ml = collisions[ml_cols].copy()\n\n#encoding categorical values\nlabel_encoder = LabelEncoder()\nfor col in collisions_ml[cat_cols].columns:\n collisions_ml[col] = label_encoder.fit_transform(collisions_ml[col])<\/code><\/pre>\nSplitting the Train & Test Data<\/h3>\n
#Create test set\n#setting X and y values\n\nX = collisions_ml.drop('crash_type', axis=1)\ny = collisions_ml['crash_type']\nX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)<\/code><\/pre>\nK-Nearest Neighbors Classification<\/h3>\n
#Classifier - K Nearest Neighbours\n#instantiate KNN Classifier\nKNNClassifier = KNeighborsClassifier(n_neighbors=3, metric = 'euclidean')\n\nKNNClassifier.fit(X_train,y_train)<\/code><\/pre>\n#Predictions\n#predict on training set\ny_train_pred = KNNClassifier.predict(X_train)\n\n#predict on test data\ny_test_pred = KNNClassifier.predict(X_test)<\/code><\/pre>\nEvaluation<\/h3>\n
\n
#Evaluate model\n# Calculate the accuracy of the model\n\n#calculating accuracy of model on training data\ntrain_accuracy = accuracy_score(y_train, y_train_pred)\n\n#calculating accuracy of model on test data\ntest_accuracy = accuracy_score(y_test, y_test_pred)\n\n#computing f1 score,precision,recall\nf1 = f1_score(y_test, y_test_pred)\nprecision = precision_score(y_test,y_test_pred)\nrecall = recall_score(y_test,y_test_pred)\n\n#comparing performances\nprint("Training Accuracy:", train_accuracy)\nprint("Test Accuracy:", test_accuracy)\nprint("Train-Test Accuracy Difference:", train_accuracy-test_accuracy)\n\n#print precision score\nprint("Precision Score:", precision)\n\n#print recall score\nprint("Recall Score:", recall)\n\n#print f1 score\nprint("F1 Score:", f1)<\/code><\/pre>\n![\"Metrics](\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2023\/08\/1rGYCFUOYhS4W9C01Sh8kbQ.png\")
Hyperparameter Tuning<\/h3>\n
\n
#Fine tuning (RandomisedSearchCV)\n# Define parameter grid\nparam_grid = {\n 'n_neighbors': [3, 7, 10],\n 'metric': ['euclidean','manhattan']\n}\n\n# instantiate RandomizedSearchCV\nrandom_search = RandomizedSearchCV(estimator=KNeighborsClassifier(), param_distributions=param_grid, cv=5)\n\n# fit to training data\nrandom_search.fit(X_train, y_train)\n\n# Retrieve best model and performance\nbest_classifier = random_search.best_estimator_\nbest_accuracy = random_search.best_score_\n\nprint("Best Accuracy:", best_accuracy)\nprint("Best Model:", best_classifier)<\/code><\/pre>\n![\"Results](\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2023\/08\/18mKFAjq5lztpaw0LETJTog.png\")
#Classifier - K Nearest Neighbours\n#instantiate KNN Classifier\nKNNClassifier = KNeighborsClassifier(n_neighbors=10, metric = 'manhattan')\n\nKNNClassifier.fit(X_train,y_train)<\/code><\/pre>\n![\"Metrics](\"https:\/\/towardsdatascience.com\/wp-content\/uploads\/2023\/08\/1Dey6fWpwH3N9EvzZuvW2_g.png\")
Conclusion<\/h2>\n
References<\/h2>\n
\n