Prediction Using Classification and Regression Trees in MATLAB

A Classification and Regression Tree(CART) is a Machine learning algorithm to predict the labels of some raw data using the already trained classification and regression trees. Initially one needs enough labelled data to create a CART and then, it can be used to predict the labels of new unlabeled raw data. 

MATLAB is amongst the best tools to execute Machine Learning algorithms due to its fast matrix computations, and MATLAB also provides a great number of inbuilt tools and functions for machine learning. In this article, we shall see how to predict the labels of some new data using Classification and Regression Trees in MATLAB.

The Predict Function 

MATLAB provides the predict() function which takes a CART and the new data whose label is to be predicted then, it returns the best fit label for the new data on the trend generated by the CART. 

Syntax

Y = predict(CART, X)

where Y is the label of unlabeled data X. CART is the name of the Classification or Regression model used to predict Y for X. Considering that X is an array, the predict function will predict labels for each dimension of X using the CART.

Predictions using Classification and Regression Trees

In this section, we shall predict using CARTs on the available data in MATLAB library using some examples.

We shall use one of the 33 sample datasets provided by MATLAB for Data Science, the carbig.mat dataset. The classes in this dataset are as follows:

Once you load the dataset using the load statement, these arrays will become available in your workspace. 

Note: When loading any sample dataset, it must be noted that you require the Statistics and Machine Learning Toolbox in the offline installation of MATLAB. These sample datasets are not available in ordinary installation of MATLAB. However, they are available in MATLAB online.

Now, let us start predicting some results.

Example 1:

Matlab

% loading the carbig dataset
load carbig.mat
% creating a new feature with only the Horsepower array
Xraw = [Horsepower,Displacement];
% generating a classification tree with Xraw and the country of origin feature
CM = fitctree(Xraw,Origin);
% predicting the origin of mean of the Xraw data
Yraw = predict(CM,mean(Xraw))

Output:

Explanation

In this example, we load the car big dataset which contains many variables. From them, we create a sub-array that only contains the organized data of the Horsepower of a car with displacement capacity. Then, we will predict the country of origin(Origin) of the cars whose horsepower and displacement are the mean of the same entities in the car big data.

Thus. we create a new sub-array Xnew and make two columns in it, Horsepower and Displacement. Then we generate a classification tree using the fitctree function and give them the label, and the country of their origin; which is also given in the dataset.

As we can see, the cars with the mean value of Horsepower and Displacement, belong to the country USA.

Now, we use the predict function to on the mean value of Xraw sub-array and predict its country of origin. 

Let’s see another example, we shall use a regression tree to predict a label for some raw data. We shall use the same dataset as previous example but, now we shall predict the acceleration of cars whose weight is the mean of ‘Car Weight in carbig dataset’. This shall provide  us a good understanding of Regression Trees.

Example 2:

Matlab

% loading carbig data
load carbig.mat

% creating a copy of Weight feature
Xraw = Weight;

% generating a regression tree
CM = fitrtree(Xraw,Acceleration);

% predicting acceleration for mean of Xraw
Yraw = predict(CM,mean(Xraw))

Output:

Explanation

As done in the previous example, we take a feature from the car big dataset (Weight) and then, generate a regression tree using the fitrtree function between Weight and Acceleration. Then we use the predict function to predict the acceleration of cars whose weight is the mean weight of cars present in the car big dataset on the basis of the trend fitted by the RM regression tree.

Conclusion

In this article, we studied how to use Classification and Regression Trees in MATLAB to predict some features. We used both classification and regression on the same dataset to predict different results. However, this choice of whether to use a classification tree or a regression tree completely depends on the user’s needs. If one needs to classify some data into categories (as shown in Example 1) one could use a Classification Tree, and if one needs to predict some feature based on the trend (as done in Example 2), they could use a Regression tree which uses trends to predict new labels.