{"id":2352,"date":"2022-04-17T06:00:48","date_gmt":"2022-04-17T06:00:48","guid":{"rendered":"https:\/\/mdr.foobrdigital.com\/?p=2352"},"modified":"2022-04-17T06:00:48","modified_gmt":"2022-04-17T06:00:48","slug":"what-is-backward-elimination","status":"publish","type":"post","link":"https:\/\/mudassirbackup.infinitycodestudio.com\/index.php\/2022\/04\/17\/what-is-backward-elimination\/","title":{"rendered":"What is Backward Elimination?"},"content":{"rendered":"\n

Backward elimination is a feature selection technique while building a machine learning model. It is used to remove those features that do not have a significant effect on the dependent variable or prediction of output. There are various ways to build a model in Machine Learning, which are:<\/p>\n\n\n\n

  1. All-in<\/li>
  2. Backward Elimination<\/li>
  3. Forward Selection<\/li>
  4. Bidirectional Elimination<\/li>
  5. Score Comparison<\/li><\/ol>\n\n\n\n

    Above are the possible methods for building the model in Machine learning, but we will only use here the Backward Elimination process as it is the fastest method.<\/p>\n\n\n\n

    Steps of Backward Elimination<\/h3>\n\n\n\n

    Below are some main steps which are used to apply backward elimination process:<\/p>\n\n\n\n

    Step-1:<\/strong> Firstly, We need to select a significance level to stay in the model. (SL=0.05)<\/p>\n\n\n\n

    Step-2:<\/strong> Fit the complete model with all possible predictors\/independent variables.<\/p>\n\n\n\n

    Step-3:<\/strong> Choose the predictor which has the highest P-value, such that.<\/p>\n\n\n\n

    1. If P-value >SL, go to step 4.<\/li>
    2. Else Finish, and Our model is ready.<\/li><\/ol>\n\n\n\n

      Step-4:<\/strong> Remove that predictor.<\/p>\n\n\n\n

      Step-5:<\/strong> Rebuild and fit the model with the remaining variables.<\/p>\n\n\n\n

      Need for Backward Elimination: An optimal Multiple Linear Regression model:<\/h3>\n\n\n\n

      In the previous chapter, we discussed and successfully created our Multiple Linear Regression model, where we took 4 independent variables (R&D spend, Administration spend, Marketing spend, and state (dummy variables)) and one dependent variable (Profit)<\/strong>. But that model is not optimal, as we have included all the independent variables and do not know which independent model is most affecting and which one is the least affecting for the prediction.<\/p>\n\n\n\n

      Unnecessary features increase the complexity of the model. Hence it is good to have only the most significant features and keep our model simple to get the better result.<\/p>\n\n\n\n

      So, in order to optimize the performance of the model, we will use the Backward Elimination method. This process is used to optimize the performance of the MLR model as it will only include the most affecting feature and remove the least affecting feature. Let’s start to apply it to our MLR model.<\/p>\n\n\n\n

      Steps for Backward Elimination method:<\/h3>\n\n\n\n

      We will use the same model which we build in the previous chapter of MLR. Below is the complete code for it:<\/a><\/a><\/a><\/p>\n\n\n\n

      1. # importing libraries  <\/li>
      2. import<\/strong> numpy as nm  <\/li>
      3. import<\/strong> matplotlib.pyplot as mtp  <\/li>
      4. import<\/strong> pandas as pd  <\/li>
      5.   <\/li>
      6. #importing datasets  <\/li>
      7. data_set= pd.read_csv(’50_CompList.csv’)  <\/li>
      8.   <\/li>
      9. #Extracting Independent and dependent Variable  <\/li>
      10. x= data_set.iloc[:, :-1].values  <\/li>
      11. y= data_set.iloc[:, 4].values  <\/li>
      12.   <\/li>
      13. #Catgorical data  <\/li>
      14. from sklearn.preprocessing import<\/strong> LabelEncoder, OneHotEncoder  <\/li>
      15. labelencoder_x= LabelEncoder()  <\/li>
      16. x[:, 3]= labelencoder_x.fit_transform(x[:,3])  <\/li>
      17. onehotencoder= OneHotEncoder(categorical_features= [3])    <\/li>
      18. x= onehotencoder.fit_transform(x).toarray()  <\/li>
      19.   <\/li>
      20. #Avoiding the dummy variable trap:  <\/li>
      21. x = x[:, 1:]  <\/li>
      22.   <\/li>
      23.   <\/li>
      24. # Splitting the dataset into training and test set.  <\/li>
      25. from sklearn.model_selection import<\/strong> train_test_split  <\/li>
      26. x_train, x_test, y_train, y_test= train_test_split(x, y, test_size= 0.2, random_state=0)  <\/li>
      27.   <\/li>
      28. #Fitting the MLR model to the training set:  <\/li>
      29. from sklearn.linear_model import<\/strong> LinearRegression  <\/li>
      30. regressor= LinearRegression()  <\/li>
      31. regressor.fit(x_train, y_train)  <\/li>
      32.   <\/li>
      33. #Predicting the Test set result;  <\/li>
      34. y_pred= regressor.predict(x_test)  <\/li>
      35.   <\/li>
      36. #Checking the score  <\/li>
      37. print(‘Train Score: ‘, regressor.score(x_train, y_train))  <\/li>
      38. print(‘Test Score: ‘, regressor.score(x_test, y_test))  <\/li><\/ol>\n\n\n\n

        From the above code, we got training and test set result as:<\/strong><\/p>\n\n\n\n

        Train Score:  0.9501847627493607\nTest Score:  0.9347068473282446\n<\/pre>\n\n\n\n
        The difference between both scores is 0.0154.<\/strong><\/p>\n\n\n\n
        Note: On the basis of this score, we will estimate the effect of features on our model after using the Backward elimination process.<\/h4>\n\n\n\n
        Step: 1- Preparation of Backward Elimination:<\/strong><\/p>\n\n\n\n