Feature Selection in Machine Learning with Python

Feature selection is the process of selecting a subset of features from the total variables in a data set to train machine learning algorithms. Feature selection is key for developing simpler, faster, and highly performant machine learning models and can help to avoid overfitting. The aim of any feature selection algorithm is to create classifiers or regression models that run faster and whose outputs are easier to understand by their users. In this book, you will find the most widely used feature selection methods to select the best subsets of predictor variables from your data. You will learn about filter, wrapper, and embedded methods for feature selection. Then, you will discover methods designed by computer science professionals or used in data science competitions that are faster or more scalable. First, we will discuss the use of statistical and univariate algorithms in the context of artificial intelligence. Next, we will cover methods that select features through optimization of the model performance. We will move on to feature selection algorithms that are baked into the machine learning techniques. And finally, we will discuss additional methods designed by data scientists specifically for applied predictive modeling. We will begin by removing useless and redundant features by examining variability and correlation. Next, we will explore how to choose features based on statistical tests such as ANOVA, chi-square, and mutual information. We will move on to selecting features by using Lasso regularization or decision tree based feature importance, which are embedded in the machine learning modeling process. Finally, we will show how to select features by recursive feature elimination, addition, or value permutation. Each chapter fleshes out various methods for feature selection that share common characteristics. First, you will learn the fundamentals of the feature selection method, and next you will find a Python implementation. The book comes with an accompanying Github repository with the full source code that you can download, modify, and use in your own data science projects and case studies.