## In Depth: Naive Bayes Classification

In this tutorial you are going to learn about the Naive Bayes algorithm including how it works and how to implement it from scratch in Python without libraries. We can use probability to make predictions in machine learning. Perhaps the most widely used example is called the Naive Bayes algorithm. Not only is it straightforward to understand, but it also achieves surprisingly good results on a wide range of problems. Discover how to code ML algorithms from scratch including kNN, decision trees, neural nets, ensembles and much more in my new bookwith full Python code and no fancy libraries. This section provides a brief overview of the Naive Bayes algorithm and the Iris flowers dataset that we will use in this tutorial. Naive Bayes is a classification algorithm for binary two-class and multiclass classification problems. It is called Naive Bayes or idiot Bayes because the calculations of the probabilities for each class are simplified to make their calculations tractable. Rather than attempting to calculate the probabilities of each attribute value, they are assumed to be conditionally independent given the class value. This is a very strong assumption that is most unlikely in real data, i. Nevertheless, the approach performs surprisingly well on data where this assumption does not hold. The Iris Flower Dataset involves predicting the flower species given measurements of iris flowers. It is a multiclass classification problem. The number of observations for each class is balanced. There are observations with 4 input variables and 1 output variable. The variable names are as follows:. Download the dataset and save it into your current working directory with the filename iris. First we will develop each piece of the algorithm in this section, then we will tie all of the elements together into a working implementation applied to a real dataset in the next section. These steps will provide the foundation that you need to implement Naive Bayes from scratch and apply it to your own predictive modeling problems. Note : This tutorial assumes that you are using Python 3. If you need help installing Python, see this tutorial:. Note : if you are using Python 2. We will need to calculate the probability of data by the class they belong to, the so-called base rate. This means that we will first need to separate our training data by class. A relatively straightforward operation. We can create a dictionary object where each key is the class value and then add a list of all the records as the value in the dictionary. It assumes that the last column in each row is the class value. Running the example sorts observations in the dataset by their class value, then prints the class value followed by all identified records.

## Naive Bayes Algorithm in Python Naive Bayes model is easy to build and particularly useful for very large data sets. There are two parts to this algorithm:. The Naive Bayes classifier assumes that the presence of a feature in a class is unrelated to any other feature. It serves as a way to figure out conditional probability. This relates the probability of the hypothesis before getting the evidence P Hto the probability of the hypothesis after getting the evidence, P H E. Go a little confused? So, according to Bayes Theorem, we can solve this problem. First, we need to find out the probability. So here we have our Data, which comprises of the Day, Outlook, Humidity, Wind Conditions and the final column being Play, which we have to predict. Starting with our first industrial use, it is News Categorization, or we can use the term text classification to broaden the spectrum of this algorithm. News on the web is rapidly growing where each news site has its own different layout and categorization for grouping news. Each news article contents is tokenized categorized. In order to achieve better classification result, we remove the less significant words i. We apply the naive Bayes classifier for classification of news contents based on news code. Naive Bayes classifiers are a popular statistical technique of e-mail filtering. They typically use a bag of words features to identify spam e-mail, an approach commonly used in text classification. Particular words have particular probabilities of occurring in spam email and in legitimate email. Nowadays modern hospitals are well equipped with monitoring and other data collection devices resulting in enormous data which are collected continuously through health examination and medical treatment. Weather is one of the most influential factors in our daily life, to an extent that it may affect the economy of a country that depends on occupation like agriculture. Weather prediction has been a challenging problem in the meteorological department for years. Even after the technological and scientific advancement, the accuracy in prediction of weather has never been sufficient. A Bayesian approach based model for weather prediction is used, where posterior probabilities are used to calculate the likelihood of each class label for input data instance and the one with maximum likelihood is considered resulting output. Here we have a dataset comprising of Observations of women aged 21 and older. The dataset describes instantaneous measurement taken from patients, like age, blood workup, the number of times pregnant. Each record has a class value that indicates whether the patient suffered an onset of diabetes within 5 years. The values are 1 for Diabetic and 0 for Non-Diabetic. I ,ve broken the whole process down into the following steps:. The first thing we need to do is load our data file. The data is in CSV format without a header line or any quotes.

## Naive Bayes Classifier: Learning Naive Bayes with Python ## Naive Bayes for text classification in Python 