A New Algorithm for Classification Based on K Nearest Neighbor and Decision Tree

Abstract

In a classification problem with binary outcome attribute, if the input attributes are both continuous and categorical, the  Nearest Neighbor (KNN) technique cannot be used. On the other hand, the Decision Tree (DT) technique handles the continuous attributes by discretization which leads to loss of information.

To overcome the limitations of the KNN and DT techniques, we propose a new technique in this study which is called  Nearest Neighbor Decision Tree (KNNDT). The proposed technique uses a combination of KNN and DT to classify the test instances. KNNDT first uses the KNN technique to select homogeneous groups of training instances by using the continuous attributes and then builds local decision trees on these homogeneous groups by using the categorical attributes.

An extensive simulation study was conducted to compare the performances of KNNDT and DT. In general, the proposed KNNDT gives better results compared to DT.