Android app for Automatic Web Page Classification : Analysis of Text and Visual Features

Abstract

Internet keeps growing everyday and with that, the creation of new web pages. Due to this fact, web pages of many different categories can be found such as News, Sports or Business. This issue has made investigators think about one innovative concept: Webpage Classification. This new approach implies the categorization of web pages to one or more category labels. Some research has been done during the last years using text and visual content extracted from the web pages to be able to classify. However, the need of being able to do such a thing in an Android app has not been investigated yet, to the best of our knowledge. Consequently, this thesis is focused in the development of an Android app which is able to classify web pages. First of all, text and visual features have to be extracted from each webpage. Four types of visual features were extracted from each web page to construct a visual features vector of 160 attributes. Concerning to the text features, a text features vector was also built for each of the webpage with 160 attributes. To do so, a “Bag-Of-Words” of one hundred and sixty words was set up from the HTML code already extracted and filtered. Thus, we end up having a full vector of 320 attributes for each webpage. A binary classification was performed trying to distinguish web pages for Adults and for Kids. Good results were obtained especially when using AdaBoost classifier with text and visual features where a 94.44% of accuracy of correct classifications was achieved.