Comprehensive Recommendation system and KNN with project-Movie Recommendation system.

What are Recommendation systems?

Recommendation systems are becoming increasingly important in today’s extremely busy world. People are always short on time with the myriad tasks they need to accomplish in the limited 24 hours. Therefore, the recommendation systems are important as they help them make the right choices, without having to expend their cognitive resources.

The purpose of a recommendation system basically is to search for content that would be interesting to an individual. Moreover, it involves a number of factors to create lists of useful and interesting content specific to each user/individual. Recommendation systems are Artificial Intelligence based algorithms that skim through all possible options and create a customized list of items that are interesting and relevant to an individual. These results are based on their profile, search/browsing history, what other people with similar traits/demographics are watching, and how likely are you to watch those movies. This is achieved through predictive modeling and heuristics with the data available.

The purpose of a recommendation system basically is to search for content that would be interesting to an individual. it involves a number of factors to create lists of useful and interesting content specific to each user. Recommendation systems are Artificial Intelligence based algorithms that skim through all possible options and create a customized list of items that are interesting and relevant to an individual.

These results are based on their profile, search/browsing history, what other people with similar traits/demographics are watching, and how likely are you to watch those movies. This is achieved through predictive modeling and heuristics with the data available.

What can be recommended?

Advertising Messages
– Movies
– Books
– Music Tracks
– News Articles
– Restaurants
– Future Friends (Social Network Sites)
– Courses in e-learning
– Jobs
– Research Papers
– Investment Choices
– TV Programs
– Citations
– Clothes
– Online Mates (Dating Services)
– Supermarket Goods

Real-World examples:

Here are some of the examples of the pioneers in creating algorithms for recommendation systems and using them to serve their customers better in a personalized manner. These are:
Group Lens:
– Helped in developing initial recommender systems by pioneering collaborative filtering model
– It also provided many data-sets to train models including Movie Lens and Book Lens
Amazon:
– Implemented commercial recommender systems
– They also implemented a lot of computational improvements
Netflix Prize:
– Pioneered Latent Factor/ Matrix Factorization models
Google- You tube:
– Hybrid Recommendation Systems
– Deep Learning based systems
– Social Network Recommendations

Various types of recommendation systems:

Popularity based recommendation systems
– Classification model based
– Content based recommendations
– collaborative filtering

1. User-based
2. Item-based

The recommendation system is classified into two type Content-Based and Collaborative based recommendation system.

(1)Content-Based:

The idea behind Content-based recommendation system is to recommend an item based on a comparison between the content of the items and a user profile.In simple words,I may get recommendation for a movie based on the description of other movies.

(2) Collaborative based:

Collaborative filtering method finds a subset of user who have similar test and preferences to the target user and use this subset for offering recommendations. In this method user with similar interest have common preferences.

Comparison between content-Based filtering & collaborative filtering:

Content based filtering — The point of content-based filtering system is to know the content of both user and item. Usually it constructs and then compare user-profile and item-profile using the content of shared attribute space. For example, for a movie, you represent it with the movie stars in it and the genres (using a binary coding for example).

For user profile, you can do the same thing based on the users likes some movie stars/genres etc

Collaborative filtering — Collaborative algorithm uses “User Behavior” for recommending items. They exploit behavior of other users and items in terms of transaction history, ratings, selection and purchase information. Other users behavior and preferences over the items are used to recommend items to the new users. In this case, features of the items are not known.

How to build a popularity based recommendation system:

we will consider the Movie Lens small data set, and focus on two files, i.e., the movie and rating
Movie has three fields namely:

1. Movie Id — It has a unique id for every movie
2. Title — It is the name of the movie
3. Genre — The genre of the movie

The ratings file has four fields namely:

1. User id — The unique id for every user who has rated one or multiple movies
2. Movie Id — The unique id for each movie
3. Rating — The rating given to a user to a movie
4. Timestamp — When was the rating given to a specific movie.

movie_data=pd.merge(ratings_data,movie_names,on='movie Id')
movie_data.head()

Implementation:

For our recommender system, we’ll use both of the techniques mentioned above: content-based and collaborative filtering. To find the similarity between movies for our content based method, we’ll use a cosine similarity function. For our collaborative filtering method, we’ll use a matrix factorization technique

#create a data frame for analysis
trend=pd.Data frame(movie_data.group by('title')['rating']mean())
trend['total number of ratings'] = pd.Data frame(movie_data.group by('title')['rating']count()) 
trend.head()

#plot rounded-up ratings with number of movies
plot.figure(file size =(10, 4))
ax=plot.bar(trend['rating']round(),trend['total number of ratings'],color='b')
plot.show()

#a bar graph describing number of reviews for first 25 movies
plot.figure(file size =(10, 4))
ax=plot.subplot()
ax.bar(trend.head(25)index,trend['total number of ratings']head(25),color='b')
ax.set.sticklabels(trend.index,rotation=40,font size='12',horizontal="right")
ax.set_title("Total Number of reviews for each movie")
plot.show()

So, In this tutorial, I have tried to cover all the required steps to Recommend any Movie/Articles/Research Books etc. I illustrated how to build a scalable collaborative filtering recommender system. I provided the model with an understanding of collaborative filtering methods and discussed the challenges of implementing them at scale. I introduced the core concepts of data flow programming model and provided implementations of both user and item-based collaborative filtering algorithms on it. I hope you find this tutorial helpful.