Data Mining Assignment -
In this assignment you are asked to explore the use of neural networks for classification and numeric prediction. You are also asked to carry out a data mining investigation on a real-world data file. You are required to write a report on your findings. You will be assessed on methodology, analysis of results and conclusions.
PART 1: CLASSIFICATION WITH NEURAL NETWORKS
This part involves the following file: heart-v1.arff
For the neural network training runs build a table with the following headings:
|
Run No
|
Architecture
|
Parameters
|
Train MSE
|
Train Error
|
Epochs
|
Test MSE
|
Test Error
|
|
1
|
23-10-5
|
lr=.0
|
0.5
|
30%
|
500
|
0.6
|
40%
|
1. Describe the data encoding that is required for this task. How many outputs and how many inputs will there be?
2. Develop a script to generate the necessary training, validation and test files. You might want to normalize the numeric attributes with Weka beforehand. Include your data preparation script as an appendix (not part of the page count).
3. Determine the "analyze" strategy that you will use.
4. Using Javanns carry out 5 train and rest runs for a network with 10 hidden nodes. Comment on the variation in the training runs and the degree of overfitting.
5. Experiment with different numbers of hidden nodes. What seems to be the right number of hidden nodes for this problem?
6. For 10 hidden nodes, explore different values of the learning rate. What do you conclude?
7. [Optional] Change the learning function to backprop-momentum. Explore different combinations of learning rate and momentum. What do you conclude?
8. Perform a run with 10 hidden nodes and no validation data. Stop training when the MSE is no longer changing. Get the classification error on the training and test data. Comment on the degree of overfitting.
9. Compare the classification accuracy of the neural classifiers with the classification accuracy of Weka J48 and Multilayer Perceptron.
Report Length Up to two pages.
PART 2: NUMERIC PREDICTIONWITH NEURAL NETWORKS
This part involves the following file: heart-v1.arff
The task is to predict the value of the Weight variable. Build a similar table of runs to the one in the previous question.
1. Describe the data encoding that is required for this task. How many outputs and how many inputs will there be? What scaling or normalization is required?
2. Modify your script from part 1 to generate the necessary training, validation and test files. You can use Weka to normalize all of the numeric attributes except for the class, ie weight attribute. You will need to write a suitable program to scale the weight to the range [0,1] and another one to reverse scale the neural net outputs to get the mean absolute error. Include your data preparation script as an appendix (not part of the page count).
3. Using Javanns carry out 5 train and test runs for a network with 5 hidden nodes. Comment on the variation in the training runs and the degree of overfitting. [Hint: When you are comparing the predictive accuracy of different models you don't have to reverse scale the output.]
4. Experiment with different numbers of hidden nodes. What seems to be the right number of hidden nodes for this problem?
5. For 5 hidden nodes, explore different values of the learning rate. What do you conclude?
6. [Optional] Change the learning function to backprop-momentum. Explore different combinations of learning rate and momentum. What do you conclude?
7. Perform a run with 5 hidden nodes and no validation data. Stop training when the MSE is no longer changing. Get the error on the training and test data. Comment on the degree of overfitting.
8. Compare the mean absolute error of the neural classifiers with the mean absolute error of Weka M5P and MultiLayer Perceptron.
Report Length Up to one page.
PART 3: DATA MINING
Choose EITHER the census data OR the movies data.
Choose - data/arff/UCI/adult.arff OR data/other/IMDB-movie-data.csv
The file adult.arff contains American census data collected several years ago. The file adult.names describes the data items. There is some further information about the data at the website.
The movie data is was collected from the IMDb web site which claims to be "the world's most popular and authoritative source for movie, TV and celebrity content".
It was collected to answer the question "How can we tell the greatness of a movie before it is released in cinema?"
IMDB-movie-data.csv has some changes from the kaggle file, mostly to make the genre information more usable.
Your task is to analyze the data with appropriate data mining techniques and identify any "golden nuggets" in the data. You are expected to use classification, clustering, association finding, attribute selection and visualization in your analysis, or to explain why a particular technique is not relevant. Be sure to give the rationale for each experiment.
Report Length Up to three pages.
Attachment:- Assignment Files.rar