Final Term Project -
This is a single person project. On the first page of the project report, indicate your name, UCID, preferred email address, and the option number you choose.
There are six options. You choose one of the six options.
Option 1 - Supervised Data Mining (Classification)
This option is to implement 2 classification algorithms of your choice on 1 dataset of your choice (each of the 2 algorithms must run on the dataset).
Your final term project documentation must indicate clearly the algorithms and dataset you used in the project.
Specific Algorithms and Tools used in the Project
- There are 12 categories of algorithms listed on the following pages (categories 13-18 are software tools). The 2 classification algorithms you choose must come from two different categories. In the same category, only one algorithm can be chosen from that category.
- In your final term project documentation, for each algorithm you choose, specify clearly the category number/name and algorithm name in that category.
Option 2 - Unsupervised Data Mining (Clustering)
Part 1 - Generate a set S of 500 points (vectors) in 3-dimensional Euclidean space. Use the Euclidean distance to measure the distance between any two points. Write a program to find all the outliers in your set S and print out these outliers. If there is no outlier, your program should indicate so. Use any programming language of your choice (specify the programming language you use in the project).
Next, remove the outliers from S, and call the resulting set S'.
Part 2 - (1) Write a program that implements the hierarchical agglomerative clustering algorithm taught in the class to cluster the points in S' into k clusters where k is a user-specified parameter value.
(2) Repeat part 1 and (1) above on two additional different datasets.
Notes on the hierarchical agglomerative clustering algorithm
In determining the distance of two clusters, you should consider the following definitions respectively:
- the distance between the nearest two points in the two clusters,
- the distance between the farthest two points in the two clusters,
- the average distance between points in the two clusters,
- the distance between the centers of the two clusters.
Use the definition that yields the best performance where the performance is measured by the Silhouette coefficient.
Option 3 -
This option is to implement a graph mining or graph clustering system. You can use any heuristics published in data mining articles and implement the heuristics using any programming language of your choice (specify the programming language you use in the project). Use any data set of your preference. The output should be displayed using a visualization tool such as Graphviz or Cytoscape, etc. (specify the name of the visualization tool you use).
Option 4 -
This option is to implement a text mining system. You can use Reuters-21578 dataset.
To parse the documents, you can use Stanford CoreNLP tools.
Represent each document by a set of keywords. Find and print out association rules among the keywords or terms in the documents.
Option 5 - Causal Discovery (Network Inference)
This option is to implement 1 causal discovery (or network inference) algorithm of your choice on 1 dataset of your choice.
Your term project documentation must indicate clearly the algorithm and dataset you used in the project.
Option 6 - Data Mining Using Hadoop in the Cloud
This option is to implement a data mining algorithm (e.g. association mining, classification, clustering, etc.) of your choice on a dataset of your choice using Hadoop and MapReduce.
Your final term project documentation must indicate clearly the algorithm, the dataset and the cloud environment you used in the project.
Infrastructure and Software -
- Cloud infrastructure: Amazon Elastic MapReduce (Amazon EMR)
- Programming software: Hadoop/MapReduce on an AWS cluster in a master slave fashion with multiple nodes using a programming language of your preference (e.g. Java)
Live Demo Prepared
- The final term project (option 6) must run on the web with appropriate software implementation.
- It should allow classmates to test on the web.
- A live demo will be arranged either in Moodle or in classroom 1-2 weeks before the end of the semester.
- Provide the website URL in both Moodle forums and your final term project report.
Attachment:- Assignment File.rar