Objectives
- To apply skills and knowledge acquired throughout the trimester in classification algorithms and machine learning process.
- To rationalize the use of machine learning algorithms to effectively and efficiently process of data in big size.
- To demonstrate ability to use R to perform email classification tasks that are common for corporate security analyst.
- To scientifically conduct and document machine learning experiments for analytics purposes.
Problem Statement
This is an individual assessment task. Each student is required to submit a report of approximately 2,000-2,500 words along with exhibits to support findings with respect to the provided spam and non-spam messages. This report should consist of:
- Overview of classifiers and evaluation metrics
- Construction of data sets, identification of features and the process of conducting classification
- Technical findings of experiment results
- Justified discussion of the performance evaluation outcomes for different classifiers
To demonstrate your achievement of these goals, you must write a report of at least 2,000 words (2,500 words maximum). Your report should consist of the following chapters:
1. A proper title which matches the contents of your report.
2. Your name and Deakin student number in the author line.
3. An executive summary which summarizes your findings.
4. An introduction chapter which lists the classification algorithms of your choice (at least 5 algorithms), the features used for classification, the performance evaluation metrics (at least 5 evaluation metrics), the brief summary of your findings, and the organization of the rest of your report. (You may find hints on features used for classification from Twitter Developer Documentation
5. A literature review chapter which surveys the latest academic papers regarding the classifiers and performance evaluation metrics of your choice. With respect to each classifier and performance evaluation metrics, you are advised to identify and cite at least one paper published by ACM and IEEE journals or conference proceedings. In addition, Your aim of this part of the report is to demonstrate deep and thorough understanding of the existing body of knowledge encompassing multiple classification techniques for security data analytics,
specifically, your argument should explain why machine learning algorithms should be used rather than human readers.
6. Technical demonstration chapter which consists of fully explained screenshots when your experiments were conducted in R. That is, you should explain each step of the procedure of classification, and the performance results for your classifiers. Note, what classifiers you presented in literature review should be what you conduct experiments.
7. Performance evaluation chapter which evaluates the performance of classifiers. You should analyse each classifier's performance with respect to the performance metrics of your choice. In addition, you should compare the performance results in terms of evaluation metrics, e.g., accuracy, false positive, recall, F-measure, speed and so on, for the selected classifiers and datasets.
8. A conclusions chapter which summarizes major findings of the study (You should use at least 5 evaluation metrics to evaluate the performance of classifiers and compare the performance of different classifiers. You can demonstrate your experiment results in the form of table and plots), discusses whether the results match your hypotheses prior to the experiments and recommends the best performing classification algorithm.
9. A bibliography list of all cited papers and other resources. You must use in-text citations in Harvard style and each citation must correspond to a bibliography entry. There must be no bibliography entries that are not cited in the report.
Attachment:- DatasetA2.zip