Milestone : Revise and Evaluate Decision Analysis Model
In this milestone, you will perform and evaluation of your decision model and revise your decision model as needed.
Evaluation examples are if you are performing a bottom-up style recursive partitioning analysis, you should report on the error rate and variable selection. You might also consider alternative variable categorizations to improve your model.
If you are performing a top-down decision tree modeling exercise, what are the threshold values that cause the tree to flip?
You should perform sensitivity analysis on the critical variables in your tree and report what those sensitivity analyses are telling you. For either style of modeling, what makes your tree stronger?
What breaks the model?
If you are performing a bottom-up style recursive partitioning analysis, you should report on the error rate and variable selection, and what you did to improve them. You might also consider alternative variable categorizations to improve your model.
You might consider creating different versions of the same variable with slightly different categories and invoking them selectively in Rattle.
You might consider making multiple models that represent different groups of variables to explain an answer to the research question slightly differently each way. You should also report shifts in the error rate and what that means when you do different things.
If you are performing a top-down decision tree model, where are the threshold values that cause the tree to flip? Are there any? You have learned about sensitivity analysis at this point in class, so you should be able to identify the critical values for key variables in your tree and report what the sensitivity analyses are telling you. What happens when you include certain decision nodes in your tree but exclude others?
Can you draw alternative trees that still answer the research question? What happens to the proportions and the outcomes? What method are you going to use to deduce the optimal path?
Generally, for any of these decision trees, what makes your tree stronger? What breaks the model? What kinds of variables do you wish you had but do not have data for? What is the best criticism of the tree that you drew? What are its limitations?
What are its strengths? You do not need to answer all of these questions exhaustively, but can use them as launching points for your writing.
Attachment:- Final Project Overview.rar