Write a report describing your analysis, following the instructions below. Your score will reflect the informativeness and conciseness of your results and quality of presentation. When you write the report, consider yourself to be presenting your analysis to someone you would like to impress, such as your boss or a consulting client. You would like to get the vital information across elegantly without wasting the "client's" time with extraneous information/discussion. Provide graphs/charts where appropriate. Throughout, think about professional presentation of your results.
Mailing marketing offers can be costly. We want to send out solicitations for donations; each solicitation costs us $0.68. Under NYU Classes->Resources->Datasets you will find a pair of data files for our problem in the correct format for Weka , mailing_hw3 and mailing_hw3_use. You will analyze (at least) three algorithms for these data: tree induction, logistic regression, and any other method of your choice. Your ultimate goal is to build the ‘best' model based on the mailing_hw3 data and then use it to target new prospects from the "Use" data based on your analysis. Since you do not know the label in the "Use" data, it is currently showing 0 for all examples - but understand that the 0 is just a placeholder, not the truth. You have a budget of $5000, and you will decide how to spend that on targeting.
A) For tree induction and logistic regression, first determine how to set a critical complexity parameter. For tree induction, turn pruning off and use the "minimum number of objects" parameter to control complexity. For logistic regression, use the "ridge" parameter to control complexity. Report the parameter value that you choose for each model, and how you determined it. Show a chart or graph where appropriate. For the third method you are free to just run it with default parameters or investigate if there is a similar complexity parameter that you can optimize.
B) Compare the three methods with respect to their generalization performance, and choose one as your method for selecting the prospects to target (from the Use data). Describe your process for selecting the method, including any results that support your choice.
C) Apply your chosen model to the Use data and select the prospects you recommend targeting. Include with your report a csv file comprising those prospects that you choose for mailing. Provide the exact row of your chosen prospects as you found it in the mailing_hw3_use file. Can you come up with an estimate on how many replies you think you will actually get for your chosen list of prospects? We will report back to you the success of your targeting based on the outcomes on the ‘use' data that we have withheld from you.
D) Make a recommendation to data science management: should we invest in more training data for this problem? Describe and explain your recommendation precisely. Support your argument as well as you can with results.
E) Consider if there are additional data that should be available that would help you create better targeting. Explain this information and how you would use it.
F) Discuss the current business scenario of having a budget of $5000 to target. Can you envision a better strategy if the objective is to maximize the total profit from the campaign? Consider again what additional information you might want that should be available easily.