Assignment - Modeling Uncertainty
The goal of this is to construct a probability model to forecast an uncertain event. There is no limitation on what the event is, as long as it can be modeled using Bayes formula with several clues.
Select an expert that will help you construct the model. Make an appointment with the expert and construct/validate the model. You should follow steps described in the lecture to construct Bayes model to make prediction. In this project, it is important that you use methodologically correct, and then test the model. Alternatively, you can obtain a dataset and with your expert select the most relevant clues from the data. Then use the data to estimate likelihood ratios and everything else that is needed to apply Bayes formula.
Prepare a narrated PowerPoint slide presentation that answers all of the following questions:
1. What problem is being addressed?
a. What are the assumptions about the problem and its causes?
b. What is the uncertain event?
c. Why would a model be useful and to whom would it be useful?
2. What clues and clue levels can be used to make the prediction?
a. Which clues are conditionally independent?
b. How did you check for conditional independence?
3. What are the likelihood ratios associated with each clue level or combination of conditionally dependent clues?
a. Show examples how did you obtain likelihood ratios from the expert.
4. Show the complete model. The model should include:
a. All clues, their levels and likelihood ratios
b. Prior odds
c. One easy way to present a complete model is in a form of a table in Excel.
5. Show the application of the model in at least one case/scenario (illustrate your calculations).
6. What is the evidence that the model is valid?
a. Is the model based on available data?
b. Did the expert consider the model simple to use?
c. Did the expert consider the model "Face valid"?
d. Does the model simulate the experts' judgment on at least 15 cases/scenarios?
e. Graph the relationship between model scores and experts' ratings.
f. Report the correlation between model score and experts' ratings.
g. If data are available, report whether the model correspond with other measures of the same concept (Construct validity)?
h. If data are available, report whether the model predicts any objective "gold standard."
7. What are conclusions from the project?
Do not exceed 12 slides, do not exceed 2 minutes per slide. You presentation will be 10 mins.