INFSCI 2130 / ISSP 2240

Decision Analysis and Decision Support Systems

Syllabus

 

 

 

 

 

 

 

Marek J. Druzdzel

School of Information Sciences

University of Pittsburgh

Spring 2000 (00–2)

 

 

 

 

OVERVIEW:

 

"The average man's judgment is so poor, he runs a risk every time he uses it." — Edgar W. Howe

 

Most real world problems involve uncertain information. Although uncertainty can be often reduced, it can be seldom eliminated and whether we are dealing with scientific, engineering, or personal problems, we are forced to make decisions that are based on incomplete knowledge. Even a deliberation of whether more information should be collected before making an actual decision is itself a decision under uncertainty. Decision making under uncertainty has been addressed in mathematics by probability theory and expected utility theory. These two together are known as decision theory. The discipline that focuses on applying decision theory in practice is known as decision analysis. Decision analysis offers a set of structured procedures that assist decision-makers in

• structuring decision problems and developing creative decision options

• quantifying their uncertainty (this includes combining available statistics with expert judgments, and their own beliefs to arrive at estimates of the probabilities of various outcomes)

• quantifying their preferences (this includes structuring their value tradeoffs and examining their attitude towards risk)

• combining their uncertainty and preferences to arrive at optimal decisions

This course provides an introductory treatment of decision analysis, along with elements of human cognition under uncertainty. The intended participants are students who want to learn more about decision making under uncertainty and tools that can be used to support it. Knowledge of these tools may prove useful in your personal decision making and in decisions that you will be making during your professional career. Should you choose to become a professional supporting decisions of others (and this is a good way to make a living), this course will lay foundations for your future studies. Most of all, and this is the reason why this course is in the information science foundations area and is cross-listed in the Intelligent Systems Program, this course should give you solid foundations for applying the ideas of decision analysis in intelligent information systems and decision support systems.

As you might have already experienced by now, being an engineer or a scientist requires intelligence, independent, creative thinking, and most of all commitment to hard working. This course reinforces this. The material is not really difficult, but you will have to invest quality time in order to master it. There will be a term project that will normally involve applying the methods learned in the course to model a real (or realistic) decision and build a decision support system to support it. The workload in this class will be heavy, but I believe that you will find it interesting and important. I require your commitment, doing the readings, coming to classes, and being their active participant. In return, I promise that you will have fun and you will learn useful skills.

YOUR RESOURCES:

The course:

 

Name : INFSCI 2130/ISSP 2240

Decision Analysis and Decision Support Systems

CRN : 20722/HONOR

Credits : 3.0

WWW : http://www.pitt.edu/~druzdzel/infsci2130.html

The instructor:

Marek J. Druzdzel

assistant professor, School of Information Sciences, Intelligent Systems Program,

and Center for Biomedical Informatics, University of Pittsburgh

Office : B-212 SIS Building (Decision Systems Laboratory)

Email : marek@sis.pitt.edu

Phone : (412) 624-9432 (office, voice mail)

FAX : (412) 624-2788

WWW : http://www.pitt.edu/~druzdzel

FTP : ftp://ftp.pitt.edu/users/d/r/druzdzel

(host ftp.pitt.edu, directory ./users/d/r/druzdzel)

Teaching Assistant:

Jian Cheng

doctoral student, Decision Systems Laboratory, School of Information Sciences

Office : 201-B SIS

Phone : (412) 624-9432

Email : jcheng@sis.pitt.edu

WWW : http://www.sis.pitt.edu/~jcheng

Meeting times and locations:

Classes (404 SIS Building):

Thursdays, 6pm-8:50pm (break 7:20pm-7:35pm)

Marek's office hours (740 SIS Building):

Wednesdays, 5pm-6pm

Thursdays, 5pm-6pm

Jian's office hours:

TBA

We can be seen at other times by appointment.

Your colleagues:

Name: _____________________________________________

Phone: _____________________________________________

Email: _____________________________________________

Name: _____________________________________________

Phone: _____________________________________________

Email: _____________________________________________

Name: _____________________________________________

Phone: _____________________________________________

Email: _____________________________________________

Name: _____________________________________________

Phone: _____________________________________________

Email: _____________________________________________

 

THE COURSE:

 

Objectives:

The primary objective of this course is to make you acquainted with a set of tools for decision making under uncertainty known under the umbrella name of decision analysis and how these relate to building decision support systems. I expect that you will learn in this course:

• How to use simple techniques for improving your own intuitive judgment and decision making under uncertainty.

• How to apply the tools of decision analysis to aid decision-making under uncertainty.

• How to employ decision-analytic methods in intelligent information systems and decision support systems.

Finally, being successful in the course should contribute to the development of your academic self-esteem.

 

 

Prerequisites:

All students in the course are expected to have taken an elementary course in probability theory and statistics. I believe that it will be helpful for you to have completed at least one course in cognitive psychology. The elements of probability theory necessary for the course will be quite basic and will be reviewed as needed. Familiarity with cognitive psychology will allow you for a better appreciation of the elements of behavioral decision theory. Familiarity with the Windows 95/98/NT environment will be needed, although this is easy to catch up on. You will also be expected to use electronic mail on a regular basis. The most important prerequisite of all, however, is your interest in the course, motivation and commitment to learning.

 

Required reading:

The principal textbook for the course is:

Robert T. Clemen "Making Hard Decisions: An Introduction to Decision Analysis." Second Edition, Duxbury Press, An Imprint of Wadsworth Publishing Company, Belmont, California, 1996, ISBN 0-534-26034-9 (current bookstore price: $83.70)

which is a nice, accessibly written textbook with many practical examples. It is available in the bookstore. I believe that you will enjoy reading it. We will be pretty much following its structure. The textbook will be supplemented with other readings (listed in this syllabus), with one copy of each on reserve in the SIS library (3rd floor SIS Building).

 

Supplementary readings:

There are several good books on decision analysis in case you want to read more. If you are serious about the topic and want to go beyond the required material, I would like to recommend looking at the World Wide Web page listing decision analysis books that I prepared for the Section on Decision Analysis of the Institute for Operations Research and the Management Science (INFORMS). The address of that page is http://www.sis.pitt.edu/~dsl/da-books.html

 

 

Decision-analytic software:

There are several computer programs that support decision-analytic approach to decision making. We will be using GeNIe, a program that we have been developing at the Decision Systems Laboratory, along with its reasoning engine, SMILEJ (Structural Modeling, Inference, and Learning Engine), and its Active-X version, SmileX. GeNIe, SMILEJ , and SmileX are available at: http://www2.sis.pitt.edu/~genie. GeNIe has a comprehensive on-line help that will supplement the textbook material. Furthermore, all influence diagrams from the textbook are available in GeNIe format as a zip file at the following location: http://www2.sis.pitt.edu/~genie/clemen.zip.

In case you would like to learn about other programs of similar type that are available on the market, I recommend looking at the World Wide Web page listing decision analysis software that I prepared for the Section on Decision Analysis of the Institute for Operations Research and the Management Science (INFORMS). The address of that page is: http://www.sis.pitt.edu/~dsl/da-software.html. The page contains electronic pointers to the developers/vendors of the software along with links to demonstration versions. Netica and Hugin may be of particular interest to you. They both come in demo versions that allow for building and saving small models. This may be sufficient for your class work.

 

 

WORK REQUIREMENTS:

 

Computer use:

You will be expected to use a text processor or a typesetting program to write your documents and use the decision-modeling environment GeNIe and possibly its reasoning engine SMILEJ , and its Active-X version, SmileX, for some of the class assignments and your term project. A calculator may prove handy in doing your homework assignments and during the exams. Most of our communication will be electronic and you will be expected to use electronic mail on a daily basis.

 

Assignments:

There will be seven homework assignments that will help you to practice the material covered in class and will help me to identify those parts of the material that you have difficulties with. Assignments will usually be done in groups of two or three students, formed during the class meetings. The assignments have to be turned in on time, and all members of the group are responsible for meeting the deadline.

 

Term project:

A major part of the training that you will receive as part of this course will result from performing a term project. The description of the project is attached to this syllabus. The project involves teamwork and requires your participation in solving a real decision problem and building a decision support system that will support it. You will have an opportunity to apply the techniques learned in the course. You will be expected to team up for the project in groups (usually large; in the past, we had groups ranging between 5 and 15 students), although it is possible to do the project individually. The deliverables are a project proposal, a mid-semester progress report, and a final report. The due dates are marked on the course schedule.

For those students who want to go an extra mile in their work or want to advance their research using decision analytic approach, I have created two other options. The first allows you to contribute to the software development effort in the Decision Systems Laboratory by implementing a useful, self-contained module of GeNIe or SMILEJ .

The second type involves solving a research problem of your choice (I will be more than happy to suggest some problems based on your skills and interests).

 

Exams:

There will be one midterm exam and one comprehensive final exam, both open book. You are advised to bring your calculator, but use of computers is not allowed (the only reason for this restriction is not to disadvantage those students who do not have laptop computers). Examples of exams given in the previous years can be found in the SIS library.

 

 

Time load:

To help you with planning your semester, I would like to give you an idea of the minimal workload in this course. Expect to spend about eight hours quality time outside of class for every class meeting. I estimate that you will need about five hours to do the readings and three hours (on the average) to do the assignments. If you keep up with readings and do the assignments well, you should not need much extra time to prepare for the exams. The term project should normally demand between twenty and thirty hours of your time. The actual load will vary, of course, depending on your background and preparation. If, for example, your knowledge of elementary college mathematics is rusty, you may need more time.

 

Grading:

Your final grade for the course will be determined as follows:

Assignments : 20%

Term project : 30%

Midterm exam : 20%

Final exam : 30%

On the top of this all, you can obtain up to 10% of the total score for in-class participation.

 

 

 

COURSE SCHEDULE

 

 

PART I: INTRODUCTION

In addition to organizing ourselves, the first class will be an introduction to decision making, to the content of this class, and to decision support systems.

January 6 [Readings: Clemen, Chapter 1; Franklin; Druzdzel&Flynn]

Getting to know each other; organization and overview of the course.

Decision making; uncertainty, preferences, and actions;

motivation for decision support; decision support systems.

Rationality, rational behavior; good decisions vs. good outcomes;

foundations of decision-analytic approach to decision support.

Identification of possible topics for class projects.

 

PART II: MODELING DECISIONS

This five-class block is arguably the most important part of this class. We will talk about modeling decisions and, in particular, structuring decision problems. We will learn two modeling tools: decision trees and influence diagrams and learn the usefulness of the latter in modeling decision problems in computer systems. While decision trees may give you some insight into modeling decisions, in this class we will primarily focus on directed graphical models, such as Bayesian networks and influence diagrams. There will be plenty of time to understand the main tools of our trade. We will also advance somewhat on our class projects by brainstorming on the rough model structure for each of the problems.

January 13 [Readings: Clemen, Chapters 2, 3]

Structuring decisions; decision modeling tools: influence diagrams,

Bayesian networks, decision trees, probability trees.

Refinement of the topics for class projects.

January 20 [Readings: GeNIe on-line help]

Introduction to GeNIe and SMILEJ ;

The clarity test; causality and decision analysis;

examples of structuring decisions.

Final decisions regarding topics and organization of class projects.

January 27 *** Project proposal due ***

*** Homework assignment 1 due (GeNIe and SMILEJ ) ***

[Readings: Clemen, Chapter 4]

Structuring decisions; solving decision models.

February 3 *** Homework assignment 2 due (decision trees & influence diagrams) ***

[Readings: Clemen, Chapter 5]

Requisite decision modeling; decision-making cycle;

robustness of a decision model, sensitivity analysis.

February 10 *** Homework assignment 3 due (sensitivity analysis) ***

[Readings: Clemen, Chapter 6; Morgan & Henrion; Philips;

von Winterfeldt & Edwards]

The goal of decision analysis; applications and pitfalls of decision analysis;

ten commandments of good decision analysis; structuring decision problems;

problem structuring aids; techniques for enhancing creativity.

 

 

 

 

PART III: MODELING UNCERTAINTY

Uncertainty is predominant in decision making. Decisions made under the condition of absolute certainty belong to a truly limited class. In this four-class block, we will review the foundations of probability theory and, in particular, its subjectivist Bayesian view. We will relate probability to human intuitive judgment and discuss ways of measuring subjective uncertainty (probability elicitation). We will quickly move through more advanced techniques involving continuous probability distributions, and the expected value of information computation.

February 17 *** Project proposal due ***

*** Homework assignment 4 due (structuring decisions) ***

[Readings: Clemen, Chapter 7]

Elements of probability theory.

Bayesian reasoning in graphical probabilistic models.

February 24 [Readings: Clemen, Chapter 8; Tversky & Kahneman; McKean;

Dawes79; Dawes80; Edland & Svenson]

Subjective probability; probability assessment; the normative

foundations of probability theory and expected utility theory.

The psychology of decision making; human judgment under uncertainty;

heuristics, biases.

March 2 *** MIDTERM EXAM ***

March 5-12 Spring recess

March 16 *** Mid-semester project progress report due ***

*** Homework assignment 5 due (probability assessment) ***

[Readings: Clemen, Chapters 9, 10, 11]

Theoretical probability distributions. Using data.

Monte Carlo simulation.

March 23 [Readings: Clemen, Chapter 12]

Expected value of information (perfect and imperfect).

 

 

PART IV: MODELING PREFERENCES

Preferences are a fundamental aspect of decision modeling. So far in the course, we have used a simple measure of goodness of an outcome, such as its monetary value. We used the expected value to make choices. This view is too simplistic and fails to model the impact of risk on our preferences. Decision theory, underlying decision analysis, postulates a set of basic principles of rational decision making and then, assuming that these principles are something that we want to subscribe to, proves that there is a unique measure of preference (called utility) and shows a way of measuring it (utility elicitation). We will cover the concept of utility, how to measure it, and how to use it in decision models. In the second class of this block we will talk about dealing with decision problems in which the outcomes of a decision process have multiple, possibly conflicting attributes.

March 30 *** Homework assignment 6 due (value of information) ***

[Readings: Clemen, Chapters 13, 14]

Utility elicitation; risk attitudes.

April 6 [Readings: Clemen, Chapters 15, 16]

Conflicting objectives: basic techniques, multi-attribute utility models.

 

 

 

 

 

PART V: GROUP DECISION-MAKING

The methods of decision analysis that we will have covered up to this point assume that there is a single decision maker, whose beliefs and preferences we are modeling. In practice, however, most of the time we are dealing with multiple decision-makers (a team, an organization, or a society) and a multitude of beliefs and preferences. This one-class block will focus on group decision making.

April 13 *** Homework assignment 7 due (utility elicitation and utility models) ***

[Readings: Kleindorfer et al.; Philips]

Combining expert opinions; group and team decision making;

organizational and societal decision making.

 

 

PART VI: CONCLUSION

Our last classroom meeting will be a grand conclusion of the course. I will have studied the final versions of your project reports and your models. We will have project demonstrations and announcement of the winner of Marek's Best Project Award. Please bring to class questions about the material that you may want to discuss before the final exam. In as much as remaining time allows us, we will talk about the course and possible ways of improving it in the future.

April 17 (Tuesday) *** Term project due ***

April 20 Discussion of the term projects, project presentations.

Announcement of Marek's Best Project Award.

Conclusion of the course. Review session for the final exam.

April 27 *** FINAL EXAM ***

 

Sources of readings:

[Clemen] Robert T. Clemen, "Making Hard Decisions: An Introduction to Decision Analysis." Duxbury Press, An Imprint of Wadsworth Publishing Company: Belmont, CA, 1991

[Dawes79] Robyn M. Dawes, "The Robust Beauty of Improper Linear Models in Decision Making." American Psychologist, 34(7), 571-582, July 1979

[Dawes80] Robyn M. Dawes, "You Can't Systematize Human Judgment: Dyslexia." In New Directions for Methodology of Social and Behavioral Sciences, 4, 67-78, 1980

[Druzdzel & Flynn] Marek J. Druzdzel and Roger R. Flynn, "Decision Support Systems." To appear in Encyclopedia of Library and Information Science, Allen Kent (ed.). Marcel Dekker, Inc., 1999 A preprint of this paper is available at http://www.pitt.edu/~druzdzel/abstracts/dss.html.

[Edland & Svenson] Anne Edland and Ola Svenson, "Judgment and Decision Making under Time Pressure: Studies and Findings." In Time Pressure and Stress in Human Judgment and Decision Making, Ola Svenson and A. John Maule (eds.), Chapter 2, pages 27-40, Plenum Press: New York, 1993

[Franklin] Fragments of Benjamin Franklin's letter to his friend Dr. Joseph Priestley on 19 September 1772. In John Bigelow (ed.), "The Complete Works of Benjamin Franklin," Vol. 4, pages 522-523, G.P. Putnam's Sons: New York, 1887

[GeNIe on-line manual] On-line manual of GeNIe, a decision modeling environment developed at the Decision Systems Laboratory and available at http://www2.sis.pitt.edu/~genie.

[Kleindorfer et al.] Paul R. Kleindorfer, Howard C. Kunreuther and Paul J.H. Schoemaker, "Decision Sciences: An Integrative Perspective." Chapters 6, 8, and 9, Cambridge University Press, 1993

[McKean] Kevin McKean, "Decisions, Decisions." Discover, pages 22-31, June 1985

[Morgan & Henrion] M. Granger Morgan and Max Henrion, "Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis." Chapter 3, "An Overview of Quantitative Policy Analysis." Cambridge University Press, 1990

[Philips] Lawrence D. Philips "Requisite Decision Modeling: A Case Study." Journal of Operational Research Society, 33, 303-311

[Tversky & Kahneman] Amos Tversky and Daniel Kahneman, "Judgment under Uncertainty: Heuristics and Biases." Science, 185(4157), 1124-1131, September 1974

[von Winterfeldt & Edwards] Detlof von Winterfeldt and Ward Edwards, "Decision Analysis and Behavioral Research." Chapter 12, Section 12.5 "Pitfalls of decision analysis: examples and lessons," pages 517-529, Cambridge University Press, 1986

 

 

TERM PROJECT

A major part of the training that you will receive as part of this course will result from working on a group project involving solving a real (or realistic) decision problem and building a decision support system that will support the problem. In the project, you will have an opportunity to apply the techniques that you will have learned in class.

Project topics:

We will identify a small number of project topics during the first few classroom meetings. The project topics will be identified and selected based on the expertise and the interest of the class members, significance of the problem, and feasibility of finding an acceptable solution within the scope of a class project (I will help with the feasibility judgment). The decision problems that we will consider will be real (or very realistic). They will be challenging in the sense that the choice from among the possible alternatives will not be obvious and the problem will be sufficiently complex to strain the limits of intuitive judgment. A challenging decision problem will have at least two of the following characteristics: uncertainty, multiple conflicting objectives, a large number of decision options, outcomes that extend over several time periods, or two or more decision makers with conflicting preferences.

Examples of challenging project topics are: evaluation of credit-worthiness of a loan applicant from the point of view of a bank, evaluation of risk related to undersigning an insurance policy from the point of view of an insurance company, evaluation of the quality of a job applicant from the point of view of a hiring company, identification of prospects for gain in a real estate deal, diagnosis of machine failures, diagnosis or therapy choice in a small area of medicine, etc. It is important that the decision problems that we identify are real, as this will allow you to appreciate the power and the limitations of the methods that you will have learned.

Project teams:

We will spend some time during our classroom meetings talking about the projects and the relevance of the material to your project. The real work, however, will happen outside the classroom. You will form moderately large teams (in the past, the teams ranged between 5 and 15 students) that will split the work and carry the project to a successful completion. The teams will be formed during the first class meetings. Each member of the team carries the responsibility for the success or failure of the entire team. While any conflict within a group is that group’s internal matter, I reserve the right to lower the grade of those students who fail to make a significant contribution to the project (as indicated by other team members).

Alternative project topics:

To accommodate those students, who want to go an extra mile in their technical work or want to advance their research using decision analytic approach, I have created two other options. The first allows you to contribute to the software development effort in the Decision Systems Laboratory by implementing a useful, self-contained module of GeNIe or SMILEJ . Examples of modules developed in the framework of this class in the past include algorithms for decision-theoretic reasoning, user interface modules such as graph layout algorithm, and a Genie agent.

The second type of alternative project involves solving a research problem of your choice. Most research fields have something to do with decision making and you can carve out an aspect of your current research that will tie nicely to this class. I will be also glad to suggest small and manageable research-oriented projects within the Decision Systems Laboratory. There have been at least two projects in the past that led to publications.

Project proposal:

The choice and framing of a decision problem will be a major factor in the success of your project. In order to prevent you from wasting your time and to make sure that your topic is challenging enough, I want every team to submit a written statement of the problem (project proposal) for my approval. The proposal, due in about a month from the starting date of the course (the deadline is marked on the syllabus), should in case of the first project type clearly state:

• relevant facts about the decision maker,

• decision maker's objectives,

• the decision problem that the team is planning to address,

• a list of available decision options,

• a list of (possibly conflicting) objectives,

• the key sources of uncertainty and the potential data sources that might reduce this uncertainty.

Details of project proposal, progress report, and final report concerning the project of an alternative type (software development and research) are quite flexible and will be negotiated on an individual basis in the beginning of the semester. Typically, the deliverable of a research-oriented project is a draft of a paper and the deliverable of a software implementation project is C++/Java/Visual Basic code with clear comments.

 

The proposal can be fairly short — a few pages usually suffice. I will let you know whether your formulation of the problem is acceptable as it stands or what you can do to improve it. If your proposal is not approved, you will have to submit another within a week. Once your proposal has been approved, you can proceed with your project.

Mid-semester project progress report:

You will be expected to submit a mid-semester project progress report containing an introduction to the final report, a worked out structure of the problem (a GeNIe graph; the numerical parameters do not need to be elicited at that point), description of the work that you have completed so far, and a detailed plan of action for the remainder of the semester. The main purpose of this report is to help you in planning your work and spreading it over the course of the semester. The deadline for submitting the mid-semester progress report is marked on the syllabus. You should attach to your progress report your original proposal with my comments.

Computer support:

You should use the GeNIe/SMILEJ /SmileX decision-modeling environment to build your model and to support your calculations. You should normally build your own user interface to SMILEJ (or SmileX) that will be customized for the user of your decision support system, in a language like C, C++, Java, Visual Basic, or make the model available and usable through an HTML page.

Project report:

The main deliverable of the project will be a brief formal report describing in detail the decision maker, the decision problem, decision options, your analysis, your recommendation, and sensitivity of this recommendation to various elements of your model. Your analysis should use quantitative methods that you will learn in this course and use GeNIe/SMILEJ /SmileX to build a model of your decision. A "common sense" type, rhetorical argument lacking a quantitative analysis cannot earn you a passing grade. The GeNIe model should contain extensive comments related to its individual elements.

Imagine being an external consultant who has been hired to help with the decision and who is expected to produce a written analysis of the problem and a simple decision support system. Your report should be not longer than 10 single-spaced pages with one-inch margin at each side. This limit includes all figures, tables, graphs, and references. Within this limit you should be concise and specific. Real decision-makers will rarely read anything that is longer than this — they are too busy. Most important facts about the problem, the model, the source of numerical parameters, and insights obtained from the model should be included directly in your GeNIe model. An electronic version of your model should be a part of your mid-semester and your final reports. You should attach to your final project report both your original proposal and your mid-semester progress report with my comments.

As an indication, your report might contain the following sections:

Problem description

This should be ideally a revised version of your project proposal.

Model description

This section should describe your view of the world that the decision-maker is facing. It should tie the decision options and all relevant variables that you have considered with the possible outcomes. In case there is no uncertainty, your model will consist of equations or logical statements that tie the decision options with the outcomes. If there is uncertainty, these equations or statements will contain probability distributions. Make sure that you provide a justification for each of the non-trivial elements of this model and make all your assumptions explicit. This section has to be completed by the mid-semester deadline.

Quantification of uncertainties

This section should describe briefly the steps that you took to quantify the uncertainties in your model. If you performed elicitation of uncertainty from the decision-maker, you should describe the method used and the judgments on which your final results are based.

Quantification of preferences

This section should describe how you assigned the values or utilities to the outcomes in your model. It should clearly specify the evaluation function and also the judgments upon which that function is based.

Evaluation

This section should describe the results of your analysis of the decision options. It should include sensitivity analysis that will identify those factors that are most crucial for your conclusions. If there are any insightful graphs that you have obtained using a modeling tool, this is the place where you should include them. The conclusion of this section should be the action that you recommend.

Suggested approach:

It is often a good idea to start with a simplified model and a rough sketch of uncertainties and preferences. I expect that for some of the project teams we will start on such a sketch in the classroom. This simple model can then be refined by elaborating on those of its elements that are important for the decision. One way to identify these elements is to use simple methods for sensitivity analysis, such as the tornado diagrams. This approach will protect you from building a huge model that will imply numerous elicitations, many of which could be avoided.

Suggested organization of the project team:

Since the project teams may be rather large (as a class, we will address three or so different projects), you will need to organize your work well so as not to waste your time and effort. I suggest that you choose a project leader who will coordinate your work, will communicate with me if necessary, and will make important decisions in case of differences of opinion that cannot be resolved inside the group. In addition, it may be a good idea to identify from among the group members those students who are domain experts, i.e., individuals who have either experience or interest in the domain and have access to information needed for the model. Ideally, it would help you a lot to get a real organization (your employer?) interested in your project and have them commit some minimal resources to it with the expectation of gain from your final result. Other team members can focus on collecting information about various uncertainties related to the model. Yet other team members can focus on the outcomes of the decision process and the preference structure over these outcomes. Small sub-groups (one or two project team members) can focus on GeNIe implementation of the model, writing reports, and building a custom interface to SMILEJ for the end user of your decision support system. Try to avoid general meetings, as these may be hard to schedule (most of us are very busy) and hard to conduct efficiently when the number of participants is large. Skillfully scheduled and coordinated smaller work meetings may get you a long way.

Evaluation criteria:

The criteria for grading your project reports are: organization and planning of your work (as expressed by your proposal and mid-semester progress report), soundness, creativity, and, finally, clarity of your writing and expressing your ideas.

Aim at excelling in your project. If you choose a problem that is of interest to a wider audience, for example a policy problem confronted by the School of Information Sciences, you may share your report with the faculty or other decision makers here or submit your project to one of the school's best paper competitions. You can also try to publish your report as an article describing a successful application of decision analysis. Research-oriented projects should quite naturally lead to a publication — two projects in the past did. Software implementation will get you acquainted with the internals of Bayesian networks and influence diagrams and will earn you DSL's gratitude and an acknowledgment inside the package. Both research and implementation work can be continued within DSL as an independent study or a thesis later, if you choose so. A winning project may help you to advance your career. It is easier and more rewarding to excel as a student than as a "mature" professional.

Marek’s Best Project Award:

All projects will be demonstrated during the last class meeting (see the course schedule for the date). The best project in class (as judged by the founder of the award) will be awarded a doughnut and a drink of the team’s collective choice along with an accompanying certificate. I reserve the right to give no award if the quality of the projects is not high enough or award the whole class with cookies if all projects turn out to be excellent.