Structuring Decision Problems Page 1 of 33 EJOR 09-02-08
Structuring Decision Problems: A Case Study and Reflections for Practitioners
Detlof von Winterfeldt
University of Southern California
and
Barbara Fasolo
London School of Economics and Political Sciences
September 2, 2008
ABSTRACT
This article reviews several approaches to problem structuring and, in particular, the three-step
structuring process for decision analysis proposed by von Winterfeldt and Edwards: 1)
identifying the problem; 2) selecting an appropriate analytical approach; 3) developing the a
detailed analytic structure. This three-step process is re-examined in the context of a decision
analysis of alternative policies to reduce electromagnetic field exposure from electric power
lines. This decision analysis was conducted for a public health organization funded by the
California Public Utilities Commission and it was scrutinized throughout by interested
stakeholders. As a result a significant effort went into structuring this problem appropriately,
with some successes and some missteps. The article extracts lessons from this experience,
updating existing guidance on structuring problems for decision analysis, and concluding with
some general insights for problem structuring.
Keywords: problem structuring; decision analysis; decision analysis application; evaluation;
stakeholders; detailed structures; mixing models; behavioural lessons
Structuring Decision Problems Page 2 of 33 EJOR 09-02-08
1. Introduction
Most operations researchers agree that structuring a problem – taking it from an initially
vague and ill defined problem to one that can be formulated, modeled and analyzed
mathematically – is the hardest yet most crucial part of an operational research (OR) analysis.
This is certainly true in decision analysis, where the emphasis of problem structuring is on
shaping some general statements by decision makers and stakeholders about their goals,
concerns, issues, and uncertainties and turning these statements into a clear and transparent
representation of the decision problem which can be mathematically formalized using principles
of decision theory (Winterfeldt and Edwards, 1986; French, 1988; Belton and Stewart, 2002).
The importance of problem formulation and structuring for OR is well reflected by the
large, and growing, literature on problem structuring methods, briefly summarized below. Some
of this literature discusses applications of problem structuring methods during the early stages of
decision analysis. Further, guidance on structuring problems for the specific purpose of
conducting decision analysis is available in the form of a three-step process (identifying the
problem; selecting an appropriate analytical approach; developing a detailed analytic structure),
originally proposed by von Winterfeldt and Edwards (1986, chapter 2).
The purpose of this paper is not to expand literature on problem structuring principles and
methods. Instead, we will use one very complex decision analysis that was conducted about ten
years ago to illustrate an application of the three-steps structuring process, and to extract lessons
for how to improve decision analytic problem structuring further. While this paper is primarily
about lessons for structuring problems into a framework amenable to decision analysis, many of
the lessons carry over to operational research modeling conducted in a facilitative mode.
Structuring Decision Problems Page 3 of 33 EJOR 09-02-08
In the followings sections we will first review some of the literature on structuring
decision problems for operational research in general and decision analysis in particular. The
subsequent sections will describe the structuring process in a specific decision analysis problem,
reflecting on successes and failures along the way. The concluding section will summarize the
lessons learned and include comments on how to improve the art and craft of structuring decision
problems for decision analysis.
2. Problem Structuring Methods in Operational Research.
Over the past 40 years, operational researchers have witnessed the growth in the theory
and practice of problem structuring methods (PSMs) – a collection of methods developed
pragmatically in the U.K. and elsewhere, including, but not limited to, Strategic Options
Development and Analysis (or SODA, Eden and Ackermann, 2001), Strategic Choice Approach
(or SCA, Friend and Hickling, 2005), and Soft Systems Methodology (or SSM, Checkland,
2000). All PSMs aim to support groups confronted with problems involving multiple actors,
multiple conflicting perspectives and key uncertainties (for an excellent survey of the most
common PSMs, and applications, see Rosenhead and Mingers, 2001; Mingers and Rosenhead,
2004). All PSMs share the ability to model the problem situation so that the people involved are
clearer about the issues at stake, and can converge on, or commit to, a potentially actionable set
of priorities. The process is often iterative and accessible to all participants, so that each
participant can contribute perspectives. Each PSM however tends to be particularly apt at
supporting a particular task faced by the group (e.g., SCA has a well developed process for
representing different sources of uncertainty while SSM has a strong focus on system redesign;
for a comparison of tasks facilitated by different PSMs, see Franco and Meadows, 2006). It is
Structuring Decision Problems Page 4 of 33 EJOR 09-02-08
because of this differential aptitude to tackle important aspects of a messy problem that PSMs
lend themselves nicely to being applied in combination. This practice of mixing methods, known
as ‘multi-methodology’ was rare in the early years of PSMs development, but is becoming
increasingly common and advocated by PSM practitioners (Mingers, 2001; Ormerod, 2005).
‘Mixing’ does not only refer to different PSMs being used in combination (e.g.
developing the IT strategy for a leading UK Supermarket chain with a combination of SODA’s
cognitive mapping, SSM and SCA’s comparison mode; Ormerod, 2005) but also, and
increasingly, PSMs being used in combination with decision analysis. For instance, elicitation of
multiple conflicting objectives and modeling of these into value trees have been successfully
‘mixed’ with SODA and cognitive mapping (e.g., Belton, Ackermann and Shepherd, 1997),
causal maps (Montibeller and Belton, 2006), and drama theory (Losa and Belton, 2006).
3. Structuring Problems for Decision Analysis
Decision analysis is a set of formal models and tools, based on utility and probability
theory, aimed at improving decision making (Clemen, 1996). Decision analysis is especially
useful when decision problems involve multiple conflicting objectives and uncertainties. To
decision analysts a “decision analysis structure” means a formal representation of a decision
problem without the actual numerical formulation and the modeling and analysis that typically
goes along with it (e.g., von Winterfeldt and Edwards, 1986). At its most basic level a decision
analysis structure is similar to what others have called a “decision frame” (e.g., Kahneman and
Tversky, 2000). A “frame” in decision analysis defines the scope of the decision problem,
including the decision maker(s) and stakeholders, their values and alternatives, the range of
consequences of concern, and the key uncertainties. The literature on structuring problems for
Structuring Decision Problems Page 5 of 33 EJOR 09-02-08
decision analysis suggests that structuring does not only involve framing the problem, but also
the two additional steps of selecting appropriate analysis structures (e.g., a decision tree structure
or a multiple objectives structure) and developing these structures in detail – before numerical
modeling and analysis begins.
These three steps, proposed in von Winterfeldt and Edwards (1986, 2007) and examined
in detail below, are not necessarily applied in a sequential manner. Spetzler (2007) emphasizes
the iterative and interactive nature of the structuring phase of decision analysis. Following work
with his colleagues at the Strategic Decision Group (SDG), he considers structuring a very
important part of the decision analysis cycle (see Figure 1). In this cycle, the decision maker(s)
– here called the “decision board” – and the decision analysts – here called the “decision team” –
interact throughout the decision process, with the early emphasis of structuring – designing the
process, refining the focus, and developing alternatives. Because of the snake-like impression of
the arrows going up and down between the decision makers and the analysts, this diagram is also
often referred to as the “snake diagram.”
Figure 1: Decision Analysis Process in a Snake Diagram (Spetzler, 2007)
.
DECISION
TEAM
1. ASSESS
Business
Situation
2. DEVELOP
Alternatives,
Information,
and Values
3. EVALUATE
Risk and
Return of
Alternatives
5. PLAN
for
Action
6. IMPLEMENT
Decisions and
Manage
Transition
• Frame
• Challenges
• Understanding
• Alternatives
• Improved
Information
• Values
Decision Plan
DECISION
BOARD
0. DESIGN
PROCESS
AGREE TO
ALTERNATIVES
REFINE
FOCUS
APPROVE
PLAN &
BUDGET
4. DECIDE
Among Alternatives
Evaluated
Alternatives
DECIDING DIRECTION CHANGING DIRECTION
Deliverables at
Major Reviews
• Well-Defined
Process
• Set up for
Success
Structuring Decision Problems Page 6 of 33 EJOR 09-02-08
Similar ideas about the importance of interactive structuring have been emphasized by
Keeney (1992) in “Value Focused Thinking” and related publications of his. Keeney emphasizes
the need to focus on values and objectives early and to generate decision alternatives using these
values and objectives.
Phillips (1984, 2007) introduced decision analysts to the concept of a “requisite decision
analysis model.” He defines a requisite model as one that “is sufficient in form and content to
resolve the issue at hand” (Phillips, 2007, p. 392). He states that a model is requisite, “if no new
intuitions arise in the group.” (p. 392). While requisite modeling can be best recognized when a
full model is developed, including elicitations of important numbers, this notion can also be
applied to decision analysis structure, implying that there can be structural representations that
are simple enough to capture the essence of a decision problem, and no more complicated that
necessary to obtain sound insights. A decision analysis structure is requisite if no additional
insights emerge that would lead to significant additions or modifications of the structure.
von Winterfeldt and Edwards (1986; 2007) proposed the following three-step procedure
for structuring decision problems in a decision analysis.
STEP 1: Identify the problem.
In the first step the following questions are answered. What is the nature of the problem?
Who is the decision maker? What are the decision maker’s values? What are the generic classes
of options? What groups are affected? What is the purpose of the analysis? At this stage only
rough formal relations are created. Simple lists of alternatives, objectives and events are the
typical products of this task. (von Winterfeldt and Edwards, 2007, p. 82).
STEP 2: Select an analytical approach.
In the second step, the analyst has to make a choice of the main analysis framework,
answering questions like: Is uncertainty the key problem, or are conflicting values more
important? Is it worthwhile to model parts of the problem with non-decision-analytic techniques
like linear programming or simulation models? How can different part models be combined
creatively? It is useful to avoid an early commitment to a specific decision analytic approach
Structuring Decision Problems Page 7 of 33 EJOR 09-02-08
(e.g., multiattribute vs. decision tree) and to explore alternatives in some detail. Often, the best
solution is to combine approaches (von Winterfeldt and Edwards, 2007, p. 82-83).
STEP 3: Develop a detailed analysis structure.
This step involves the more familiar territory of trees, diagrams, and networks (von
Winterfeldt and Edwards, 2007, p. 83).
There are several analytic structures available for this step, depending on the nature of the
problem:
• Evaluation problems
o Means-ends networks
o Objectives hierarchies
o Consequence tables
• Decision problems under uncertainty
o Decision trees
o Influence diagrams
• Probabilistic inference problems
o Event trees
o Fault trees
o Belief networks
These three steps have important commonalities with other PSMs (e.g. Morton,
Ackermann and Belton, 2003; Rouwette and Vennix, 2006). Step 1, for instance, is at the very
heart of all of these methods, which emphasize the importance of probing for the real problem,
understanding the purpose of the analysis and the group constituency (problem-owners vs.
stakeholders) as well as finding the most suitable approach or combination of approaches to meet
the clients’ needs. There is more variance with regard to how PSMs address and implement Steps
2 and 3. For instance, with SODA, key issues of concern (be they issues surrounding conflicting
views, uncertainty, or both) are modeled and displayed graphically by means of ‘cognitive maps’
– a network of concepts capturing an individual (or a group’s) mental model of the issues (Eden
and Ackermann, 2001). On the other hand, with SCA, values are structured separately from
uncertainties, in terms of distinct, but interconnected, ‘areas’ (decision areas, comparison areas,
uncertainty areas, and decision graphs, Friend and Hickling, 2005). SSM practitioners structure
Structuring Decision Problems Page 8 of 33 EJOR 09-02-08
problems by drawing ‘rich pictures’, building root definitions, conducting CATWOE
(Customers, Actors, Transformation, Worldview, Owners and Environment) analyses, and multi-
level thinking (PQR, Checkland, 2000)
An important aspect to note is that in PSMs structuring allows ‘local, partial’ solutions
rather than global solutions that imply a ‘merging’ of different views (Mingers & Rosenhead,
2004). This means that values and uncertainties are structured in qualitative ‘incommensurable’
form. This distinguishes the structuring of PSMs from the structuring activities in decision
analysis, which aims at developing a quantitative model of the decision maker’s values
(multiattribute utility problems) and perceptions of uncertainties (uncertainty problems), where
the two are integrated.
4. Three steps structuring in the EMF case
In the followings sections, we will investigate how each of these three steps was carried
out in a major decision analysis of alternative policies to reduce the potential health risks posed
by exposure to electromagnetic fields (EMF) from power lines. The focus is the three-step
structuring process and lessons learned from it; the resulting results and recommendations are
described elsewhere (von Winterfeldt et al., 2004).
Step 1: Identify the Problem
1.a What is the nature of the problem? When decision makers approach a decision
analyst for help, they often only have a general idea of what their problem is. For example, an
R&D organization may request help with prioritizing R&D projects and using this prioritization
to allocate scarce funds. A government agency concerned with homeland security may be
interested in determining the best ways to protect commercial airlines from terrorist attacks. An
Structuring Decision Problems Page 9 of 33 EJOR 09-02-08
individual may worry about his or her future and wants help in thinking through possible career
paths. In most cases, the initial discussions between the decision maker and the analyst change
the nature of the problem – in some cases substantially.
Initially, the EMF problem appeared to be relatively straightforward. Almost thirty years
ago, Wertheimer and Leeper (1979) published an article suggesting a statistical association
between proximity of homes to electrical power lines and the incidence of childhood leukemia.
Since then several hundred million dollars of research funds were spent to determine the possible
relationship between EMF exposure and human health effects. In the late 90s, when the analysis
described in this article was underway, a major effort was launched in California to determine
whether the concern was sufficiently severe to warrant mitigations efforts (von Winterfeldt et al.,
2004). Official opinions ranged from dubious to cautious. In 1996, the National Research
Council stated “there is no conclusive evidence that EMF causes cancer” (NRC, p. 4). In 1999,
the National Institute of Environmental Health Sciences stated that “The scientific evidence
suggesting that EMF exposures pose any health risk is weak,” but that “EMF exposures cannot
be recognized as entirely safe, because of weak scientific evidence that exposures may pose a
leukemia hazard” (NIEHS, p. 1 and 2).
While some epidemiological studies support an EMF-cancer link, two major problems
stand in the way of drawing firm scientific conclusions. First, the energy created by electric or
magnetic fields is so small that it is hard to create a plausible biophysical mechanism that can
explain the relationship, leading some skeptics to dismiss the possibility out of hand. This
skepticism is validated by the fact that initial data supporting any specific biological hypotheses
so far could not be replicated. Second, alternative hypotheses have been proposed. For example,
several studies are underway to examine the possibility of a selection bias in past
Structuring Decision Problems Page 10 of 33 EJOR 09-02-08
epidemiological studies. Other studies test an alternative explanation, namely that the increased
incidence of childhood leukemia is due to currents on the metallic surfaces of bathtubs.
In 1993, the California Public Utilities Commission of California (CPUC) initiated a
program to fund a series of studies to explore its policy options for addressing the
electromagnetic field issue. Funds were made available to the California Department of Health
Services (CDHS) to select and support scientific studies to clarify the issues. In 1994 the CDHS
issued a Request for Proposal (RfP) for a study called the “Power Grid and Land Use Policy
Analysis” that asked for research on cost-effective ways to reduce electromagnetic fields
exposure, in case a strong EMF-health link would emerge in future research.
The first author’s company, Decision Insights, Inc., responded to this RfP and won the
award to conduct the study. Recognizing the need for health, engineering, and cost expertise,
several consultants were hired in these areas, including two prominent researchers from Carnegie
Mellon University who had conducted previous work in this area, an electrical engineer and a
health physicist. In addition, cost analyses were subcontracted. The lead work, which used
primarily decision analysis tools, was conducted by the core staff of Decision Insights, Inc.
1.b. What is the Problem Environment and who are the Stakeholders? An initial meeting
was held with the sponsor (CDHS) and consultants – only to be faced with a relatively hostile
group of stakeholders, who advised the sponsor of the study. These stakeholders included
residents who were convinced that the EMF issue had eroded their property values, utility
representatives who thought that the EMF issue was blown out of proportion, health specialists,
union representatives, and representatives of the CPUC – the agency who we had considered the
ultimate client of our study. As it turned out the primary role of the CPUC representative was to
Structuring Decision Problems Page 11 of 33 EJOR 09-02-08
listen while the task of guiding our project was performed by the representatives of the CDHS,
who often deferred to the stakeholders’ wishes when it came to shaping our study.
Learning about the EMF problem environment was another important task at this stage.
Some of our consultants and collaborators had already participated in previous studies, but the
analysis team at Decision Insights, Inc. had only superficial experience with the issues. Learning
about exposure assessment, epidemiology, and health effects was a difficult process, but it was
also important to establish credibility with the decision makers and stakeholders. For example,
early in the project, the project staff participated in a two-day crash course in mapping magnetic
exposures using various devices to locate and measure magnetic fields around power lines and in
houses. They also attended many meetings and conferences on the EMF issue, including the
annual conference on bioelectromagnetics. They learned just enough about the topic to be able
to publish a paper in the journal “Bioelectromagnetics” (von Winterfeldt and Trauger, 1994).
While the need to ‘learn’ about the content of the problem is a common experience for
facilitators of decision workshops (e.g., Papamichail, Alves, French, Yang, and Snowdon, 2007),
it is also very hard work. Decision analysts and operations researchers, who usually are experts
in modeling, simulation, and analysis, cannot afford to immerse themselves in too many
substantive application topics. As a result, DA and OR practitioners specialize in selected areas.
1.c. Who are the Decision Makers and what are their Values? Initially we thought that
the decision maker was the CPUC, because this organization provided the funds for the study
and had the ultimate power to implement any recommendations. However, the CDHS and its
representatives were directly in charge of managing our study. The stakeholder advisory group
was intended to be just that – advisory to the CDHS, but it was clear from the beginning that we
needed some level of consent from this group in order to proceed with our plans.
Structuring Decision Problems Page 12 of 33 EJOR 09-02-08
As a result of our meetings with the staff of the CDHS and its stakeholder advisory
group, we began to think of the CDHS and its stakeholder advisory group as our real “clients.”
They were not the decision makers, but if we could not address their concerns and convince them
about the value of the analysis insights and recommendations, the real decision maker, the
CPUC, would likely ignore our work.
Table 1 lists some of the stakeholders in this decision problem and their key concerns.
We developed this table after several meetings with the staff of the CDHS and its stakeholder
advisory group. The intent was to consider all individuals, groups or organizations that actively
participate in or are affected by possible EMF policies. Many of the stakeholders were
represented in the stakeholder advisory group. It is also evident that these stakeholders have
very different, and often conflicting, values and concerns.
1.d. What is the Purpose of the Analysis? An important initial step in communicating
with the clients of an analysis is to explore why the analysis is needed, who is interested in the
results, and how the results are likely to be used. It is tempting to accept the client’s formal
statements at this point, as expressed, for example, in a request for proposal. But it is almost
always educational and sometimes crucial to probe for the “real” reasons for the request.
The original purpose for the study, described in the request for proposal issued by the
CDHS was very general. It suggested that the analysis was to support many different decision-
makers, including regulators, the utilities, and environmental and residents’ groups. The request
for proposal suggested exploring a variety of alternatives for reducing EMFs, including standard
setting, engineering fixes, and land use restrictions. Regarding objectives, the request for
proposal suggested that the analysis should consider a broad range of concerns including health,
cost, property values, environmental justice, and others.
Structuring Decision Problems Page 13 of 33 EJOR 09-02-08
Table 1: Stakeholders and Their Concerns with EMF Exposure
Stakeholder Major Concerns Examples
Utilities Service Pacific Gas and Electric
Reliability Southern California Edison
Cost San Diego Gas and Electric
Los Angeles Water and Power
Regulators Safety California Public Utilities Commision
Health California Energy Commission
Reliability California EPA
Cost California Independent Systems Op.
City Councils
Rate-Payers Utility Rates Ratepayers' Association
Residents EMF-Exposure Citizens Concerned about EMF
Property Values Undergrounders
Rent
Environmental Environmental Impacts Sierra Club
& Advocacy
Groups
Health Environmental Defense Fund
Unions Worker Safety Electric Utilities Union
Worker Health
Salaries
Research EMF Research Base Electric Power Res. Institute
Agencies Competing Research Nat'l Institute for Env. Health Science
U.S. EPA
Cal. Energy Commission
Cal. Dept. of Health Services
Professional Enhance Profession Bioelectromagnetic Society
Organizations Physics Society
American Industrial Health Council
National Brain Tumor Foundation
Parent-Teacher Organizations


The CPUC appeared to be asking a question that it did not know an answer to: What
policies should we implement if the EMF-cancer link is proven to be true? The CDHS, on the
other hand, had a stronger interest in clarifying the EMF issue itself – i.e., to determine what
Structuring Decision Problems Page 14 of 33 EJOR 09-02-08
factors affected the choices of a policy, including factors related to equity and social justice. The
residents living near power lines and the utilities had probably already made up their minds. In
the case of the residents, they strongly preferred to place the power lines underground to
eliminate the health effects and to increase (or restore) property values. The utilities preferred
the low cost or no cost options to reduce electromagnetic fields and strongly opposed
undergrounding. In many ways, the purpose of the analysis was to support the CPUC and the
CDHS with an objective analysis and results that would enable them to referee this conflict.
1.e. What is the generic class of alternatives? One of the most productive steps in
decision analysis is to identify the decision alternatives and the key objectives that the decision
maker tries to achieve. After alternatives are defined, a vague problem often turns into an
opportunity, as interesting options emerge that are not just reactive to the problem, but include
ideas that create value proactively (Keeney, 1992). In the EMF context an example is the
inclusion of options that reduce EMF exposure and at the same time improve service reliability
and improved uses of the right-of-ways.
On the surface, the RfP asked to create and assess “policy options” in response to a
possible EMF-cancer link. While undergrounding and minor adjustments to the overhead system
were the major options on the table, the RfP also asked to investigate “land use” options, which
could broadly be interpreted as routing alternatives to avoid populated areas and as increases in
the right-of-way (ROW) of existing and planned power lines.
At this step in the structuring process we made two key decisions: First, to analyze eight
types of decision problems (see Table 2) with their own sets of alternatives; second, to analyze
within each decision problem a specific scenario e.g. a 15 mile stretch of an existing 115 kV
power line on a right-of-way with a 50 foot clearance on each side, with typical housing and
Structuring Decision Problems Page 15 of 33 EJOR 09-02-08
population density of a major city. This decision was made early in the structuring process, since
it seemed impossible to estimate the consequences of mitigation strategies at a high level – e.g.,
undergrounding all lines in the State of California. This made the initial analysis simpler, but
created complications for drawing conclusions for the California as a whole.
Table 2: Types of EMF Decision Problems
Grid Component Existing New
Transmission Lines
Distribution Lines
Substations
Home Grounding Systems
.
While the alternative differed somewhat across the eight decision problems, they could
usually be grouped into three categories: No changes to the source of EMF exposure, moderate
changes, and major changes. For example for existing transmission lines, we considered no
change, compaction of overhead lines and undergrounding; for new transmission lines we
considered regular routing and design, compact design, undergrounding. For new lines we also
considered route changes and wider right-of-ways, but these alternatives turned out to be more
costly and less effective in reducing EMF exposure than undergrounding.
Step 2: Select an Analytical Approach:
It was clear from the start that we were facing a problem with significant uncertainties.
The main uncertainty was whether EMF exposure was a health hazard or not. Early in the
project we sketched out a simple decision tree that captured these uncertainties – see Figure 2.
Structuring Decision Problems Page 16 of 33 EJOR 09-02-08
Figure 2: Initial Decision Tree for the EMF Problem
The problem clearly involved multiple objectives as well. The options varied on many
objectives, including health effects, costs, impacts on property values, and impacts on service
reliability. For example, undergrounding the power lines reduced potential health effects
substantially, but cost around $1-3 million per mile. In addition, undergrounding increased
property values (by eliminating the visual and noise impacts of the lines) and affected service
reliability. The decision tree in Figure 2 distinguished only among health effects. While
childhood leukemia was the major concern at the time, we deliberately wanted to be open to
other possible health endpoints and included other suspected health endpoints. As the project
evolved additional objectives were included, primarily on the request of the stakeholders (von
Winterfeldt et al., 2004).
One of the challenges of this step was to estimate health effects, assuming that EMF
exposure poses a hazard. To do so, we had to first estimate exposures to the population near
power lines given the “no change” and the possible mitigation options. With a “no change”
Structuring Decision Problems Page 17 of 33 EJOR 09-02-08
option, one could, in principle, take direct exposure measurements, though, in practice, it was
infeasible to take exposure measurements along all possible power lines in California. It was
much harder to estimate exposures with proposed mitigation options.
A significant effort was therefore spent on modeling exposures for different mitigation
options. One of our consultants, Jack Adams, was an electrical engineer with experience in this
type of modeling and he developed a simulation model that allowed us to make these estimates.
This simulation model also had the capability to estimate exposures for different metrics (e.g.,
long-term average exposure, peak exposure, etc.), which had been an issue that was raised by
several stakeholders.
Step 3: Develop a Detailed Analysis Structure
We used four of the eight analytic structures proposed by von Winterfeldt and Edwards
(2007): Objectives hierarchies, consequence tables, decision trees and influence diagrams. In
addition, we used a simulation model to predict EMF exposures. We connected all structures
through a model developed with the Analytica software.
3.a. Objectives Hierarchies.
Decision analysts distinguish between means, ends, and process objectives (see Keeney, 1992).
Ultimately, one would like to consider the ends objectives in an evaluation of alternatives.
However, decision makers and stakeholders usually express a mix of objectives, and it is useful
to list all of them at an early stage. In the EMF case, we developed the objectives and associated
measures in many discussions with the stakeholders, both in separate meetings and in meetings
with the stakeholder advisory group. Perhaps the most interesting and challenging aspect of this
application was that stakeholder clearly wanted to include objectives that were likely to favor
their preferred alternative. For example, residents living near power lines wanted to include the
Structuring Decision Problems Page 18 of 33 EJOR 09-02-08
reduction of car accidents due to removing utility poles as a result of undergrounding a power
line. Utility representatives wanted to include the noise and disruption due to undergrounding.
It was clear to us that many of these objectives would not lead to a meaningful distinction
between the mitigation options, but we included them anyway to satisfy the concerns of the
stakeholders. This led to a proliferation of irrelevant attributes in the value tree and substantial
work in attempting to assess the consequences of the alternatives with respect to these irrelevant
attributes. However, it also led to much goodwill by the stakeholder whose concerns were
represented.
An example of a set of objectives for utility representatives is shown in Table 3, an
example for residents and environmental groups is shown in Table 4. The complete set of ends
objectives (criteria) and associated consequence measures is shown in Table 5.
Structuring Decision Problems Page 19 of 33 EJOR 09-02-08
Table 3: Means, Ends, and Process Objectives for Utility Representatives
Ends Objectives Means Objectives
Health and Safety Means Affecting Aesthetics
Public Health (EMF) Routing of Powerlines
Worker Health (EMF) Reliability
Indirect Risks Pole and Tower Height
Due to Routing Number and Type of Poles and Towers
Due to Reduced Reliability Number and Configuration of Lines
Environment Means Affecting Ease and Cost of Maintenance
Aesthetics Frequency of Maintenance
Cost Ease of Access
Land Time for Maintenance
Construction Training of Crew
Maintenance Means Affecting Outages
Local Development Number of Outages
Growth Duration of Outages
Infrastructure Means Affecting Property Values
Reliability Service Reliability
Outages Cost of Service
Indirect Impacts of Outages Power Availability
Cost
Lost Revenue Process Objectives
Possible Damages
Environmental Impacts Public Acceptance
Crime, Public Safety Adaptability to Deregulation
Property Values
Due to EMF
Due to Other Causes
Planning and Regulatory Concerns
Adaptability to Deregulation
Impact on Long-Term Local Planning
Compliance with Regulations
Structuring Decision Problems Page 20 of 33 EJOR 09-02-08
Table 4: Means, Ends, and Process Objectives for Residents and Environmental Group
Ends Objectives Means Objectives
Public Health Risks Means Affecting Property Values
Leukemia Stigmatization
Brain Cancer Means Affecting Cost
Breast Cancer Impacts of Risk Avoidance
Electrocutions Impacts of Liability and Law Suits
Other Health Endpoints Means Affecting Outages
Worker Health Risks Storm Hazards
From EMF Exposure Fires
From Other Causes
Distrubution of Risks
Children vs. Adults
Voluntary vs. Involuntary Process Objectives
Minorities vs. Others
Across Socioecon. Groups EMF Management
Property Value Loss Flexibility
Visual Impacts and Aesthetics Practicality
Justice and Fairness - Outcomes Credibility of Information
Fair Distribution of Costs Avoid "Alarming" People
Fair Distribution of Risks Local Autonomy
Costs Impacts on Property Rights
Direct Costs Local Control
Social Costs Impacts on Land Use
Due to EMF
On Housing
Due to Property Devaluation
Service Reliability
Outages
Consistency with Existing Regulations
Structuring Decision Problems Page 21 of 33 EJOR 09-02-08
Table 5. Combined Objectives and measures for evaluating policies for reducing exposure
to electromagnetic fields from power lines
Criteria Measures
Health Effects - EMF
Leukemia
Brain Cancer For cancer incidence: Number of cases
Breast Cancer For fatal cancer: Life-years lost
Alzheimer's Disease For Alzheimer’s: Number of cases
Health Effects - Accidents
Fires
Pole Collisions For fatalities: Life-years lost
Electrocutions For injuries: Number of cases
Construction
Cost
Total Project Cost 1998 dollars
O&M 1998 dollars
Power Losses 1998 dollars
Service Reliability
Contingencies Number of contingency hours
Customer Interruptions Number of person-hours of interruption
Property Impacts
Property Values 1998 dollar change in property values
Fire Losses 1998 dollars
Pole Collision Losses 1998 dollars
Environmental Impacts
Aesthetics Aesthetics point scale
Tree Losses Number of trees lost
Air Pollution Percent change of fossil fuel generation
Noise and Disruption Person-days of noise and disruption
Socioeconomic Impact
Gross Regional Product 1998 dollars
Employment Percent change in employment
Implementation Concerns
Equity and Env. Justice Qualitative judgment
Practicality Qualitative judgment
Compliance Qualitative judgment
Structuring Decision Problems Page 22 of 33 EJOR 09-02-08
3.b. Consequence Tables. A consequence table shows decision alternatives and
objectives as rows and columns and consequences in each cell. We used consequence tables
extensively to show how the options stacked up against the objectives (see Table 6).
Table 6: Example Table of Non-EMF Consequences for 69 kV Transmission Line Retrofit1
Alternatives
Criteria No Change
Raise Pole
Height Underground Split Phase
Fire Fatalities (Years of Life Lost) 0.82 0.82 0.00 0.82
Fire Injuries (Number) 0.36 0.36 0.00 0.36
Collision Fatalities (Years of Life Lost) 3.18 3.18 0.80 3.18
Collision Injuries (Number) 0.06 0.06 0.02 0.06
Electrocutions - Public (Years of Life Lost) 1.00 1.00 0.18 1.00
Construction Fatalities (Years of Life Lost) 0.00 0.01 3.96 0.01
Construction Injuries (Number) 0.00 0.06 20.10 0.06
Electrocutions - Workers (Years of life Lost) 0.67 0.67 0.21 0.67
Total Project Cost (1998 Dollars) $0 $1,655,000 $11,640,000 $2,321,000
Operation and Maintenance Cost (1998 Dollar) $945,000 $945,000 $787,500 $945,000
Conductor Losses (1998 Dollars) $6,542,000 $6,542,000 $8,137,000 $3,271,000
Property Values (1998 Dollars) $0 $0 -$12,640,000 $0
Property Loss - Fires (1998 Dollars) $57,850 $57,850 $0 $57,850
Property Loss - Collisions (1998 Dollars) $16 $16 $4 $16
Outages - Contingencies (Hours) 138 138 36 138
Outages – (Customer Interruptions Customer-Hours) 275000 275000 71260 275000
Aesthetics (Constructed Scale) 0 0 -30 0
Trees (Equiv.Number of Trees Lost) 0 0 -120 0
Air Pollution (1998 Dollars) $0 $0 -$98,460 -$8,038
Noise and Disruption (Person Days) 0 1517 35390 758
1All estimates are for 35 years. Dollar estimates are in 1998 dollars and not discounted. The estimate for
total project cost assumes no financing.
At the structuring stage, the table had not been filled out with consequence estimates. In fact,
developing consequence estimates took the better part of two years of data collection, modeling,
and analysis.
3.c. Decision Trees. The decision tree in Figure 2 was only a start. When we began to
examine other uncertainties that were relevant to assessing the potential health consequences of
EMF exposure, two uncertainties were mentioned often by scientists and stakeholders: what
exposure metric was associated with possible health effects (e.g., average and peak exposures),
and how large the health effects were, if any. This led to an expansion of the decision tree as
Structuring Decision Problems Page 23 of 33 EJOR 09-02-08
shown in Figure 3. The consequences in Table 6 are affected only by the decisions at the root
node of this tree. The health effects are, in contrast, affected by the specific path through the
complete tree.
As it turned out in later stages of the analysis, the exposure metric did not matter much
(all exposure metrics were highly correlated) and the risk ratio was parameterized in the model.
Thus, for practical purposes, the decision tree reverted again to the one shown in Figure 2. This
is another example where additional detail led to extra work that could have been avoided. In
particular, since preliminary analyses indicated that the different metrics were highly correlated,
we could have added a minor study to confirm this finding, thereby eliminating the complexities
of conducting the analysis with multiple exposure metrics.
Figure 3: Expanded EMF Decision Tree
Structuring Decision Problems Page 24 of 33 EJOR 09-02-08
3.4 Influence Diagrams. We used influence diagrams extensively, primarily to visually
lay out the numerical models that we used to calculate the consequences. Figure 4, for example,
shows the influence diagram to calculate health effects. While we used the influence diagrams
for deterministic calculations and for communication purposes, we did not use the probabilistic
relationships and algorithms. All influence diagrams were programmed in Analytica.
Figure 4: Example Influence Diagrams for Health Effects in the EMF Problem
3. e. Other Analysis Structures. As mentioned earlier, we spent a fair amount of effort on
simulating the exposures under different scenarios and mitigation options. The overall structure
of how the exposure model fit into our analysis is shown in Figure 5.
E x pec ted
N um ber o f
F a ta l B ra in
C anc e rs P e r
Y ea r (A du lt )
Los s o f L ife
E xpec tanc y
(A du lt )
E x pec ted
Los s o f life
E x pec tanc y
(To ta l)
E x pec ted
A nnua l Los s
o f L ife
E x pec tanc y
R is k
R a t ios
B as e R a tes
Inc rem en ta l
R is k
P o ten t ia l
N um ber o f
F a ta l B ra in
C anc e rs P e r
Y ea r (A du lt )
B ac k g round
R is k
To ta l R is k
M axim um R is k
R a t ios
S lope o f
D os e -R es pons e
F unc t ion
D egree o f
C e rta in ty:
H az a rd
EMF Exposure
Results (Adults)
EMF Source
Risk
Structuring Decision Problems Page 25 of 33 EJOR 09-02-08
Exposure
Calculations
(C++)
Specification of
Exposure Measures &
Line Configurations
(VISUAL BASIC)
Decision Analysis
Model
(ANALYTICA)
User
Figure 5: Exposure Modeling and Decision Analysis
Because the exposure model was difficult to run by novice users, we created a Visual
Basic interface (see Figure 6) that allowed users to specify current and improved line
configurations. A typical output of this simulation is shown in Figure 7, where exposure to
EMF, measured as TWA (Time-Weighted Average) in mG is plotted against the distance from it
(measured in feet).
Structuring Decision Problems Page 26 of 33 EJOR 09-02-08
Figure 6: Visual Basic Interface for the EMF Exposure Model
0.00
2.00
4.00
6.00
8.00
10.00
12.00
14.00
16.00
18.00
20.00
-350 -250 -150 -50 50 150 250 350
Distance (ft.)
E
x
p
o
s
u
re
(
T
W
A
i
n
m
G
)
No Change Raise Pole Height Underground Split Phase
Figure 7: Example Output of the EMF Exposure Model
(TWA means Time-weighted Average, mG means MilliGauss; the undergrounding exposure tops out at 100 mG in a
narrow range of the x-axis; the y-axis was cut off to better demonstrate the differences between the other
alternatives)
Structuring Decision Problems Page 27 of 33 EJOR 09-02-08
5. Conclusions
This paper described the structuring activities associated with a major decision analysis
of alternative policies to mitigate the possible health effects of electromagnetic fields exposure
from electric power lines. The structuring activity was extensive, involving many stakeholders
and experts. The resulting analysis and recommendations are described in von Winterfeldt et al.
(2004).
In sum, the analysis concluded that moderate mitigation alternatives were generally
preferred over not mitigating at all or resorting to expensive undergrounding, rerouting or other
land use alternatives. This message was not welcomed by either the utility representatives (who
preferred no change) or the residents (who preferred undergrounding). In spite of this, the
stakeholders praised the analysis for being responsive to their concerns, for making the models
and tools accessible to them, and for providing valuable insights. In the end, the California
Department of Health Services, using its own judgments of probabilities of an EMF hazard,
leaned toward proposing moderate mitigation. However, the California Public Utility
Commission, the ultimate user of the analysis, decided not to implement a policy that would
require utilities to modify existing power lines. However, prior to this analysis, they had adopted
a “low or no cost” mitigation rule for new transmission lines, and the analysis clearly supported
this decision.
Several ‘behavioural lessons’ emerged from the structuring activity in this analysis.
Balance detailed structures with what’s important for decision making
Perhaps the most important lesson is that the analyst team succumbed to the pressure to
develop overly detailed structures. To use Phillips’ terms, we did not develop, but could have
developed, much simpler structures that would have been “requisite.” This occurred both in the
Structuring Decision Problems Page 28 of 33 EJOR 09-02-08
decision trees, which later showed the irrelevance of the exposure metrics, and in the objectives
hierarchy, which included many objectives that later turned out to have small impacts (e.g., pole
crashes) or failed to differentiate among alternatives (e.g., outages).
This might have happened for a couple of reasons: Firstly, stakeholders tended to request
the inclusion of objectives that favoured their preferred alternative – an example is the residents’
request to include risks from pole collisions (favouring their preferred ‘undegrounding’ option).
Just as people tend to seek for information that confirms their belief or favourite positions (a
tendency known to psychologists as ‘confirmation bias’), so we find that people tend to propose
objectives on which the preferred option would score favourably.
Secondly, we noticed that the proliferation of redundant attributes was often the result of
stakeholders challenging the analysis and requesting more detail in areas they were very familiar
with – an example is the request by utility representatives for detail on project costs; this can lead
to unnecessarily detailed models. This ‘familiarity bias’ has been shown recently to affect
investment decisions (Huberman, 2001) and portfolio choices (e.g. Massa and Simonov, 2006)
but its implications for structuring of decision trees and value trees is yet to be explored.
The lesson for practitioners is that it is important to be responsive to both requests, but to
also make sure that analysis resources are properly allocated in the process. A good way of
doing this is to perform some credible bounding calculations to eliminate criteria and concerns
early. Another possibility is to use swing weighting methods to convince stakeholders that their
favorite criteria will not affect the decision (see Ewing et al., 2006).
Mix analytic structures
Another important lesson from this application was the value of step 2, where analysts
are encouraged to combine decision analysis approaches – in this case decision trees, multiple
Structuring Decision Problems Page 29 of 33 EJOR 09-02-08
objectives, and simulation approaches. The practice of mixing methods was still relatively new at
the time the analysis was conducted but has over the past decade become increasingly praised
and encouraged (see Ormerod, 2001; Mingers, 2001).
In many applications of OR in general or of decision analysis in particular there appears
to be a tendency to “fit the problem to the tool at hand.” As a result a problem is quickly labeled
as a “multicriteria problem” or a “decision tree problem” and dealt with only with these specific
models and tools. Mixing and matching alternative approaches – even drawing in models from
other disciplines is a skill that is rarely taught in a typical OR class.
Iterate and simplify
Last, this application emphasized the value of iterative problem structuring. This was true
throughout the three steps. For instance, the nature of the problem, the composition of the
stakeholders and decision makers groups, and the purpose of the analysis changed and became
clearer over several meetings. Steps that were not iterated (e.g., the development of an objectives
hierarchy or the decision tree) resulted in additional work that may not have been necessary with
iterations. For example, laying out all 25 objectives of all stakeholders was appropriate at the
initial stage of the analysis, but some simple back-of-the-envelope analyses could have justified
the elimination of at least two thirds of them.
The interesting and novel lesson for us was to notice that iterations often led to less
detail, not more. For instance, the decision analytic structures often changed, mostly through
simplification, once the preliminary numerical model results were obtained. This is an interesting
finding that echoes work in the behavioural decision making arena, suggesting conditions when
‘less can be more’ and decision making can be ‘fast and frugal’ (Katsikopoulos and Fasolo,
2006). For example, the significant concern with alternative exposure metrics that was expressed
Structuring Decision Problems Page 30 of 33 EJOR 09-02-08
in the initial stages of the analysis dissolved once the high correlations between exposure metrics
were demonstrated. As a result, the alternative metrics were removed from the decision tree. The
same outcome could have been achieved earlier, and with less effort, if we had run the complete
model with different exposure metrics before including all the metrics in the decision tree.
General lessons for structuring decision problems.
This case study offered a set of interesting structuring issues and pitfalls and a few
lessons how to avoid them. Perhaps the most important general lesson from this and other studies
on structuring decision problems is that the structuring task should be conducted in a close
dialogue between the analysts, the decision maker(s) and the stakeholders. This dialogue should
be highly interactive (many meetings and exchanges) and iterative (leaving options for re-
structuring throughout the process). The snake diagram in Figure 1 emphasizes the interactive
nature. It could be useful to add another diagram that shows the spiral nature of the process,
starting with rough problem representation and drilling down into detail as the process evolves.
Another important general lesson is the value of maintaining a focus on solving the
problem, not on forcing a particular analytic structure onto the problem. This may even lead to
the conclusion that the analytic structure originally considered for solving the problem is not
appropriate for the problem at all (and possibly requires expertise outside of the current analysis
team).
A final lesson concerns the mix of facilitation skills, which are largely social, and
analysis skills, which are largely technical, when structuring problems. The social skill enhances
the likelihood that the decision makers and stakeholders will participate in the process, provide
important inputs, and appreciate the results; the technical skills assure that the analysis is logical,
Structuring Decision Problems Page 31 of 33 EJOR 09-02-08
simple, doable, and still relevant to the concerns of the decision maker and stakeholders. A good
structure emerges when both social and technical skills are combined.
References
Belton, V., Ackermann, F. & Shepherd, I. (1997). Integrated support from problem structuring
through to alternative evaluation using COPE and VISA. Journal of Multi-Criteria
Decision Analysis. v6. 115-130.
Belton, V. & Stewart T. (2002). Multiple Criteria Decision Analysis: An Integrated Approach.
Kluwer: Dordrecht.
Checkland P (2000). Soft Systems Methodology: A 30-year Retrospective. Systems Research
and Behavioral Science. 17, 11–58.
Clemen, R. Making hard decisions. (1996), Belmont, CA: Duxbury (second edition).
Eden, C. & Ackermann, F., (2001). SODA: the principles. In: Rosenhead J and Mingers J (eds).
Rational Analysis for a Problematic World Revisited: Problem Structuring Methods for
Complexity, Uncertainty and Conflict. Wiley, Chichester, pp 21–41.
Edwards, W., Miles, R.F., & von Winterfeldt, D. (Eds.) (2007). Advances in decision analysis:
From foundations to applications. Cambridge, UK: Cambridge University Press, pp. 81-
103.
Ewing, P., Tarantino, W., and Parnell, G. (2006). Use of decision analysis in the Army base
realignment and closure (BRAC) 2005 military value analysis, Decision Analysis, Vol. 3,
pp. 33-49.
Franco, L.A., & Meadows, M. (2007). Exploring new directions for research in problem
structuring methods: on the role of cognitive style, Journal of the Operational Research
Society, vol. 58, no. 12, pp. 1621-1629.
French, S. (1988) , Decision Theory: An Introduction to the Mathematics of Rationality,
Chichester: Ellis Horwood.
Friend J & Hickling A (2005). Planning Under Pressure: The Strategic Choice Approach.
Elsevier: Oxford.
Huberman, G. (2001). Familiarity Breeds Investment, Review of Financial Studies, Oxford
University Press for Society for Financial Studies, vol. 14(3), pp 659-80.
Kahneman, D. & Tversky, A. (2000) Choices, values, and frames. Cambridge, UK: Cambridge
University Press.
Structuring Decision Problems Page 32 of 33 EJOR 09-02-08
Katsikopoulos, K.V. & Fasolo, B. (2006). New tools for decision analysts. IEEE Transactions on
Systems, Man, and Cybernetics, Part A 36(5). 960-967.
Keeney, R.L. (1992). Value focused thinking. Cambridge, MA: Harvard University Press.
Losa, F. B. & Belton, V. (2006) - Combining MCDA and conflict analysis: an exploratory
application of an integrated approach, Journal of the Operational Research Society
(JORS), 57 (5), 510-525.
Massa, M & Simonov, A. (2006). Hedging, Familiarity and Portfolio Choice. Review of
Financial Studies, Vol. 19, No. 2, pp. 633-685
Mingers, J. (2001). Multimethodology: Mixing and Matching Methods. In: Rosenhead J and
Mingers J (eds). Rational Analysis for a Problematic World Revisited: Problem
Structuring Methods for Complexity, Uncertainty and Conflict. Wiley, Chichester, pp.
289-310
Mingers, J. & Rosenhead, J. (2004). Problem structuring methods in action. European Journal of
Operational Research 152(3): 530-554
Montibeller, G. & Belton, V. (2006) Causal maps and the evaluation of decision options - a
review. Journal of the Operational Research Society, 57(7), pp. 779-791.
Morton, A., Ackermann, F. & Belton, V. (2003) Technology-driven and model-driven
approaches to group decision support: focus, research philosophy, and key concepts.
European Journal of Information Systems, 12 (2). pp. 110-126.
National Research Council. (1996). Public health effects of exposures to residential electric and
magnetic fields. Washington, D.C.: National Academy Press.
National Institute for Environmental Health Sciences (1999). Health Effects from Exposure to
Power-Line Frequency Electric and Magnetic Fields. Research Triangle Institute, NC:
EMF Rapid Program (Executive Summary, Pages 1-2).
Ormerod R. (2001). Mixing Methods in Practice. In: Rosenhead J and Mingers J (eds). Rational
Analysis for a Problematic World Revisited: Problem Structuring Methods for
Complexity, Uncertainty and Conflict. Wiley, Chichester, pp.311-336
Ormerod, R. (1995). Putting soft OR methods to work: Information systems strategy
development at Sainsbury’s. Journal of the Operational Research Society 46 (3), 277–
293.
Papamichail, K.N., Alves, G, French, S., Yang, J. & Snowdon R. (2007). Facilitation practices in
decision workshops, Journal of the Operational Research Society, vol. 58 (5), pp. 614-
632.
Structuring Decision Problems Page 33 of 33 EJOR 09-02-08
Phillips, L. D. (1984). A theory of requisite decision models. Acta Psychologica, 56, pp. 29-48.
Phillips, L.D. (2007). Decision conferencing. In Edwards, W., Miles, R.F., & von Winterfeldt,
D. (Eds.) Advances in decision analysis: From foundations to applications. Cambridge,
UK: Cambridge University Press, pp. 375-399.
Rosenhead J. & Mingers J. (2001). Rational Analysis for a Problematic World Revisited. John
Wiley & Sons: Chichester.
Rouwette, E.A.J.A. & Vennix J.A.M. (2006). System dynamics and organizational interventions.
Systems Research and Behavioral Science 23(4), 451-466.
Spetzler, C.S. (2007). Building decision competency in organizations. In Edwards, W., Miles,
R.F., & von Winterfeldt, D. (Eds.) Advances in decision analysis. Cambridge, UK:
Cambridge University Press, pp. 451-468..
von Winterfeldt, D. & Edwards, W. (1986). Decision analysis and behavioral research.
Cambridge: Cambridge University Press.
von Winterfeldt, D. & Trauger, T. (1994). Managing electromagnetic fields from residential
electrode grounding systems: A pre-decision analysis. Bioelectromagnetics, 17, 71-84.
von Winterfeldt, D., Eppel, T., Adams, J., Neutra, R., & Delpizzo, V. (2004). Managing potential
health risks from electric power lines: A decision analysis caught in controversy. Risk
Analysis, 24, pp. 1487-1502.
von Winterfeldt, D. & Edwards, W. (2007). Defining a decision analytic structure. In Edwards,
W., Miles, R.F., & von Winterfeldt, D. (Eds.) Advances in decision analysis. Cambridge,
UK: Cambridge University Press, pp. 81-103.
Wertheimer, N. and Leeper, E. (1979). Electric wiring configurations and childhood leukemia in
Rhode Island. American Journal of Epidemiology, 109, 273-284.