The lengthy and sequential analysis involved in many integrated impact assessments gives opportunity for uncertainties to develop and grow.  If unchecked, these can in some cases build to a level where they swamp the results, making them of little practical value.  At the very least this implies a wastage of effort in completing the assessment; at worst, it can lead to poor decision-making and ineffective or even harmful action. 

It is therefore essential not only to identify and categorise potential uncertainties at the design stage of an assessment (see link to Characterising uncertainties, below), but also to evaluate and track them as the assessment proceeds - and if appropriate to compare them with a specified limit for uncertainty for the final results.  When large, or unexpected uncertainties are dentified, their cause needs to be investigated, and in most cases some frm of remedial action (e.g. use of better data or a different model) taken.  However, it is not always appropriate immediately to terminate the assessment if the uncertainty limit is exceeded, for in some cases uncertainties may cancel out!  At least some degree of tolerance might thus need to be applied.  In addition, even very uncertain assessments can be informative, if only in showing where further monitoring or research needs to be done.  

Monitoring uncertainties in the assessment

Identifying uncertainties is not always easy.  Nevertheless, in well designed integrated impact assessments it is at least facilitated by basing assessment on a clear and detailed conceptual model of the system being studied, and by following the causal chain.  In this way, clear expectations about the results can be built up, and the results viewed and evaluated at various intermediate stages in the assessment. 

In an ideal world, uncertainties at these intermediate stages in the analysis can be evaluated by comparing the results with independent reference data.   Thus, estimated pollutant concentrations can be compared with current, monitored concentrations to determine whether they show a consistent distribution and lie within a plausible range.  Similarly estimated exposures can be compared with survey data, from analogous situations, and excess risks can be checked against those found in previous studies.  A range of statistical analyses are available for this purpose (see link to Methods for uncertainty analysis, below).

In many instances, however, external validation of the results in these ways is not possible, simply because relevant data from matching situations do not exist.  In these cases, uncertainties may only be spotted by comparing against prior expectations of what the results should look like at key stages in the analysis.  Wherever possible, these expectations should have been specified in advance (e.g. during the Design phase)  - for example, as an anticipated distribution or range of likely values, and/or as a direction of effect (e.g. expected direction of change in the outcome variable from one scenario to another).  Another useful tool is sensitivity analysis: rerunning specific elements of the assessment with changed input variables (or different models) to determine how robust the results are to small differences in the analytical conditions.   

Where even this is not possible, then a more subjective estimate of uncertainty needs to be given based on the general credibility of the preceding data and methods that have led to the result at that stage.   One useful approach in these situations is to use an uncertainty scorecard.  This should be designed to indicate the sources and levels of uncertainty at each stage (location) in the causal chain, and show (in at least a qualitative way) how these accumulate as the analysis proceeds (see link to Methods of uncertainty analysis, below).  To set the uncertainty estimates into broader context, a useful trick is finally to compare the overall level of uncertainty with that associated with other, better known risks (e.g. smoking and lung cancer, or solar radiation and skin cancer), for this gives the tlimate user a yardstick against which to make judgements.