Designing effective assessment reports
Some form of written report (or set of reports) is almost inevitably a primary product of any integrated environmental health impact assessment. It would be misleading and banal (and probably impossible) to provide detailed guidance, or rigid templates, on what form these reports should take. How results are presented depends on the nature of the assessment and the specific (and often varied) user interests. It is therefore wholly inappropriate to force reports into a standard format, unless the type of issue and assessment is similar, and the stakeholders and their interests the are same. Instead, the scope, format and style of the reports should be discussed with the stakeholders early in the process of designing the assessment, and should reflect their priorities and needs.
That said, it is useful to outline the purpose of assessment reports, provide a checklist of the materials that should be considered for inclusion, and give some examples of how the material can be presented.
What are assessment reports intended to do?
Assessment reports clearly serve a number of different roles, not only for those doing the assessment and those on whose behalf it has been done, but also for other, would-be future assessors or commissioners of new assessments. For the assessors, for example, the report may have a number of contractual functions, as well as providing a detailed catalogue of what has been done and what results have been obtained.
For the 'customers' of the assessment, the report is both an insight into the results and process, and a signpost to the implications that follow and the actions that might be needed. For them, therfore, it needs to:
- describe the results of the assessment, in a useful and intelligible manner;
- show how the assessment was done, and explain and justify the decisions made on the way;
- indicate what the results mean in relation to the question which was originally posed;
- spell out any limitations or other considerations that need to be borne in mind when interpreting and using the results.
Because integrated assessment is a relatively new field, and one that (perhaps wisely) has not yet established a dominant paradigm or set of formal practices, the assessment report is also a valuable learning tool - a means of helping others work out what assessments involve, what sorts of issues they can be applied to, and what sort of results they can provide. Whilst informing this wider body of interested people will rarely be the primary objective of the reports, there are clearly benefits in ensuring that the reports are disseminated as widely as possible. And for this audience, it will also be helpful (as far as this can be done without jeopardy) to provide some reflection on the lessons learned - the things that went well, the things that failed, the problems that were encountered and how they were overcome.
What should be included?
Given these objectives, and given the general process of IEHIA that has been proposed here, it follows that a number of key elements are likely to be needed in the assessment report. The table below summarises some of the main elements that should be considered for inclusion, and suggests how some of these might best be dealt with.
| Item | Description | Comment |
| Context | ||
| The question and the neeed | A reprisal of the question that is being addressed, and the reason it was raised, together with a summary of who the main stakeholders are. | |
| Scope and content of the assessment | ||
| The issue | A detailed description of the issue, as it was ultimately defined, both textually and in the form of a diagram of the causal model. | May be supplemented (e.g. in an Appendix) by a description of how the final causal model was agreed, showing also (some of) the precursor or intermediate versions. |
| Scenario and type of assessment | An outline of the scenario(s) used as a basis for assessment, including the assumptions involved; specification of what type of assessment was done. | Appendices may need to be provided giving further details of the scenarios, and justifying their use (e.g. in the light of serious limitations or assumptions). |
| Geographical and temporal scope | Description of the study area, population and timeframe for the assessment. | Maps, statistical data and photographs should be included or appended, where appropriate. |
| Environmental exposures | Description of the main agents (e.g. hazards, pollutants), their sources, release and transport pathways, and exposure routes considered in the assessment. | May be supplemented by an explanation of which other exposures were excluded from consideration and why. |
| Health outcomes | Specification (including ICD codes if appropriate) of health outcomes selected for assessment. | May be supplemented by an explanation of which other health outcomes were excluded from consideration and why. |
| Assessment methodology | ||
| Exposure assessment | Description of data and methods used to model causal chain from source to exposure. |
Appendix should provide copy of assessment protocol, together with a clear explanation and justification of any deviations from the original plan. Appendices may also provide details of individual data sets and models used in the assessment. |
| Health effect assessment | Description of how health effects were estimated, including presentation and justification of exposure-response functions used and how baseline health data were derived. | |
| Impact assessment | Specification of the impact measures used, and description of how they were derived (including weights, aggregation procedures). | |
| Uncertainty analysis | Outline of methods uded to characterise and assess uncertainties. | |
| Results | ||
| Main findings | Description of main findings, including quantitative information (e.g. as tables, graphs, maps) and an outline of the main uncertainties involved. |
Use of 'indicator scorecards' to summarise the results of different scenarios is often helpful (see link to left). More detailed, quantitative results (including results of sensitivity analyes) should be provided in an Appendix. |
| Interpretation | ||
| Implications | Summary of main conclusions arising from the assessment, and their implications (e.g. ranking of different scenarios in terms of scale of impact). | Note: reports should in most cases stop short of making recommendations (e.g. about which policy measure to adopt), unless this is specified in the contract. |
| Caveats | Listing and explanation of limitations to assessment, or other factors that need to be borne in mind when applying the results. | |
| Lessons learned | Outline of important lessons regarding process of the assessment, that might aid future assessments (e.g. issues faced and ways in which they were overcome; continuing gaps in knowledge) | |
| Additional materials | ||
| Executive summary | Brief overview of main content, highlighting the qustion, nethodological approach and findings. | |
| Assessment protocol | Full copy of the original protocol, including information on any changes made during the assessment, and the reasons for doing so. | Should usually be provided as an appendix or on-line. |
| Stakeholder engagement | Listing of stakeholders engaged in the assessment, and an outline of how stakeholder consultation was done . | |
| References/ bibliography | Full listing of references cited, and indications of further reading. | |
It should be stressed that reporting will rarely be adequately served by a single, comprehensive report. Instead, it is often more effective to produce a number of different sets of materials, at different levels of detail, designed for different audiences and purposes. Given this, it is crucial that these materials are properly harmonised, so that they give consistent messages (albeit in different language or by different means), and so that users can move between them reasonably seamlessly.
Reports can also be made available through different media, including paper, digital and on-line versions. On-line versions can in most cases be enhanced by judicious use of hyperlinking (e.g. to link summary gindings to more detailed information in tables or appendices), though care is needed to ensure that this does not impair the linear logic of the report. In many cases, also, digital media offer better options to view and explore complex materials such as detailed causal diagrams and mapped results.
Examples
As has been noted, integrated environmental health impact assessment is relatively young, so most assessors are likely to be finding their way about how best to design and structure reports. Unfortunately, this also means that there are as yet few examples of assessment reports to use as a starting point. The INTARESE and HEIMTSA projects, which underlay the development of this Toolbox, however, carried out a number of case studies, aimed at testing and illustrating the approach. Because these were essentially experimental studies, done in a research context, they do not for the most part relate to real 'client-led' issues - though they all deal with major policy questions in environmental health. For the same reason, they also did not involve full stakeholder engagement and are often limited in their scope. They nevertheless provide useful worked examples both of assessments and of assessment reports (broadly following the structure outlined above). A key component of all the reports is also a reflection on the lessons learned, which may help both to forewarn assessors what might lie in store, and give clues about how both assessment procedures and the reports can be improved.
Reports are included in the Toolkit section of the Toolbox (see link below).
Examples of assessment reports
print version