Report on the criteria for good indicators selection

by D. Krause-Jensen, A. Bruhn, A. Borja


A new DEVOTES Deliverable (D3.2: Report on the criteria for good indicators selection) was produced by a very large team of institutions and colleagues. The study is initiated by a detailed literature survey and critical examination and synthesis of selection criteria for good indicators of the status of marine ecosystems. On this basis we provide a short-list of ‘criteria for a good indicator’ to be applied in the DEVOTES project. These criteria and an associated scoring system build on an ICES’ broadly accepted list of 16 criteria simplified to a set of 8 key criteria by omitting those that were correlated/linked with the selected 8 key criteria. We hereby obtain the benefit of applying broadly accepted criteria and at the same time having a simple criteria list.


The set of eight key criteria for quality analyses of indicators criteria is:

–         scientific basis;
–         ecosystem relevance;
–         responsiveness to pressure;
–         possibility to set targets within the indicator response;
–         precautionary capacity/early warning/anticipatory capability;
–         quality of sampling method: concrete/measurable, accurate and precise outputs;
–         cost-effective implementation;
–         part of an existing or current ongoing monitoring or data;

The deliverable further provides an overview of all the metadata on biological indicators and associated pressures available for analyses within the DEVOTES project for each of the descriptors for Marine Strategy Framework Directive (MSFD): ‘Biodiversity (1)’, ‘Seafloor integrity (6)’ and ‘Food webs (4)’ and also specifies which of these available data are prioritized for quality analysis within DEVOTES.

The study proceeds with quality analyses of 36 prioritized indicators based on the identified data sets and criteria, using the DEVOTool, developed in our project. The quality analyses represent:
–         the 4 regional seas: the Baltic Sea, the Atlantic Sea, the Mediterranean Sea and the Black Sea;
–         various indicator components: phytoplankton (4 indicators), zooplankton (3 indicators), macroalgae (8 indicators), seagrasses (7 indicators), benthic invertebrates (11 indicators), fish (1 indicator), marine mammals (1 indicator) and a “biological value” indicator, which includes several ecosystem components in its calculation;
–         various types of pressure: Aggregate extraction, Fishery, Nutrient/organic pressure (Eutrophication) and sediment dumping.

All indicator tests follow the same basic structure with short summaries of the performance according to each of the 8 main indicator criteria. The various tests provides a thorough overview of how specific types of indicators and data sets can be analyzed and scored according to a common set of key criteria and the analyses thereby identified forces and weaknesses of the range of tested indicators. Overall, based on the 36 indicator tests, the criteria “Ecosystem relevance”, “Scientific basis” and “Existing/ongoing data” were the most commonly met criteria, lack of “Possibility to set targets”, “Responsiveness to pressures”, “Quality of the method (concreteness, etc.)”, and “Cost-effectiveness” were identified as weaknesses of a number of indicators, and only few indicators possessed “Anticipatory capacity”..

While it proved possible to apply the eight key criteria across a broad range of ecosystems, components and pressures, we identified an element of subjectivity in interpreting the criteria. In order to increase the objectiveness of testing and scoring the various criteria and the transparency of the process we therefore conclude the study by standardizing the method further through a framework that for each criterion defines a suite of evaluation steps involving definition of a nul-hypothesis and appropriate analytical methods for testing the criteria. We exemplify the framework by applying it to a selection of the analyzed indicators. Overall, the study thereby offers a stringent framework for assessing the quality of potential MSFD indicators to enable an objective selection of the most suitable.
This Deliverable will be shortly available at the DEVOTES web page.