Feasibility Studies, Impact Assessment, Workshops

In the analysis and definition of the possible actions to be undertaken by the JPI Oceans, some preliminary actions like feasibility studies, impact assessment and workshops are intended to help evaluating and orienting the decisions.

The Voluntary Guidelines on Framework Conditions for Joint Programming in Research of the European Research Area Committee, Groupe de Programmation Conjointe (ERAC-GPC) consider the ex-ante impact assessment as part of the "Evaluation of Joint Programme". Dealing with the judgment of interventions according to the expected results, impacts and needs to be satisfied, the evaluation of a Programme is defined as a means for an effective evidence-based policymaking. The evaluation in general terms is usually one of the phases of the programming cycle, both in terms of ex-ante evaluation to define the vision and interventions, both in terms of final review at the end of the cycle to assess the results and plan future initiatives. Concerning the Joint Programming, the Framework Conditions distinguish among different levels of evaluation, relating to assessing the validity of the general policy concept; its implementation within single JPIs; the individual projects conducted within a JPI. The document points out some aspects of the evaluation process:

• have a clear, logical and well-laid-out hierarchy of the objectives to be achieved, as well as a standardised approach to presenting the rationale and motivations for each of the foreseen interventions. In this case the ex-ante impact assessment will help to define the objectives chain and assess their effectiveness, as well as evaluate the relevance, efficiency and sustainability of an intervention;
• identify meaningful parameters and outputs indicators to be monitored and evaluation methodologies, taking into account the level of risks and uncertainties intrinsic in research activities;
• define the management of the programme and of the information flow;
• define an ex-post evaluation based on criteria, key performance indicators and synergies between the different levels, to assess the results of individual research projects, the success of a specific Joint Programming Initiative in addressing its target challenge, the Joint Programming concept, as an effective way for cross-border collaboration.

The Voluntary Guidelines does not go further in analysing the ex-ante impact assessment.

The JPI to CO-WORK organised a specific workshop aimed at discussing JPIs current practices and options relating to the impact assessment and ex-ante evaluation. JPI to CO-WORK starts taking into consideration the definition of the terms "impact assessment" and "ex-ante evaluation" adopted by the Commission. Following the Commission practice, "impact assessment" is intended as a legal requirement for any new legislation or initiative proposed by the Commission itself, focusing on the goals and their likely economic, environmental and social impacts. The "ex-ante evaluation" is a requirement for new and renewed programmes and actions with budgetary implications. The Commission considers the impact assessment an instrument to support the political decision-making and to ensure that the initiatives are prepared on the basis of transparent, comprehensive and balanced evidence. It is based on the following principles: make the people responsible for policy development also responsible for assessing the impact of what they propose, in order to improve the quality of the proposals themselves; an integrated approach which analyses benefits and costs, and addresses possible economic, social and environmental impacts of the initiatives; coherence of initiatives across policy areas, guaranteed by the inclusion of relevant expertise within the Commission and inputs from the stakeholders; accountability and transparency of the system, including the active involvement of the stakeholders; explanation of the necessity to adopt or not adopt an action, and its appropriateness.

The Commission impact assessment procedure identifies a number of main questions, that can be transposed in different context:

• "What is the nature and scale of the problem, how is it evolving and who is most affect ted by it?
• What are the views of the stakeholders concerned?
• Should the Union be involved?
• If so, what objectives should it set to address the problem?
• What are the main policy options for reaching these objectives?
• What are likely economic, social and environmental impacts of those options?
• How do the main options compare in items of effectiveness and coherence in solving the problems?
• How could future monitoring and evaluation be organised?".

KEY ANALYSTICAL STEPS IMPACT ASSESSMENT

1. Identifying the problem
   • Describe the nature and extent of the problem
   • Identify the key players/affected populations
   • Establish the drivers and underlying causes
   • Is the problem in the Union's remit to act? Does it pass the necessity and value added test?
   • Develop a clear baseline scenario, including, where necessary, sensitivity analysis and risk assessment
      
2. Define the objectives
   • Set objectives that correspond to the problem and its root causes
   • Establish objectives at a number of levels, going from general to specific/operational
   • Ensure that the objectives are coherent with existing EU policies and strategies, such as the Lisbon and Sustainable Development Strategies, respect for Fundamental Rights as well as the Commission's main priorities and proposals
      
3. Develop main policy options
   • Identify policy options, where appropriate distinguishing between options for content and options for delivery mechanisms (regulatory/non-regulatory approaches)
   • Check the proportionality principle
   • Begin to narrow the range through screening for technical and other constraints, and measuring against criteria of effectiveness, efficiency and coherence
   • Draw-up a shortlist of potentially valid options for further analysis
      
4. Analyse the impacts of the options
   • Identify (direct and indirect) economic, social and environmental impacts and how they occur (causality)
   • Identify who is affected (including those outside the EU) and in what way
   • Assess the impacts against the baseline in qualitative, quantitative and monetary terms. If quantification is not possible explain why
   • Identify and assess administrative burden/simplification benefits (or provide a justification if this is not done)
   • Consider the risks and uncertainties in the policy choices, including obstacles to transposition/compliance
      
5. Compare the options
   • Weigh-up the positive and negative impacts for each option on the basis of criteria clearly linked to the objectives
   • Where feasible, display aggregated and disaggregated results
   • Present comparisons between options by categories of impacts or affected stakeholder
   • Identify, where possible and appropriate, a preferred option
      
6. Outline policy monitoring and evaluation
   • Identify core progress indicators for the key objectives of the possible intervention
   • Provide a broad outline of possible monitoring and evaluation arrangements

The ex-ante evaluation is also a method for supporting the preparation of proposals for new actions. In the Commission perspective it is a process aimed at gathering information and carrying out analyses that allow to better define objectives and to ensure their feasibility, to use cost-effective instruments and reliable evaluation mechanisms. Following the Commission rules, the ex- ante evaluation is mandatory for new and renewed programmes or actions having resource implications, while in a wider approach is commonly used to address a project, a programme or a policy. In the new programme or actions programming, the ex-ante evaluation should in particular identify: the need to be met in the short or long term; the objectives to be realized; the expected results and the indicators needed to measure them; the added value of Community involvement; the risks linked with the proposals and the alternative options available; the lessons learned from similar experiences in the past; the volume of appropriations, human resources and other administrative expenditures to be allocated with regard to the cost-effectiveness principle; the monitoring system to be set up.

In details the key elements of an ex-ante evaluation are summarized as follows:

1. Problem analysis and needs assessment

• Explaining how the specific problems relate to the overall political goals or principles, and how problems and factors relate to each others;
• Analysing the situation, motivations and interests of the key actors;
   

2. Objective setting and related indicators

• Translating the high-level policy goals into tangible, quantified and measurable objectives;
• Defining the qualitative/quantitative indicators that help to monitor the progress and report on the objectives to be achieved;
   

3. Alternative delivery mechanisms and risks assessment

• Ensuring the appropriateness of the instruments chosen for the implementation of the intervention;
• Analysing the possible risks and identifying the possible countermeasures;
   

4. Added value of Community involvement

• Assessing the rationale for taking action at EU level, and its complementarily to and coherence with other interventions;
• Identifying synergies with other interventions;
   

5. Lessons from the past

• Critically analyzing past actions, experiences, reports (etc.);
• Define how these can be useful in order to improve the current programming;
   

6. Planning future monitoring and evaluation

• Planning the necessary arrangements for collecting data and relevant factors for analyzing the achievements;
• Analyzing the soundness and reliability of the proposed methods and instruments for collecting the data;
• Ensuring the operability of the monitoring system.
   

Feasibility studies are specific instruments dedicated at analyzing in details and assessing the achievability of pre-defined goals, the feasibility of a specific intervention, from a technical point of view and cost-benefit analysis. Furthermore, the organisation of periodic workshops can help monitor and support the implementation of the foreseen actions and eventually re-orient and adapt them.

The impact assessment, ex-ante evaluation and feasibility studies focus on identifying the problem addressed and assess the different options that could be implemented to solve the problem/need. They are often used to define similar practices and as synonyms, and sometimes they can overlap. The benefits deriving from the introduction of these supporting actions, even if considered in their strict sense or as a part of the same overall process, are manifest in terms of improvement of the quality, the relevance and comprehensiveness of the programming phase.

The JPI Oceans, in the process of defining the specific objectives to be reached, and in selecting among the several possible typologies of actions the most appropriate to reach the pre-defined goals, shall put into practice these supporting actions both in terms of ex-ante appraisal than ex-post evaluation.

The JPI to CO-WORK in its analysis on the JPIs experiences, underlines that each JPI would take advantage from the ex-ante impact assessment in terms of introducing a systematized and standardized procedure for national decision making processes, and as ex-post evaluation in terms of thinking through the rationales and the impacts of interventions. Furthermore the specific characteristics of the JPI, namely the bottom-up approach and the inclusion of the stakeholders in the definition and implementation of the JPI, are inherent in the JPI procedures.

The JPI to CO-WORK investigated the JPIs attempt to characterize the impact, in terms of outcomes of the JPIs and what they intend to achieve and in terms of impact they plan to have on the societal challenge that is being addressed. The JPI to CO-WORK identifies some major common issues emerged from the various JPI:

1. Creating a critical mass of players/funders and countries;
2. Developing a shared vision of the challenge and adopting a long-term planning;
3. Developing a coordination platform;
4. Prioritizing research topics and identifying the field to be developed and the possible gaps;
5. Jointly funding the activities and implementing large scale projects;
6. Implementing research, especially by opening up current research (including sharing data and results, allowing access to research databases, creating integrated data and information base, fostering quality research, etc.);
7. Encourage external coordination with other initiatives, defining common collaboration strategies towards "third countries" and with the Commission;
8. Involving industry stakeholders;
9. Addressing societal challenges through interdisciplinarity;
10. Addressing societal challenges through cooperation and coordination between researchers and policy makers;
11. Networking.

In the characterization of the impact and in the definition of the indicators to measure it, each JPI should take into consideration all the abovementioned aspects and the broad range of activities. The JPI Neurodegenerative Diseases Research - JPND, developed a monitoring and evaluation framework, which identifies four types of indicators: input indicators, output indicators, outcome indicators and impact indicators. Input indicators are defined in relation to the resources used to implement the JPI, like for instance human resources and amount of funding, while output indicators relate to goods, services, technology and knowledge directly produced due to the JPI activities. These two indicators are considered the core indicators for monitoring the progress of the JPI, due to their short-term measurability compared to the outcome and impact indicators which are due to produce effect in a long-time perspective. Outcome indicators are defined as less tangible that the previous ones, indicating the initial results of the intervention and the reason for the programme. Finally impact indicators measure the long-term socio-economic changes caused by the intervention⁶.

Foresight

“Foresight” is a special case, where it is usually addressed as a "framework condition" in its broad sense, using the concept of Forward-looking activities aiming at "inspiring future oriented strategic decision-making, providing fresh insight into current trends and possible disruptive events, building shared visions of the future challenges". Indeed, in JPI Oceans, Foresight is not only a framework condition, but also one of the possible typologies of actions that can implemented to reach its goals.

Emergency issues

Europe is the world's largest market in crude oil imports, representing about one third of the world total. Ninety percent of oil and refined products are transported to and from Europe by sea. Inevitably, some of this makes its way into the sea. Whether by accident or normal ship operation, the marine environment is degraded. Large oil spills at sea constitute a threat to the environment, placing enormous demands on the national authorities responsible for response and clean-up operations. Rapid and radical degradation of the world' seas and oceans is triggering increasing calls for more effective approaches to protect, maintain and restore marine ecosystems. A broad spectrum of land and ocean based activities, coupled with continued growth of the human population and migration to coastal areas, is driving unanticipated, unprecedented and complex changes in the chemistry, physical structure, biology and ecological functioning of oceans worldwide. Symptoms of complex and fundamental alterations to marine ecosystems abound, including increases in: zones of hypoxic or anoxic water, abrupt changes in species composition, habitat degradation, invasive species, harmful algal blooms, marine epidemics, mass mortalities, fisheries collapses, etc. Fishing practices, coastal development, land based chemical and nutrient pollution, energy practices, aquaculture, land use and land transformation, water use and shipping practices combine to alter the structure and functioning of marine ecosystems globally and pose an insidious long term threat to the marine and coastal environment. Therefore, there is an urgent search for effective actions to prevent or reverse widespread declines and to protect, maintain, and restore ocean ecosystems and its resources.

These actions, though typically adopted at local or regional sea basin, need a global perspective and cross-border cooperation, as well as for exchange of best practices, frontier and basic research in order to capture the benefits from past experiences, emerging science and technology opportunities.