JPI Oceans has set out on an ambitious journey: To contribute to a more effective and coordinated research and innovation landscape in the European marine and maritime field. Bringing together a large partnership of more than 20 participating countries, the JPI Oceans represents a truly substantial effort to make an impact in the European Research Area. Aligning the member states' programmes and strategies, as well as financial and other resources promises to generate significant impacts on synergies and reducing gaps and fragmentation, and may contribute to a more effective approach to tackle grand, societal challenges.

JPI Oceans has developed a Strategic Research and Innovation Agenda (SRIA) and an Implementation Plan (Iplan) to take the strategy forward, and an Operational Plan (Oplan) is being developed to ensure consistent implementation of near term prioritized activities. Evaluation is an inherent part of these efforts, consistent with the framework conditions laid down by Groupe de haut niveau pour la Programmation Conjointe (GPC). Guidelines for evaluation and monitoring have been developed through the CSA-project supported by FP7 named CSA Oceans.

The underpinning element of the evaluation approach will be the monitoring and the information system that will be built up through tagging of the activities of JPI Oceans. Further, a large set of indicators has been identified for the three categories of actions planned for JPI Oceans:

• Policy actions, i.e. processes of coordination and alignment within and among the partner countries, governance of the partnership and activities;
• Structuring actions, i.e. actions intended to have an impact on the alignment of the European research and innovation landscape in the marine and maritime domain. These structuring actions can have different typologies (link to the webpage on typologies of actions): for some of them (as an example the joint calls), their evaluation has been long experienced, for some others a careful approach has to be adopted.
• Pilot actions, i.e. actions of an experimental nature intended to ensure learning and if possible effective implementation of up-scaled structuring actions (pilot actions are pilot versions of the structuring actions).

These indicators will provide the basis of the dedicated evaluations and assessments that will be conducted as the partnership is going forward. The approach will therefore make up a mix of internal monitoring and information systems construction, targeted evaluation of experimental pilot actions, and evaluations of the various actions that will be implemented with a focus on the established criteria and indicators. Hence, this approach is tightly connected to the hierarchy of steering documents (vision document, strategic research and innovation agenda, implementation plan and the short term operational plan). These documents make up a governance system with a decision cycle supported by the evaluation approach. Results from the various evaluation activities will be fed back into the decision making process with a view to ensure that JPI Oceans is a learning organization, and that there is a dynamic consistency between the priorities laid down in these steering documents.

In 2016, GPC has proposed an evaluation framework for the joint programming, mainly focused on the process of alignment of national programmes. This framework is usually addressed to evaluate JPIs and adopts some indicators which can be sometimes misleading of the peculiar characteristics of some JPIs and of the different typologies of actions, other than joint calls, which can be adopted. In this regard, from the very beginning, some of the main goals of JPIs were identified in reducing fragmentation, avoiding unnecessary duplications and structuring the European research area in its complexity. 

JPI Oceans pilot actions were devised as a test of cooperation between participating countries, in order to tackle some specific challenges. These actions have identified their specific objectives and developed their activities: they can be considered as a beta test for the effectiveness and efficiency of the fit-to-purpose approach and the interconnections between the research levels. They aimed at providing effective and efficient solutions. JPI Oceans pilot actions show some of the aspects of the joint programming process. In fact, the implementation somehow reflects priorities, commitments and management, and the activities could indeed introduce structuring effects or also show transformative impacts.

Therefore, what should be investigated when evaluating joint programming, are the counterfactual implication of the JPIs and the peculiarities of the process as a whole, even if some impacts are difficult to be quantitatively measured. In the context of the evaluation framework proposed by the GPC, many of those peculiarities, are apparently missing. This includes coordination, reducing fragmentation, increasing efficiency and testing the structuring of the European Research Area.  

Through the analysis of JPI Oceans pilot actions (reported in the Template “Performance indicators of JPIs in the evaluation framework of GPC: lesson learnt from JPI Oceans”), common aspects were preliminary identified, at different level of relevance, for the evaluation: support to policy, efficiency/defragmentation, multi-disciplinarity, emergent issues and counterfactual. They are clearly in tune with the drivers which justified JPIs to be launched, but they are difficult to be framed in evaluation criteria identified by GPC so far.