Assessing design principles for climate services training courses: educational design principles assessment of six C3S Blended Training courses within the Copernicus Climate Change Service

Climate change has an important impact on the water sector. Climate extremes will become more frequent and intense in the future, resulting in more and intense floods and droughts. In order to facilitate adaptation, climate services are developed to provide tailored information. These climate services are supposed to provide scientific climate information to enhance users’ knowledge to support their decisions and actions (Vaughan et al., 2018). Hecht (1980) recognized the value of climate service's value even 40 years ago; in those days these services were driven primarily by national meteorological agencies and climate research institutes as climate information providers (Vaughan & Dessai, 2014). But, researchers observe that climate services’ influence on the user's decision and actions are limited, and they refer to a ‘usability gap’ (Lemos et al., 2012). Lourenço et al. (2016) attribute this usability gap to climate service providers’ science-driven or supply-driven inclination. This usability gap results in a growing need for tailored climate services with user-demand-driven approaches that consider socio-economic aspects and adaptation options (Dilling & Lemos, 2011; Kirchhoff et al., 2013; Lourenço et al., 2016). The gap calls for user involvement and a shift towards a collaborative knowledge creation process known as co-production (Vincent et al., 2018).

The use of collaborative approaches in climate service design is growing. Advances are also being made in unraveling stakeholders’ relationships and influences to identify common challenges and solutions (García et al., 2019; Rojas et al., 2020). Co-production as a collaborative process helps to develop actionable knowledge that assists decision-making (Vincent et al., 2018). Co-production not only requires an iterative and interactive collaboration but learning within and between users and providers (Nicolescu, 1997; Mauser et al., 2013; Schuck-Zöller et al., 2018). The results of co-produced climate services are described as ‘demonstrably usable’ knowledge developed through interactive and iterative scientific processes (Lemos et al., 2012; Kuusaana & Bukari, 2015). Karpouzoglou et al. (2015) distinguish between first- and second-generation climate services. The difference is that second-generation climate services are based on co-production (Karpouzoglou et al., 2015), making scientific information usable for decision-making and creating user-driven products. Vincent et al. (2018) add to this description that co-production of climate services leads to a result that is decision-driven, process-based and time-managed, developed within an inclusive, collaborative, and flexible process. Although co-production of climate services is discussed in different terms, it is commonly agreed that co-production is a learning process to create user-driven climate services and, therefore, tailored to the user needs. Yet, most literature on co-production in climate service is fragmented and context-specific, making a complete understanding of the nature of tailored climate services difficult (Weichselgartner & Arheimer, 2019).

In general, climate services distinguish between users and producers. While researchers agree that different users have different needs, the literature commonly does not differentiate between types of users (Swart et al., 2017). In practice, producers and users are heterogeneous. Similarly, to specific fields like water governance, climate services require multiple layers and scales of stakeholders: from international, national, regional, and/or local authorities, regulators, and civil society from public and private sectors, including a wide range of scientific disciplines (Weichselgartner & Arheimer, 2019; Rojas et al., 2020). Weichselgartner & Arheimer (2019) distinguish climate service providers and producers in providing climate service to the user. Climate service providers produce raw climate data, while the producer processes or adds value to the raw data to provide the required information to the user. The producer that adds value is also called a purveyor (Máñez et al., 2014). The World Meteorological Organization (WMO) uses different terminology. It considers the National Meteorological and Hydrological Services (NMHSs) as climate service producers and distinguishes between different types of users: ‘users’, ‘intermediary users’, and ‘end-users’. While the ‘end-users’ are the decision-makers to whom the information is provided to, ‘users’ interpret, analyze and process climate and weather information, adding their sector-specific knowledge to produce a useable, tailored, and integrated climate service. The ‘intermediary users’ are the National Meteorological and Hydrological Services’ partners, working together to produce climate services (WMO, 2010). Therefore, the distinction and understanding in the literature of users and providers/producers are ambiguous terms, embracing a vast background and expertise of different levels and sectors.

The WMO (2010) states that capacity building is a critical climate service pillar for improving information provided to users. Capacity building for climate services includes, in its curriculum, climate data management, monitoring and prediction, service delivery, and product communication to users. Although WMO differentiates between users, intermediate users, and end-users, its capacity-building pillar suggests to distinguish between climate service providers and capacity building for users (Domingos Freires, 2016). Capacity building for users focuses on understanding climate variability and change, and interpretation and use of climate services provided. Capacity building for climate service providers aims to deliver accurate and reliable information and communication to the user (WMO, 2012). This division potentially maintains the breach between users and providers rather than integrated within the co-production of the information, making it questionable if these capacity-building programs can foster more tailored climate services.

The heterogeneity within climate service producers and users, combined with the unbalanced emphasis on the supply of data and limited understanding of the nature of tailored climate services, does present a challenge for capacity building and its design. Therefore, the main research question is ‘What are the design principles for capacity building for tailored climate services?’ We use the term climate services as the tailoring of decision and climate-relevant information that provides actionable knowledge to relevant users (Weichselgartner & Arheimer, 2019). We also understand capacity building as obtaining and improving skills and knowledge through various means such as workshops, trainings, dialogues, and others. But answering the question is not only relevant for capacity building as such, but the outcomes have wider implications because their findings suggest the principles for changing the cognitive basis and skills required for the creation of climate services in general.

The empirical basis for this study is a capacity-building project in the frame of the Copernicus Climate Change Service (C3S) program. The C3S project aimed at improving the quality and usability of tailored climate services by offering courses in EU member countries. The courses proved to be an opportunity to test, integrate, and adjust principles for capacity building for tailored climate services, given its international, open access, and wide-scale around Europe. This paper presents an evaluation of the educational design principles from the six first-year C3S Blended Training courses. The results from the assessment of those six first courses in the first year allow us to define concrete implications for the principles and correspondingly adjust and test them again on the following ones. These implications may improve capacity building to promote the provision of user-relevant climate information and, therefore, tailored climate services in the considered most climate-sensitive sectors – agriculture, health, disaster reduction, and water (World Meteorological Organization, 2010).

To understand the working of design principles, we use the constructive alignment approach (Biggs, 2003) and the three curriculum perspectives of Van den Akker (2004) as analytical frameworks. The consistencies and differences within the course design as well as its comparison with the current challenges on capacity building for climate services offer a unique opportunity to draw conclusions for future capacity building and its inclusion on the design of tailored climate services, together with ideas for future research.

Before describing the study, the next sections present the context of the study, which is the C3S Blended Training courses and the two conceptual frameworks used to analyze the educational design of these courses.

The ‘C3S Blended Training courses’ are part of the C3S. The European Centre for Medium-Range Weather Forecasts (ECMWF) operates the C3S on behalf of the European Union. The C3S is a climate service that offers quality-assured information about the past, current, and future states of the climate in Europe and worldwide. The C3S provides monthly climate bulletins with maps and guidance on current climate change indicators. The C3S also provides diverse demonstrator projects to present key sectorial themes to address climate-related issues that businesses or communities are facing across Europe. The main component of the C3S is the Climate Data Store (CDS). The CDS is a one-stop platform that provides free available climate data to enable climate change adaptation and mitigation strategies for policy-makers and businesses (ECMWF, no date a).

The User Learning Services (ULS) is C3S’ capacity building component. The ECMWF outsourced the ULS development to Wageningen University and Research (WUR). WUR won the service by a tender (ECMWF, 2017) and has developed the ULS as a service contract from 2018 until 2021. The service contract included two other project partners: Leeds University, which is tasked with developing an initial training requirement report and with monitoring and evaluation, and the Royal Netherlands Meteorological Institute (KNMI), which is tasked with developing online lessons and assisting the courses as master trainers. The ULS offers online lessons and blended training courses through a free access learning platform. Individuals can choose between 28 online lessons, which are subdivided into two categories: understanding of climate data (e.g., the data discovery, climatic data sources, and sectorial applications uncertainty) and use of the CDS (ECMWF, no date b). These lessons are included as learning activities within the ‘C3S Blended Training courses’. Additionally, to the individual learning, 30 training courses have been organized on location in different European countries.

The ‘C3S Blended Training courses’ are designed with a blended approach that combines online learning lessons, two webinars, and a 1-day place-based classroom on location with hands-on activities. An essential educational activity in the courses was developing a case study aimed at designing a climate service for an adaptation challenge related to the learner. The course encouraged learners to bring their own adaptation challenges as a basis for this case study. The case studies encompassed a variety of fields relevant to their context. However, most of the case studies not only focused on climate-sensitive sectors, and specifically on the nexus between water and agriculture, but also related water to other sectors like energy and ecology. Case study's titles ranged from ‘Seasonal weather forecasts for agriculture’ in Italy, ‘Characteristics of the wave regimes in the Baltic Sea’ in the training in Latvia, ‘Effects of climate change on river fish populations of Gipuzkoa’ in the training in Spain to ‘adaption to climate change in the Tagus River’ in Portugal. The final case studies developed by learners are presented at the course end and submitted through the learning platform.

The ‘C3S Blended Training courses’ were designed by WUR curriculum designers and implemented by master trainers from the service contract and local trainers from the in-country courses. Those courses were internally monitored and evaluated by the University of Leeds and annually evaluated by trainers. While this evaluation from trainers was based on the informal exchange of insights, this paper aims to develop an educational, scientific-based analysis of the first-year courses.

This study assessed the six courses performed in the first year of the service contract using two conceptual frameworks relevant for educational design: the constructive alignment model of Biggs (1996) and the curriculum perspectives of Van Den Akker (2013). Constructive alignment is a pedagogical principle that defines and aligns what students should be learning, how they are learning, and how they are assessed (see Figure 1). If a learning outcome aims to develop a specific skill, activities need to align with practising this skill and assessment methods should allow learners to demonstrate they have met the intended result. Constructive alignment thus allows for identifying the consistencies and inconsistencies among the learning goals, activities, and assessments.

A learning goal is a clear statement of what a learner is expected to do or know after following the educational entity (e.g., module, training, etc.). Teaching is designed to engage students in learning activities that enhance their abilities for attaining those learning outcomes. Assessment tasks are designed to measure how well learning goals have been achieved. Assessments may include traditional exams and also presentations, assignments, or group work (Biggs, 1996).

The second framework used in the analysis of the C3S Blended Training courses contains the curriculum perspectives of Van den Akker. Van Den Akker (2013) understand ‘curriculum’ in the context of education as the course, trajectory, or ‘learning plan’. Given this definition, a curriculum is represented in different forms: the intended, implemented, and attained curriculum. Conventionally, the intended form relates to how the curriculum developers designed (intended) the curriculum to be (in various roles). The implemented form refers to the world of teachers and trainers, that is, how teachers implemented the training, and the attained relates to learners and students and how they experienced their learning in the course. To understand how to change a curriculum, Van Den Akker (2013) highlight the need to clarify those forms to define inconsistencies between them and change the curriculum.

The six C3S Blended Training courses are the first courses developed within the ULS between September 2018 and July 2019 around Europe: The Netherlands, Italy, Spain, Serbia, Latvia, and Portugal. These first courses had the same intended curriculum design in terms of goals, activities, and assessments, though their implementations could differ to a certain degree. However, for the implemented and attained curriculum, we did not aim to analyse individual training courses but instead to identify commonalities in the implemented designs. In this study, we distinguish between local trainers and master trainers. Master trainers are experts in the field of climate change. Local trainers come from the origin country from the training, and their involvement in the learning activities depends on their background and expertise. Two master trainers involved in the service agreement contract implemented each course together with one to three local trainers from each in-country course. The courses had an average of 27 participants. Each participant submitted an individual case study proposal at the beginning of the course, of which four to six cases per course were selected to develop as a team. The case studies in The Netherlands were not considered in this study as they were not submitted to the learning platform, and there was no record of them. A summary of the courses is shown in Table 1.

Different data sources and methodologies were used to assess the intended, implemented, and attained goals, activities, and assessments (see Table 2). The intended domain is based on the written formal intentions reflected in documents and materials from the curriculum developers (Van Den Akker 2013). The intended goals, activities, and assessments were evaluated by comparing contract service documents, including the call for tenders from the ECMWF and the service contract proposal from WUR. The results from the document analysis were discussed in individual interviews with the four proposal developers from WUR as they designed the curriculum. The implemented goals, activities, and assessments were extracted from reported training requirements by Leeds University, local training documents and interviews with trainers. The report was not published but stored as part of the training documents of the service contract.

We interviewed nine local trainers, at least one local trainer per course, and four master trainers from Wageningen University and KNMI. The attained domain relates primarily to students’ experiences and learned outcomes from learners perceived by both master trainers and local trainers. The attained goals, activities, and assessments were compared with what learners learned by analyzing 38 final case studies submitted at the end of the course as a short presentation in PowerPoint and 73 individual case study proposals submitted at the beginning of the course. Monitoring and evaluation carried out by Leeds University were excluded from this study as these were focused on assessing the learner satisfaction instead of the knowledge and skills achieved.

The analysis was conducted in three phases. First, the collected data was analyzed qualitatively using directed content analysis. In direct content analysis, codes were defined derived from the conceptual framework (Hsieh & Shannon, 2005) on goals, activities, and assessments. Second, potential themes and categories were deduced. In the last phase, all data was organized using a coding tree, including themes and categories. Finally, every code (see Table 3) was typified by the intended, implemented, and attained curriculum (see Conceptual Framework) (Van Den Akker 2013).

The intended vision and mission of the ULS was to (1) increase the data uptake from the CDS and (2) ensure the best use of data from the CDS. Indirectly, its mission was to set conditions for getting more CDS data users. However, there were differences in the perception of these intended goals as curriculum designers differed in their perception of ‘best use of data’. Although some interviewees pointed towards selecting and using the data as ‘a way to avoid wrong conclusions best’, others highlighted the data selection towards decision-making. These different perceptions implied vague intended learning goals without a clear definition of the cognitive levels. We could distinguish between learning for (1) processing and visualization tools, (2) quality and uncertainty in climate data, and (3) climate impacts and risks at the local level for specific sectors and domains. These diverse angles related to different target groups. However, the curriculum designers pointed out that the definition of ‘target group’ as a data user was controversial and unclear from the beginning. While the tender mentioned a wide range of target groups explicitly as data users (sectorial users, policy-makers, planners, and academic and science communities), curriculum designers reduced this by excluding policy-makers, as they were considered as end-user in comparison with the other target groups. The curriculum designers highlighted that the backgrounds were too different to provide only one training that ‘fits them all’. The tender was also pointing to the prerequisite to assess the user's needs to deliver a tailored capacity building. Without a clear understanding of the user's needs and the initial differences in perception, the intended learning goals became ambiguous and, consequently, limiting further specification of the course design.

The intended course design aimed at reaching the learning goal through ‘blended learning’ with various topics and learning activities grounded in adult learning theory. The proposal did not refer to any specific definition of this concept but mentions the use of case studies as a tool to ‘capitalize on their own experience’ by selecting ‘preferred learning situations’. Blended learning was a prerequisite demanded by the ECMWF as a learning approach that could allow professionals to combine their busy agendas with education. The curriculum designers used ‘adult learning theory’ principles, which became apparent via the inclusion of a case study, to enable adult learners to use their context by bringing their own experiences and applying the knowledge acquired. The case study was a learning activity intended as an individual learning experience to connect context with relevant information. The use of such a case study implies that an understanding of ‘best use of data’ as data selection towards decision-making was chosen. Yet, the theoretical knowledge offered in the online lessons had a ‘best use of data’ approach geared at data quality, as this was an ECMWF tender requirement. The ‘must-in’ topics from the ECMWF tender focused on understanding data and use of CDS such as essential climate variables, seasonal forecasting, climate projections, or CDS Toolbox, among others. Curriculum designers added topics on user engagement and communication in their proposal, although there was no further explanation. Although user engagement and communication were broad and unspecific, they were useful to connect the learner's case study contexts with ‘best use of data’ for decision-making. However, a direct connection between the offered topics and the related intended target group's needs was missing. The inclusion of abstract concepts like user engagement and communication in the course was a consequence of the ambiguity in the intended goals, target groups, vision, and mission of the C3S Blended Training course.

The intended assessment was not a priority with curriculum designers. The proposal mentioned a regular quality assurance through monitoring and evaluation of learner satisfaction after each course instead of assessing knowledge and skills achieved by individual learners. Traditional exams were considered unsuitable because of the course length and the uncertainty on learners’ initial knowledge and skills.

Different misalignments were identified within the constructive alignment check on goals, activities, and assessments for the intended curriculum. First, the lack of intended assessment led to ineffective or less relevant learning activities and goals. Second, the case study as a learning activity was not aligned with the proposed focus on data handling offered in the courses and asked for by the ECMWF tender. Yet, the use of this case study is aligned with the intended mission and vision ‘best use of data’ towards decision-making.

Before the start of the implementation of the course, a study on training requirements was conducted to be able to better tailor them to the learner's needs. A meta-data analysis and survey between different potential data users was developed to assess the users. The report showed four main points that the C3S Blended courses should consider: (1) the increase of data use is influenced by awareness of what is available and on the understanding of certain concepts; (2) there is a need to better address the complexities around climate services and information provided; (3) personal interest in the use and provision from climate data needs to be addressed; and (4) more attention is needed for how to use the data from the CDS. It is important to consider that the personal interest on the last two points was from the perspective of data users’ needs rather than a representation of the intended target group. Although this study guided the design of courses and defined learning goals, activities, and potential outcomes in the implementation phase, the relation with the different target groups remained unclear.

The implemented goals shifted from an increase of data uptake and the best use of data to (1) understanding climate change and impacts, (2) getting acquainted with climate data and terminology, and (3) exploring the CDS potential. The trainers pointed out that this shift in goals came from the juvenile status of the CDS, limiting data availability during the courses, but also as a coping mechanism for the uncertain learner's profile and prior knowledge. The trainers frame this shift as a simplification of the learning goals.

Concerning the learning activities and content of the implemented curriculum, we could distinguish three main learning activities: (1) online lessons, (2) lectures during courses on location, and (3) the case study. The online lessons and lectures focused on understanding climate data (e.g., data resources and discovery, seasonal forecast, models, and reanalysis) and exploring the CDS. A guest lecture was included as a learning activity to contextualize climate impacts. The intended content on user engagement and communication was not implemented either in online lessons or lectures. In the case studies, learners would apply the theoretical knowledge acquired through the online lessons and lectures into a real case. Learners could bring in their individual case study proposals, of which trainers selected case studies for groups to work on. The selection was based on sectorial diversity, representation of local interests, and data availability. The case study activities included: theoretical discussions and brainstorms of relevant adaptation issues, potential users, data information needs, and climate pertinent data for the case. The development and actual use of climate data for the case study was not compulsory for the learners. Only learners with programming skills were encouraged to use data. Trainers emphasized the theoretical discussions and brainstorms over the actual use of data.

Local trainers set the final presentation and submission of the case study as a formative assessment to evaluate knowledge and skills achieved by learners. Based on their perception of the implemented course and their evaluation of the case study, trainers also submitted a report to give feedback and suggestions for improvements.

Different misalignments were identified in the implementation phase. First, the case study as a learning activity was still misaligned with the intended goals. While developing a case study on a climate service for adaptation requires wide trans-sectorial and multidisciplinary knowledge, the learning goals focused on the data side. Second, the processes and concepts for designing climate services were not addressed in other learning activities of the course. However, the implementation of the case study as a formative assessment corrected this second intended misalignment.

Local trainers perceived that participants in the trainings attained different learning goals. For example, the local Spanish trainer pointed out that learners with management positions in the private sector aimed to understand climate data and potentially include it in their business. Similarly, a Portuguese trainer pointed out that non-climatic related researchers sought to include climate data in their research. However, trainers observed that, where learners could manage complex data and had coding skills, they lacked theoretical knowledge on climate data. Thus, the attained goals became flexible, relevant, and interest-based, tailored to the different categories of learners.

Regarding the learning activities, the individual case study evolved into group case studies. The trainers perceived a vast spectrum of learners from data users to end-users and general-interested learners. The learner's backgrounds varied in sectors, job titles, and prior knowledge on adaptation, climate data, and climate services. Learners with limited experience and prior knowledge could not carry out the case study alone and spontaneously created groups to learn from each other. By learning from this experience and using the learners’ heterogeneity, trainers transformed the individual intended case study into a group learning activity.

Local trainers also perceived that groups with different backgrounds were better able to connect climate data with the real-life or policy context than groups with a more homogeneous background. The Portuguese trainer gave a specific ‘good’ example from a case study on agriculture. He argued that, as the group contained business background learners, they better could address and connect the user needs compared with other groups. Trainers perceived the heterogeneity at the beginning of the courses as challenging to provide a course that ‘fits them all’. However, trainers highlighted how their perception changed and saw heterogeneity as an advantage. The heterogeneous groups had more inside knowledge on the case from different perspectives and expertise and experienced more interactive multi-way engagement activities than the homogenous groups.

The case studies were used to assess increases in knowledge and skills, and increases were seen, particularly in the understanding of contextualized climate impacts, climate data, and the CDS. Case studies showed the correct use of scenarios and data set selection from the CDS. Trainers observed a shift from a data-driven initial individual proposal into user-driven case studies after changing to group work in the case studies compared with the first individual proposal with the group case studies. The majority of the first assignments focused on using data and its potential purpose (supply-driven) instead of first identifying the user need (demand-driven). For example, the first assignment adaptation case study (in Italy) was defined as ‘Extreme events like heat waves are on the rise in the changing climate scenarios’. Compared with this first assignment, the final adaptation case study included the purpose and use of the information produced.

However, most final case studies did not consider which decisions are being supported by the information provided, resulting in misalignment between the selected user and the information provided. For example, initially in a Portuguese case on ‘adaption to climate change in the Tagus River’, the learners focused on a better understanding of ‘the dynamics of climate change at a transnational river basin scale, and to develop the appropriate elements for an adaptation strategy that would tackle the main water stress risks and potential cross-sectoral interactions’. The first assignment showed the participant rationale: adaptations strategies are directly related to understanding the climate change impacts. The climate service's input was providing information on the climate change impacts on the river basin. But this data orientation missed the decision-maker(s) context for those adaptation strategies and the complexity of stakeholders involved in the same issue. However, the final case study evolved by distinguishing between multiple stakeholders, from politicians in multi-level governance, citizens, NGOs, farmers and agricultural entities, and the energy sector. All stakeholders except those on governance were considered users of the climate services. Although those stakeholders initially were considered having homogeneous interests and power, learners started to recognize the relevance of differentiating between information needs. But the course's supply of climate information was not prepared for this.

Trainers identified three main reasons for this misalignment in the attained courses: the lack of context expertise, the lack of user within the group, and the lack of knowledge and skills provided during the course's learning activities. Trainers argued that without the context-based knowledge from an expert (e.g., hydrologist) and/or the user (e.g., a water manager or policymaker from a municipality), it proved to be challenging to assess the needs and to identify users. Trainers proposed improvements by including learning activities that explicitly link different users with different decisions and, therefore, information. Trainers also suggested including expert's and users within learning group activities.

Although the earning goals were flexibly adjusted in the attainment phase, based on background and interest from the learner, the resulting learning goals were yet covering the required knowledge and skills for climate services in the case study. Therefore, the lack of guidance on knowledge and skills in climate service development could be a relevant factor in the misalignment between user information.

The combined results described in the previous results sections in terms of constructive alignment and curriculum perspectives are presented in a matrix (see Table 4).

The learning goals changed or evolved significantly from their intended to implemented to attained forms. The intended goals proved overambitious and assumed a one-size-fits-all approach, which turned out not to be feasible for the heterogeneous target group. The implemented and the attained goals became more tailored to the needs and prior knowledge of the different types of learners. Initially, curriculum designers intended a slightly different understanding of the ‘best use of data’ from its direct meaning on improving the supply of reliable and accurate climate information to best information supplied for decision-making. Although this might be perceived as a minor difference, its consequence is significant in the target group. While the ‘best use of data’ direct meaning suggests that the target groups are actors that directly manipulate data, the ‘best’ information supplied for decision-making relates to a more complex network of actors. Vincent et al. (2018) argue that tailored climate services require considering a broader spectrum of stakeholders from users and a more comprehensive range of multidisciplinary and cross-sectoral stakeholders. Given this initial ambiguity and uncertainty in the target group, trainers shifted the implemented learning goals towards a lower cognitive level set of more general learning goals. However, in the attained curriculum, the ambiguous goals became flexibly adjusted to the target group. The C3S Blended Training courses decided to open itself to this flexibility and embrace it. Trainers highlighted the advantage of opening the course to a heterogeneous target group: because participants learned from each other and experienced more interactive engagement activities and co-production than in the homogenous groups.

The collaborative learning activities among providers and users as implemented by the C3S trainers correspond with calls for more such interactions in literature (Weichselgartner & Kasperson, 2010; Lemos et al., 2012; Vincent et al., 2018; Weichselgartner & Arheimer, 2019). Although improving data availability and better inclusion of the future change in hydrometeorological extremes is essential for climate services, it is insufficient for providing tailored climate services. It requires inclusive and collaborative learning processes between providers and users (Weichselgartner & Kasperson, 2010). Therefore, as the tool to promote the knowledge and skills to achieve inclusive and collaborative learning processes for tailored climate services, capacity building should also explicitly address them. Furthermore, a better understanding of the different stakeholders and users’ prior knowledge, experiences, and expectations are essential for more tailored climate services (Weichselgartner & Arheimer, 2019) and, therefore, also crucial for understanding user needs in terms of ‘learning needs’ (Rauthmann, 2017). Understanding learning needs allows tailored capacity building providing flexible learning goals to the different learning profiles in an inclusive program.

The learning activities not only shifted in content but also in the way the case studies were used in the trainings. The intended learning content aimed for a broad selection of topics, including user engagement and communication. The University of Leeds report on training requirements highlighted the need to address complexities around climate change and climate services. However, the implemented learning activities largely missed that content. But the content and the case study, shifted from individual to group work, driven by the need to cater to the heterogeneity of learners.

The learning activities included discussions and brainstorms. However, the actual use of climate data was limited to learners with programming skills. While learners highly skilled in managing data can benefit from getting acquainted with the CDS and further enhancing their knowledge on climate data, other learners sought the basic understanding of adaptation and climate data for their business. The cases studies promoted collaboration and co-production. Incorporating user engagement processes, including online lessons as an explicit learning goal in capacity building, could be essential to learn the principles of tailored climate services. However, understanding co-production as a type of user engagement is still limited (Alexander & Dessai, 2019). Further research on how co-production relates to climate services could assist in its translation into future capacity building.

Using the case study result for a formative assessment on which feedback was given during the presentation and submission helped produce more user-driven case studies. However, most of the case studies missed the definition of decisions from users, leading to a misalignment between users and information provided.

In this study, the educational design principles used in the C3S Blended Training courses on climate services were assessed. By evaluating the educational design of the trainings through the lenses of constructive alignment (Biggs, 2003) and the intended–implemented–attained curricula (van den Akker 2013), we could analyze how the educational design could be improved to better foster capacity building for tailored climate services. These educational analyses focused on the learning goals, learning activities, and assessments, as well as on the needs of the different learners.

This study shows that capacity building on climate services attracts a heterogeneous background of learners with diverse knowledge and skills. Learners can be roughly divided in service providers, usually with a scientific expert background, and service users, usually operating in policy- and planning-oriented contexts. But within these main groups, many subdivides can be made. Defining standardized learning goals for such a heterogeneous group led to inconsistencies in course design and implementation and consequently to lower then desired attainment levels. Capacity-building design requires understanding the learner's profile to align the goals, activities, and assessments and accommodate for the contribution of different learners in dealing with tailored climate services.

Focusing the learning activities on the use and understanding of climate data does not directly translate into tailored climate services. Learners succeeded in increasing their theoretical knowledge of climatic data. Still, they missed connecting this theoretical knowledge with knowledge needs relevant for specific decision-making contexts. There proved to be a gap between information offered and user information needs. Therefore, capacity building should include content and exercises for designing decision-driven climate service in the goals, activities, and assessment strategies. The case studies, submitted by learners, initially were not meant to close that gap, but under influence of learners seeking collaboration during the trainings and starting to work together in the cases, the case studies largely proved to be a helpful method in closing that gap.

Capacity-building programs designs, like the C3S, are usually not based on educational design principles. This study highlights the usefulness of educational research approaches in the design and implementation of training for climate services for the clear purpose of transitioning to better, tailored, and actionable information. But as climate services themselves aim at conveying information and knowledge and can be regarded as a form of training; the use of educational research approach is expected to be helpful in improving climate service's designs in general. Although this study offers some first ideas to align the educational design, further research is necessary to assess their effectiveness to design tailored climate services.

Collaboration among learners with different backgrounds in the trainings caused a shift from individual learning towards multidisciplinary and cross-sectorial group work in the case studies. This change was not envisaged in the intended learning goals and activities. Collaborative learning should not only be a case study-related activity, but should be included in the content, goals, learning activities, and assessment strategies. Not only in trainings, but also in real-life climate services-boundary crossing among stakeholders and disciplines is required. Therefore, capacity building for climate services should not only focus on providing different stakeholders with relevant knowledge, but also prioritize stimulating different stakeholders to learn and co-create across each other's boundaries. However, literature on climate service still fails to recognize the difficulties associated with organizing collaboration as a design principle for tailored climate services. Yet, more in-depth knowledge is needed to assess to what extent the inclusion of collaborative learning could promote the co-production of tailored climate services within capacity-building programs.

The first-year C3S Blended Training courses reflect the challenges of dealing with a heterogeneous learner group combined with the unbalanced emphasis on supply-driven learning approaches and limited understanding of the principles of tailored climate services. Further research on the nature of tailored climate services and learner profiles for designing capacity-building programs becomes essential. The lack of understanding of these factors impacts the definition of learning goals, teaching and learning activities, and assessment within a capacity-building program (Biggs, 1996). Developing an aligned program in which goals, activities, and assessments are geared towards decision-based and tailored climate service might strongly increase the effectiveness of capacity building for tailored climate services. Those improved climate services might potentially provide actionable knowledge to assist decision-making in adaptation, as it is in the case of the water sector among other sectors in general.

Data cannot be made publicly available; readers should contact the corresponding author for details.