Skip booklet index and go to page content

Measuring Environment Canada's Research and Development Performance (2014)

Principle: Relevance

Relevance - Support current and future needs of users

Why we measure

The linkages between science users and science producers, both within Environment Canada (EC) and beyond, must be robust in order for EC’s science to effectively and efficiently deliver on the Department’s mandate and key federal priorities. It is for this reason that the Science Strategy identifies the enhancement of such linkages as one of the key mechanisms to put the Strategy’s mission, vision and principles into practice.

How we measure

To measure the relevance of EC’s research and development (R&D) in supporting the current and future needs of users, Program Alignment Architecture (PAA) program survey respondents were asked to identify themselves as users and/or producers of EC’s science, rank their most significant linkages to other users/producers, and assess their degree of satisfaction with Departmental science.

Note that the survey is meant to gauge opinions across science-related EC programs. Its limitations should be considered when interpreting results; for example, each program’s response is counted equally (meaning that differences in program size and resources are not accounted for).

Results

Environment Canada is a science-based department, with all survey respondents (36 lowest-level PAA programs) using or producing science.

Out of 36 lowest-level PAA programs under Strategic Outcomes 1, 2 and 3 who responded to the survey, all either use or produce science. The majority (72%) both use and produce science. Out of the programs that use science, nearly all of them (97%) use related science activities (RSA) and about two thirds (68%) use R&D. Out of the programs that produce science, nearly all of them (93%) produce RSA and about two thirds (68%) produce R&D. Overall, 81% of program respondents are involved in using and/or producing R&D, on its own or in conjunction with RSA.

The 2009 and 2014 surveys reveal the same general finding: R&D is performed and used throughout the Department. In 2009, EC’s organizational structure was different and Outcome Project leads (not PAA program leads) were surveyed. 80% of past survey respondents indicated a connection to R&D as a user, producer or funder, and almost 30% of respondents played all three roles.

Figure 2: Proportion of PAA programs surveyed that use or produce science (RSA/R&D breakdown), 2014

Proportion of PAA programs surveyed that use or produce science (RSA/R&D breakdown), 2014 (See long description below)

Description of Figure 2

Proportion of PAA programs surveyed that use or produce science (RSA/R&D breakdown), 2014. 22% use science; 6% produce science, 72% both produce and use science. 19% use and/or produce RSA only; 81% use and/or produce R&D (often in addition to RSA).

 

Source: Internal survey of lowest-level PAA leads (2015)

For the rest of this report, survey findings are restricted to the 23 R&D-using programs and the 19 R&D-producing programs. Programs that use or produce only RSA are excluded.

Environment Canada’s science is regarded by Departmental science users/producers as highly relevant to Department and government-wide priorities while also supporting the needs of key public- and private-sector clients.

The figure below lists the most common users and producers of EC’s science, as identified by R&D-producing and using PAA programs within EC. Each bar shows the percentage of PAA programs that identify that partner as one of their program’s top three science users or producers.

Figure 3: Most common producers and users of EC’s science, 2014

Most common producers and users of EC’s science (See long description below)

Description of Figure 3

Most common producers and users of EC’s science, 2014. Among top three users identified by R&D-producing programs: Other programs within EC, my own program, other federal departments or agencies, provincial or municipal governments, academic institutions, private companies, non-governmental organizations, foreign governments, international institutions or scientific bodies, other. Among top three producers identified by R&D-using programs: Other programs within EC, my own program, other federal departments or agencies, provincial or municipal governments, academic institutions, private companies, non-governmental organizations, foreign governments, international institutions or scientific bodies, other.

Source: Internal survey of lowest-level PAA leads (2015)

Note that 21% of R&D-producing programs ranked international institutions or scientific bodies among their top three users. This is consistent with EC’s role as a prominent contributor to international fora; for example, four EC scientists were lead authors or review editors for a recent Intergovernmental Panel on Climate Change (IPCC) report, Climate Change 2014: Impacts, Adaptation and Vulnerability.

The 2009 and 2014 surveys reveal the same general finding: EC’s science remains highly relevant to Departmental and governmental priorities. In 2009, “Other EC Outcome Projects” were identified as top science users and producers, and “Other federal departments and agencies” were top science users.Footnote 1 Present results are similar, with other EC programs and federal departments and agencies among the top users and producers.

Internal users of Environment Canada’s R&D are satisfied with how Departmental science meets their needs, with the majority (83%) assigning it a positive or neutralranking (3 or higher out of 5).

As part of the survey, R&D-using PAA programs rated how well EC’s science meets their needs. As shown in the figure below, the majority (65%) feel EC’s science meets their needs well or very well, and 17% are neutral. Results cannot be compared over time, as user satisfaction was not assessed in the 2009 survey.

Figure 4: Satisfaction of EC’s R&D users, 2014

Satisfaction of EC’s R&D users, 2014 (See long description below)

Description of Figure 4

Satisfaction of EC’s R&D users, 2014. The majority (65%) feel EC’s science meets their needs well or very well, and 17% are neutral.

Source: Internal survey of lowest-level PAA leads (2015)

Environment Canada undertakes various initiatives to enhance science user/producer linkages. An inter-branch Assistant Deputy Minister (ADM) forum established by the Assistant Deputy Ministers of S&T Branch and the Environmental Stewardship Branch is one example of these initiatives in which we strive to improve the alignment and communication of the Department’s science in relation to our regulatory efforts.

Relevance - Deliver on the Department's mandate and key federal priorities

Why we measure

In addition to robust linkages between science users and producers, the Department’s R&D activities stay relevant by focusing on the priority areas identified in the Science Strategy. This focus helps EC’s science play a meaningful role in delivering on the Department’s mandate and on key federal priorities.

How we measure

To measure the alignment of EC’s R&D with its stated priorities, the Department’s journal publications were sorted by priority area. Additionally, the PAA survey asked respondents to rate the level of support their program provides to each component of EC’s mandate, the Science Strategy priorities and the Department’s Strategic Outcomes.

Results

Environment Canada’s science delivers on all components of the Department’s mandate, is aligned with Science Strategy priority areas and helps provide a clean, safe and sustainable environment for Canadians.

R&D-producing programs (19 in total) support each of the six components of EC’s mandate:

  • Preserve and enhance the quality of the natural environment, including water, air, soil, flora and fauna (79% of R&D producers agree their science supports this component)
  • Conserve and protect Canada's water resources (63%)
  • Coordinate environmental policies and programs for the federal government (53%)
  • Conserve Canada’s renewable resources (26%)
  • Enforce rules relating to boundary waters (21%)
  • Forecast daily weather conditions and warnings, and provide detailed meteorological information to all of Canada (11%)

R&D-producing programs also indicate that EC’s science is relevant to all Strategic Outcomes (SO):

  • Clean: Threats to Canadians and our environment from pollution are minimized (79% of R&D producers agree their science supports this SO)
  • Safe: Canadians are equipped to make informed decisions on changing weather, water and climate conditions (42%)
  • Sustainable: Canada's natural environment is conserved and restored for present and future generations (84%)

In the survey, R&D-producing respondents were asked to indicate the degree to which their program’s science supports each Science Strategy priority area. A large share of respondents (84%) indicated support for Contaminants & Stressors (PA1), while Conservation & Protection (PA4) and Climate Change (PA3) had the most overall support (95% and 89% respectively). A lower percentage of respondents (58%) indicated support for Weather (PA2), which is consistent with the percentage of programs supporting the weather-focused Strategic Outcome 2 in the 2014-15 PAA.

Environment Canada’s science is focused on environmental issues of current significance to the Government of Canada, with the large majority of publications (85%) supporting at least one Science Strategy priority area.

Figure 5 shows that, between 2004 and 2013, the majority of EC’s publications supported one or more of the four Science Strategy priority areas (based on a study sample size of 5,711 EC papers). Of those papers, 48% were associated with Contaminants & Stressors (PA1); 37% with Weather (PA2); 21% with Climate Change (PA3); and 29% with Conservation & Protection (PA4). Note that the total equates to a sum larger than 100% as some publications were associated with more than one priority area.

Figure 5: Share of EC’s publications linked to Science Strategy priority areas, 2004-2013

Share of EC’s publications linked to Science Strategy priority areas, 2004-2013 (See long description below)

Description of Figure 5

Figure 5 shows that, between 2004 and 2013, the majority of EC’s publications supported one or more of the four Science Strategy priority areas (based on a study sample size of 5,711 EC papers). Of those papers, 48% were associated with Contaminants and Stressors (PA1); 37% with Weather (PA2); 21% with Climate Change (PA3); and 29% with Conservation and Protection (PA4). Note that the total equates to a sum larger than 100% as some publications were associated with more than one priority area.

 

Source: Observatoire des sciences et des technologies, Web of Science (2014)

Overall, EC’s productivity in each priority area has been sustained over the past decade. For example, EC produced between 204 and 272 publications per year related to PA1 Contaminants and Stressors. Note that scientific peer-reviewed publications are only one of many science outputs at EC.

Date modified: