Skip booklet index and go to page content

Measuring Environment Canada's Research and Development Performance (2014)

Principle: Excellence

Excellence - Maintain stature, reputation and productivity

Why we measure

EC is regarded as a world leader in environmental science. That reputation for excellence is built on the Department’s research and development (R&D), which is conducted by highly skilled researchers and scientists working with integrity, rigour, and creativity.

How we measure

The excellence of Environment Canada's (EC) R&D is measured by the number and scientific impact of EC’s publications in peer-reviewed journals.

Results

Environment Canada is a highly productive institution, publishing around 700 scientific peer-reviewed papers per year.

Based on 2012 Science-Metrix data, EC ranks second in Canada (behind the University of British Columbia) and twentieth in the world in terms of number of environmental science publications. In a previous study, EC had ranked first in Canada and seventh in the world (Science-Metrix 2007). This decline in rank is primarily due to the fact that some other institutions have seen significant growth in the number of environmental science publications, while EC’s scientific publication output has remained relatively stable, as shown in the figure below. EC has produced around 700 scientific peer-reviewed papers per year over the past decade.

Figure 14: Number of EC’s publications, 2003–2013

Number of EC’s publications, 2003–2013 (See long description below)

Description of Figure 14

Number of EC’s publications, 2003–2013. EC has produced around 700 scientific peer-reviewed papers per year over the past decade.

Source: Scopus data compiled by Science-Metrix for years 2003-2009 (2012) and by EC for years 2010-2013 (2014)

The scientific impact of Environment Canada's publications is well above world average as measured by Average Relative Citation (ARC) and Average Relative Impact Factor (ARIF).

EC excels in quality as well as quantity of publications. Based on an OST study of 5,711 EC papers from 2004-2013, EC's papers are cited 50 per cent more than world average (ARC 1.5)Footnote 1 and are published in more impactful journals than world average (ARIF 1.17).Footnote 2 This data represents an increase from the ARC of 1.4 presented in the 2009 report for papers from 2003-2007.

Based on the figure below, the ARC of EC’s publications decreased from 2004 to 2009, but has increased since then. This apparent fluctuation in scientific impact cannot be explained with the data currently available. ARIF has remained relatively stable since 2004.

Even at their lowest point, EC’s ARC and ARIF are well above world average. This sustained high quality of EC’s peer-reviewed scientific publications contributes to the institution’s credibility and reputation.

Figure 15: Scientific impact of EC’s publications, 2004-2013

Scientific impact of EC’s publications, 2004-2013 (See long description below)

Description of Figure 15

EC excels in quality as well as quantity of publications. Based on an OST study of 5,711 EC papers from 2004-2013, EC's papers are cited 50 per cent more than world average (ARC 1.5) and are published in more impactful journals than world average (ARIF 1.17). This data represents an increase from the ARC of 1.4 presented in the 2009 report for papers from 2003-2007. Based on the figure below, the ARC of EC’s publications decreased from 2004 to 2009, but has increased since then. This apparent fluctuation in scientific impact cannot be explained with the data currently available. ARIF has remained relatively stable since 2004. Even at their lowest point, EC’s ARC and ARIF are well above world average. This sustained high quality of EC’s peer-reviewed scientific publications contributes to the institution’s credibility and reputation.

Source: Observatoire des sciences et des technologies, Web of Science (2014)

Within Environment Canada, the Department’s science is most impactful in Climate Change (PA3) and least impactful when it does not fall under a priority area (as measured by ARC).

Papers published in Contaminants & Stressors (PA1) and Weather (PA2) are highly impactful, with ARCs above both world and EC average. The ARC of papers published in Conservation & Protection (PA4) is above world average, but below EC average.

Figure 16: Scientific impact of EC’s publications by priority area (ARC), 2004-2013

Scientific impact of EC’s publications by priority area (ARC), 2004-2013 (See long description below)

Description of Figure 16

Scientific impact of EC’s publications by priority area (ARC), 2004-2013. Papers published in Contaminants & Stressors (PA1) and Weather (PA2) are highly impactful, with ARCs above both world and EC average. The ARC of papers published in Conservation & Protection (PA4) is above world average, but below EC average.

Source: Observatoire des sciences et des technologies, Web of Science (2014)

Compared to other institutions, Environment Canada’s science is world-leading in Contaminants & Stressors (PA1) and Weather (PA2). Productivity in Climate Change (PA3) and Conservation & Protection (PA4) are potential areas for improvement.

EC’s productivity, scientific impact and specialization in each priority area were compared with top national and international institutions. Productivity was measured by number of publications. Scientific impact was measured by ARC. Specialization was measured by a specialization index, based on the intensity of institutions’ publications in a particular field (i.e., priority area).

Figure 17: EC’s productivity, scientific impact and specialization by priority area, 2004-2013

 Productivity - Rank in CanadaProductivity - Rank in worldScientific impact - Rank in CanadaScientific impact - Rank in worldSpecialization - Rank in CanadaSpecialization - Rank in world
Contaminants & Stressors (PA1)1st4th6th10th1st2nd
Weather (PA2)1st16th5th16th2nd3rd
Climate Change (PA3)3rdn/a8thn/a2ndn/a
Conservation & Protection (PA4)4thn/a8thn/a4thn/a

Source: Observatoire des sciences et des technologies, Web of Science (2014)

Note:  Top Canadian and international institutions were identified by their productivity in each priority area. Arank of “n/a” means that EC was not in the top 25 most productive institutions. No further comparison was done.

Excellence - Adhere to internationally recognized standards and processes

Why we measure

In addition to its strong publication record, Environment Canada maintains excellence by adhering to internationally recognized standards and processes, which assure the integrity of EC science within the scientific community worldwide.

How we measure

EC’s adherence to these standards was measured by reference to the accreditation of Departmental labs and systems under the Canadian Association for Laboratory Accreditation Inc. (CALA) and International Organization for Standardization (ISO), and by EC’s contribution to the establishment of a new ISO standard.

Results

Environment Canada upholds the importance of international standards by securing accreditation of its laboratories through the Canadian Association for Laboratory Accreditation Inc.

The accreditation and governance of an institution’s facilities are important components when examining the excellence of its science. EC currently has 15 laboratories accredited through CALA, including 11 “wet” labs and 4 “dry” labs.Footnote 3 The Department’s consolidation of the S&T Branch in 2005 contributed to more effective capital planning, investment, and overall enterprise management of the Department’s laboratory assets.

One way in which EC adheres to internationally recognized standards and processes in its laboratories is by securing accreditation by CALA where possible. CALA is recognized internationally as a not-for-profit accreditation body. It serves public and private sector testing laboratories in Canada and abroad, with laboratory accreditation programs in the fields of environmental, food, mineral, and petroleum testing. CALA accreditation provides assurance that EC laboratories are competent to carry out the environmental testing that supports environmental monitoring programs and regulations.

According to the CALA Directory, there are currently twelve EC facilities accredited by CALA (including mobile laboratories and external laboratory space). This represents an increase from the 2009 report, which listed nine EC facilities certified under CALA.

Environment Canada contributes to international scientific standards and processes through a variety of fora.

MSC’s weather warning system is certified to the International Organization for Standardization (ISO) 9001:2008 Standard through a Quality Management System. This standard provides a framework that helps MSC identify and understand customers’ needs, and put in place processes to meet those needs. A key component of the certification is a commitment to continuously improving the quality of the information provided to Canadians.

EC not only adheres to international standards, it contributes to them in a variety of ways:

  • EC leads the development of an ISO standard on Environmental Technological Verification.

Through the S&T Branch, EC manages Canada’s Environmental Technology Verification (ETV) Program. The Program offers independent validation of environmental performance claims for innovative technologies, processes, and products. The ETV process is based on sound science, high-quality data, and recognized protocols.

In an effort to harmonize similar such initiatives globally, there was international interest in developing a common protocol for ETV.  As a result, Canada is currently leading the development of an ISO standard on ETV.  This standard, expected to be published in 2016, will facilitate recognition of environmental technologies and their capabilities across jurisdictions. International recognition of a common ETV protocol will foster faster and more widespread adoption of technologies to help solve environmental challenges.

  • EC develops standardized biological testing methods and contributes to international inter-laboratory validation activities.

Standardized biological testing methods for monitoring and controlling toxins and contaminants are essential for protection of the Canadian environment.  Since 1990, EC has published 24 standardized test methods and 7 supporting national guidance documents for performing biological testing in water, sediment and soil. Each method was primarily developed to help fulfill an existing or future regulatory need in Canada (i.e., Canadian Environmental Protection Act, Fisheries Act, provincial regulations).

A high degree of national and international consistency has been achieved due to the collaboration of laboratories during the development of each test method.  Each method undergoes a rigorous process of development and validation, including two rounds of peer review and numerous rounds of inter-laboratory validation.  The research and standardization process can take up to several years to complete.

Date modified: