The Status of Federal Scientific Integrity Policies - April 2022

Background

In 2018, the Governance Committee for the implementation of the Government-wide Scientific Integrity Policy^{Footnote 1} (the Governance Committee), whose membership includes the Chief Science Advisor, the Secretary of the Treasury Board (TB), and the President of the Professional Institute of the Public Service of Canada (PIPSC), published the Model Policy on Scientific Integrity^{Footnote 2}. The model was drafted as a guide for 24 (now 25^{Footnote 3}) federal departments and agencies (henceforth DDAAs) that were required to implement a scientific integrity policy (SIP) according to a 2016 Memorandum of Agreement between TB and PIPSC. Since January 2020, implicated DDAAs have been surveyed annually to determine their progress in implementing DDAA scientific integrity policies and procedures. Results of the 2020 and 2021 surveys were reported in The Status of Federal Scientific Integrity Policies^{Footnote 4} report, published in 2021. The current report summarizes results from the third annual survey conducted between early December 2021 and late January 2022.

The 2022 survey

As with the 2020 and 2021 surveys, the 2022 survey invited DDAAs to provide information on 13 compliance measures, each of which is associated with a non-discretionary provision of the model SIP^{Footnote 5}. Since DDAAs may adopt, adapt or even replace the model policy as they see fit, some compliance measures may not apply if a DDAA chose to eliminate the corresponding non-discretionary provision or made it discretionary in their policy.

The 2022 survey also included several new questions. A fourteenth compliance measure designed to provide information on progress in developing SIP performance monitoring and evaluation plans was introduced. In addition, the survey includes two additional questions about potential amendments to SIPs, as well as implementation actions taken by DDAAs from 2021 to 2022 (Appendix A).

As in the past, the 2022 report is based on a thorough review of all evidence provided by DDAAs to support survey responses. In cases where the evidence provided was deemed insufficient to justify the submitted response, DDAAs were contacted to discuss their responses and evidence. In some cases, these discussions resulted in modified DDAA survey responses.

2022 results

Twenty-two of the 25 implicated DDAAs responded to the survey in time for their results to be included in the following analysis; one DDAA responded too late to be included. The two non-responding DDAAs provided some basic information on the status of their policies in response to follow-up inquiries. Twenty-two of the 25 DDAAs, representing 2 733 RE employees and 9 528 SP employees, respectively^{Footnote 6}, now have a SIP in effect (Fig. 1); the results reported below are based on the 21 DDAAs who responded to the survey and who have a SIP in effect.

In support of responsible conduct of research, two key non-discretionary provisions of the model SIP are subjected to peer review of all technical communications and, where appropriate, approval of proposed research by a research ethics board^{Footnote 7}. Eighteen of 21 DDAAs (representing 12 000 employees in the RE and SP groups) now have peer review requirements in place, up from 15 in 2021. Fifteen (unchanged since 2021) DDAAs (representing 10 432 employees in the RE and SP groups) have, where appropriate, a requirement for research ethics board approval. For one and two DDAAs respectively, requiring peer review and, where appropriate, research ethics board approval was considered not applicable. One DDAA decided not to report on the requirement for research ethics board approval.

The model SIP includes non-discretionary provisions facilitating the federal science and research community in providing advice on regulations and policy, on departmental research programs, and on prioritizing investments in research. Sixteen of the 21 (representing 11 631 employees in the RE and SP groups) DDAAs have processes for soliciting advice on policies and regulations, while 14 of the 21 (representing 11 440 employees in the RE and SP groups) have processes for soliciting advice on research programs and the prioritization of departmental research investment. For two, three, and four DDAAs respectively, a formal process for soliciting researcher/scientist advice on regulations and policy, on departmental research programs, and on prioritizing investments in research was considered not applicable as the model SIP provisions that pertain to these actions have been removed or rendered discretionary in the departmental policy.

As of January 30, 2022, 20 DDAAs have notified their employees about the policy, and all DDAA’s with SIPs in effect now require the appointment of Scientific Integrity Lead (SIL), and all 20 have appointed a SIL. In all cases, the SIL is responsible for overseeing allegations of scientific integrity breach. In several DDAAs, the SIL is also charged with promoting departmental scientific integrity.

Since the 2020-21 survey, DDAAs have made significant progress implementing processes for peer review of all technical communications, responsible research education, notifying external collaborators and contractors, additional tools in support of scientific integrity policies, and soliciting advice on policy, regulations, and research programs. Slightly improved since the 2021 survey, almost 40% of DDAAs have yet to implement processes for notifying external collaborators and contractors about SIPs. Furthermore, no DDAA has implemented a monitoring plan that will provide information on the extent to which their SIP has achieved its objectives (Fig. 2).

No reporting DDAA has amended its policy in the past year. Eleven DDAAs have produced additional procedures, policies, guidelines, tools, training, or professional development opportunities relevant to their policies. These include:

• engaging staff and raising awareness of departmental scientific integrity policies and their application through open-houses, learning sessions, workshops, and videos;
• developing and implementing policy or guidelines for a range of actions, including investigation of alleged breaches; external communications; and dissemination of research and scientific findings;
• conducting internal reviews of their SIPs to propose potential revisions/amendments;
• developing SIP training materials;
• including SIP implementation within existing departmental science mentorship programs;
• investigating scientific integrity culture via employee surveys.

The evolution of federal scientific integrity policies

Many DDAAs have had SIPs in effect for more than two years. This collective experience with implementation has helped identify potential areas for improvement of the model policy itself, as well as associated implementation guidance. Moreover, since 2018 when the model policy was first developed, the federal government has made progress on several other agendas with potential implications for scientific integrity. Finally, the global pandemic has highlighted several issues – notably the importance of rapid sharing of scientific information - whose importance has taken on new relevance since the model policy was first developed.

To this end, the Governance Committee is soliciting input from DDAAs about issues that might be effectively addressed in the next iteration of the model policy. Two have already been identified, including:

1. navigating the potential tensions among open science, research security, and scientific integrity; and
2. the appropriate solicitation, gathering, communication, and use of Indigenous Knowledge.

Next steps

The 2021-2022 survey results demonstrate that federal DDAAs have made, and continue to make, considerable progress on scientific integrity. Building on this progress, over the next year, the Governance Committee will:

• Provide ongoing advice on the development and implementation of departmental SIP performance monitoring and evaluation plans. As noted above, no department has implemented such plans;
• Continue to monitor and evaluate SIP compliance and assist DDAAs in resolving outstanding issues, especially those related to peer review and ethics board approval of departmental research;
• Explore with DDAAs the implications of departmental SIPs on federal Open Science and Research Security initiatives (and vice versa) to ensure consistency and complementarity. This work may result in candidate provisions for inclusion inversion 2.0 of the Model Policy on Scientific Integrity;
• Continue to work with DDAAs to develop and provide education and training opportunities on several issues related to scientific integrity, especially the role of researchers and scientists in evidence-informed decision-making;
• Work with the I-STEM Cluster^{Footnote 10} and other partners to develop model policy provisions concerning the appropriate solicitation, gathering, communication, and use of Indigenous Knowledge.

Though much work remains, the Governance Committee is pleased with the progress made by federal departments and agencies on scientific integrity, especially in light of current urgent demands and continued associated resource reallocations arising from the COVID-19 pandemic. The committee is keenly aware that this progress reflects the commitment of federal departments and agencies, Ministers, and employees to ensure that government decisions are informed by scientific evidence and that Canadians are informed about important scientific issues that affect them.

Appendix A: Scientific Integrity Policy (SIP) Compliance Survey Questions

Compliance Measure (CM)

CM1. What is the status of your organization’s SIP?

CM2. Has your organization notified employees about the SIP?

CM3. Has your organization implemented additional procedures policies, guidelines, tools, training or professional development opportunities in support of the SIP?

CM4. Does your organization have a process for reporting and recording instances of policy conflict or incompatibility with the SIP?

CM5. Does your organization require that all technical communications undergo peer review?

CM6. Does your organization require that, where appropriate, research or scientific projects be reviewed and approved by a Research Ethics Board (REB)?

CM7. Has your organization taken steps to notify/inform contractors or extramural collaborators of the SIP and encourage them to comply with its articles?

CM8. Has your organization appointed a Science Integrity Lead?

CM9. Does your organization have a process for bringing forward and investigating breaches of scientific integrity?

CM10. Does your organization have a mechanism and/or procedure for soliciting researcher/scientist advice on deartmental policy and/or regulations?

CM11. Does your organization have a mechanism and/or procedure for soliciting researcher/scientist advice on its research programs?

CM12. Does your organization have a mechanism and/or procedure for systematically soliciting researcher/scientist assistance in identifying and prioritizing federal investment in research?

CM13. Does your organization have measures in place to support education, training and/or professional development in any of the following areas: responsible conduct in research; research eithcs; and the annotation, management and archiving of research and scientific data?

CM14. Does your organization have a monitoring plan for the Departments and Agencies (DDAA) SIP that will provide information on the extent to which the policy has achieved its objectives (policy performance)?

2021-2022 Survey Additional Questions:

1. Did your organization complete the 2020-21 compliance survey?
2. Have there been any amendments to your organization’s Scientific Integrity Policy (SIP) since the 2020-21 compliance survey?
3. Has your organization developed any additional procedures, policies, guidelines, tools, training or professional development opportunities relevant to the SIP since the 2020-21 survey?