Evaluation of Diversity and Inclusion
Table of Contents
Alternate Formats
Assistant Deputy Minister (Review Services)
Key Findings and Recommendations
Key Findings |
Recommendations |
---|---|
Relevance |
|
1. The diversity and inclusion initiatives directly support federal government mandated and legislated roles and responsibilities, and are fully aligned with government and departmental priorities. |
|
Performance—Achievement of Expected Outcomes (Effectiveness) |
|
2. While progress has been made with respect to EE and diversity and inclusion, this progress has been slow and not all goals have been met. |
|
3. More recent information and data are needed to report on the current state of diversity inclusion within DND and the CAF. |
|
4. There lacks a common understanding of the overall vision, objectives, outcomes and indicators of success for diversity and inclusion across the DND and CAF. |
1. Develop, in collaboration with the Defence Team, a clear vision and a common understanding of the diversity and inclusion objectives, including measurable outcomes and indicators, and communicate this to stakeholders across all levels of DND and the CAF. |
5. The overall governance and structures for diversity and inclusion, including roles, responsibilities and accountabilities, are unclear. This lack of clarity has resulted in gaps and duplication of efforts with respect to diversity and inclusion initiatives across DND and the CAF. |
2. Implement a Defence Team approach for the governance and structures of the diversity and inclusion, including clear roles and responsibilities for organizations and stakeholders across DND and the CAF. |
6. Current performance measurement activities are largely focused on EE data and do not allow DND and the CAF to report on the success of diversity and inclusion initiatives. |
3. Once the diversity and inclusion outcomes have been developed and approved, implement a performance measurement strategy to enable the collection, reporting and communication of results to inform changes to policies and procedures. |
Table 1. Key Findings and Recommendations. This table lists the key findings along with associated recommendation for the evaluation.
Table 1 Details - Key Findings and Recommendations.
Note: Please refer to Annex A – Management Action Plan for the management responses to the ADM(RS) recommendations.
1.0 Introduction
1.1 Context for the Evaluation
This report presents the findings and recommendations of the Evaluation of Diversity and Inclusion, which are activities and initiatives of the DTMP. This targeted evaluation was conducted in accordance with the TB Policy on Results, and supports the Five-Year DND and CAF Departmental Evaluation Plan (2017/18 to 2022/23), as approved by PMEC in March 2018.Footnote 1 The evaluation is intended to support CMP and ADM(HR-Civ) in their efforts to design and deliver aligned HR diversity and inclusion initiatives for both the civilian and military personnel of DND and the CAF.
Since 2014, there have been two evaluations and eight audits conducted by ADM(RS) and the Office of the Auditor General of Canada (OAG) on the CMP and ADM(HR-Civ) portfolios, related to the management of civilian and military personnel.Footnote 2 These previous evaluations and reviews did not examine the results of DND and CAF diversity and inclusion initiatives.
1.2 Program Profile
1.2.1 Program Background
The DTMP encompasses eight program components that relate to the management of civilian and military personnel.Footnote 3 The DTMP supports the DRF CR 3, Defence Team, which aims to “[r]ecruit, develop and support an agile and diverse Defence Team, within a healthy workplace free from harmful behaviour; support military families; and meet the needs of all retiring military personnel, including the ill and injured” and “[s]trengthen Canadian communities by investing in youth.” Footnote 4
The diversity and inclusion activities and initiatives of the DTMP support DND and the CAF in building workforce diversity, fostering workplace inclusiveness, and working towards high performing and operationally effective organizations. More specifically, the program results are achieved through personnel policy development and implementation, planning and coordination, governance, performance management, reporting, research, the creation and allocation of compensation and benefits, and management of military careers. The program also implements, monitors and reports on legislative directives and requirements (e.g., Diversity, Operation HONOUR, etc.).
The DTMP is comprised of the following eight program segments:
- Military Compensation and Benefits
- Military Career Management
- Military Personnel Research, Development and Analysis
- Military Personnel Command Governance and Operations
- Military Personnel Command Strategic Development
- Military Personnel Generation and Individual Training and Education
- Foster A Climate of Dignity and Respect
- Civilian Diversity and Inclusion
The DND and CAF HR-related diversity and inclusion activities and initiatives refer to the afore-mentioned two program segments, “Civilian Diversity and Inclusion” and “Military Personnel Command Strategic Development” and are linked to the following two DTMP immediate outcomes:Footnote 5
- Civilian workplace is respectful, diverse and inclusive (civilian)
- CAF personnel programs and policies foster a climate of dignity and respect (military)
The logic model of the DTMP can be found in Annex B.
1.2.2 Diversity and Inclusion Description
The diversity and inclusion activities and initiatives support the federal government as well as DND and CAF priorities related to fostering a diverse workforce and an inclusive work environment free of employment barriers. As such, the approach focuses on enabling a culture shift whereby DND and the CAF are viewed as employers of choice that value diversity and inclusion, and where diversity and inclusion enable organizations to be high performing and operationally effective. The diversity and inclusion activities and initiatives thus support the DND and CAF Departmental Result aimed at creating a “Defence Team [that] reflects the values and diversity of Canadian society.”Footnote 6
More specifically, the objectives of diversity and inclusion are to further the concept of workforce diversity in consideration of:
- the Designated Group Members (DGM) of the EEA (women, Indigenous peoples, visible minorities and people with disabilities);
- the Canadian Human Rights Act, related to the prohibited grounds of discrimination (race, national or ethnic origin, colour, religion, age, sex, sexual orientation, gender identity or expression, marital status, family status, genetic characteristics, disability or conviction for an offence for which a pardon has been granted or in respect of which a record suspension has been ordered); and
- any other (demographic) elements such as education, life experiences, skills and abilities, learning styles and much more.
Furthermore, the objectives for diversity and inclusion are also aimed towards supporting a work environment where military and civilian personnel feel included and empowered to bring their whole selves to work and contribute to their fullest potential.
Diversity and inclusion is operationalized through the following main activities:
- Legislative and regulatory compliance: By reviewing and developing DND and CAF HR‑related policies, programs, plans and initiatives, including performance measures, mainly aimed towards EE, diversity and inclusion.
- Governance and leadership: By providing a governance structure for the diversity and inclusion requirements which enables the provision of direction, guidance, advice and services with regards to the implementation of the approach and other related initiatives.
- Attraction, recruitment, training and retention: By conducting and leveraging research, designing and implementing training and awareness programs and activities, including promotional activities, to attract, recruit, train and retain military and civilian personnel.
The logic model of Diversity and Inclusion can be found in Annex C.Footnote 7
1.2.3 Stakeholders
The Functional Authorities for “employment equity and diversity” are CMP and ADM(HR-Civ). The Director Human Rights and Diversity (DHRD) in CMP and the Director Workforce Programmes and Services (DWPS) from ADM(HR-Civ) are the functional stakeholders responsible for the design and delivery of diversity and inclusion activities and initiatives, including its performance measurement.Footnote 8
Other diversity and inclusion stakeholders include:
- Director Integration of Gender Perspectives (DIGP), Strategic Joint Staff (SJS)
- Director Gender, Diversity and Inclusion (DGDI), Corporate Secretary (Corp Sec)
- Diversity and Inclusion Defence Champions
- Champions of Gender and Diversity for Operations
- Champions for Gender-based Analysis Plus (GBA+)
- Champion for Women, Peace and Security (WPS)
- Defence Employment Equity Champions
- Champion for Women
- Champion for Indigenous peoples
- Champion for Visible minorities
- Champion for People with disabilities
- Defence Advisory Groups (DAG) at the local, regional and national levels
- Defence Women’s Advisory Organization
- Defence Aboriginal Advisory Group
- Defence Visible Minority Advisory Group
- Defence Advisory Group for Persons with Disabilities
- Employment Equity Officers
- Gender Advisors and Gender Focal Points
An overview of the diversity and inclusion related organizations and stakeholders, including a summary of their roles, responsibilities and mandates, can be found in Annex D.
1.3 Evaluation Coverage
1.3.1 Evaluation Scope
The initial scope of the evaluation was developed through interviews with senior management at the Director General and Director levels within the Functional Authorities. As a result, it was determined that the evaluation would assess the success of the diversity and inclusion initiatives and its alignment between DND and the CAF. This emphasis was reiterated in a letter from the CMP, proposing that “the evaluation related to [the DTMP] be confined to a review of the CAF Diversity Strategy with a view to better alignment with National Defence’s priorities in this area.”Footnote 9 The evaluation scope was tabled and approved at the PMEC meeting in March 2018. The following two diversity and inclusion immediate outcomes were included in the scope of the evaluation:
- Diversity and inclusion initiatives are aligned between DND and the CAF
- Diversity and inclusion considerations are integrated into DND and the CAF at all levels
Additional in-depth scoping interviews were conducted with CMP, ADM(HR-Civ), SJS and Corp Sec staff to identify priority needs and further refine the scope of the evaluation. Given that the plans and priorities for diversity and inclusion are fairly new and constantly evolving, it was determined that the evaluation would adopt a forward-looking approach and focus on assessing the extent to which the current organizational structures and governance facilitate the achievement of outcomes to date and in the future. The evaluation also explored the availability of performance measures by examining the implementation of the DTMP PIP.
1.3.2 Program Resources
Given that diversity and inclusion initiatives are activities associated with the larger DTMP, expenditure data and resources (FTEs and spending) directly associated with diversity and inclusion activities are not available at this time. However, this information is available for the DTMP. DTMP was established as a program in fiscal year (FY) 2017/18, therefore, historical expenditure data was not available. Table 2 shows that the planned spending budget of the program is $594.9 million in FY 2018/19 and will be slightly increasing to $610.7 million for FY 2020/21. Table 2 also shows that the planned Full-Time Equivalent (FTE) for the program was 1,829 military and civilian members of DND and the CAF in FY 2018/19, and slightly increasing to 1,849 for FY 2020/2021.Footnote 10
2018/19 | 2019/20 | 2020/21 | |
---|---|---|---|
Planned Spending ($M) |
594.9 |
601.2 |
610.7 |
Planned FTEs (n) |
1,829 |
1,839 |
1,849 |
Table 2. DTMP Planned Spending and Planned FTEs, FY 2018/19 to FY 2020/21.Footnote 11 This table displays the planned spending and planned FTE for the DTMP Program for FYs 2018/19 to 2020/21.
Table 2 Details - DTMP Planned Spending and Planned FTEs, FY 2018/19 to FY 2020/21.
1.3.3 Issues and Questions
In accordance with the TB Policy on Results, the evaluation addressed issues related to the relevance and effectiveness of the diversity and inclusion activities and initiatives.
- Relevance: The evaluation assessed the extent that diversity and inclusion align with, and contribute to, federal government and departmental priorities.
- Effectiveness: The evaluation assessed progress towards the achievement of diversity and inclusion outcomes to date; the extent that the Functional Authorities (shared between the CMP and ADM(HR-Civ)) have an appropriate governance and structures to support the achievement of outcomes to date and in the future; and the state of the performance measures and data to inform the diversity and inclusion related outcomes.
An evaluation matrix listing each of the evaluation issues, questions and indicators, with associated research methods, is provided in Annex E. The methodology used to gather evidence in support of the evaluation questions can be found in Annex F.
2.0 Findings and Recommendations
2.1 Relevance—Alignment with Government Priorities
Key Finding 1: The diversity and inclusion initiatives directly support federal government mandated and legislated roles and responsibilities, and are fully aligned with government and departmental priorities.
This key finding is based on a comprehensive document review of federal government and departmental roles, responsibilities and priorities in the context of diversity and inclusion.
2.1.1 Alignment with Federal Government Roles and Responsibilities
Firstly, the diversity and inclusion approach is in alignment with the Canadian Charter of Rights and Freedoms and the Canadian Human Rights Act, aimed at improving workplace conditions, particularly for individuals that are discriminated against or disadvantaged due to race, national or ethnic origin, color, religion, sex, age, sexual orientation, gender identity or expression, family status, genetic characteristics or disabilities.Footnote 12 Secondly, diversity and inclusion aligns with the EEA of equality in the workplace and proactive employment for women, Indigenous peoples, persons with disabilities and members of visible minorities.Footnote 13 In addition, the approach aligns with Art. 22(d) of the Public Service Employment Act, which requires departments to adapt their hiring strategies to facilitate and improve the recruitment of equity-seeking groups.Footnote 14 Lastly, diversity and inclusion aligns with other complimentary legal frameworks such as the Official Languages Act and Canadian Multiculturalism Act, which protect and promote linguistic and cultural diversity in Canadian institutions.Footnote 15
On the international front, the Government of Canada (GC) is also committed to the implementation of the United Nations Security Council Resolution 1325 on Women, Peace and Security (UNSCR 1325) through Canada’s National Action Plan (CNAP). Diversity and inclusion activities and initiatives align with federal government commitments to the UNSCR 1325 and address the objectives set out in CNAP to increase women’s representation, as well as active and meaningful participation in peace operations, peace processes, management of conflict situations and decision making.Footnote 16
2.1.2 Alignment with Federal Government and Departmental Priorities
The GC prioritizes diversity and inclusion, as seen in several key government documents. First, diversity is one of the five main government priorities presented in the Speech from the Throne (2015), which highlights the positive socioeconomic impacts of diversity and specific initiatives towards Indigenous people, women, immigrants, veterans and official language minorities.Footnote 17
Organizational Benefits of Diversity and Inclusion
Recent research suggests that given the right context, diversity and inclusion will increase employee overall productivity and performance in certain roles, thus increasing overall organizational effectiveness.Footnote 18 In fact, organizations that reflect their social environment are better able to attract and retain more diverse employees, thus perpetuating the cycle.Footnote 19 In a management and practice framework that creates an inclusive climate, diversity consistently leads to increased organizational outcomes.Footnote 20
Second, the Minister’s mandate letter supports government commitments to diversity and inclusion, including “transparent, merit based appointments to help ensure gender parity and that Indigenous Canadians and minority groups are better reflected in positions of leadership.”Footnote 21 Finally, the latest Federal Budget (2018) supports gender equality through gender equality goals and gender budgeting.Footnote 22
Additionally, in 2016 the GC renewed its support to GBA+ and launched a new action plan to strengthen its implementation across all federal departments in an effort to support women and other diverse groups by putting gender at the center of the decision-making process.Footnote 23 Diversity and inclusion contributes to the government’s commitment to address historically unjust federal policies and practices that contribute to systemic discrimination against Canadian LGBTQ2 community members, including federal public servants and CAF members.Footnote 24 In sum, the objectives of diversity and inclusion are directly aligned with government priorities, through the integration of diversity and inclusion initiatives in the revision and development of HR-related policies, programs and services.
Specific to DND and the CAF, the objectives set out in Canada’s defence policy, Strong, Secured, Engaged (SSE), promote an institution-wide culture that embraces diversity and inclusion.Footnote 25 To fully leverage Canada’s diversity, the SSE outlines six initiativesFootnote 26 towards greater workplace diversity and inclusion, the integration of GBA+, and improving recruitment and retention of under-represented populations. The DND and CAF Departmental Plan (2018/19) mirrors these initiatives through the Departmental Result 3.3 which states that the “Defence Team reflects the values and diversity of Canadian society.”Footnote 27 In sum, supporting recruitment priorities related to diversity, and improving the management of diversity and inclusion-related programs, services and initiatives within the workforce and the work environment are core departmental priorities that align with the Defence Team approach for diversity and inclusion.
2.2 Performance—Achievement of Expected Outcomes (Effectiveness)
This section of the report relies on information from previous studies and reports, along with interviews with key informants to assess the current state of diversity and inclusion and progress towards the achievement of outcomes, the need for clearly defined diversity and inclusion outcomes, and the existence of organizational structures and governance to support diversity and inclusion initiatives.
2.2.1. State of Diversity and Inclusion within DND and the CAF
Key Finding 2: While progress has been made with respect to EE and diversity and inclusion, this progress has been slow and not all goals have been met.
Key Finding 3: More recent information and data are needed to report on the current state of diversity inclusion within DND and the CAF.
These key findings are based on a review of research and academic documents focusing on diversity and inclusion. This review was conducted in an effort to gather the most relevant studies in the area of diversity and inclusion and provide evidence to assess the progress that has been made towards building workforce diversity and fostering workplace inclusiveness. Three main themes were revealed from the review: EE; recruitment and retention; and occupational training and career development. These themes are discussed in greater detail in the following section.
Employment Equity Statistics
Table 3 shows the DND and CAF EE representation rates for the past five FYs, along with the annual goals for DND and the long-term EE goal for the CAF.
CAF |
||||||||
---|---|---|---|---|---|---|---|---|
Year |
Women |
Indigenous Peoples |
Visible Minorities |
Persons with Disabilities |
||||
2013/14 |
14.9% |
2.3% |
5.3% |
|||||
2014/15 |
14.9% |
2.5% |
6.1% |
1.3% |
||||
2015/16 |
14.9% |
2.6% |
7.0% |
1.5% |
||||
2016/17 |
15.2% |
2.7% |
7.6% |
1.4% |
||||
2017/18 |
15.4% |
2.8% |
8.4% |
1.3% |
||||
2026 Goal |
25.1% |
3.5% |
11.8% |
n/a |
||||
DND |
||||||||
Goal |
Rate |
Goal |
Rate |
Goal |
Rate |
Goal |
Rate |
|
2013/14Footnote 29 |
- |
- |
- |
- |
- |
- |
- |
- |
2014/15 |
39.1% |
40.9% |
2.5% |
3.1% |
9.1% |
7.5% |
4.6% |
5.7% |
2015/16 |
39.1% |
40.0% |
2.7% |
3.1% |
8.4% |
7.3% |
4.6% |
6.1% |
2016/17 |
39.8% |
40.9% |
2.7% |
3.2% |
8.5% |
7.5% |
4.6% |
6.1% |
2017/18 |
39.5% |
40.0% |
2.6% |
3.1% |
8.7% |
7.8% |
4.6% |
5.4% |
Table 3. Employment Equity Representation Rates and Goals, FY 2013/14 to FY 2017/18.Footnote 30 This table displays the DND and CAF EE representation rates and goals, for FYs 2013/14 to 2017/18.
Table 3 Details - Employment Equity Representation Rates and Goals, FY 2013/14 to FY 2017/18.
As indicated in Table 3, there has been some progress in achieving the EE goals. Nevertheless, some areas need further improvement.Footnote 31 The CAF has seen increases from FYs 2016/17 to 2017/18 in women’s representation (+0.2%), Indigenous peoples’ representation (+0.1%) and visible minority representation (+0.8%), with the aim to achieve set goals by 2026. This progress, however, has been minimal, which will make achieving 2026 goals challenging. DND estimates of DGMs, based on self-identification rates and current goals are: Women (40% versus 39.5%), Visible Minorities (7.8% versus 8.7%), Persons with Disabilities (5.4% versus 4.6%), and Aboriginal Peoples (3.1% versus 2.6%). DND’s report notes under-representation based on occupational categories as follows: Women (management, administrative and foreign services, technical management, scientific and professional), Visible Minorities (technical, operational), Indigenous Peoples (operational, scientific and professional), and Persons with Disabilities (operational).Footnote 32
Perceptions of the Defence Team on Diversity and Inclusion
This evaluation did not collect primary data on the Defence Team’s perceptions of diversity and inclusion. Instead, the intent was to review past research and surveys such as the Employment Systems Review (2013), Your Say Survey (2011) and the Diversity Climate Survey (2007), which documented CAF members’ perceptions and views on areas related to EE, diversity and inclusion. Given that these reviews and surveys are dated, it was determined that they may not reflect the current views of the Defence Team on the state of diversity and inclusion within DND and the CAF. More recent survey data was not available at the time of this evaluation. Since 2013/14, DND and CAF research agenda has shifted away from the study of the diversity and inclusion climate and its integration in the organizational culture, to other specific issues such as harassment and discrimination and the recruitment and retention of various DGMs, which were not the focus of the current evaluation.
It should be noted that the synthesized findings from the older internal and external research report on the state of diversity and inclusion within the CAF highlighted some areas of concern. However, it appears that there may be a positive trend as a more recent study indicated that overall attitudes and perceptions towards EE and diversity in the CAF are generally positive and have improved over time.Footnote 33 ,Footnote 34 In order to obtain the current views (as of 2019) of the Defence Team with respect to diversity and inclusion, and to assess the results and impacts of the diversity and inclusion-related activities and initiatives, new studies and survey data would be required (also see Section 2.3). The Defence Team Workplace Well-Being Survey was conducted in 2018/19 and may be able to shed light on current perceptions, but the results were not available at the time of this evaluation.
As for DND, the 2017 Public Service Employee Survey revealed that 67 percent of DND employees believe that the department works to ensure that everyone is accepted as an equal member of their team, and that 73 percent believe that the department implements activities and practices that support a diverse workplace.Footnote 35 These overall results are closely matched with those of the public service, with only marginal differences.
2.2.2. Understanding Diversity and Inclusion: Definitions and Outcomes
Key Finding 4: There lacks a common understanding of the overall vision, objectives, outcomes and indicators of success for diversity and inclusion across DND and the CAF.
Stakeholders consulted, from senior management to the staff level, indicated that there was a lack of a common understanding of the diversity and inclusion activities and initiatives, and its objectives across DND and the CAF at all levels. Moreover, stakeholders felt that the absence of a lexicon that provides common definitions of diversity and inclusion and its related concepts has resulted in confusion and a lack of cohesiveness among organizations and stakeholders across DND and the CAF. As such, key informants highlighted the need to develop a strategic message that would promote both a common understanding and clearly defined concepts for diversity and inclusion. Key informants further reinforced the need for increased organizational knowledge on both the diversity and inclusion approach and its relationship to other similar or related programs, such as: EE, GBA+, and Women, Peace and Security (WPS).
While strategic documents such as the DND and CAF Departmental Plan and SSE provide clear overall intent for diversity and inclusion, key informants indicated that the short, medium and longer term outcomes that support these departmental plans and priorities have not been clearly outlined (developed and defined) and communicated across DND and the CAF. As such, the lack of clear outcomes for diversity and inclusion leads to difficulty for stakeholders in developing and implementing policies, programs, strategies and initiatives that would effectively work towards achieving the overall desired intent for diversity and inclusion. While a logic model was developed by the evaluation team to support the evaluation, this logic model has not been approved by senior management as representing the overall diversity and inclusion theory of change. One key informant clearly articulated and summarized the overall main finding:
“The challenge is in the lack of a [common] understanding of what diversity and inclusion means. A short and concise definition would help to move diversity and inclusion forward. While the intention is clear, it is not clear how it should be [operationalized or] applied on a day-to-day basis as well as how to measure it. Clarity and the outcomes to be measured are the big issues. We have difficulty in communicating something that is not understood.”
The evidence suggests that the lack of understanding of what diversity and inclusion entails (e.g., definitions and outcomes) has resulted in a lack of clear and consistent direction and guidance from senior management on how to progress on this agenda. As such, inefficiencies and uncertainties were documented by the evaluation team, as stakeholders are constantly attempting to define their roles and responsibilities related to diversity and inclusion.
ADM(RS) Recommendation
1. Develop, in collaboration with the Defence Team, a clear vision and a common understanding of the diversity and inclusion objectives, including measurable outcomes and indicators, and communicate this to stakeholders across all levels of DND and the CAF.
OPI: CMP, ADM(HR-Civ)
OCI: SJS, Corp Sec, Defence EE Champions, Diversity and Inclusion Defence Champions, DAGs
2.2.3. Organizational Structures and Governance (Alignment)
Key Finding 5: The overall governance and structures for diversity and inclusion, including roles, responsibilities and accountabilities, are unclear. This lack of clarity has resulted in gaps and duplication of efforts with respect to diversity and inclusion initiatives across DND and the CAF.
Organizational Structures
The Functional Authority in the current structure is split between the CMP and ADM(HR‑Civ) which has created issues and challenges for functional stakeholders to design and deliver aligned HR policies, programs, plans and initiatives for the Defence Team. Stakeholders mentioned that it is not clear whether the relationship between the Functional Authorities is a true and equal partnership or if one organization has the lead.
The diversity and inclusion activities and initiatives are currently designed and delivered by two separate Functional Authorities and, as a result, issues with governance and clarity of roles, responsibilities and accountabilities is evident. In addition to the issues previously mentioned, these challenges further impede the alignment of the overall approach, leading to additional negative impacts on the delivery of diversity and inclusion activities and initiatives and, thus, the achievement of outcomes.
Collaboration efforts among DND and CAF functional stakeholders are challenging because stakeholders are pursuing separate directions and priorities from their respective chains of command. In addition, the diversity and inclusion activities and initiatives are managed at different organizational levels with varying degrees of influence and support, including capacity. Finally, the current approach in the design and delivery of the diversity and inclusion related activities and initiatives does not facilitate regular communication. Information is shared on an ad hoc basis, which has led to work being completed in silos.
Governance
Two oversight governance committees were identified by stakeholders to have the potential to oversee and enable alignment for the DND and CAF diversity and inclusion approach. This includes the following committees:
The Employment Equity and Diversity Departmental CommitteeFootnote 36 (EEDDC) which is a component of the DND and CAF EE and Diversity Governance structure. The EEDDC has the mandate to “promote alignment and cohesion amongst the departmental Functional Authorities for Employment Equity and Diversity and the Defence Champions.”
The Defence Team Human Resources CommitteeFootnote 37 (DT HRC) was recently established as the official DND and CAF governance oversight committee for all common military and civilian HR areas of responsibility. Diversity and inclusion falls under its subcommittee, the Human Resources Representative, Inclusive and Respectful Workplace CommitteeFootnote 38 (HR RIRWC). The mandate of the HR RIRWC is to act as a “consultative body,” and its objectives are to further the Defence Team’s HR Strategy through the achievement of workforce diversity and workplace inclusiveness.
Despite their existence, evidence suggests that these governance committees are not implemented or used as intended. In addition, stakeholders were not in agreement on the appropriateness of these committees to oversee the activities and initiatives for diversity and inclusion. These issues have led to further challenges associated with developing an integrated, coordinated and aligned Defence Team approach for diversity and inclusion.
Roles and responsibilities
In addition to the functional stakeholders under the Functional Authorities, other stakeholdersFootnote 39 for diversity and inclusion across DND and the CAF include:
- Director Gender Diversity and Inclusion (GBA+), Corp Sec
- Director Integration Gender Perspectives (GBA+), SJS
- Defence Champions (for Diversity and Inclusion, and EE)
- DAGsFootnote 40
There was agreement among most diversity and inclusion stakeholders, including the functional stakeholders, that roles and responsibilities are not well defined or clearly communicated to them and to other organizations within DND and the CAF. The lack of clarity with respect to roles and responsibilities has resulted in diluted accountabilities. Functional stakeholders indicated that they do not understand how other DND and CAF stakeholders’ mandates support or complement diversity and inclusion. As such, duplication of effort exists among functional stakeholders, including with other DND and CAF organizations and stakeholders. In addition, key informants identified a lack of clarity in the roles and responsibilities of stakeholders and their relationships, and a lack of knowledge related to the objectives and outcomes for diversity and inclusion as well as around the activities and initiatives to further these objectives and outcomes.
With the growing importance of diversity and inclusion within DND and the CAF, many stakeholders indicated limited capacity in many areas, including financial, time, expertise and support, and identified the need to align organizational capacity with the diversity and inclusion plans and priorities in order to fulfill their roles and responsibilities.
ADM(RS) Recommendation
2. Implement a Defence Team approach for the governance and structures for diversity and inclusion, including clear roles and responsibilities for diversity and inclusion organizations and stakeholders across DND and the CAF.
OPI: CMP, ADM(HR-Civ)
OCI: SJS, Corp Sec, Defence EE Champions, Diversity and Inclusion Defence Champions, DAGs
2.3 Performance—Design and Delivery (Performance Measurement)
Consultations and interviews with key stakeholders were conducted to gather evidence of the data currently being collected to measure the success of the diversity and inclusion activities and initiatives. This key finding is based on key stakeholder knowledge, experiences and perspectives, along with a review of documents, such as the DTMP PIP.
Key Finding 6: Current performance measurement activities are largely focused on EE data and do not allow DND and the CAF to report on the success of diversity and inclusion initiatives.
All functional stakeholders interviewed mentioned that only EE data is currently being collected and there are no plans to integrate other performance measures and data related to diversity and inclusion into the DTMP PIP. A common challenge cited among stakeholders is the difficulty they face in measuring the outcomes of diversity and inclusion activities. However, functional stakeholders did mention that this challenge is not unique to DND and the CAF, as other federal government departments are also experiencing difficulties in establishing meaningful measures to capture the results of diversity and inclusion initiatives. Currently, the only performance measures and data captured in the DTMP PIP are the public Departmental Results Indicators which support both the Departmental Results 3.3, “[t]he Defence Team reflects the values and diversity of Canadian society”, and are related to the EE statutory requirements:Footnote 41
- Percentage of the Defence Team that self-identify as a woman
- Percentage of the Defence Team that self-identify as a visible minority
- Percentage of the Defence Team that self-identify as an indigenous person
The evaluation also explored the extent to which other departmental stakeholders are measuring the performance of diversity and inclusion initiatives. It should be noted that Diversity and Inclusion Defence Champions are currently defining their role, which includes developing a strategic action plan, as well as a framework to measure performance. As for the integration of GBA+ into all defence activities, the DGDI has initiated the development of a logic model and is working towards developing a framework and data analytics for measuring their programming. In terms of the DAGs, key informants indicated the absence of an official performance monitoring framework, and common reporting system (i.e., database) and guidelines to capture data on systemic issues and barriers.
The evaluation also investigated the possibility of exploring established organizations, internal and external to DND and the CAF, which may already have measures and captured data that could be successfully integrated into the DTMP PIP and used to inform the diversity and inclusion activities and initiatives. These organizations are listed in Table 4.
Internal |
External (other federal government departments) |
---|---|
|
|
Table 4. Organizations and Structures with Relevant Measures and Data. This table displays the internal and external organizations and structures to DND and the CAF that have performance measures and data that could be used to inform the diversity and inclusion and the DTMP PIP.
Table 4 Details - Organizations and Structures with Relevant Measures and Data.
As a complement to the afore-mentioned findings, the evaluation team conducted a review of academic literature and organizational documents for best practices in diversity and inclusion performance measurement. This review can be found in Annex G.
ADM(RS) Recommendation
3. Once the diversity and inclusion outcomes have been developed and approved, implement a performance measurement strategy to enable the collection, reporting and communication of results to inform changes to policies and procedures.
OPI: CMP, ADM(HR-Civ)
Annex A—Management Action Plan
The following provides the Functional Authority Management Action Plan (MAP) in response to the ADM(RS) Evaluation of Diversity and Inclusion in the Defence Team.
This MAP will propose steps that will institutionalize and improve the performance of the Diversity and Inclusion approach in the Defence Team in order to respond to the ADM(RS) evaluation. Effective Diversity and Inclusion approach requires an understanding of what diversity and inclusion entails, its objectives, outcomes and indicators for success. It is only through this preliminary understanding and foundation that a careful examination of the overlaps and gaps in the existing Defence Team diversity and inclusion programming can occur, and innovative approaches to implementing diversity and inclusion as a core institutional value can be leveraged.
This proposed MAP seeks to effectively balance the Defence Team hierarchy and reporting order, while generating an approach that brings together the currently siloed and decentralized stakeholders when and where feasible, acknowledging that accepted differences exist between DND and the CAF in certain areas, including that of functional authority which could result in the identification and implementation of distinct and specific priorities, objectives, strategies and action plans.
ADM(RS) Recommendation
1. Develop, in collaboration with the Defence Team, a clear vision and a common understanding of the diversity and inclusion objectives, including measureable outcomes and indicators, and communicate this to stakeholders across all levels of DND and the CAF.
Management Action
At the strategic level, ensuring a clear and coordinated effort for the Diversity and Inclusion activities and initiatives is essential for the Defence Team. Fostering a work environment and culture that leverages the diverse skills of CAF and DND personnel will aid the Defence Team in reflecting the society we serve. Providing care and programs for Defence Team members will ensure the operational effectiveness and security of Canada as we work toward improving the Total Health and Wellness of each individual. A multi-phased approach will ensure coordinated and cohesive progress toward the end state of a Defence Team that reflects the values of Canadian society.
MAP 1.1
Deliverable: Definition and establishment of objectives via a Defence Team Diversity and Inclusion Strategic Framework.
OPI: CMP and ADM(HR-Civ)
OCI: DHRD and DGWD as co-chairs of the Human Resources Representative and Inclusive Committee (DT HR-RIC), a subcommittee responsive to DT HRC. This subcommittee will evaluate the function/portfolio of Diversity and Inclusion program across the Defence Team. Subcommittee membership will include the multitude of stakeholders engaged in, and working toward, diversity and inclusion-related activities. Unions and DAGs are also included.
Target Date: Defence Team Strategic Framework endorsed by DT HRC November 2019. Further refinement by the DT HR-RIC to begin winter 2020.
ADM(RS) Recommendation
2. Implement a Defence Team approach for the governance and structures of diversity and inclusion, including clear roles and responsibilities for organizations and stakeholders across DND and the CAF.
Management Action
Following the definition and establishment of the DND/CAF objectives for the function/portfolio of Diversity and Inclusion (MAP 1.1), the functional authorities, along with key stakeholders, will assess the governance structure of the diversity and inclusion activities. This is particularly pertinent given capacity and resource restraints noted throughout the ADM(RS) evaluation and experienced by the functional authorities.
MAP 2.1
Deliverable: Creation of a clear governance structure for the Diversity and Inclusion portfolio.
OPI: DT HRC and Corp Sec
OCI: DT HR-RIC
Target Date: Initial operational capability November 2019, Full operational capability March 31, 2021
MAP 2.2
Deliverable: Develop or clarify authorities, responsibilities and accountabilities for all diversity and inclusion Defence Team stakeholders.
OPI: CMP and ADM(HR-Civ)
OCI: DT HR-RIC, Unions
Target Date: March 31, 2021
ADM(RS) Recommendation
3. Once the diversity and inclusion outcomes have been developed and approved, implement a performance measurement strategy to enable the collection, reporting and communication of results.
Management Action
As per the ADM(RS) recommendation, there is a need at this time for a Performance Measurement (PM) Strategy to measure the performance of the delivery of diversity and inclusion activities and initiatives. Following the creation and recommendation of a Defence Team approach for diversity and inclusion governance (MAP 2.1, 2.2), a PM Strategy will be developed for the organizational governance recommended by the functional authorities for Diversity and Inclusion following stakeholder consultations. This PM Strategy will be aligned with strategic direction, assess performance and include measures of output. It will include a logic model, key performance questions and key performance indicators, and be based on the ADM(RS) evaluation Logic Model included in the report. Furthermore, this PM Strategy will aspire to consolidate ongoing efforts related to Diversity and Inclusion reporting and measurement, including but not limited to the Employment Equity Plans.
MAP 3.1
Deliverable: Development of a PM Strategy to account for the Defence Team Diversity and Inclusion program.
OPI: CMP and ADM(HR-Civ)
OCI: DT HR-RIC, DAGs, Unions
Target Date: December 31, 2021
Note: Civilian Action Plan completed November 2019 which captures some initial performance measures and CAF 2017 Action Plan is under revision.
Annex B—Logic Model of the Defence Team Management Program
Figure B-1 Details - Logic Model of the Defence Team Management Program.
Annex C—Logic Model of Diversity and Inclusion
Figure C-1 Details - Logic Model of Diversity and Inclusion.
Annex D—Overview of the Diversity and Inclusion Stakeholders
This annex provides an overview of the diversity and inclusion related organizations and stakeholders, including a summary of their roles, responsibilities and mandates.
Director Human Rights and Diversity, Military Personnel Command
DHRD is responsible for ensuring that all current and future CAF policies and programs are in compliance with the Canadian Charter of Rights and Freedoms and the Canadian Human Rights Act through initiating and developing policies, directives and instructions. DHRD also analyzes, develops and monitors CAF personnel policies and programs related to human rights issues and EE, ensuring that that CAF policies and programs are implemented in accordance with the EEA to achieve CAF representation goals and to provide a better work environment for all members.Footnote 42
Director Workforce Programmes and Services, ADM(HR-Civ)
DWPS is responsible for developing and implementing workforce strategies and programs in support of a diverse and inclusive workplace that nurtures talent and builds community. This includes supporting L1s in implementing and integrating diversity and inclusion into various programs as well as providing direction, advice and guidance to senior management in support of EE, diversity and inclusion and official languages practices. DWPS will continuously improve HR process to address gaps and strengthen a diverse and inclusive environment in DND. In addition, DWPS develops learning and awareness products for senior management to embrace their role. DWPS is also responsible for other areas such as community management, performance management and talent management.Footnote 43
Director Gender, Diversity and Inclusion, Corporate Secretary
DGDI supports the advancement of the DND diversity and inclusion agenda by providing corporate leadership in implementing GBA+ through the provision of expert advice across L1s and the development of reporting frameworks and strategic plans for integrating the GBA+ into DND planning, policies and priorities. DGDI addresses ideas, questions, challenges or areas of resistance raised by Gender Focal Points through new tools and training as well as providing timely and responsive advice to Gender Focal Points on challenging GBA+ cases. As the Centre of Expertise for GBA+, DGDI is responsible for designing and implementing GBA+ information repositories.Footnote 44
Director Integration of Gender Perspectives, Strategic Joint Staff
DIGP supports CAF’s strategic and operations goals by providing corporate leadership in developing, implementing, monitoring, evaluating and reporting frameworks and strategic plans for integrating UNSCR 1325, NATO policy, CNAP, and integrating GBA+ into CAF planning, operations, doctrine training, professional military education and institutions. This includes defining short and long-term approaches to GBA+ implementation into policies, systems and processes which support delivery of CAF operational effects. DIGP develops measures of effectiveness and accountability practices to report on the effectiveness of the frameworks and campaign plans for fully integrating WPS initiatives.Footnote 45
Defence Champions
Diversity and Inclusion Defence Champions
The Diversity and Inclusion Champions consist of a Leadership Team of Defence Champions for Gender and Diversity for Operations, which is supported by an Executive Team consisting of Defence Champions for WPS and for GBA+.
The expectations for the Diversity and Inclusion Defence Champions is that they will move beyond the traditional Champion role (of being a voice of support) and instead, serve as active participants and leaders in effecting change as well as complement the roles of the Defence EE Champions. The Diversity and Inclusion Defence Champions were appointed for operations, and will provide leadership, advice and support to institutionalize diversity as an operational capability. They will advance cultural and institutional change with activities in both military and civilian operations, and provide a focal point that coordinates activity, advances key initiatives and supports the governance of the Champions for Gender and Diversity for Operations, WPS and GBA+ portfolios.Footnote 46
Defence Employment Equity Champions
The Defence EE Champions are senior-level managers and officers appointed by the Deputy Minister (DM) and the Chief of the Defence Staff (CDS) and are part of the EE governance framework. The Defence EE Champions ensure that EE is integrated into all decision making and business planning by advocating and promoting EE programs and best practices as well as fostering a representative, equitable and inclusive workforce and workplace through supporting their own DAG and enhancing visibility. The Defence EE Champions also provide advice and feedback to senior committees.Footnote 47
Defence Advisory Groups
The mandate of the DAGs is to provide advice and insight to leadership towards EE implementation, and report systemic employment issues and barriers to the functional stakeholders. When appropriate, the DAGs will seek their respective Defence EE Champion’s support in resolving the systemic issues and barriers. Each DAG (local, regional and national) is co-chaired by volunteer and elected military and civilian members and was created (endorsed by the DM) to be representative of both CAF members and DND employees as CAF members are not represented by unions.Footnote 48
Employment Equity Officers
EE Officers operate under the diversity and inclusion stakeholders (DHRD, DWPS) as the point of contact for EE related issues by providing insight and information with regards to diversity and EE as it applies to civilian staffing shortages, and distribute information regularly on diversity, EE initiatives, best practices and training opportunities. EE Officers are also a point of contact for the DAGs, and attend DAG meetings to monitor ongoing issues relating to regulations, policy changes, trends and areas of concern.Footnote 49
Gender Advisor
Gender Advisor is a military or civilian officer serving the Commander of a specific mission, whose primary responsibility is the overall integration of gender perspectives into mission planning, execution and evaluation. The Gender Advisor monitors, coordinates with, and supports Gender Field Advisors and Gender Focal Points.Footnote 50
Defence Team Joint Responsibility Centre
The Defence Team Joint Responsibility Centre (JRC) is a virtual community of practice (often referred to as a Centre of Expertise) for GBA+, which consists of four pillars (i.e., DHRD, DWPS, DGDI, and DIGP) and with the overall support of the Diversity and Inclusion Defence Champions. The JRC also has a network of Focal Points within each of the DND and CAF L1 organizations, including the four CAF Gender Advisors currently supporting SJS, CANSOFCOM, CJOC and CMP. The JRC provides expert support to all civilian and military L1 organizations, Functional Authorities and Defence Champions on the implementation of GBA+ and the WPS for all DND and CAF lines of business. The primary line of efforts is the institution, while also supporting operations. The JRC thus strengthens the application of GBA+ analysis in policy, programs, initiatives and corporate documents, and provides a forum to share best practices and strategies.Footnote 51 ,Footnote 52 ,Footnote 53
Gender Focal Point (or GBA+ Focal Point)
Focal Points “demonstrate individual GBA+ competency and build organizational capacity” and are responsible for the following: Articulating the GC requirements to integrate GBA+; Demonstrating value-added of GBA+; Challenging GBA+; and Supporting quality assurance of GBA+ analyses.Footnote 54 These individuals provide first-line advice to those within their respective L1 organizations who are conducting GBA+ analyses. Gender Focal Points are there to provide guidance, support and to ensure that GBA+ has been conducted appropriately.Footnote 55
Annex E—Evaluation Matrix
Evaluation Issues and Questions |
Indicators |
Research Methods |
|
---|---|---|---|
Document & Literature |
Interviews |
||
Relevance |
|||
|
1.1. Evidence that diversity and inclusion is aligned with the federal government roles, responsibilities and commitments |
X |
|
1.2. Evidence of consistency between diversity and inclusion and the federal government policies, priorities and commitments |
X |
||
1.3. Evidence of consistency between diversity and inclusion and the departmental policies, priorities and expected results | X |
||
Performance – Achievement of Expected Outcomes (Effectiveness) |
|||
|
2.1. Evidence of the current state of diversity and inclusion within DND and the CAF (achievement of outcomes) | X |
|
|
3.1. Evidence that the roles and responsibilities of stakeholders for diversity and inclusion are clearly defined and communicated | X |
X |
3.2. Evidence that key stakeholders have the capacity to carry out their roles and responsibilities for diversity and inclusion | X |
||
3.3. Evidence that the current structure is appropriate in overseeing diversity and inclusion (e.g., is there a need for a more aligned and/or centralized approach?) | X |
||
3.4. Evidence on challenges and/or limitations related to the alignment of the current structure and/or centralization approach for diversity and inclusion | X |
||
3.5. Evidence that appropriate mechanisms are in place to identify, address and share systemic issues and barriers that are related to diversity and inclusion | X |
X |
|
Effectiveness – Design and Delivery (Performance Measurement) |
|||
|
4.1. Evidence on the data that is currently being collected to measure the success of diversity and inclusion | X |
X |
4.2. Evidence of lessons learned and best practices in measuring the state and success of diversity and inclusion | X |
X |
Table E-1. Evaluation Matrix. This table outlines the evaluation issues and questions for this report along with the research methods used.
Table E-1 Details - Evaluation Matrix.
Annex F—Evaluation Methodology and Limitations
1.0 Methodology
The Evaluation of Diversity and Inclusion considered multiple lines of evidence to assess relevance and effectiveness. The scope and focus of this evaluation was to assess the success of the diversity and inclusion initiatives and its alignment between DND and the CAF. As such, the evaluation focused on the organizational structures and governance as key factors that enable the achievement of the diversity and inclusion outcomes, and whether the DTMP PIP integrates performance measures and data to inform the diversity and inclusion outcomes.
This methodology established a consistent approach to data collection and analyses, including the development of research instruments. This enabled data triangulation and contextualization to ensure evaluation findings and recommendations were accurate, credible, and strengthened validity.
1.1 Overview of Research Methods
The research methods used were selected based on an assessment of both the data that was available and required to address the evaluation issues, questions and indicators, as outlined in the evaluation matrix. The following two research methods were used to gather complimentary qualitative and quantitative data, discussed in greater detail in the following sections:
- Document and literature review
- Key informant interviews
1.2 Details on Data Collection Methods
1.2.1 Document and literature review
Document review component
A preliminary document review was conducted as part of the planning phase of the evaluation, to gather foundational understanding and knowledge of diversity and inclusion across DND and the CAF. This included the roles and responsibilities of the functional authorities and stakeholders, and other key collaborating organizations and stakeholders. The preliminary document review provided valuable insight in developing the logic model and the evaluation matrix.
A comprehensive document review was conducted as part of the conduct phase of the evaluation, focused on the relevance and, to a lesser extent, effectiveness.
- Relevance: extensive evidence was gathered towards answering the evaluation question and indicators to assess the degree to which diversity and inclusion align with and contribute to federal government and departmental priorities.
- Effectiveness: some complimentary evidence was also gathered towards answering the evaluation questions and indicators related to the governance and structures, including on performance measurement. Following is a non-exhaustive list of documents that were reviewed during the scoping and conduct phase of the evaluation:
- Legal and policy documents: EEA, Canadian Human Rights Act, Canadian Multiculturalism Act, Official Languages Act, Canadian Charter of Rights and Freedom, SSE, Mandate Letters of Cabinet Ministers, Budgets, Speeches from the Throne, etc.
- Corporate documents: Departmental Plan, DRF, Orders, directives and regulations related to DND and CAF members, Employment Equity Plan and Annual Reports, Terms of References and/or the Mandate Letters (DAGs, Champions, EE Officers, Gender Advisors and Focal Points, etc.), DM and CDS Directives, CANFORGENS, etc.
- Other documents: Status of Women Action Plan on GBA+, UNSCR 1325 and the CNAP, Prime Minister’s apologies to the LGBTQ2 Canadians, etc.
Literature review component
Comprehensive literature reviews were conducted as part of the conduct phase of the evaluation, mainly to inform the performance (effectiveness) of diversity and inclusion. Following is a non-exhaustive list of the internal and external literature that was reviewed as part of this exercise.
- A primary literature review was conducted to gather extensive evidence towards exploring the current state of the diversity and inclusion within DND and the CAF.
- Internal research from DND and the CAF: Over 40 research papers and articles (i.e., scientific letters) were reviewed to gather the views of the Defence Team on the EE, and diversity and inclusion (e.g., challenges and barriers, attitudes, organizational culture and climate, etc.). Other research papers and articles were reviewed where feasible and appropriate (e.g., CFC, RMC).
- Research on the public service: Public Service Employee Survey, Joint Union and Management Task Force on Diversity and Inclusion, Twenty-Fifth Annual Report to the Prime Minister on the Public Service of Canada, etc.
- Other scientific research from the academic sector to gather evidence towards establishing a correlation link between diversity and inclusion, and a high performing and operationally effective organization.
- This brief literature review of scientific articles built on the findings discussed in the 2018 Final Report of the Joint Union/Management Task Force on Diversity and Inclusion in section 6, which focused on diversity. Relevant peer-reviewed articles published in the last eight years were sourced from available databases, using the keywords diversity, inclusion, organizational outcomes, performance, and effectiveness.
- A secondary literature review was conducted to gather current best practices and lessons learned related to measuring the performance of diversity and inclusion. As such, the specific intent of this complimentary review was to assist the Functional Authorities in planning for the development of measurement frameworks to enable the collection of data which would provide insights on the progress and results associated with the diversity and inclusion activities and initiatives.
- Due to the relative newness of diversity and inclusion as an organizational focus, an exploratory approach was taken in sourcing information. Keywords were initially sourced from the interviews, and external organizational documents with a private, public and social focus. A matrix of article summaries was completed, and similar themes were grouped and then further researched in order to gain an in-depth review of the topic. A literature review of scientific research was then sourced through available databases using the keywords diversity, inclusion, performance measurement, frameworks, metrics, best practices, dimensions, scorecard and benchmarking.
1.2.2 Key informant interviews
The key informant interviews were designed to gather in-depth information – including opinions, explanations, examples and factual information – with respect to the performance (effectiveness) evaluation issue and its related questions and indicators. This research method was an important source of qualitative information, and assisted the evaluation in the understanding and interpretation of other qualitative and quantitative data stemming from the document and literature reviews.
During the planning phase of the evaluation, a total of 13 scoping interviews were conducted with 24 key stakeholders in order to inform the scope and focus of the evaluation. These scoping interviews and discussions were used to conduct an evaluability assessment of available performance measures and data and have informed the research methods to be used for conducting the evaluation. In sum, these scoping interviews provided the evaluation with a foundational understanding of current issues, needs and priorities, which was used to complement the development of the logic model for the diversity and inclusion approach, evaluation matrix, and the comprehensive key informant interview guides.
During the conduct phase, a total of 17 semi-structured qualitative interviews were conducted with 19 key stakeholders. The first few interviews were also used as pre-tests to inform the guides regarding flow, terminology, clarity and timing. The interviews were mostly conducted in-person and one-on-one, averaging 60 minutes in length. Some follow-up interviews were conducted over conference call.Footnote 56
The scoping interviews and key informant interviews were conducted with key stakeholders at the Director General, Director, Manager and Staff levels within the following organizations:
- CMP
- ADM(HR–Civ)
- Corp Sec
- SJS
- Other (subject matter experts): Select sample of Diversity and Inclusion Defence Champions, Defence EE Champions, National Co-Chairs of the DAGs and EE Officers.
Process
To support this line of inquiry, the evaluation developed, in collaboration with the functional stakeholders, a sample of approximately 25 key informants in order to allow for non-participation and or refusals.Footnote 57 Moreover, a bilingual invitation email with detailed information on the purpose and objectives of the evaluation, including appropriate confidentiality and privacy clauses, were used to recruit interview respondents. Before the beginning of each interview, the confidentiality and privacy clauses, as well as the interview guide, were further explained to the interviewees to foster a climate that encouraged respondents to be open and candid.
Detailed notes were taken during the interviews and then transcribed electronically. The summarized notes were entered into an evidence matrix, and tailored to each of the evaluation issues, questions and indicators for all respondents. The interview notes were shared with the interviewees and compared with a view to reaching a common record, if requested by the respondents. When necessary, interviewees provided detailed written responses to the interview questions before or after the interview was completed. Once all interview summary notes were entered into the evidence matrix, a content analysis was performed on the qualitative data within and across groups of interviewees. These findings were then triangulated with the evidence from other research methods, and used to develop specific sections of the evaluation report and construct empirical key findings.
2.0 Limitations
Table F-1 lists and describes the limitations and the mitigation strategies that were employed for the Evaluation of Diversity and Inclusion.
Limitations |
Mitigation Strategies |
---|---|
Time constraints
|
In consultation with the functional stakeholders, the decision was made to focus the evaluation solely on the governance and structures for diversity and inclusion, as both DND and the CAF diversity and inclusion strategies and frameworks could not be part of the review given the new direction to develop a joint Defence Team Strategy. This permitted the evaluation to focus its efforts and reduce the number of evaluation issues, questions and indicators, including the amount of research methods used to gather data. |
Methodological
|
The evaluation resorted to mostly qualitative research methods to gather evidence towards the evaluation issues, questions and indicators. However, efforts were made to maximize the internal validity of this study by using research methods and sources of information that enabled triangulation and contextualization of both quantitative and qualitative evidence-based findings (e.g., document review, literature review, key informant interviews). A large number of key informant interviews were also conducted with diverse stakeholders and at different organizational levels. |
Attribution to Outcomes
|
Measuring the attribution to outcomes and net impacts is not the goal of this evaluation. Specifically, the evaluation focuses on assessing the governance and structures of the diversity and inclusion approach, as those are key factors to enable the achievement of outcomes. While the evaluation documented general contributions to outcomes, it did not directly assess the outcomes and impacts of the diversity and inclusion approach on the Defence Team and the organizational culture. |
Table F-1. Evaluation Limitations and Mitigation Strategies. This table displays the limitations of the evaluation and the corresponding mitigation strategy.
Table F-1 Details - Evaluation Limitations and Mitigation Strategies.
Annex G—Review of Best Practices in Measuring Diversity and Inclusion
This Annex is intended to be a performance measurement guideline for diversity and inclusion, to be integrated into the DTMP PIP. Due to the relative newness of diversity and inclusion as an organizational focus, an exploratory approach was taken in sourcing information. External organizational documents had a private, public and social focus, and research articles were sourced through available databases.Footnote 58 The review is divided into best practices commonly found in successful diversity and inclusion performance measurement frameworks, and lessons learned: planning for measurement; methods and metrics; monitoring; and measuring change post-implementation.
1.0 Planning for Measurement
What are our gaps and goals? What is our strategy? Available organizational documentations agree that planning for measurement is an essential element of successful diversity and inclusion frameworks. Involve management and staff in the initiative from the start by creating a diversity and inclusion committee or advisory. Once roles and responsibilities are established, the committee can move towards identifying current gaps in diversity and inclusion. Next, develop a strategy to tackle the identified gaps, and narrow the focus to key areas or elements. During this process, ensure that these key areas or elements of focus are also aligned with organizational objectives and culture. Lastly, identify tangible and measurement actions and goals that will ultimately bridge the identified gaps of diversity and inclusion.Footnote 59
2.0 Methods and Metrics
2.1 Determining Best Practices
What does a successfully diverse and inclusive organization look like? The first step in measurement methodology is to determine what the starting line or benchmark is. Although not essential, benchmarking is an effective and competitive way to approach change, and is a common organizational practice in performance and measurement frameworks. A good benchmarking model should be clear and logical. Keep in mind that benchmarking is context and time dependent.Footnote 60
There are two main methods of benchmarking. The first, external benchmarking, involves looking at what other organizations in your field are doing well in terms of diversity and inclusion, and using that as a goal to work towards. Tracing causal relations to identify these best practices can be achieved through a mixed methods approach: multivariate analysis can help locate potential best practice organizations, and a subsequent case study can analyze the content of the supposed best practice. Note external benchmarking is prone to error and, once again, will not take into account the varying contexts of organizations.Footnote 61
The second method is internal benchmarking, often used when there is no other pre-established benchmark. Internal benchmarking is a reduced version of performance measurement, with many of the same steps. It involves identifying and consulting stakeholders to form key areas of measurement and focus, measurement activities and gap analysis, followed by an action plan for implementation and continuous improvement.Footnote 62
2.2 Types of Measurement
What do we measure and how? The literature agrees that a mixed methods approach with internal and external measures may provide the most valid depiction of diversity and inclusion. Following are examples of measures taken directly from the Canadian Institute of Diversity and Inclusion toolkit.Footnote 63
Internal Measures | External Measures |
---|---|
|
|
Table G-1. Performance Measurement Measures. The table displays examples of internal and external performance measures that an organization could use to gather information and data on diversity and inclusion.
Table G-1 Details - Performance Measurement Measures.
2.3. Dimensions of Measurement
How can we measure diversity and inclusion? Whereas diversity is usually measured via quantitative measures, inclusivity is usually approached with surveys and questionnaires. There exists several validated measures; however, the sensitivity to context makes generalizability a challenge.Footnote 64 Golden and colleagues (2010; 2011) identified the following dimensions for measuring diversity and inclusion: diversity sensitivity, integrity with difference, interacting with difference, valuing difference, team inclusion, managing conflict over difference, embedding inclusion, group membership, group affection, room for authenticity, and value in authenticity.Footnote 65 In medicine, there exists the widely used “Diversity Engagement Survey” that assesses the dimensions of common purpose, trust, appreciation of individual attributes, sense of belonging, access to opportunity, equitable reward and recognition, cultural competence, and respect.Footnote 66 These dimensions should be taken as a guide in formulating organization specific and relevant metrics.
3.0 Monitoring
3.1 Performance Frameworks – Scorecards
How can we quickly see and show our progress? Tracking progress in performance measurement helps focus efforts and create a shared understanding at all organizational levels and the public. Different frameworks exist: the balanced scorecard, the sustainable balanced scorecard, the dynamic multi-dimensional performance framework, and the success dimensions. Scorecards began in the world of business and are now an accepted best practice to track diversity and inclusion efforts, especially when it comes to “soft factors”. They are a visual depiction or snapshot of an organization’s status in relation to specific goals, usually in the form of an easy-to-read chart. Organizational strategies are boiled down into dimensions, and aligned indicators are developed. The dimensions and indicators are also aligned with the overall organizational strategy, and there is usually a temporal component or different timeframes considered.Footnote 67
3.2 Organizational Performance Measurement
How can we compare our progress to other organizations? Scorecards are tailored to each organization. An additional method to keep track of progress is the use of organizational performance indicators, which will:
- help determine the relevance of performance to key stakeholders;
- take into account conflicting strategies and practices;
- reaffirm knowledge of timeframes between activities and performance;
- provide measures that are sufficiently specific and robust; and
- encourage deeper understanding of the relationship between measures.
Indicators are then developed via triangulation methods and longitudinal analysis.Footnote 68
4.0 Measuring Change Post-Implementation
Measurement following the implementation of a new change, such as diversity and inclusion, is often based on the data and results of initial benchmarking, internal and external measurements, scorecards and organizational performance measurement. These frameworks and tools provide an in-depth view of how the change has progressed at different points in time.
4.1 The New World Kirkpatrick Model
How do we measure learning? The New World Kirkpatrick Model is a post program implementation framework widely used in evaluation for assessing learning. According to the Kirkpatrick Partners website, there are four levels of assessment. Level 1 measures “reaction” - the degree to which participants found the training favorable, engaging and relevant to their jobs. Level 2 measures “learning” - the degree to which participants acquire the intended knowledge, skills, attitude, confidence and commitment based on their participation in the training. Level 3 measures “behavior” - the degree to which participants apply what they learned during training when they are back on the job. And level 4 measures “results” - the degree to which targeted outcomes occur as a result of the training and the support and accountability package.Footnote 69
5.0 Lessons Learned
This annex is intended as an overview of best practices in diversity and inclusion performance measurement. Table G-2 outlines lessons learned, some unique to diversity and inclusion, pulled from various organizational and research sources to keep in mind while moving through the performance measurement process.Footnote 70
Lessons learned | |
---|---|
|
|
Table G-2. Lessons Learned in Performance Measurement. This table displays some unique diversity and inclusion lessons learned with regards to performance measurement.
Table G-2 Details - Lessons Learned in Performance Measurement.
References
- Anand, G., & Kodali, R. (2008). Benmarking the benchmarking models, Benchmarking: An International Journal, 15(3), 257-291.
- Anderson, R., & Billings-Harris, L. (2010). Trailblazers: How top business leaders are accelerating results through inclusion and diversity. John Wiley & Sons, N.J.: Hoboken.
- April, K. & Blass, E. (2010). Measuring diversity practice and developing inclusion, Dimensions, 1(1), 59-66.
- Bentley University (2017). Assessing diversity metrics and women’s advancement. (pdf format) https://www.bentley.edu/files/2017/03/17/Bentley%20CWB%20Diversity%20Metrics%20Research%20Report%20Winter%202017.pdf
- Canadian Institute of Diversity and Inclusion (2014). Locking in your leadership: toolkit for developing a diversity and inclusion strategy. (pdf format) https://ccdi.ca/media/1075/20140910-ccdi-report-di-strategy-toolkit.pdf
- Church, A., & Rotolo, C. (2013). Leading diversity and inclusion efforts in organizations: should we be standing behind our data or our values (or both)?, Industrial and Organizational Psychology, 6(3), 245-248.
- ConAgraFoods. (2009). Diversity best practices: diversity primer, (Pdf format: Chapter 15 Research and Measurement, Greg Morris, 225-237).
- Derven, M. (2014). Diversity and inclusion by design: best practices from six global companies, Industrial and Commercial Training, 46(2), 84-91.
- De Vries, M. (2010). Performance measurement and the search for best practices, International Review of Administrative Sciences, 76(2), 313-330.
- Federal Reserve Board (2018). Diversity and Inclusion Strategic Plan 2016-19. (website) https://www.federalreserve.gov/publications/2016-diversity-inclusion-plan.htm
- Figge, F., Hahn, T., Schaltegger, S., Wagner, M. (2002). The sustainability balanced scorecard - linking sustainability management to business strategy, Business Strategy and the Environment, 11(5), 269-284.
- Jansen, W., Otten, S., Van Der Zee, K., & Jans, L. (2014). Inclusion: conceptualization and measurement, European Journal of Social Psychology, 44, 370-385.
- Jones, S. (2017). Examining the Business Case for Diversity and Inclusion, San Diego Business Journal, 38(35), 1-6.
- Journeault, M. (2016). The integrated scorecard in support of corporate sustainability strategies, Journal of Environmental Management, 182, 214-229.
- Kaplan, R., Norton, D. (1992). The balanced scorecard measures that drive performance, Harvard Business Review, 70(1), 71-79.
- Kaplan, R., Norton, D. (1993). Putting the balanced scorecard to work, Harvard Business Review, 71(5), 134-147.
- Kaplan, R., Norton, D. (1996). Linking the balanced scorecard to strategy, California Management Review, 39(1), 53-79.
- Kaplan, R., Norton, D. (2000). Having trouble with your strategy? then map it, Harvard Business Review, 78(5), 167-176.
- Kirkpatrick Partners (2018). The new world Kirkpatrick model. (website) https://www.kirkpatrickpartners.com/Our-Philosophy/The-New-World-Kirkpatrick-Model
- Kreitz, P. (2008). Best practices for managing organizational diversity, The Journal of Academic Librarianship, 34(2), 101-120.
- Lafever-Ayer, L. (2013). Making diversity business critical, Strategic HR Review, 12(3), 145-150.
- Lim, N., Haddad, A., & Daugherty, L. (2014). Implementation of the DoD Diversity and Inclusion Strategic Plan: A Framework for Change through Accountability (Chapter 2 (p9-16) and Chapter 6 (p55-58)). RAND Corporation.
- Maltz, A., Shenhar, A., & Reilly, R. (2003). Beyond the balanced scorecard: refining the search for organizational success measures, Long Range Planning, (36), 187-204.
- Person, S., Jordan, G., Allison, J., Fink Ogawa, L., Castillo-Page, L., Conrad, S., Nivet, M., & Plummer, D. (2015). Measuring diversity and inclusion in academic medicine: the diversity engagement survey, Academic Medicine, 90(12), 1675-1683.
- Richard, P., Devinney, T., Yip, G., & Johnson, G. (2009). Measuring organizational performance: towards methodological best practice, Journal of Management, 35(3), 718-804.
- Rush University Medical Center (n.d.). Diversity & Inclusion. (pdf format) https://www.aamc.org/download/451104/data/rushuniversitymedicalcenter.pdf
- Shenhar, J. & Dvir, D. (1996). Long term success dimensions in technology-based organizations (Chapter 32). Handbook of Technology Management. McGraw Hill, New York.
- The City of Edmonton (2005). Diversity & inclusion framework & implementation plan. (pdf format) https://www.edmonton.ca/city_government/documents/Final_Diversity__Inclusion_Framework__Implementation_Plan.pdf
- Thompson, S. (2017) Defining and measuring ‘inclusion’ within an organization, Institute of Development Studies, 1-15.
- Turnbull, H., Greenwood, R., Tworoger, L. & Golden C. (2010). Skill deficiencies in diversity and inclusion in organizations: developing an inclusion skills measurement, Academic Strategic Management Journal, 9(1), 1-14.
- Turnbull, H., Greenwood, R., Tworoger, L. & Golden C. (2011). The inclusion skills measurement profile: validating and assessment for identification of skill deficiencies in diversity and inclusion, Journal of Organizational Culture, 15(1), 11-24.
Annex H—Mechanisms to identify systemic issues and barriers in DND and the CAF
In November 2018, functional stakeholders and Diversity and Inclusion Defence Champions convened at the DAG Co-Chair Council Meeting to identify and discuss the DAG-related areas of improvement. The issue of how EE and the DAG fit within diversity and inclusion were raised and discussed. A revised DAG Terms of reference is being drafted to refine roles and responsibilities.
Mechanisms that can be used to identify, report, share or address systemic issues and barriers are important. They contribute to the effectiveness of the diversity and inclusion approach and support functional stakeholders in developing, reviewing and implementing informed HR‑related initiatives to overcome diversity and inclusion employment barriers and challenges. As such, these mechanisms and tools are contributing to the overall achievement of diversity and inclusion outcomes.
Table H-1 identifies and describes the DND and CAF mechanisms that can be used and leveraged to either identify, report, share or address systemic issues and barriers that are related to diversity and inclusion. Identified issues which are preventing some mechanisms to be fully functional are also discussed in greater detail.
Functional Stakeholders | Mechanism: Develop and review HR-related policies and procedures to ensure they are free of systemic employment issues and barriers. |
---|---|
Issues: Capacity limitations have impeded their ability to fulfill their roles and responsibilities for diversity and inclusion, including providing adequate support to the DAGs. |
|
Defence Advisory Groups |
Mechanism: DAG members bring issues and barriers to the attention of the local and regional DAG Co-Chairs, who then report to the DAG National Co-Chairs to determine if they are systemic. Systemic issues and barriers are then brought to the attention of the relevant diversity and inclusion stakeholders to address. |
Issues: There is a lack of integration and collaboration between the DAG Co-Chairs and functional stakeholders. Moreover, while a feedback and reporting structure exists, it has not been properly implemented and utilized as intended. Capacity in terms of funding, time and support was also found to be lacking among the DAG Co-Chairs, which has further impeded their ability to fulfill their roles, responsibilities and mandates. |
|
Gender-based Analysis Plus |
Mechanism: The implementation of GBA+ through L1 Action Plans holistically identifies and work towards addressing systemic issues and barriers for all DND and CAF lines of business, including HR. |
Issues: Capacity limitations within the DIGP (SJS) have impacted its ability to further the advancement of the implementation of GBA+ for CAF planning and operations. |
|
Defence Champions (Employment Equity, Diversity and Inclusion) |
Mechanism: Defence Champions provide leadership support and bring necessary attention to ensure systemic issues and barriers are adequately identified, reported, shared and addressed. |
Issues: Lack of integration and collaboration of the Defence EE Champions with the championship structure of the Diversity and Inclusion Defence Champions. |
|
DND and CAF Research |
Mechanism: Internal groups such as DRDC and DGMPRA conduct research related to diversity and inclusion, and support the needs of the diversity and inclusion stakeholders. |
Issues: Capacity limitations among functional stakeholders is impeding their ability to conduct more responsive research and analysis and direct the research areas of DRDC and DGMPRA to support the overall diversity and inclusion approach. |
|
Sexual Misconduct and Response Centre (SMRC) |
Mechanism: The SMRC was established as a resource for victim and leadership support outside of the Forces and staffed by experts. |
Issues: Design and implementation issues that affect the effectiveness of the SMRC were identified through a recent report conducted by the OAG. However, commitments by the CAF leadership were made to address these recommendations. |
|
Integrated Conflict and Complaint Management (ICCM) |
Mechanism: The ICCM integrates the previously separate existing harassment, grievance, human rights and Alternative Dispute Resolution (ADR) conflict resolution systems and mechanisms. |
Issues: Duplication in the ICCM and CMP roles and responsibilities exist and require further clarification. |
|
Chain of Command or Management |
Mechanism: The chain of command or management address issues at the lowest level possible, and report systemic issues to the functional stakeholders. |
Table H-1. Mechanisms for Identifying and Addressing Systemic Issues and Barriers. This table displays the mechanisms that can identify, report, share or address systemic employment issues and barriers related to diversity and inclusion.
Table H-1 Details - Mechanisms for Identifying and Addressing Systemic Issues and Barriers.
Page details
- Date modified: