Language selection

Government of Canada / Gouvernement du Canada

Search

ARCHIVED – Enhanced Language Training Initiative: Formative Evaluation

We have archived this page and will not be updating it.

You can use it for research or reference.

We have archived this page and will not be updating it.

You can use it for research or reference.

Interview Guide

Employers

This interview is being undertaken as part of the evaluation of the Enhanced Language Training (ELT) Initiative. Goss Gilroy Inc. has been hired to carry out the evaluation over the period of January to April 2007. Several methodologies are being used for the evaluation including a survey of service providers, focus groups with clients, case studies of funded projects, and interviews with Citizenship and Immigration Canada (CIC), provincial representatives, service providers, employers and other stakeholder representatives. It is as part of the case studies that you are being asked to participate. Please note that anything you say will be kept confidential. Your comments will not be associated to you but rather reported in aggregate. This interview will take about 30 minutes to complete.

Introduction

1. Please describe your company in terms of its main services/products, and main types of occupations.

2. Please describe your familiarity with the <project name> being developed/delivered by the <name of SPO>. (Prompt if necessary.)

Questions Specific to <Project Name>

We would like to ask you a series of question about the <project name> project specifically.

3. How did you first get involved with the project?

4. What was your role in the project?

  1. Was this role appropriate?
  2. Did you have any responsibilities for this project (e.g., placement targets, reporting requirements)?
  3. How could your involvement have been improved?

5. Does/did your participation in this project include a job placement or internship component?

  1. In your opinion, how useful is/was this for the immigrants? (E.g., Canadian work experience, skill development, orientation to Canadian work environment.)
  2. How useful is/was this for your company? (E.g., access to qualified employees.)
  3. Does your company have a policy on diversity? If so, how does your participation in the ELT program fit into this policy?

6. What other impacts of the project can you identify:

  1. For the immigrants who participated?
  2. For your company?

7. Would your company participate in a similar project again? Why or why not?

Questions Related to ELT Overall

The next set of questions pertain to ELT overall.

8. To what extent is labor market integration of immigrants a challenge? Why do you believe this?

  1. Is it more of a challenge in relation to particular occupations, regions/cities?

9. Are there other workforce integration programs available to immigrants?

  1. Can you briefly describe them?
  2. Who delivers that program/those programs?

10. Do you have any other comments you would like to make at this time?

Thank you very much for your time!

Evaluation of ELT
Draft Focus Group Moderator’s Guide

Program Clients

Hello, my name is ______________________ and I work for Goss Gilroy Inc., a management consultant firm located in Ottawa. We have been hired by Citizenship and Immigration Canada (or CIC) to carry out the evaluation of the Enhanced Language Training Program (or ELT). You have been invited here today as a client of ELT to take part in a discussion called a focus group.

We would like to you to focus your comments on the <project name> project, delivered by the <SPO name>. Are all of you familiar with this program?

With me is my colleague _________________ from Goss Gilroy. S/he will be taking notes so we can remember what you said. [If applicable: There is also someone from CIC’s evaluation unit here today, ___________________. They are NOT from the group that manages ELT, but rather from the part of CIC that is responsible for making sure the government’s settlement programs are working the way they should.]

Please note that anything you say will be kept anonymous. Your name will not be provided to CIC and your comments will not be associated with you but rather reported overall (for example, some said… a few felt…).

A focus group is essentially a loosely structured conversation. I will ask a number of questions you are all invited to make a comment either in response to my question or in response to something someone else says. There are no right or wrong answers – it is likely that each of you has had a different experience or has a different opinion – we want to hear from all of you.

This discussion is expected to last about two hours. We’ll take a break about halfway through. Are there any questions before we begin?

Introduction (15 minutes)

1. Let’s go around the table so we can introduce ourselves. Please tell me your first name, when you arrived in Canada and your home country.

2. Which of you are still enrolled in <project name>?

  1. For those of you who are no longer enrolled, what are you doing now? (Prompt: work placement/internship component, working, unemployed, student, other.)

ELT Program Delivery (60 minutes)

For the next part of our discussion, I’d like to talk about the <project name> and the training/work support services you have received from <SPO name>.

3. How did you first hear about the <project name> project?

  1. What were you told about the project?

4. Were you able to enroll in the language training component right away? Was there a waiting list?

5. Can you please describe the <project name> to me?

  1. How often did you attend class?
  2. Where were classes held?
  3. Who taught the class? Same person every time?
  4. Did you have speakers or field trips? Did employers come to the class?
  5. How long did it last?
  6. Was there a test at the end?

6. How satisfied were you with your teacher? Did they appear to know about your profession (if applicable) and the material in the course?

7. Were there things that made it difficult for you to participate in the class? Why? [Prompt: required transportation, child minding, not relevant to your profession, lacked language ability.]

8. Were services such as transportation assistance (e.g., bus tickets) or child minding offered to you by the organization?

  1. Did these services help you attend the class?

9. [If applicable] I understand that this project included a “bridge-to-work” component that included: working on-site at an employer, an employer talking to your class, to you personally, visit to a work site, information on how to look for job, other. [Note to moderator: discuss bridge-to-work components as appropriate for the project and adapt the language to correspond with the terminology used by the SPO and the project.] Please describe it.

  1. Do you recall participating in any or all of these activities?
  2. Were any of these activities more useful or helpful than others? Which ones and why?
  3. If you actually spent time working on site at an employer, where did you work? Are you still working there? If not, why?
  4. If you received information about how to look for work, how useful was it? Was it tailored for your profession?

10. Were there things that made it difficult for you to participate in the work component? Why? [Prompt: required transportation, child minding, not relevant to your profession, lacked language ability.]

11. Were services such as transportation assistance (e.g., bus tickets) or child minding offered to you by the organization?

  1. Did these services help you participate in the work component?

ELT Success (45 minutes)

During the last part of the discussion, I’d like us to talk about what kind of impact or different the <project name> project has made for you.

12. Did the project meet your expectations? Why or why not? Please explain.

13. To what extent would you say the <project name> project helped you to: [Note to moderator: Probe for how they achieved these outcomes/ provide examples.]

  1. Improve your language skills?
  2. Get more Canadian work experience?
  3. Learn more about what it’s like to work in Canada?
  4. Learn how to look for work?
  5. Apply for jobs in your profession?
  6. Find people or organizations that can help you further (e.g., mentors, networks, contacts)? How have these people/organizations helped you?

14. Since participating in this project, have you:

  1. Increased your self-confidence to compete in the Canadian labor market?
  2. Found more jobs you can apply for? That is, where you meet the minimum requirements?
  3. Gone on more job interviews?
  4. Found a job in your profession that uses your skills?
  5. Found a job not in your profession but that uses your skills?
  6. Found a job in your profession that does not use your skills?
  7. Found a job not in your profession and does not use your skills?

15. Do you think you would have achieved these outcomes (in Q12 and Q13) if you had not participated in <project name>?

16. Do you have any other comments you would like to make?

Those are all the questions I have for you.
Thank you for your time and cooperation.

Site Visit Template – Information on Specific Delivery and/or Development Projects

To be filled out by the team members during the site visits and to be shared with the experts reviewing the project tools/materials to provide the context for their review. Data may be collected from:

  • Review of SPO documents
  • Interviews with SPO management
  • Interviews with teachers
  • Focus groups with participants
  • Observation of facilities and/or classes
Issue Data [note the source of the information]
Courses

Nature of courses (e.g., objectives, target group, how course promoted, length of course, hours in-class, number of contact hours, in-class methods (e.g., discussions, group work, lecture), allocation of time to listening/reading/writing/speaking)

 

Delivery (e.g., number in class, number of instructors, intake procedures (e.g., continuous intake, how do they deal with arrival/departures in mid-course, how they deal with dropouts and changing class sizes), use of external (including workplace-related) resources, balance between language training, workplace orientation and work placement (general or specific occupation), use of different teaching methods (e.g., scenarios, samples, videos, speakers, field trips, self-study), availability of teacher office hours)

  

Qualifications of instructors/teachers(e.g., ESL/TESL certification, university degree/ specialization, teaching credentials, teaching experience (children/teenagers/ adults), experience working with newcomers, experience developing curricula, experience in occupation, if appropriate, mother tongue, other)

  
Facilities and resources

Quality of class facilities (e.g., lighting, blackboard, supplies, resources available to students)
Other resources: capacity to offer transportation, childcare or childminding, accommodation for people with disabilities if requested.

  

Availability of teaching resources (e.g., resources for teachers, technology, language labs, equipped classroom, supplies, printer/photocopier, office space, resources from host organization)

  

Institutional support for the program from SPO (e.g., access to resources, support for teachers, funding)

  
Participants/ Assessments

Profile of participants (e.g., gender, country of origin, CLB levels, involvement in other newcomer programs)

 

Selection/placement (how participants selected, how tested and placed in classes, use of intake assessments)

 

Progress assessments (e.g., tools used, timing and frequency (e.g., student assessment throughout delivery or at the end)

  
Tools/materials

Tools in use (e.g., course-specific materials developed by SPO, resources from other ELT projects, off-the-shelf resources, teacher-developed materials, lesson plans) [Collect copies of tools, or references to tools, if possible.]

  

How tools selected, if not developed by SPO (e.g., who involved in selection criteria for selection, how identified, access to other tools developed under ELT)

 

How tool developed, if appropriate (e.g., in-house or external resources, how needs assessed, involvement of experts and/or employers, approach to testing)

  

Qualifications of development staff (e.g., ESL/TESL certification, university degree/ specialization, teaching credentials, teaching experience (children/teenagers/ adults), experience working with newcomers, experience developing curricula, experience in occupation, if appropriate, mother tongue, other)

  
Bridge-to-Work

Nature (e.g., type of program (e.g., work placement, mentorship, orientation, job shadowing, internship, employment), hours in bridge-to-work, availability of mentors/quality of mentorship (e.g., frequency of meetings)

  

Promotion/selection (e.g., how course promoted, method to identify/screen/assess/match participants, community outreach activities, employer engagement activities, advisory committee, etc)

 
Other

Challenges in project delivery

 

Feedback gathered from participants/available (for classes and bridge-to-work); participant evaluation of program or measures in place to assess quality and effectiveness of services

 

Other comments

 

Review of ELT Tools/Materials and Processes:Expert Review Workbook

This tool will be used by the team’s experts to review the tools/materials being used in the delivery of ELT projects. The review will be conducted after the completion of the case studies and the experts will be provided with:

  • Team notes on how the tools/materials were developed (for tools/materials developed by the SPO) and how they are being delivered;
  • Copies of tools/materials developed by the SPO;
  • Links to, or names of, any tools that are not developed by the SPO and cannot be provided to the team (e.g., online resources, books).
Tools/materials

Description of what is being reviewed (e.g., assessment tools, placement tools, teaching manuals, student manuals, lesson plans):

 

 

 

 

 

 

 

 

 

Issue Criteria Observations
Process
(for materials developed by SPO)
  • Extent to which clients needs taken into account in the development of the tools/materials
  
  • Extent and timing of involvement of the following groups in the development of the tools:
    • People with experience teaching second languages;
    • People with experience teaching adults;
    • People with experience in the specific occupation, if appropriate
  
  • Nature and adequacy of the pilot testing before implementation
  
 
  • Appropriateness of the content for the objectives of ELT (e.g., covering CLB levels 7 – 10 or 0 – 10, as required, focus on profession-specific language training, providing labor market orientation etc.)
  
Content
  • Clearly stated learning outcomes and goals
  
  • Appropriateness of the content for:
    • CLB levels being taught
    • Occupation focus, if appropriate
    • Target audience (e.g., language and cultural background, age, gender)
  
  • Clarity of the language, vocabulary and style of writing. Appropriateness of the language for its targeted readers
  
  • Appropriateness of the organization of the material (e.g., logical flow, conciseness, amount of content)
  
  • Appropriateness of the balance between speaking, listening, reading and writing
 
  • Generalizability of the tools to other contexts
 
  • For testing tools, alignment of the testing tool with the teaching materials
  
  • Overall assessment of the content
  
Format and Design
  • Consistency of the format and layout with standards for this type of resource (e.g., sufficient white space, headings and subheadings, readability of font)
  
  • Appropriateness of the format (print, video etc.) for the objectives of the tool
  
  • Appropriate use of graphics (pictorial illustrations, tables, and line drawings)
 
  • Overall assessment of the design and layout
  
Instructional/ learning strategies/media
  • Appropriateness of the medium (print, video, online etc.) for the objectives of the tool
  
  • Appropriateness of the balance between speaking, listening, reading and writing
  
  • Sufficient opportunities for learner practice/ interaction
  
  • (For Teacher Resources) Material includes sufficient and useful suggestions for presenting content and organizing activities
  
  • Overall assessment of learning/teaching strategies
  
Other comments
  • Any suggestions for improvements to the tool/ material
  
  • Any suggestions for the delivery of these materials
  
  • Other comments
  

Page details

Date modified: