Can calculated ratings improve the dynamic factors identification and analysis: Revised?

Research Highlights: Calculating ratings from endorsed indicators provide greater predictive power and reliability.

The Dynamic Factors Identification and Analysis – Revised (DFIA-R) is a key component of Correctional Service of Canada’s (CSC) Offender Intake Assessment process. Implemented in 2009, it is designed to assess dynamic risk. The DFIA-R is comprised of 100 indicators across seven domains – employment/education, marital/family, associates, substance abuse, community functioning, personal/emotional, and attitudes – assessed by parole officers. Based on their professional judgment, an overall dynamic need rating is generated as well as a rating on each domain. Recent research has demonstrated that the overall rating and each of the domain ratings are predictive of revocations of conditional release across offender groups (Stewart, Wardrop, Wilton, Thompson, Derkzen & Motiuk, 2017). The present study examined whether calculated ratings could improve the predictive validity of the tool over the structured professional judgment provided by the parole officers at intake.

The study included 15,487 non-Indigenous men, 4,640 Indigenous men, and 1,195 women who had completed DFIA-R assessments that included the assessment of indicators and had been released. Because of the low number of revocations for women offenders we were unable to disaggregate by Indigenous status.

Calculated domain ratings on the DFIA-R were based on the proportion of indicators endorsed on each domain for each offender group. Using this method, no endorsed indicators led to a rating of no need; fewer than 33% of indicators endorsed produced a low need rating; 33% to 66% of indicators endorsed were rated as moderate need, and more than 66% of indicators endorsed produced a high need rating.

For the overall rating of need, two calculated methods were used. The first was based on the mean of the proportions of indicators endorsed across the domains and the second weighted each domain by its strength of association with outcome based on the survival analyses (hazard ratios) for each offender group. Applying the first method, offenders with fewer than 25% of indicators endorsed across the domains were rated low need, 25% to 50% endorsed were rated medium need, and greater than 50% rated high need.

An adjusted calculation for women set the cut-offs at less than 33%, 33%-66%, and greater than 66% for low, moderate or high ratings respectively. We did not control for covariates given the need to maximize statistical power.

Results indicated that across all domains and for non-Indigenous men, Indigenous men, and women, ratings based on the proportion of indicators endorsed produced greater differentiation in predicting revocations among offenders rated as no, low, moderate and high need than the parole officers’ ratings. Predictive validity was improved for all three study groups using these two methods. Both calculated methods of overall need ratings produced stronger predictions of revocations than ratings made by parole officers. For women offenders, the adjusted calculated method further improved the prediction of the overall ratings. Split-half reliability indicated that parole officers’ ratings of overall need had lower reliability than the calculated ratings.

Calculated ratings based on proportion of indicators endorsed provided greater predictive power and reliability for the DFIA-R measure than the parole officer ratings for men, women, and Indigenous men and should therefore be considered as a recommended innovation in a future revision to the tool.

Wilton, G., Stewart, L., & Motiuk, L. (2017). Can the predictive validity of the Dynamic Factors Identification and Analysis – Revised be improved by calculated ratings? Research Report (R-400). Ottawa, Ontario: Correctional Service of Canada.

To obtain a PDF version of the full report, or for other inquiries, please e-mail the Research Branch or contact us by phone at (613) 995-3975.

You can also visit the Research Publications section for a full list of reports and one-page summaries.