Employment Insurance (EI) Machine Learning Workload

Overview

Employment and Social Development Canada (ESDC) developed and deployed a machine learning system to forecast and triage Employment Insurance (EI) recalculation workloads within its Benefits and Integrated Services Branch (BISB). The system, known as the Machine Learning Workload project, uses a supervised Random Forest classification model trained on historical EI production data to predict the likely outcome of benefit recalculations and prioritise associated work items for processing by officers.

The recalculation of Employment Insurance benefits involves conducting a comprehensive review and reassessment of a claim in response to new information or changes in circumstances. The primary objective of this review is to ensure accurate benefit calculations, determining whether they result in overpayment, underpayment, or no change to the claimant's benefit rate. Since the introduction of Web and Electronic Records of Employment (ROEs) in 2013, approximately 300,000 recalculation work items have been generated annually. During the COVID-19 pandemic, this number surged to over 1.7 million due to the implementation of simplification measures associated with the Emergency Response Benefit (ERB), simplified EI, and the subsequent return to regular EI processing. This created a substantial backlog of claim reviews, many of them for claims established before March 2020, which competed for resources with more current and pressing work.

The ML model was developed by the Employment Insurance Program Performance unit within ESDC in consultation with several internal stakeholders across the EI programme. A Random Forest algorithm was applied to data drawn from EI production systems. The model specifically focuses on terminated or dormant claims and classifies claim recalculations into three categories of predicted outcomes: increase in benefit rate (indicating underpayment), decrease in benefit rate (indicating overpayment), and no change in benefit rate. This classification enables the programme to close recalculations that are predicted to have no impact on the claimant and to prioritise those with a greater likelihood of resulting in a change of benefit, thereby directing officer attention to cases that matter most.

The Algorithmic Impact Assessment (AIA) completed for this system under Canada's Directive on Automated Decision-Making yielded an Impact Level of 1, the lowest level on the scale, with a current score of 30, a raw impact score of 30, and a mitigation score of 29. The AIA classified the system as partial automation, meaning it contributes to administrative decision-making by supporting an officer through assessments, recommendations, intermediate decisions, or other outputs, rather than making fully autonomous decisions. The assessment noted that the model's decisions are reversible and that impacts are most likely to be brief. The system was assessed as having little to no impact on the equality, dignity, privacy, autonomy, health, and economic interests of individuals.

The system uses personal information from EI production systems as input data, with the highest security classification being Protected A. Data is controlled by the federal government, collected by ESDC itself, and the system interfaces with other IT systems while using data from multiple different sources. The system operates within a closed environment with no connections to the Internet, Intranet, or any other external system. De-identification is achieved through the creation of a primary key. The system does not require the analysis of unstructured data.

Before deployment, strict adherence to the Treasury Board of Canada Secretariat (TBS) guidelines for Automated Decision Systems was required. The TBS has been regulating the use of ML within the Government of Canada since April 1, 2020, and compliance with the TBS Directive for ADS requires a rigorous approval process to ensure adherence to established standards. Risk mitigation measures include randomised manual spot checks conducted by agents to review claims with no anticipated change in benefits. If a client makes an inquiry about their file, officers proceed with the full recalculation process. The Integrity Service Branch (ISB) is responsible for reviewing undeclared contentious issues to ensure accuracy and fairness. Additionally, the workload process is modified to prioritise active claims, allowing for efficient allocation of resources.

Internal stakeholder consultations were conducted with Data Governance, Programme Policy, Legal Services, Communications Services, and the Access to Information and Privacy Office. Accountability for the design, development, maintenance, and improvement of the system has been formally assigned within ESDC. The system provides an audit trail that records all recommendations and decisions, identifies all key decision points, maintains a current log of changes to the model and system, identifies the system version used for each decision, and enables human override of system decisions with logging of override instances. A recourse process is established for clients who wish to challenge decisions. A concept case was prepared and presented to the Government of Canada Enterprise Architecture Review Board.

The model was reported to accurately identify claims with no change in benefits at a rate of 90 percent. The worst-case scenario identified in the AIA occurs when a claim eligible for a higher benefit rate is erroneously classified as having no change. By July 2023, the implementation of the Machine Learning model proved successful, resulting in the completion of over 40,000 claims. Although the original backlog reduction project page is no longer updated, the project continues under the name Employment Insurance Machine Learning Workload. The system is part of ESDC's broader 2023-2026 Data Strategy, which emphasises data-driven approaches to improving programme delivery and operational efficiency.

The Integrity Services Branch at ESDC has also developed broader analytics capabilities including predictive models, anomaly detection algorithms, and the Integrity Technology-Reinforced Anomaly Program (iTRAP), which has identified over 70,000 instances of potential identity theft and prevented approximately one billion dollars in benefits from being paid fraudulently since its deployment in late 2021. The ML Workload project represents one component of this wider data-driven transformation of integrity and operational functions within Canada's Employment Insurance programme.

Classification

Programme Details

Machine learning system using a Random Forest model to predict the outcome of Employment Insurance recalculation work items, triaging them to prioritise cases likely to result in a change of benefit and closing those with no anticipated impact on claimants.

Implementation Details

Risk & Oversight

Deployment & Outcomes

Sources

1. SRC-001-CAN-001 Canada School of Public Service (2023) Government of Canada Data Conference 2023: ESDC Integrity Services Branch's Use of Quantitative and Qualitative Data (Transcript). Available at: https://www.csps-efpc.gc.ca/video/data-conference2023/integrity-services-eng.aspx (Accessed: 31 October 2025).
   https://www.csps-efpc.gc.ca/video/data-conference2023/integrity-services-eng.aspx
2. SRC-002-CAN-001 Employment and Social Development Canada (2024) Algorithmic Impact Assessment - Machine Learning Workload (Employment Insurance). Open Government Portal. Available at: https://open.canada.ca/data/dataset/6b429c8e-ee5b-451a-883f-b6180ada9286/resource/2694367e-babf-4d87-a852-827e4178141d/download/1-aia_ei_ml_workload_en.pdf (Accessed: 31 October 2025).
   https://open.canada.ca/data/dataset/6b429c8e-ee5b-451a-883f-b6180ada9286/resource/2694367e-babf-4d87-a852-827e4178141d/download/1-aia_ei_ml_workload_en.pdf
3. SRC-003-CAN-001 Employment and Social Development Canada (2024) Reducing Employment Insurance Backlog: A Machine Learning Approach. Open Government Portal. Available at: https://open.canada.ca/data/en/info/24d2cab2-6a0d-4234-9239-b6ce102ebabd (Accessed: 31 October 2025).
   https://open.canada.ca/data/en/info/24d2cab2-6a0d-4234-9239-b6ce102ebabd
4. SRC-004-CAN-001 Government of Canada (2025) Amendments to the Directive on Automated Decision-Making. Available at: https://www.canada.ca/en/government/system/digital-government/policies-standards/policy-service-digital-announcements/amendments-directive-automated-decision-making.html (Accessed: 31 October 2025).
   https://www.tbs-sct.canada.ca/pol/doc-eng.aspx?id=32592
5. SRC-006-CAN-001 Kaye, K. (2024) 'AI Governance on the Ground: Canada's Algorithmic Impact Assessment Process and Algorithm has evolved', World Privacy Forum, 14 August. Available at: https://www.worldprivacyforum.org/2024/08/ai-governance-on-the-ground-series-canada/ (Accessed: 30 March 2026).
   https://www.worldprivacyforum.org/2024/08/ai-governance-on-the-ground-series-canada/
6. SRC-005-CAN-001 Treasury Board of Canada Secretariat (2024) Guide on the Scope of the Directive on Automated Decision-Making. Ottawa: Government of Canada. Available at: https://publications.gc.ca/collections/collection_2024/sct-tbs/BT48-46-2024-eng.pdf (Accessed: 31 October 2025).
   https://publications.gc.ca/collections/collection_2024/sct-tbs/BT48-46-2024-eng.pdf