We have located links that may give you full text access.
Artificial intelligence exceeds humans in epidemiological job coding.
Commun Med (Lond) 2023 November 5
BACKGROUND: Work circumstances can substantially negatively impact health. To explore this, large occupational cohorts of free-text job descriptions are manually coded and linked to exposure. Although several automatic coding tools have been developed, accurate exposure assessment is only feasible with human intervention.
METHODS: We developed OPERAS, a customizable decision support system for epidemiological job coding. Using 812,522 entries, we developed and tested classification models for the Professions et Catégories Socioprofessionnelles (PCS)2003, Nomenclature d'Activités Française (NAF)2008, International Standard Classifications of Occupation (ISCO)-88, and ISCO-68. Each code comes with an estimated correctness measure to identify instances potentially requiring expert review. Here, OPERAS' decision support enables an increase in efficiency and accuracy of the coding process through code suggestions. Using the Formaldehyde, Silica, ALOHA, and DOM job-exposure matrices, we assessed the classification models' exposure assessment accuracy.
RESULTS: We show that, using expert-coded job descriptions as gold standard, OPERAS realized a 0.66-0.84, 0.62-0.81, 0.60-0.79, and 0.57-0.78 inter-coder reliability (in Cohen's Kappa) on the first, second, third, and fourth coding levels, respectively. These exceed the respective inter-coder reliability of expert coders ranging 0.59-0.76, 0.56-0.71, 0.46-0.63, 0.40-0.56 on the same levels, enabling a 75.0-98.4% exposure assessment accuracy and an estimated 19.7-55.7% minimum workload reduction.
CONCLUSIONS: OPERAS secures a high degree of accuracy in occupational classification and exposure assessment of free-text job descriptions, substantially reducing workload. As such, OPERAS significantly outperforms both expert coders and other current coding tools. This enables large-scale, efficient, and effective exposure assessment securing healthy work conditions.
METHODS: We developed OPERAS, a customizable decision support system for epidemiological job coding. Using 812,522 entries, we developed and tested classification models for the Professions et Catégories Socioprofessionnelles (PCS)2003, Nomenclature d'Activités Française (NAF)2008, International Standard Classifications of Occupation (ISCO)-88, and ISCO-68. Each code comes with an estimated correctness measure to identify instances potentially requiring expert review. Here, OPERAS' decision support enables an increase in efficiency and accuracy of the coding process through code suggestions. Using the Formaldehyde, Silica, ALOHA, and DOM job-exposure matrices, we assessed the classification models' exposure assessment accuracy.
RESULTS: We show that, using expert-coded job descriptions as gold standard, OPERAS realized a 0.66-0.84, 0.62-0.81, 0.60-0.79, and 0.57-0.78 inter-coder reliability (in Cohen's Kappa) on the first, second, third, and fourth coding levels, respectively. These exceed the respective inter-coder reliability of expert coders ranging 0.59-0.76, 0.56-0.71, 0.46-0.63, 0.40-0.56 on the same levels, enabling a 75.0-98.4% exposure assessment accuracy and an estimated 19.7-55.7% minimum workload reduction.
CONCLUSIONS: OPERAS secures a high degree of accuracy in occupational classification and exposure assessment of free-text job descriptions, substantially reducing workload. As such, OPERAS significantly outperforms both expert coders and other current coding tools. This enables large-scale, efficient, and effective exposure assessment securing healthy work conditions.
Full text links
Related Resources
Get seemless 1-tap access through your institution/university
For the best experience, use the Read mobile app
All material on this website is protected by copyright, Copyright © 1994-2024 by WebMD LLC.
This website also contains material copyrighted by 3rd parties.
By using this service, you agree to our terms of use and privacy policy.
Your Privacy Choices
You can now claim free CME credits for this literature searchClaim now
Get seemless 1-tap access through your institution/university
For the best experience, use the Read mobile app