Early risk detection of self-harm and depression severity using BERT-based transformers: iLab at CLEF eRisk 2020

Rodrigo Martínez-Castaño; Amal Htait; Leif Azzopardi; Yashar Moshfeghi

Early risk detection of self-harm and depression severity using BERT-based transformers: iLab at CLEF eRisk 2020

Rodrigo Martínez-Castaño, Amal Htait, Leif Azzopardi, Yashar Moshfeghi

Research output: Chapter in Book/Published conference output › Conference publication

Abstract

This paper briefly describes our research groups’ efforts in tackling Task 1 (Early Detection of Signs of Self-Harm), and Task 2 (Measuring the Severity of the Signs of Depression) from the CLEF eRisk Track. Core to how we approached these problems was the use of BERT-based classifiers which were trained specifically for each task. Our results on both tasks indicate that this approach delivers high performance across a series of measures, particularly for Task 1, where our submissions obtained the best performance for precision, F1, latency-weighted F1 and ERDE at 5 and 50. This work suggests that BERT-based classifiers, when trained appropriately, can accurately infer which social media users are at risk of self-harming, with precision up to 91.3% for Task 1. Given these promising results, it will be interesting to further refine the training regime, classifier and early detection scoring mechanism, as well as apply the same approach to other related tasks (e.g., anorexia, depression, suicide).

Original language	English
Title of host publication	Working Notes of CLEF 2020 - Conference and Labs of the Evaluation Forum
Publisher	CEUR-WS.org
Number of pages	16
Volume	2696
Publication status	Published - 25 Sept 2020
Event	Early Risk Prediction on the Internet: CLEF Workshop - Thessaloniki, Greece Duration: 22 Sept 2020 → 25 Sept 2020 https://early.irlab.org/2020/index.html

Publication series

Name	CEUR Workshop Proceedings
Volume	2696
ISSN (Electronic)	1613-0073

Workshop

Workshop	Early Risk Prediction on the Internet: CLEF Workshop
Abbreviated title	ERISK 2020
Country/Territory	Greece
City	Thessaloniki
Period	22/09/20 → 25/09/20
Internet address	https://early.irlab.org/2020/index.html

Bibliographical note

Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). Acknowledgements & Funding: The first author would like to thank the following funding bodies for their support: FEDER / Ministerio de Ciencia, Innovaci ́on y Universidades, Agencia Estatal de Investigaci ́on / Project (RTI2018-093336-B-C21), Conseller ́ıa de Educacion, Universidade e Formaci ́on Profesional and the European Regional Development Fund (ERDF) (accreditation 2019-2022 ED431G-2019/04, ED431C 2018/29, ED431C 2018/19). The second and third authors would like to thank the UKRI’s EPSRC Project
Cumulative Revelations in Personal Data (Grant Number: EP/R033897/1) for their support. The authors would also like to thank David Losada for arranging this collaboration.

Keywords

Self-Harm
Depression
Classification
Social Media
Early Detection
BERT
XLM-RoBERTa

Access to Document

Martinez-Castano et al_Early Risk Detection_Self-harm_using BERT_paper_50
Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
Final published version, 562 KBLicence: CC BY 4.0

https://ceur-ws.org/Vol-2696/paper_50.pdfLicence: CC BY 4.0

Cite this

Martínez-Castaño, R., Htait, A., Azzopardi, L., & Moshfeghi, Y. (2020). Early risk detection of self-harm and depression severity using BERT-based transformers: iLab at CLEF eRisk 2020. In Working Notes of CLEF 2020 - Conference and Labs of the Evaluation Forum (Vol. 2696). Article 50 (CEUR Workshop Proceedings; Vol. 2696). CEUR-WS.org. https://ceur-ws.org/Vol-2696/paper_50.pdf

@inproceedings{30e7399e691040e8ad356cb981f606a3,

title = "Early risk detection of self-harm and depression severity using BERT-based transformers: iLab at CLEF eRisk 2020",

abstract = "This paper briefly describes our research groups{\textquoteright} efforts in tackling Task 1 (Early Detection of Signs of Self-Harm), and Task 2 (Measuring the Severity of the Signs of Depression) from the CLEF eRisk Track. Core to how we approached these problems was the use of BERT-based classifiers which were trained specifically for each task. Our results on both tasks indicate that this approach delivers high performance across a series of measures, particularly for Task 1, where our submissions obtained the best performance for precision, F1, latency-weighted F1 and ERDE at 5 and 50. This work suggests that BERT-based classifiers, when trained appropriately, can accurately infer which social media users are at risk of self-harming, with precision up to 91.3% for Task 1. Given these promising results, it will be interesting to further refine the training regime, classifier and early detection scoring mechanism, as well as apply the same approach to other related tasks (e.g., anorexia, depression, suicide).",

keywords = "Self-Harm, Depression, Classification, Social Media, Early Detection, BERT, XLM-RoBERTa",

author = "Rodrigo Mart{\'i}nez-Casta{\~n}o and Amal Htait and Leif Azzopardi and Yashar Moshfeghi",

note = "Copyright {\textcopyright} 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). Acknowledgements & Funding: The first author would like to thank the following funding bodies for their support: FEDER / Ministerio de Ciencia, Innovaci{\' }on y Universidades, Agencia Estatal de Investigaci{\' }on / Project (RTI2018-093336-B-C21), Conseller{\' }ıa de Educacion, Universidade e Formaci{\' }on Profesional and the European Regional Development Fund (ERDF) (accreditation 2019-2022 ED431G-2019/04, ED431C 2018/29, ED431C 2018/19). The second and third authors would like to thank the UKRI{\textquoteright}s EPSRC Project Cumulative Revelations in Personal Data (Grant Number: EP/R033897/1) for their support. The authors would also like to thank David Losada for arranging this collaboration.; Early Risk Prediction on the Internet: CLEF Workshop, ERISK 2020 ; Conference date: 22-09-2020 Through 25-09-2020",

year = "2020",

month = sep,

day = "25",

language = "English",

volume = "2696",

series = "CEUR Workshop Proceedings",

publisher = "CEUR-WS.org",

booktitle = "Working Notes of CLEF 2020 - Conference and Labs of the Evaluation Forum",

url = "https://early.irlab.org/2020/index.html",

}

Martínez-Castaño, R, Htait, A, Azzopardi, L & Moshfeghi, Y 2020, Early risk detection of self-harm and depression severity using BERT-based transformers: iLab at CLEF eRisk 2020. in Working Notes of CLEF 2020 - Conference and Labs of the Evaluation Forum. vol. 2696, 50, CEUR Workshop Proceedings, vol. 2696, CEUR-WS.org, Early Risk Prediction on the Internet: CLEF Workshop, Thessaloniki, Greece, 22/09/20. <https://ceur-ws.org/Vol-2696/paper_50.pdf>

Early risk detection of self-harm and depression severity using BERT-based transformers: iLab at CLEF eRisk 2020. / Martínez-Castaño, Rodrigo; Htait, Amal; Azzopardi, Leif et al.
Working Notes of CLEF 2020 - Conference and Labs of the Evaluation Forum. Vol. 2696 CEUR-WS.org, 2020. 50 (CEUR Workshop Proceedings; Vol. 2696).

Research output: Chapter in Book/Published conference output › Conference publication

TY - GEN

T1 - Early risk detection of self-harm and depression severity using BERT-based transformers

T2 - Early Risk Prediction on the Internet: CLEF Workshop

AU - Martínez-Castaño, Rodrigo

AU - Htait, Amal

AU - Azzopardi, Leif

AU - Moshfeghi, Yashar

N1 - Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). Acknowledgements & Funding: The first author would like to thank the following funding bodies for their support: FEDER / Ministerio de Ciencia, Innovaci ́on y Universidades, Agencia Estatal de Investigaci ́on / Project (RTI2018-093336-B-C21), Conseller ́ıa de Educacion, Universidade e Formaci ́on Profesional and the European Regional Development Fund (ERDF) (accreditation 2019-2022 ED431G-2019/04, ED431C 2018/29, ED431C 2018/19). The second and third authors would like to thank the UKRI’s EPSRC Project Cumulative Revelations in Personal Data (Grant Number: EP/R033897/1) for their support. The authors would also like to thank David Losada for arranging this collaboration.

PY - 2020/9/25

Y1 - 2020/9/25

N2 - This paper briefly describes our research groups’ efforts in tackling Task 1 (Early Detection of Signs of Self-Harm), and Task 2 (Measuring the Severity of the Signs of Depression) from the CLEF eRisk Track. Core to how we approached these problems was the use of BERT-based classifiers which were trained specifically for each task. Our results on both tasks indicate that this approach delivers high performance across a series of measures, particularly for Task 1, where our submissions obtained the best performance for precision, F1, latency-weighted F1 and ERDE at 5 and 50. This work suggests that BERT-based classifiers, when trained appropriately, can accurately infer which social media users are at risk of self-harming, with precision up to 91.3% for Task 1. Given these promising results, it will be interesting to further refine the training regime, classifier and early detection scoring mechanism, as well as apply the same approach to other related tasks (e.g., anorexia, depression, suicide).

AB - This paper briefly describes our research groups’ efforts in tackling Task 1 (Early Detection of Signs of Self-Harm), and Task 2 (Measuring the Severity of the Signs of Depression) from the CLEF eRisk Track. Core to how we approached these problems was the use of BERT-based classifiers which were trained specifically for each task. Our results on both tasks indicate that this approach delivers high performance across a series of measures, particularly for Task 1, where our submissions obtained the best performance for precision, F1, latency-weighted F1 and ERDE at 5 and 50. This work suggests that BERT-based classifiers, when trained appropriately, can accurately infer which social media users are at risk of self-harming, with precision up to 91.3% for Task 1. Given these promising results, it will be interesting to further refine the training regime, classifier and early detection scoring mechanism, as well as apply the same approach to other related tasks (e.g., anorexia, depression, suicide).

KW - Self-Harm

KW - Depression

KW - Classification

KW - Social Media

KW - Early Detection

KW - BERT

KW - XLM-RoBERTa

UR - https://pureportal.strath.ac.uk/en/publications/8bbc9fa9-e365-46b8-97c9-11fda4c880af

M3 - Conference publication

VL - 2696

T3 - CEUR Workshop Proceedings

BT - Working Notes of CLEF 2020 - Conference and Labs of the Evaluation Forum

PB - CEUR-WS.org

Y2 - 22 September 2020 through 25 September 2020

ER -

Early risk detection of self-harm and depression severity using BERT-based transformers: iLab at CLEF eRisk 2020

Abstract

Publication series

Workshop

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this