Gang violence is a severe issue in major cities across theU.S. and recent studies have found evidence of social me-dia communications that can be linked to such violence incommunities with high rates of exposure to gang activity. Inthis paper we partnered computer scientists with social workresearchers, who have domain expertise in gang violence, toanalyze how public tweets with images posted by youth whomention gang associations on Twitter can be leveraged to au-tomatically detect psychosocial factors and conditions thatcould potentially assist social workers and violence outreachworkers in prevention and early intervention programs. Tothis end, we developed a rigorous methodology for collectingand annotating tweets. We gathered 1,851 tweets and accom-panying annotations related to visual concepts and thepsy-chosocial codes:aggression,loss, andsubstance use. Thesecodes are relevant to social work interventions, as they repre-sent possible pathways to violence on social media. We com-pare various methods for classifying tweets into these threeclasses, using only the text of the tweet, only the image of thetweet, or both modalities as input to the classifier. In partic-ular, we analyze the usefulness of mid-level visual conceptsand the role of different modalities for this tweet classifica-tion task. Our experiments show that individually, text infor-mation dominates classification performance of thelossclass,while image information dominates theaggressionandsub-stance useclasses. Our multimodal approach provides a verypromising improvement (18% relative in mean average preci-sion) over the best single modality approach. Finally, we alsoillustrate the complexity of understanding social media dataand elaborate on open challenges. The annotated dataset willbe made available for research with strong ethical protectionmechanism.

Multimodal Social Media Analysis for Gang Violence Prevention

Rossano Schifanella;
2019-01-01

Abstract

Gang violence is a severe issue in major cities across theU.S. and recent studies have found evidence of social me-dia communications that can be linked to such violence incommunities with high rates of exposure to gang activity. Inthis paper we partnered computer scientists with social workresearchers, who have domain expertise in gang violence, toanalyze how public tweets with images posted by youth whomention gang associations on Twitter can be leveraged to au-tomatically detect psychosocial factors and conditions thatcould potentially assist social workers and violence outreachworkers in prevention and early intervention programs. Tothis end, we developed a rigorous methodology for collectingand annotating tweets. We gathered 1,851 tweets and accom-panying annotations related to visual concepts and thepsy-chosocial codes:aggression,loss, andsubstance use. Thesecodes are relevant to social work interventions, as they repre-sent possible pathways to violence on social media. We com-pare various methods for classifying tweets into these threeclasses, using only the text of the tweet, only the image of thetweet, or both modalities as input to the classifier. In partic-ular, we analyze the usefulness of mid-level visual conceptsand the role of different modalities for this tweet classifica-tion task. Our experiments show that individually, text infor-mation dominates classification performance of thelossclass,while image information dominates theaggressionandsub-stance useclasses. Our multimodal approach provides a verypromising improvement (18% relative in mean average preci-sion) over the best single modality approach. Finally, we alsoillustrate the complexity of understanding social media dataand elaborate on open challenges. The annotated dataset willbe made available for research with strong ethical protectionmechanism.
2019
International Conference on Web and Social Media
Munich, Germany
11-14 June
Proceedings of the Thirteenth International Conference on Web and Social Media
AAAI
114
124
978-1-57735-806-0
https://arxiv.org/abs/1807.08465
Philipp Blandfort, Desmond Upton Patton, William R. Frey, Svebor Karaman, Surabhi Bhargava, Fei-Tzin Lee, Siddharth Varia, Chris Kedzie, Michael B. Gaskell, Rossano Schifanella, Kathleen R. McKeown, Shih-Fu Chang
File in questo prodotto:
File Dimensione Formato  
1807.08465.pdf

Accesso aperto

Tipo di file: PREPRINT (PRIMA BOZZA)
Dimensione 2 MB
Formato Adobe PDF
2 MB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/2318/1720247
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 17
  • ???jsp.display-item.citation.isi??? ND
social impact