Predicting Fatalities with Pre-trained Temporal Transformers: A Time Series Regression Approach

Theabilitytopredictwithprecisionevents, suchasthenumberofbattle-relatedfatalities, is not only of academic interest, but it holds significant implications for policy-making and conflict prevention too. This broad interest has enlightened our research, which, specifically, delves into the use of temporal transformers as a new approach to predict the number of battle-related deaths, at the country-level and over a forecast temporal horizon spanning from 3 to 14 months. OurArtificialIntelligence-EarlyWarningSystem(AI-EWS),proposedforthe2023/24 VIEWS prediction competition [Hegre et al., Forthcoming], leverages a multi-headed at- tentionmechanismasoutlinedbyVaswanietal.[2017]. Wechosethetemporaltransform- ers due to their proven efficacy in time series representation learning, as demonstrated in Zerveas et al. [2021]. The model incorporates residual connections from input to output, preserving linear activation, a method supported by empirical evidence for its effectiveness in time-series forecasting [Zeng et al., 2023]. The following section details the methodology used to harness this model for time series regression.