Comparing Transformers and Linear models for precipitation forecast in Rio de Janeiro

Resumo

Precipitation represents a critical meteorological phenomenon that exerts a substantial influence on different geographic regions, as well as playing a fundamental role in various human activities. Notably, Rio de Janeiro experiences unstable weather conditions that lead to sudden and intense rainfall. Consequently, forecasting such precipitation patterns, particularly extreme events, is of fundamental importance in mitigating adverse impacts. Artificial Neural Networks (ANNs) present a promising path for predicting time series data, with transformer architectures emerging as an efficient option. Recognized for their versatility across diverse tasks, transformers have demonstrated effectiveness in time series forecasting, with the Autoformer model emerging as a standout performer, achieving state-of-the-art performance levels. However, the computational demands inherent in transformer-based models, including significant time and memory requirements, have led to the exploration of simpler alternatives. Linear models, such as DLinear, have been proposed as computationally efficient alternatives, capable of providing predictive performance comparable or superior to transformers. The objective of this study was to evaluate and contrast the predictive effectiveness of the linear model with the transformer-based approach in precipitation forecasting tasks for data from Rio de Janeiro. The dataset was obtained through the INMET meteorological system, covering historical records from 2002 to 2023, from four meteorological stations distributed in Rio de Janeiro, Brazil. When it comes to precipitation forecasting, the presence of data imbalance, particularly with regard to extreme events characterized by precipitation exceeding 25 mm, represents a significant challenge. In the scope of this work, the dataset was used in unbalanced form. To train the models, the dataset was partitioned into training (60%), validation (20%) and test (20%) subsets. Both models were instantiated with equal parameters, including sequence length and prediction length of 96, batch size of 32, 20 epochs, and utilizing EarlyStopping and ReduceLROnPlateau callbacks with a patience parameter of 3. The mean squared error (MSE) served as the primary metric for optimizing the loss function during training and evaluating predictive performance. Finally, the study seeks to evaluate the quality of models for predicting precipitation in the Rio de Janeiro region. By evaluating meteorological data, the study attempts to contribute to the understanding of models performance in precipitation forecasting tasks. Our analysis sought to demonstrate insights into which architecture to choose when it comes to precipitation with an unbalance dataset.