publications

2024

1. 

   Evaluating explainability for machine learning predictions using model-agnostic metrics

   Cristian Munoz ,  Kleyton Costa ,  Bernardo Modenesi , and 1 more author

   arXiv preprint arXiv:2302.12094, 2024

   Abs HTML PDF

   The rapid integration of artificial intelligence (AI) into various industries has introduced new challenges in governance and regulation, particularly regarding the understanding of complex AI systems. A critical demand from decision-makers is the ability to explain the results of machine learning models, which is essential for fostering trust and ensuring ethical AI practices. In this paper, we develop nine distinct model-agnostic metrics designed to quantify the extent to which model predictions can be explained. These metrics measure different aspects of model explainability, ranging from local importance, global importance, and surrogate predictions, allowing for a comprehensive evaluation of how models generate their outputs. Furthermore, by computing our metrics, we can rank models in terms of explainability criteria such as importance concentration and consistency, prediction fluctuation, and surrogate fidelity and stability, offering a valuable tool for selecting models based not only on accuracy but also on transparency. We demonstrate the practical utility of these metrics on classification and regression tasks, and integrate these metrics into an existing Python package for public use.

2023

1. Bank of England

   Deep learning model fragility and implications for financial stability and regulation

   Rishabh Kumar ,  Adriano Koshiyama ,  Kleyton Costa , and 6 more authors

   2023

   Abs PDF

   Deep learning models are being utilised increasingly within finance. Given the models are opaque in nature and are now being deployed for internal and consumer facing decisions, there are increasing concerns around the trustworthiness of their results. We test the stability of predictions and explanations of different deep learning models, which differ between each other only via subtle changes to model settings, with each model trained over the same data. Our results show that the models produce similar predictions but different explanations, even when the differences in model architecture are due to arbitrary factors like random seeds. We compare this behaviour with traditional, interpretable, ’glass-box models’, which show similar accuracies while maintaining stable explanations and predictions. Finally, we show a methodology based on network analysis to compare deep learning models. Our analysis has implications for the adoption and risk management of future deep learning models by regulated institutions.

2. Applied Sci.

   Solid Waste Management in Peru\’s Cities: A Clustering Approach for an Andean District

   Katherine Quispe ,  Mayra Martı́nez ,  Kleyton Costa , and 6 more authors

   Applied Sciences, 2023

   PDF
3. Arxiv

   Anomaly Detection in Global Financial Markets with Graph Neural Networks and Nonextensive Entropy

   Kleyton Costa

   arXiv preprint arXiv:2308.02914, 2023

   Abs PDF

   Anomaly detection is a challenging task, particularly in systems with many variables. Anomalies are outliers that statistically differ from the analyzed data and can arise from rare events, malfunctions, or system misuse. This study investigated the ability to detect anomalies in global financial markets through Graph Neural Networks (GNN) considering an uncertainty scenario measured by a nonextensive entropy. The main findings show that the complex structure of highly correlated assets decreases in a crisis, and the number of anomalies is statistically different for nonextensive entropy parameters considering before, during, and after crisis.

4. IE/UFRJ

   Effects of Monetary Policy on Income Distribution: Evidence from Brazil

   André Melo Modenesi ,  Kleyton Costa ,  Bernardo Modenesi , and 3 more authors

   2023

2022

1. Energies

   Statistical and Artificial Neural Networks Models for Electricity Consumption Forecasting in the Brazilian Industrial Sector

   Felipe Silva ,  Kleyton Costa ,  Paulo Canas Rodrigues , and 2 more authors

   Energies, 2022

   Abs PDF

   Forecasting the industry’s electricity consumption is essential for energy planning in a given country or region. Thus, this study aims to apply time-series forecasting models (statistical approach and artificial neural network approach) to the industrial electricity consumption in the Brazilian system. For the statistical approach, the Holt–Winters, SARIMA, Dynamic Linear Model, and TBATS (Trigonometric Box–Cox transform, ARMA errors, Trend, and Seasonal components) models were considered. For the approach of artificial neural networks, the NNAR (neural network autoregression) and MLP (multilayer perceptron) models were considered. The results indicate that the MLP model was the one that obtained the best forecasting performance for the electricity consumption of the Brazilian industry under analysis.

2. ANPEC

   Redes Neurais Artificiais para Análise e Previsão de Fenômenos Econômicos: Avaliando o Grau de Monetização da Economia Brasileira

   Kleyton Vieira Sales Costa ,  Antonio Jose Alves Junior ,  Felipe Leite Coelho Silva , and 2 more authors

   ANPEC, 2022

   Abs PDF

   Este estudo tem como objetivo avaliar a capacidade dos métodos de explicabilidade para modelos de inteligência artificial (XAI) na seleção de variáveis de resposta para os fenômenos econômicos. Para alcançar esse objetivo foi construído um fluxo de trabalho considerando modelos de aprendizado de máquina (Random Forest e Multilayer Perceptron Regressor) e modelos de regressão múltipla convencionais (Modelo Linear Generalizado, Regressão Bayesiana, Regressão Robusta e Regressão Regularizada - Ridge). A variável econômica dependente selecionada foi o grau de monetização da economia brasileira no período entre janeiro de 2001 e janeiro de 2021. O comportamento do grau de monetização pode ser impactado através do risco percebido pelos agentes econômicos em relação ao comportamento futuro da economia e, por esse fator, foram selecionadas variáveis usualmente utilizadas nas previsões dos agentes econômicos, sendo elas: taxa Selic, taxa de crescimento do PIB, taxa de câmbio e taxa de inflação. Os resultados encontrados através do mensuração do valor SHAP indicam que as variáveis mais importantes para prever o grau de monetização da economia são, respectivamente: a taxa Selic, a taxa de câmbio e a taxa de inflação. Por fim, aplicou-se um estudo comparativo entre modelos de redes neurais (Recurrent Neural Network, Long Short-term Memory e Gated Recurrent Unit) artificiais para a previsão do grau de monetização considerando as variáveis selecionadas pelo método SHAP como os valores de entrada. O estudo mostrou que o modelo Recurrent Neural Network tem a melhor capacidade preditiva dentre os modelos selecionados

3. Preprint

   Central bank digital currencies: a high-level overview

   Kleyton Costa ,  Antônio José Alves Júnior ,  Felipe Silva , and 2 more authors

   Available at SSRN 4271644, 2022

   Abs PDF

   In the aftermath of the private digital currencies’ revolution the debate on the costs/benefits of issuing a public digital currency gained prominence. This is a relevant public policy debate and, therefore, should not be led only by central bankers but also by academics. The aim of this paper is to discuss the properties and motivations of issuing a CBDC. A central bank-issued digital currency (CBDC) is a digital form of central bank money that differs from the current digital monetary forms made available by central banks – bank reserves and settlement accounts. When compared to other monetary forms (paper money, bank deposits, etc.) CBDC meets four key properties for a currency, which may be widely accessible, digital, central bank-issued, and token-based. Some countries have already launched guidelines and have been expanding their research and testing with digital currencies issued by their central banks, such as China (e-RMB), Brazil (Real Digital), and Sweden (e- krona). The conclusions indicate that the most relevant challenges for issuing a CBDC are: technical, operational limitations, legislative, and managerial barriers.

4. Nature

   A machine learning approach to analyse ozone concentration in metropolitan area of Lima, Peru

   Natalı́ Carbo-Bustinza ,  Marisol Belmonte ,  Vasti Jimenez , and 7 more authors

   Nature Scientific Reports, 2022

   PDF

2021

1. CNMAC

   Analisando a Capacidade de Previsão entre Diferentes/Classes de Modelos: um Estudo Comparativo para os ı́ndices de Produção Industrial no Brasil

   Felipe Leite Coelho Silva ,  and  Kleyton Vieira Sales Costa

   Proceeding Series of the Brazilian Society of Computational and Applied Mathematics, 2021
2. CNMAC

   Artificial Neural Networks and Conditional Heteroscedastic Models for Forecasting Exchange Rate

   Kleyton Vieira Sales Costa ,  and  Felipe Leite Coelho Silva

   Proceeding Series of the Brazilian Society of Computational and Applied Mathematics, 2021
3. Monography

   A MOEDA DIGITAL EMITIDA PELO BANCO CENTRAL: UMA INVESTIGAÇÃO DAS SUAS PROPRIEDADES E EFEITOS MACROECONÔMICOS

   Kleyton Vieira Sales Costa

   2021

   Undergraduate Thesis

2020

1. Arxiv

   A Systematic Comparison of Forecasting for Gross Domestic Product in an Emergent Economy

   Kleyton Costa ,  Felipe Leite Coelho Silva ,  Josiane da Silva Cordeiro Coelho , and 1 more author

   arXiv preprint arXiv:2010.13259, 2020