Neural-Bayesian Models for Agile Business Environments
DOI:
https://doi.org/10.5281/zenodo.20591715Keywords:
Real-time decision support; dynamic environments; hybrid models; neural networks; Bayesian methods; uncertainty quantification; risk assessment; online learning; explainable AI Understanding .Abstract
Growing algorithmic complexity and stringent response requirements inhibit the wide-scale deployment of neural networks for business decision support. While Bayesian methods for quantifying modeling uncertainty introduce latency overheads typically incompatible with real-time operation, hybrid architectures promise the best of both worlds. Yet, existing ensembles lack the necessary expediency. Combined with the need for accurate risk assessment in dynamic environments, a neural-Bayesian model is developed and evaluated for critical support, specifically for decision-making in dynamic environments.
Real-time deployment requires fast inference with low-latency data updates. Meeting these criteria within a hybrid architecture requires special consideration of component data streams and training protocols. In domain-relevant environments, temporal data sparsity enables a direct mapping from the input feature set to the neural subsystem and the early exit of superfluous data from the neural model. The real-time objective also allows for a non-standard online learning procedure involving offline model pretraining, updating during active drift only, and online drift detection with rollback for sudden change. For both systems, the decision-theoretic formulation of risk mitigates the challenge of accurate calibration without the requirement for a dedicated calibration set.
References
[1] Blei, D. M., Kucukelbir, A., & McAuliffe, J. D. (2017). Variational inference: A review for statisticians. Journal of the American Statistical Association, 112(518), 859–877.
[2] Blundell, C., Cornebise, J., Kavukcuoglu, K., & Wierstra, D. (2015). Weight uncertainty in neural networks. In Proceedings of the 32nd International Conference on Machine Learning (ICML) (pp. 1613–1622).
[3] Carvalho, C. M., & West, M. (2007). Dynamic matrix-variate graphical models. Bayesian Analysis, 2(1), 69–98.
[4] Cobb, J., & Ward, M. (2022). Real-time analytics and decision intelligence in digital business transformation. Business Horizons, 65(6), 707–718.
[5] Curi, C., Krause, A., & Krause, A. (2020). Neural contextual bandits without regret. In Proceedings of the 37th International Conference on Machine Learning (ICML).
[6] Depeweg, S., Hernández-Lobato, J. M., Doshi-Velez, F., & Udluft, S. (2018). Decomposition of uncertainty in Bayesian deep learning for efficient and risk-sensitive learning. In Proceedings of the 35th International Conference on Machine Learning (ICML).
[7] Dew, J., Read, J., & Carr, N. (2023). Decision intelligence systems: Integrating AI, analytics, and business decision-making. MIS Quarterly Executive, 22(3), 199–213.
[8] Gal, Y., & Ghahramani, Z. (2016). Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In Proceedings of the 33rd International Conference on Machine Learning (ICML) (pp. 1050–1059).
[9] Ghosal, S., & van der Vaart, A. (2017). Fundamentals of nonparametric Bayesian inference. Cambridge University Press.
[10] Gretton, A., Borgwardt, K. M., Rasch, M. J., Schölkopf, B., & Smola, A. (2012). A kernel two-sample test. Journal of Machine Learning Research, 13, 723–773.
[11] Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT Press.
[12] Graves, A. (2011). Practical variational inference for neural networks. In Advances in Neural Information Processing Systems.
[13] Gulshan, V., Peng, L., Coram, M., Stumpe, M. C., Wu, D., Narayanaswamy, A., Venugopalan, S., Widner, K., Madams, T., Cuadros, J., Kim, R., Raman, R., Nelson, P. C., Mega, J. L., & Webster, D. R. (2016). Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA, 316(22), 2402–2410.
[14] Harrison, P. J., & West, M. (1999). Bayesian forecasting and dynamic models (2nd ed.). Springer.
[15] He, H., & Garcia, E. A. (2009). Learning from imbalanced data. IEEE Transactions on Knowledge and Data Engineering, 21(9), 1263–1284.
[16] Hüllermeier, E., & Waegeman, W. (2021). Aleatoric and epistemic uncertainty in machine learning: An introduction to concepts and methods. Machine Learning, 110(3), 457–506.
[17] Jospin, L. V., Buntine, W., Boussaid, F., Laga, H., & Bennamoun, M. (2022). Hands-on Bayesian neural networks—A tutorial for deep learning users. IEEE Computational Intelligence Magazine, 17(2), 29–48.
[18] Jordan, M. I., Ghahramani, Z., Jaakkola, T. S., & Saul, L. K. (1999). An introduction to variational methods for graphical models. Machine Learning, 37(2), 183–233.
[19] Kapoor, S., & Narayanan, A. (2022). Leakage and the reproducibility crisis in ML-based science. Patterns, 3(9), 100496.
[20] Kendall, A., & Gal, Y. (2017). What uncertainties do we need in Bayesian deep learning for computer vision? In Proceedings of the 31st International Conference on Neural Information Processing Systems (NeurIPS) (pp. 5574–5584).
[21] Kingma, D. P., & Welling, M. (2014). Auto-encoding variational Bayes. In Proceedings of the 2nd International Conference on Learning Representations (ICLR).
[22] Koller, D., & Friedman, N. (2009). Probabilistic graphical models: Principles and techniques. MIT Press.
[23] Lakshminarayanan, B., Pritzel, A., & Blundell, C. (2017). Simple and scalable predictive uncertainty estimation using deep ensembles. In Proceedings of the 31st International Conference on Neural Information Processing Systems (NeurIPS) (pp. 6405–6416).
[24] Langford, J., Li, L., & Zhang, T. (2007). The epoch-greedy algorithm for multi-armed bandits with side information. In Advances in Neural Information Processing Systems.
[25] Lim, B., Arık, S. Ö., Loeff, N., & Pfister, T. (2021). Temporal fusion transformers for interpretable multi-horizon time series forecasting. International Journal of Forecasting, 37(4), 1748–1764.
[26] Lütkepohl, H. (2005). New introduction to multiple time series analysis. Springer.
[27] Makridakis, S., Spiliotis, E., & Assimakopoulos, V. (2018). Statistical and machine learning forecasting methods: Concerns and ways forward. PLOS ONE, 13(3), e0194889.
[28] Makridakis, S., Spiliotis, E., & Assimakopoulos, V. (2020). The M4 competition: Results, findings, conclusion and way forward. International Journal of Forecasting, 36(1), 54–74.
[29] McElreath, R. (2020). Statistical rethinking: A Bayesian course with examples in R and Stan (2nd ed.). CRC Press.
[30] Murphy, K. P. (2022). Probabilistic machine learning: An introduction. MIT Press.
[31] Papamakarios, G., Nalisnick, E., Rezende, D. J., Mohamed, S., & Lakshminarayanan, B. (2021). Normalizing flows for probabilistic modeling and inference. Journal of Machine Learning Research, 22(57), 1–64.
[32] Pearl, J. (2009). Causality: Models, reasoning, and inference (2nd ed.). Cambridge University Press.
[33] Peters, J., Janzing, D., & Schölkopf, B. (2017). Elements of causal inference: Foundations and learning algorithms. MIT Press.
[34] Rasmussen, C. E., & Williams, C. K. I. (2006). Gaussian processes for machine learning. MIT Press.
[35] Rue, H., Martino, S., & Chopin, N. (2009). Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 71(2), 319–392.
[36] Salinas, D., Flunkert, V., Gasthaus, J., & Januschowski, T. (2020). DeepAR: Probabilistic forecasting with autoregressive recurrent networks. International Journal of Forecasting, 36(3), 1181–1191.
Additional Files
Published
Data Availability Statement
The study uses publicly available datasets: financial time series from Yahoo Finance, Twitter sentiment data, and COVID-19 infection statistics from Johns Hopkins University, spanning January 2019 onward. The Iris Recognition (2022) dataset is also referenced. No proprietary or custom dataset was created by the authors.
