Investigating the Role of Activation Functions in Predicting the Price of Cryptocurrencies during Critical Economic Periods
DOI:
https://doi.org/10.34021/ve.2024.07.04(4)Keywords:
cryptocurrencies; deep learning; forecasting; activation functions; COVID-19; Russian-Ukrainian conflictAbstract
Accurate cryptocurrency price forecasting is crucial due to the significant financial implications of prediction errors. The volatile and non-linear nature of cryptocurrencies challenges traditional statistical methods, revealing a gap in effective predictive modelling. This study addresses this gap by examining the impact of activation functions on neural network models during critical economic periods, specifically aiming to determine how optimising activation functions enhances accuracy in neural network models, including RNN, GRU, LSTM, and hybrid architectures. Using data from January 2016 to June 2022—encompassing stable periods, the COVID-19 pandemic, and the onset of the 2022 Ukraine conflict—we analysed price trends under various market conditions. Our methodology involved testing three activation functions (ReLU, sigmoid, and Tanh) across these models. Both univariate and multivariate analyses were conducted, with the latter incorporating additional metrics such as opening, highest, and lowest prices. The results indicate that optimising activation functions enhances prediction accuracy. Among the models, GRU demonstrated the highest accuracy, whereas RNN was the least efficient. Multivariate models outperformed univariate ones, highlighting the benefits of incorporating comprehensive data. Notably, the Tanh activation function led to the greatest improvements, particularly in underperforming models such as RNN. These findings underscore the critical role of activation function selection in enhancing the predictive power of neural networks for cryptocurrency markets. Optimising activation functions can lead to more reliable forecasts, facilitating better trading decisions and risk management. This study highlights activation functions as key parameters in neural network modelling, encouraging further exploration. Future research could investigate different economic periods and cryptocurrency behaviours to assess model robustness. Additionally, examining a broader range of cryptocurrencies may reveal whether the benefits of activation function optimisation are consistent across various assets. Incorporating external factors such as macroeconomic indicators or social media sentiment could further enhance models and improve forecasting accuracy.
Downloads
References
Ji, Q., Bouri, E., Lau, C. K. M., & Roubaud, D. (2019). Dynamic connectedness and integration in cryptocurrency markets. International Review of Financial Analysis, 63, 257-272. https://doi.org/10.1016/j.irfa.2018.12.002
Zhang, H., Ho, T. B., Lin, M. S. (2004). A Non-Parametric Wavelet Feature Extractor for Time Series Classification. In Advances in Knowledge Discovery and Data Mining; Springer: Berlin/Heidelberg, Germany, 2004; pp. 595–603. https://doi.org/10.1007/978-3-540-24775-3_71
Ong, B., Lee, T. M., Li, G., & Chuen, D. L. K. (2024). Evaluating the potential of alternative cryptocurrencies. In Handbook of digital currency (pp. 79-132). Academic Press. https://doi.org/10.1016/B978-0-323-98973-2.00022-8
Ellahie, A. (2024). Disclosure in Initial Coin Offerings. In The Palgrave Encyclopedia of Private Equity (pp. 1-7). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-030-38738-9_217-1
Hoang, D., & Wiegratz, K. (2023). Machine learning methods in finance: Recent applications and prospects. European Financial Management, 29(5), 1657-1701. https://doi.org/10.1111/eufm.12408
Apicella, A., Donnarumma, F., Isgrò, F., & Prevete, R. (2021). A survey on modern trainable activation functions. Neural Networks, 138, 14-32. https://doi.org/10.1016/j.neunet.2021.01.026
Dubey, S. R., Singh, S. K., & Chaudhuri, B. B. (2022). Activation functions in deep learning: A comprehensive survey and benchmark. Neurocomputing, 503, 92-108. https://doi.org/10.1016/j.neucom.2022.06.111
Singh, B., Patel, S., Vijayvargiya, A., & Kumar, R. (2023). Analyzing the impact of activation functions on the performance of the data-driven gait model. Results in Engineering, 18, 101029. https://doi.org/10.1016/j.rineng.2023.101029
Szandała, T. (2021). Review and comparison of commonly used activation functions for deep neural networks. Bio-inspired neurocomputing, 203-224. https://doi.org/10.1007/978-981-15-5495-7_11
Sun, X., Liu, M., & Sima, Z. (2020). A novel cryptocurrency price trend forecasting model based on LightGBM. Finance Research Letters, 32, 101084. https://doi.org/10.1016/j.frl.2018.12.032
Wang, Y., Wang, C., Sensoy, A., Yao, S., & Cheng, F. (2022). Can Investors’ Informed Trading Predict Cryptocurrency Returns? Evidence from Machine Learning. Research in International Business and Finance, 101683. https://doi.org/10.1016/j.ribaf.2022.101683
Oyedele, A. A., Ajayi, A. O., Oyedele, L. O., Bello, S. A., & Jimoh, K. O. (2023). Performance evaluation of deep learning and boosted trees for cryptocurrency closing price prediction. Expert Systems with Applications, 213, 119233. https://doi.org/10.1016/j.eswa.2022.119233
Akyildirim, E., Goncu, A., & Sensoy, A. (2021). Prediction of cryptocurrency returns using machine learning. Annals of Operations Research, 297(1), 3-36. https://doi.org/10.1007/s10479-020-03575-y
Borges, T. A., & Neves, R. F. (2020). Ensemble of machine learning algorithms for cryptocurrency investment with different data resampling methods. Applied Soft Computing, 90, 106187. https://doi.org/10.1016/j.asoc.2020.106187
Zhang, Z., Dai, H. N., Zhou, J., Mondal, S. K., García, M. M., & Wang, H. (2021). Forecasting cryptocurrency price using convolutional neural networks with weighted and attentive memory channels. Expert Systems with Applications, 183, 115378. https://doi.org/10.1016/j.eswa.2021.115378
Alonso-Monsalve, S., Suárez-Cetrulo, A. L., Cervantes, A., & Quintana, D. (2020). Convolution on neural networks for high-frequency trend prediction of cryptocurrency prices using technical indicators. Expert Systems with Applications, 149, 113250. https://doi.org/10.1016/j.eswa.2020.113250
Cavalli, S., & Amoretti, M. (2021). CNN-based multivariate data analysis for bitcoin trend prediction. Applied Soft Computing, 101, 107065. https://doi.org/10.1016/j.asoc.2020.107065
Chen, Z., Li, C., & Sun, W. (2020). Bitcoin price prediction using machine learning: An approach to sample dimension engineering. Journal of Computational and Applied Mathematics, 365, 112395. https://doi.org/10.1016/j.cam.2019.112395
Jaquart, P., Dann, D., & Weinhardt, C. (2021). Short-term bitcoin market prediction via machine learning. The journal of finance and data science, 7, 45-66. https://doi.org/10.1016/j.jfds.2021.03.001
Alkhodhairi, R. K., Aljalhami, S. R., Rusayni, N. K., Alshobaili, J. F., Al-Shargabi, A. A., & Alabdulatif, A. (2021). Bitcoin Candlestick Prediction with Deep Neural Networks Based on Real Time Data. CMCCOMPUTERS MATERIALS & CONTINUA, 68(3), 3215-3233. https://doi.org/10.32604/cmc.2021.016881
Chen, W., Xu, H., Jia, L., & Gao, Y. (2021). Machine learning model for Bitcoin price prediction using economic and technology determinants. International Journal of Forecasting, 37(1), 28-43. https://doi.org/10.1016/j.ijforecast.2020.02.008
Mudassir, M., Bennbaia, S., Unal, D., & Hammoudeh, M. (2020). Time-series forecasting of Bitcoin prices using high-dimensional features: a machine learning approach. Neural computing and applications, 1-15. https://doi.org/10.1007/s00521-020-05129-6
Mallqui, D. C., & Fernandes, R. A. (2019). Predicting the direction, maximum, minimum and closing prices of daily Bitcoin price using machine learning techniques. Applied Soft Computing, 75, 596-606. https://doi.org/10.1016/j.asoc.2018.11.038
Jang, H., & Lee, J. (2017). An empirical study on modeling and prediction of bitcoin prices with bayesian neural networks based on blockchain information. IEEE Access, 6, 5427-5437. https://doi.org/10.1109/ACCESS.2017.2779181
Al-Nefaie, A. H., & Aldhyani, T. H. (2022). Bitcoin Price Forecasting and Trading: Data Analytics Approaches. Electronics, 11(24), 4088. https://doi.org/10.3390/electronics11244088
Cocco, L., Tonelli, R., & Marchesi, M. (2021). Predictions of bitcoin prices through machine learning based frameworks. PeerJ Computer Science, 7, e413. https://doi.org/10.7717/peerj-cs.413
Tapia, S., & Kristjanpoller, W. (2022). Framework based on multiplicative error and residual analysis to forecast bitcoin intraday-volatility. Physica A: Statistical Mechanics and its Applications, 589, 126613. https://doi.org/10.1016/j.physa.2021.126613
Dutta, A., Kumar, S., & Basu, M. (2020). A gated recurrent unit approach to bitcoin price prediction. Journal of risk and financial management, 13(2), 23. https://doi.org/10.3390/jrfm13020023
Lahmiri, S., & Bekiros, S. (2019). Cryptocurrency forecasting with deep learning chaotic neural networks. Chaos, Solitons & Fractals, 118, 35-40. https://doi.org/10.1016/j.chaos.2018.11.014
Serrano, W. (2022). The random neural network in price predictions. Neural Computing and Applications, 34(2), 855-873. https://doi.org/10.1007/s00521-021-05903-0
Uras, N., Marchesi, L., Marchesi, M., & Tonelli, R. (2020). Forecasting Bitcoin closing price series using linear regression and neural networks models. PeerJ Computer Science, 6, e279. https://doi.org/10.7717/peerj-cs.279
Sebastião, H., & Godinho, P. (2021). Forecasting and trading cryptocurrencies with machine learning under changing market conditions. Financial Innovation, 7(1), 1-30. https://doi.org/10.1186/s40854-020-00217-x
Poongodi, M., Sharma, A., Vijayakumar, V., Bhardwaj, V., Sharma, A. P., Iqbal, R., & Kumar, R. (2020). Prediction of the price of Ethereum blockchain cryptocurrency in an industrial finance system. Computers & Electrical Engineering, 81, 106527. https://doi.org/10.1016/j.compeleceng.2019.106527
Zoumpekas, T., Houstis, E., & Vavalis, M. (2020). Eth analysis and predictions utilizing deep learning. Expert Systems with Applications, 162, 113866. https://doi.org/10.1016/j.eswa.2020.113866
Patel, M. M., Tanwar, S., Gupta, R., & Kumar, N. (2020). A deep learning-based cryptocurrency price prediction scheme for financial institutions. Journal of information security and applications, 55, 102583. https://doi.org/10.1016/j.jisa.2020.102583
Peng, Y., Albuquerque, P. H. M., de Sá, J. M. C., Padula, A. J. A., & Montenegro, M. R. (2018). The best of two worlds: Forecasting high frequency volatility for cryptocurrencies and traditional currencies with Support Vector Regression. Expert Systems with Applications, 97, 177-192. https://doi.org/10.1016/j.eswa.2017.12.004
Fabozzi, F. J., Fallahgoul, H., Franstianto, V., & Loeper, G. (2024). Asymptotic Properties of ReLU FFN Sieve Estimators. Studies in Nonlinear Dynamics & Econometrics, (0). https://doi.org/10.1515/snde-2023-0072
Konak, F., Bülbül, M. A., & Türkoǧlu, D. (2024). Feature selection and hyperparameters optimization employing a hybrid model based on genetic algorithm and artificial neural network: Forecasting dividend payout ratio. Computational Economics, 63(4), 1673-1693. https://doi.org/10.1007/s10614-023-10530-z
Fraszka-Sobczyk, E., & Zakrzewska, A. (2024). The impact of foreign stock market indices on predictions volatility of the WIG20 index rates of return using neural networks. Computational Economics, 1-14. https://doi.org/10.1007/s10614-024-10662-w
Kayim, F., & Yilmaz, A. (2022). Time series forecasting with volatility activation function. IEEE Access, 10, 104000-104010. https://doi.org/10.1109/ACCESS.2022.3211312
Tripathi, B., & Sharma, R. K. (2023). Modeling bitcoin prices using signal processing methods, bayesian optimization, and deep neural networks. Computational Economics, 62(4), 1919-1945. https://doi.org/10.1007/s10614-022-10325-8
Sbrana, A., & Lima de Castro, P. A. (2024). N-BEATS perceiver: a novel approach for robust cryptocurrency portfolio forecasting. Computational Economics, 64(2), 1047-1081. https://doi.org/10.1007/s10614-023-10470-8
Bai, Y., Xie, J., Liu, C., Tao, Y., Zeng, B., & Li, C. (2021). Regression modeling for enterprise electricity consumption: A comparison of recurrent neural network and its variants. International Journal of Electrical Power & Energy Systems, 126, 106612. https://doi.org/10.1016/j.ijepes.2020.106612
Xiao, H., Chen, Z., Cao, R., Cao, Y., Zhao, L., & Zhao, Y. (2022). Prediction of shield machine posture using the GRU algorithm with adaptive boosting: A case study of Chengdu Subway project. Transportation Geotechnics, 37, 100837. https://doi.org/10.1016/j.trgeo.2022.100837
Ortu, M., Uras, N., Conversano, C., Bartolucci, S., & Destefanis, G. (2022). On technical trading and social media indicators for cryptocurrency price classification through deep learning. Expert Systems with Applications, 198, 116804. https://doi.org/10.1016/j.eswa.2022.116804
Dai, Y., Zhou, Q., Leng, M., Yang, X., & Wang, Y. (2022). Improving the Bi-LSTM model with XGBoost and attention mechanism: A combined approach for short-term power load prediction. Applied Soft Computing, 109632. https://doi.org/10.1016/j.asoc.2022.109632
Zhao, L., Li, Z., Qu, L., Zhang, J., & Teng, B. (2023). A hybrid VMD-LSTM/GRU model to predict non-stationary and irregular waves on the east coast of China. Ocean Engineering, 276, 114136. https://doi.org/10.1016/j.oceaneng.2023.114136
Islam, M. S., & Hossain, E. (2021). Foreign exchange currency rate prediction using a GRU-LSTM hybrid network. Soft Computing Letters, 3, 100009. https://doi.org/10.1016/j.socl.2020.100009
Faru, S. H., Waititu, A., & Nderu, L. (2023). A Hybrid Neural Network Model Based on Transfer Learning for Forecasting Forex Market. Journal of Data Analysis and Information Processing, 11(2), 103-120. https://doi.org/10.4236/jdaip.2023.112007
Nti, I. K., Adekoya, A. F., & Weyori, B. A. (2020). A systematic review of fundamental and technical analysis of stock market predictions. Artificial Intelligence Review, 53(4), 3007-3057. https://doi.org/10.1007/s10462-019-09754-z
Wang, Y., Li, Y., Song, Y., & Rong, X. (2020). The influence of the activation function in a convolution neural network model of facial expression recognition. Applied Sciences, 10(5), 1897. https://doi.org/10.3390/app10051897
CoinMarketCap. (2025). www.coinmarketcap.com
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
