Department of Electrical and Electronics Engineering, Sri Sivasubramaniya Nadar College of Engineering, Chennai, India.
Abstract
Accurate district-level forecasting of sugarcane yield is critical for effective planning and management of this important crop. This study develops a novel hybrid Hidden Markov Model–Random Forest (HMM–RF) framework for forecasting sugarcane yield across India. The framework integrates hidden agro-climatic production regimes identified using a Gaussian Hidden Markov Model with the nonlinear predictive capability of Random Forest regression. The model was developed using 16,172 district-year observations covering 1966–2017 across 311 districts and 20 states. Historical productivity, climatic, agronomic, irrigation, fertilizer, infrastructure, and economic variables were used as predictors, together with temporal features derived exclusively from preceding observations. Model evaluation employed leakage-free chronological validation, with 1966–2010 used for training and 2011–2017 reserved for testing. The proposed HMM–RF achieved the best testing performance, with R² = 0.9754, RMSE = 5.56 tonnes per hectare MAE = 1.55 tonnes per hectare and MAPE = 4.11%. Compared with the benchmark Random Forest model, HMM–RF reduced RMSE by 22.1% and MAE by 41.6%. Statistical validation further confirmed significant performance differences among the evaluated models. SHAP analysis showed that historical productivity variables were the dominant predictors, while HMM-derived production regimes, climatic variables, and management-related factors provided complementary predictive information.
2.Chlingaryan, A., Sukkarieh, S. and Whelan, B. 2018. Machine learning approaches for crop yield prediction and nitrogen status estimation in precision agriculture: A review. Comput. Electron. Agric. 151:61–69. https://doi.org/10.1016/j.compag.2018.05.012
5.Hyndman, R.J. and Athanasopoulos, G. 2021. Forecasting: Principles and practice. 3rd ed. OTexts, Melbourne, Australia. https://otexts.com/fpp3/
6.International Crops Research Institute for the Semi-Arid Tropics. 2024. District Level Database (DLD). ICRISAT Development Center, Hyderabad, India. https://data.icrisat.org/district-level-data/
7.Jabed, M.A. and Murad, M.A.A. 2024. Crop yield prediction in agriculture: A comprehensive review of machine learning and deep learning approaches, with insights for future research and sustainability. Heliyon 10(24):e40836. https://doi.org/10.1016/j.heliyon.2024.e40836
8.James, G., Witten, D., Hastie, T. and Tibshirani, R. 2021. An introduction to statistical learning: With applications in R. 2nd ed. Springer, Cham, Switzerland. https://doi.org/10.1007/978-1-0716-1418-1
9.Jeong, J.H., Resop, J.P., Mueller, N.D., Fleisher, D.H., Yun, K., Butler, E.E., Timlin, D.J., Shim, K.-M., Gerber, J.S., Reddy, V.R. and Kim, S.-H. 2016. Random forests for global and regional crop yield predictions. PLoS One 11(6):e0156571. https://doi.org/10.1371/journal.pone.0156571
11.Khosravani Shariati, S.A. and Abbasi, A. 2025. Machine learning-based winter wheat yield prediction using multisource data. Agric. Water Manage. 322:109951. https://doi.org/10.1016/j.agwat.2025.109951
12.Kouame, A.K.K., Heuvelink, G.B.M. and Bindraban, P.S. 2025. Unraveling drivers of maize (Zea mays L.) yield variability in Ghana: A machine learning approach. Comput. Electron. Agric. 237:110647. https://doi.org/10.1016/j.compag.2025.110647
13.Kuhn, M. and Johnson, K. 2019. Feature engineering and selection: A practical approach for predictive models. 1st ed. Chapman and Hall/CRC, Boca Raton, FL. https://doi.org/10.1201/9781315108230
15.Rabiner, L.R. 1989. A tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE 77(2):257–286. https://doi.org/10.1109/5.18626
16.Shawon, S.M., Ema, F.B., Mahi, A.K., Niha, F.L. and Zubair, H.T. 2025. Crop yield prediction using machine learning: An extensive and systematic literature review. Smart Agric. Technol. 10:100718. https://doi.org/10.1016/j.atech.2024.100718
18.Sun, J., Di, L., Sun, Z., Shen, Y. and Lai, Z. 2019. County-level soybean yield prediction using deep CNN-LSTM model. Sensors 19(20):4363. https://doi.org/10.3390/s19204363
19.Wilson, G.T. 2016. Time series analysis: Forecasting and control, 5th ed., by G.E.P. Box, G.M. Jenkins, G.C. Reinsel and G.M. Ljung. J. Time Ser. Anal. 37(5):709–711. https://doi.org/10.1111/jtsa.12194
Articles in Press, Accepted Manuscript Available Online from 13 September 2026
D,G S and R,J . (2026). Forecasting Sugarcane Yield at the District Level in India using a Hybrid Hidden Markov-Random Forest Model. (e29268). Journal of Agricultural Science and Technology, (), e29268
MLA
D,G S , and R,J . "Forecasting Sugarcane Yield at the District Level in India using a Hybrid Hidden Markov-Random Forest Model" .e29268 , Journal of Agricultural Science and Technology, , , 2026, e29268.
HARVARD
D G S, R J. (2026). 'Forecasting Sugarcane Yield at the District Level in India using a Hybrid Hidden Markov-Random Forest Model', Journal of Agricultural Science and Technology, (), e29268.
CHICAGO
G S D and J R, "Forecasting Sugarcane Yield at the District Level in India using a Hybrid Hidden Markov-Random Forest Model," Journal of Agricultural Science and Technology, (2026): e29268,
VANCOUVER
D G S, R J. Forecasting Sugarcane Yield at the District Level in India using a Hybrid Hidden Markov-Random Forest Model. J. Agric. Sci. Technol.. 2026;():e29268.
Journal of Agricultural Science and Technology