Hierarchical Ensemble-based Feature Selection for Time Series Forecasting

This paper is about a novel ensemble approach for feature selection based on hierarchical stacking in cases of non-stationarity and a limited number of samples with a large number of features. Our approach exploits the co-dependency between features using a hierarchical structure. Initially, a machine learning model is trained using a subset of features, and then the model’s output is updated using another algorithm with the remaining features to minimize the goal. This hierarchical structure allows for flexible depth and feature selection, resembling post-processing techniques in computer vision. By exploiting feature co-dependency hierarchically, our proposed approach overcomes the limitations of traditional feature selection methods and feature importance scores. The effectiveness of the approach is demonstrated on synthetic and real-world datasets, indicating improved accuracy and stability compared to traditional methods and state-of-the-art approaches. The relatively high performance on different datasets proves the scalability and robustness of the hierarchical ensemble-based approach.