Forecasting for a time series signal carrying single pattern characteristics can be done properly using function mapping-based principle by a well-designed artificial neural network model. But the performances degraded very much when time series carried the mixture of different patterns characteristics. The level of difficulty increases further when there is a need to predict far time samples. Among several possible mixtures of patterns, the trend-cycle time series is having its importance because of its occurrence in many real-life applications like in electric power generation, fuel consumption and automobile sales. Over the mixed characteristics of patterns, a neural model, suffered heavily in getting generalized learning, in result poor performances appeared over test data. To overcome this issue in this work, a decomposition-based approach has been applied to separate the component patterns of trend and cyclic patterns, and a dedicated model has been developed for predicting the individual data patterns. The linear characteristic of the trend data pattern has been modeled through a linear regression model while the nonlinearity behavior of cyclic pattern has been model by an adaptive radial basis function neural network. The final predicted outcome has been considered as the linear combination of individual model outcomes. The Gaussian function has been considered as the kernel function in the radial basis function neural network because of its wider and efficient applicability in function mapping. The performance of the neural model has been improved very much by providing the adaptive value of spreads and centers of basis function along with weights values. In this paper, two different applications of forecasting in the area of electric power demand by the individual house and month-wise annual power generation have been considered. Based on house characteristics parameters, the power demanded by a house have been considered which carried a moderate complexity of function mapping problem while in another case, total power generation needed to be predicted on the monthly basis for a year from just the previous year observation, which carried the mixed behavior of trend and cyclic pattern. For house power demand forecasting the adaptive kernel-based radial basis function has shown very satisfactory performances and much better against static kernel radial basis function and multilayer perceptron neural network. The integrated approach of neural model and linear regression has shown very efficient outcomes for the mixture pattern while individual neural models were failed to do so. The coefficient of determination (R2) has been applied to estimate the quality of predicted outcomes and comparison.
In the last few decades, a popular research topic is the analysis of time series which has attracted attention a great deal. A complex system that is characterized by observed data uses a time series analysis recognized as a powerful tool. In numerous fields like pattern recognition, clustering, classification, and prediction, the time series analysis approach is used by researchers. A dynamic role is played by time series analysis, time series forecasting in various practice applications. Finance, energy, electricity load and tourism are the example of various applications of forecasting. However, even though it’s difficult but there is always a need to improve the performance of the forecast. An impressive result is obtained by the forecast model combinations for accuracy and reliability than the result of single forecasting methods used. The forecasting models combination provides better accuracy in forecast outcomes and carried low variability along with simplicity in model designing. Superior performance by forecast combination can occur because of the following four points (a) individual forecast model may have the best-trained model but maybe not very efficient in predicting the future values (b) Component forecasts generally characterize the data-generating process of the time series data from different and to a certain extent of complementary perspectives, (c) with the adaptive strategy for forecast combination can use the partial solution cooperatively, (d) various obvious limitations associated with an individual model like a structural break, uncertainty in the model and inappropriate specification of the model, etc. can be mitigated with the use of forecast combination. In short, the supportive interaction of individual forecasts in forecast combination reduces the limitations of individual forecast up to great extent and makes the final forecast more accurate and robust. In general, there is a rising tendency that has been observed in electric energy demands and generation because of growth in the economy and technological evolution. There is also some seasonal fluctuation embedded in this trend which carried the characteristics of cyclic. Over the last several decades, electric energy demand has significantly increased to power generation. Electricity demand depends upon economic variables, national circumstances, and as well as on climatic conditions. In most electricity systems, the residential sector is one of the main providers to the peak loads, and electricity demand in other domains grows at a smooth pace and is comparatively steady. Energy consumption in the residential sector depends upon the building envelope, and the occupant’s consumption behaviors, which are affected by seasonal factors and show holistic fluctuation. Economic activities have made tremendous progress, in few domains like public security manufacturing, transportation, and the required energy demand has been directed by the state’s economic situation and the policy, it also includes market supply and demand signals. As we know a resource that cannot be stored is electric power. Hence the amount of consumption should be equal to supply. The functions in different fields of society will be affected due to the insufficient supply. Hence the electric utility industry is seriously influenced by neither the shortage of consumption nor shortage of supply. Forecasting the electric demand plays a significant role by guaranteeing the usability of the electric energy by the electricity users wherever and whenever required. Forecasting also provides an advanced estimate requirement of electricity to electricity companies to ensure optimal electric energy allocation to prevent the electricity power network collapse in large areas and to avoid wastage of resources. By forecasting the guidelines and advanced estimate helps the electric companies to further develop and expand. From the above analysis, the future demand for electricity in the developing trend is affected by various arbitrary factors such as manufacturing, Policies in Industry, societal changes, economic development, seasonal issues, etc. Developing a single prediction model is not realistic by taking all the factors influential into account. There might be errors or even model failure which is unavoidable by ignoring any one of these elements. When such a complicated situation is faced involving non-negligible factors, questioning how to improve forecasting accuracy becomes increasingly significant. There is need of advanced intelligence tools such as machine learning and statistical forecasting techniques to forecast the demand for electricity in both residential areas and industries where energy consumption is high. Till now, conventional prediction methods can be roughly divided into several types: regression methods (in this model the limitation is depended on the reliability and availability of variables which are independent over the forecasting time, thus demanding further efforts in data estimation and collection), time series methods (a typical model in time series is ARIMA, which require the past and historical data to forecast the variable of interest of its future evolution behavior). The above sad methods not only need a huge amount of historical data but also need typical distribution to use in statistical methods to analyze system characteristics. One such tool is artificial neural networks (ANNs), which can outperform traditional time series forecasting techniques such as multiple regression or autoregressive integrated moving average (ARIMA) models.
ANNs are able to adapt nonlinearity and estimate complex relationships without wide data or knowledge, unlike traditional predicting/forecasting approaches. A neural network is an interconnected network of artificial neurons, each connection is associated with a different weight. In contrast to traditional forecasting approaches, neural networks can estimate complex mapping functions between their input and output spaces without any pre-specified model. The ability of neural networks to provide an estimated function that comes from its weighted connectionist structure and is modified through a learning algorithm.
In this paper, the workflow has been divided into different sections. Section 2 contains the literature survey while Section 3 represents the forecasting problem definition. The proposed model of neural network has been discussed in Section 4 while the experiment corresponding to function mapping-based electricity demand forecasting has presented in Section 5. Decomposition process-based trend-cyclic time-series data prediction has been discussed in Sections 6 and 7 while the conclusion has presented at the end.
In the past, several researchers have given attention to time series forecasting using different methods. It has been proposed many strategies on a seasonal pattern exhibiting time series forecasting in [
For forecasting application, there is always an obvious choice is to have supervised learning based neural network model which are usually very good for curve fitting and function mapping. For a predefined set of inputs and targets, a learning mechanism is applied to minimize the error between the outputs and targets for a defined set of inputs by obtaining the optimal set of neural network weights. Once the network has achieved enough learning, the acquired knowledge can be utilized to perform the interpolation. Radial basis function has often been utilized to map the non-linear characteristics. Functionally in RBF (Radial Basic Function), there are generally three layers namely the input layer which carried the inputs which may be already preprocessed if needed, the hidden layer whose neurons carried the nonlinear transfer functions, and finally, the output layer whose neurons accumulated the output of the hidden layer neurons to generate the output. In mathematical form, the output of the RBF neural network can be expressed as
where
where,
where
One of the adjustable parameters of the network is obtained with its output layer weights mostly by a fixed center method RBFNN. This approach is easy, but to perform sampling of input adequately, a designated enormous variety of centers should be obtained from the input data set. This produces a reasonably massive network. For the fixed centered RBF neural network, the responsibility of learning is completed by adjusting the output layer weights values which cause difficulty in the learning as well as slow progress. It can be observed that the characteristics of the Gaussian kernel are ruled by its center value and spreads. Considering the basis function parameters also as variables in the process of learning can provide significant benefits in the learning process. Hence in this work along with output layer weights, centers, and spreads of Gaussian basis function also changes to make the learning optimal and faster. The error function of RBF NN can be defined by
Use of gradient can be very efficient in the search of better parameters values and can be defined through
where
Experiments have been given to predicting the output based on two different approaches: (i) prediction based on the development of mapping function (ii) prediction of complex time-series data carried the trend and cyclic characteristics together.
In this work, two different areas of research have focused on, development of better neural architecture and how to develop a better predictor model from this architecture under different complexity of forecasting. To obtained the comparative quality of neural architecture, their learning characteristics and the final mean error have been considered while performances stability has been decided by obtaining the average performance under 10 trials with standard deviation. The quality of predicted outcomes has been determined by the coefficient of determination (R2) which is given by
where
First, consider a forecasting problem from a mapping point of view where for given inputs in one domain to mapped the output in the other domain. The complexity of the problem appears in the forming of the mapping function. The area of application has been considered as the demand for power supply by houses contains various indexes of power consumption like the size of houses, the capacity of air conditioning machines, and the appliance index of the house which included various machines and devices cause of power consumption. The details of the data set have given in Appendix A. Broadly there parameters are good enough to capture a close approximate of the real demand of power consumption by individual houses with this 3 parameters model has to develop to predict the demand of power consumption. To get the details of different possible models by neural network the choice of feed-forward architecture is obvious with supervising learning. Hence, two different architectures, multilayer perceptron-based feed-forward (MLP) and radial basis function neural network have been considered. Gradient-based learning has been applied in both neural models. The size of both networks has been considered as [3, 3, 1] there were uni-polar sigmoid transfer functions in the MLPNN active nodes while Gaussian function in the RBF NN. The learning and momentum rate values were 0.3 in the weight up-gradation of MLPPNN and RBF. Low value ensures smoothness in the learning process which is needed in the appropriate mapping function development. The two different strategies have been applied in model development with RBFNN. First, one, called static RBF (SRBF) where spreads and center of Gaussian function remain fixed with the progress of learning and only output layer weights updated while other models are called adaptive RBF (ARBF), where all the three parameters were updated (as given in section 3). All three network models have given chance to evolved up to 1000 iterations and to mitigate the random variation, 10 independent trials have been given with individual models. The convergence characteristics over 10 independent trials for MLPNN, SRBF, and ARBF have shown in
The convergence for SRBFNN over 10 trials has shown the complexity of learning because of the availability of single parameters (output layer weights) for error minimization. Convergence is very fast but having diversity in nature and heavily depends upon the initialization process of available parameters in the network that is why some convergence was mature while some were immature as shown in
The convergence characteristics of ARBF in
Over the test data set, performances of ARBF were superior and very close to expected values as shown in
The obtained numeric values of standard deviation in the absolute error, mean absolute error (MAE), and estimated R2 values for all the three models over data set for training and test have shown in
MLP | SRBF | ARBF | ||||||
---|---|---|---|---|---|---|---|---|
MAE | Std. Dev | R^{2} | MAE | Std. Dev | R^{2} | MAE | Std. Dev | R^{2} |
0.0324 | 0.0235 | 0.9093 | 0.5005 | 0.3277 | 6.1419 | 0.0244 | 0.0224 | 0.9828 |
0.0329 | 0.0239 | 0.9078 | 0.1160 | 0.1265 | 1.1511 | 0.0249 | 0.0222 | 0.9785 |
0.0332 | 0.0242 | 0.9056 | 0.4906 | 0.2337 | 7.9471 | 0.0248 | 0.0206 | 0.9769 |
0.0322 | 0.0236 | 0.9119 | 0.4106 | 0.3260 | 4.9059 | 0.0250 | 0.0213 | 0.9709 |
0.0320 | 0.0238 | 0.9139 | 0.2120 | 0.1524 | 3.3169 | 0.0229 | 0.0211 | 0.9900 |
0.0321 | 0.0234 | 0.9126 | 0.0706 | 0.0847 | 0.8701 | 0.0240 | 0.0203 | 0.9993 |
0.0337 | 0.0248 | 0.9019 | 0.0631 | 0.0455 | 0.9648 | 0.0247 | 0.0206 | 0.9762 |
0.0322 | 0.0234 | 0.9109 | 0.0813 | 0.0945 | 0.8539 | 0.0196 | 0.0184 | 0.9773 |
0.0327 | 0.0238 | 0.9087 | 0.0473 | 0.0523 | 0.9879 | 0.0242 | 0.0214 | 0.9721 |
0.0320 | 0.0238 | 0.9140 | 0.1691 | 0.1396 | 2.5987 | 0.0246 | 0.0224 | 0.9831 |
0.0325 | 0.0237 | 0.9095 | 0.1576 | 0.1227 | 1.2367 | 0.0232 | 0.0204 | 0.9791 |
MAE | Std. Dev | R^{2} | |
---|---|---|---|
MLP | 0.0255 | 0.0206 | 1.2152 |
SRBF | 0.1655 | 0.0951 | 4.2892 |
ARBF | 0.0171 | 0.0122 | 1.0115 |
MLP | SRBF | ARBF | ||||||
---|---|---|---|---|---|---|---|---|
MAE | Std. Dev | R^{2} | MAE | Std. Dev | R^{2} | MAE | Std. Dev | R^{2} |
0.0255 | 0.0206 | 1.2127 | 0.3837 | 0.2496 | 8.4195 | 0.0176 | 0.0122 | 0.9862 |
0.0263 | 0.0209 | 1.2127 | 0.0711 | 0.0539 | 1.1830 | 0.0197 | 0.0156 | 0.9872 |
0.0268 | 0.0210 | 1.2122 | 0.5050 | 0.1266 | 15.6371 | 0.0180 | 0.0140 | 1.0184 |
0.0249 | 0.0203 | 1.2159 | 0.2268 | 0.2294 | 5.0025 | 0.0180 | 0.0121 | 1.0094 |
0.0241 | 0.0201 | 1.2224 | 0.1942 | 0.1213 | 5.8201 | 0.0158 | 0.0111 | 1.0169 |
0.0251 | 0.0204 | 1.2144 | 0.0337 | 0.0288 | 1.2048 | 0.0166 | 0.0115 | 1.0357 |
0.0275 | 0.0211 | 1.2107 | 0.0566 | 0.0322 | 1.3491 | 0.0160 | 0.0125 | 1.0364 |
0.0251 | 0.0204 | 1.2149 | 0.0620 | 0.0375 | 1.0479 | 0.0158 | 0.0115 | 1.0164 |
0.0260 | 0.0207 | 1.2135 | 0.0284 | 0.0240 | 1.1107 | 0.0161 | 0.0105 | 1.0201 |
0.0242 | 0.0202 | 1.2233 | 0.1522 | 0.0995 | 4.3092 | 0.0180 | 0.0123 | 0.9896 |
0.0255 | 0.0205 | 1.2150 | 0.1073 | 0.0438 | 2.0968 | 0.0167 | 0.0114 | 1.0098 |
The proposed models have been applied in a more complex environment where time-series data (carried the trend and cycle pattern together) have been considered for prediction. To make the prediction more complex there is no immediate relation establishment made by considering the problem of predicting data which are far over a while. Hence there is no benefit of placing the time-lag model where only a next time data sample enters in the input domain in association with previously established correlation by other input samples. In this work, the prediction of net power generation every year by U.S electric companies has been considered for the application and the details have given in Appendix B. The present problem of forecasting has been defined as: “forecast the net power generation month wise for a year from just previous year information”. There were 10 years of data from 1985 to 1995 having the net power generation from the month of January to December every year. Let’s first investigate the characteristic of data as shown in
Over this time series data a neural predictor model ARBF, as discussed in Section 3.1, has been developed which has an architecture size of [12, 10, 12] over the training of the first 8 years of data. The predicted performance over the training and test data have shown in
To overcome this limitation a decomposition method has been proposed which worked in two stages. In the first stage, the trend pattern as observed by considering the mean year power generation as shown in
First, there is the normalization of data which makes the upper limit of data as 1 while positive bias has applied further by adding the positive value to make all value positive only. The same sign of data will make the learning process faster. The same previously considered ARBFNN has been applied to form the cyclic pattern neural predictor. In the test phase, both predictors’ outputs combined. Post-processing is the reverse process of pre-processing and carried the function of the level downshifting and de-normalization. The developed predictor from this process has given a name as DARBFNN which is a hybrid predictor of simple regression and ARBFNN.
The comparative prediction performances by different models over the training data have shown in
The numeric performance comparisons by different models over the training data and test data have shown in
MAE | Std. Dev | R^{2} | |
---|---|---|---|
MLP | 17.4219 | 11.0384 | 0.3736 |
SRBF | 29.1765 | 30.4068 | 4.7719 |
ARBF | 0.8927 | 0.7814 | 0.9961 |
DARBF | 4.6926 | 4.1138 | 0.9552 |
MAE | Std. Dev | R^{2} | |
---|---|---|---|
MLP | 18.7283 | 14.1239 | 0.0089 |
SRBF | 146.0433 | 34.9241 | 41.3499 |
ARBF | 32.1408 | 20.0807 | 2.3040 |
DARBF | 8.5999 | 5.4114 | 0.5554 |
The restriction of static basis function parameters is caused by slow and unsatisfactory learning in the conventional RBF neural network. To overcome this issue adaptive approach has been proposed in the basis function parameters along with weights as result there were three degrees of freedom in error minimization. The prediction based on function mapping was very satisfactory with the use of MLPNN and ARBFNN based predicting models while these models had shown difficulties in prediction of mixed pattern of trend and cyclic in a time series data. The difficulty of the individual predictor has been minimized by the decomposition approach where the component of mixed time series has been separated and the corresponding predictor has been developed. Forecasting application of power demand and power generation has been considered. With the experimental analysis, benefits of ARBF have been confirmed against SRBF or MLPNN in function mapping based forecasting while has been used as a successful component predictor for cyclic pattern prediction in association with linear regression which has been used for increasing trend predictor to predict the outcomes of mix pattern time series. In future work, to develop a better predictor model through linear regression, the coefficients of linear regression can be explored in solution space through computational intelligence to increase the level of accuracy.
This research work has been completed in Manuro Tech Research Pvt. Ltd., Bangalore, India, under the program of Innovation with Computational Intelligence (ICI).