Normalization (Min–Max Scaling):

Xscaled(t)=(X(t)−Xmin)/(Xmax−Xmin)

Z-score Standardization:

Xnorm(t)=(X(t)−μX)σX

Sliding Window Representation:

St={X(t−L+1),…,X(t)}

Forget Gate:

ft=σ(Wf·[ht−1,xt]+bf)

Input Gate:

it=σ(Wi·[ht−1,xt]+bi)

Candidate Cell State:

Ĉt=tanh(WC·[ht−1,xt]+bC)

Updated Cell State:

Ct=ft⊙Ct−1+it⊙Ĉt

Output Gate:

ot=σ(Wo·[ht−1,xt]+bo)

Hidden State:

ht=ot⊙tanh(Ct)

Lévy Flight Update Rule:

X(k+1)=X(k)+α·Lévy(λ)

Fitness Function (RMSE Minimization):

RMSE=sqrt((1/N)Σ(yt−ŷt)2)

Population Diversity Update:

P(g+1)=P(g)+β·(Pbest−Prand)

Exploration–Exploitation Switching:

Ifr<p:X(k+1)=X(k)+α·Lévy

Else:X(k+1)=X(k)+γ·(Xbest−X(k))

Mean Absolute Error (MAE):

MAE=(1/N)Σ|yt−ŷt|

Mean Absolute Percentage Error (MAPE):

MAPE=(100/N)Σ|(yt−ŷt)/yt|

Coefficient of Determination (R²):

R²=1−[Σ(yt−ŷt)²]/[Σ(yt−y-)²]

Meteorological and solar irradiance data were sourced from two established databases, NASA POWER and PV Watts. The dataset contains points of GHI, temperature, wind speed, relative humidity, and timestamps in hours.

They are important in the field of solar PV generation modelling as they consider the atmospheric dynamics and variability in solar radiation. Alternatively, methods employing multiple meteorological variables have been proposed in the latest years to improve forecasting accuracy particularly in short-term forecasting applications as discussed in previous studies.^1–4 See Table 2 for a detailed comparative review of related hybrid approaches

.

The data subsequently traversed an inspection pipeline that implemented validation procedures to ensure its cleanliness for utilization in the ML models. The procedure was as follows:

Data Cleansing and Management of Absent Values: Linear interpolation, which succeeded in ensuring consistency in time integrity,²⁵ was employed to address missing occurrences. To mitigate bias in our model, we conduct Z-score analysis (|z| > 3)²⁶ and substitute outliers with the mean numbers of their respective neighborhood based on data type and vendor.

Standardization of Attributes: Given that the model learns inside a local context constrained between 0 and 1, all features were normalized to the relevant range using Min-Max scaling, which is effective for adjusting attribute ranges that differ from the input.²⁷

Partitioning of Training and Testing Data: In accordance with the acceptable literature, the data is divided into learning (70%) and validation (30%) sets.²⁸

Temporal Structuring: Input data were transformed into temporal windows of 24–48 hours to accommodate possible Dependencies across time scales, facilitating the integration of the LSTM/GRU models.^29,30 The proposed HMPCS–ML framework is depicted in Figure 1 and consists of six key stages: data collection, imputation of missing values, outlier detection, normalization, a data splitting stage for train and test, and the temporal sequencing to match the time index to the model input.

To this end, the performance of the proposed hybrid framework was evaluated in two different machine learning (ML) models:

Long Short-Term Memory (LSTM): LSTM networks are one of the recurrent neural networks (RNN) variations capable of learning long-term temporal dependencies in sequential data. Furthermore, they retain the temporal information, making them most suitable for time forecasting issues, such as photovoltaic (PV) power forecasting, in which some meteorological variables exhibit diurnal and seasonal periodontitis.^31,32 To address the vanishing gradient problem, a disadvantage of classical recurrent neural networks (RNNs), LSTMs employ gated mechanisms (input, forget, and output gates), which enable them to learn both long-term and short-term dependencies effectively.³³

Light Gradient Boosting Machine (Light GBM): It is a tree-based ensemble learning algorithm that utilizes gradient boosting decision trees (GBDT), which employs an ensemble of weak learners. Light GBM can achieve fast training and efficient memory usage. Light GBM supports low-latency inference and is known to perform better in the high-dimensional tabular data space than deep neural architectures.³⁴ Because of this feature, it can consider both categorical and continuous features simultaneously, making it a powerful associative model compared to LSTM in solar forecasting tasks. Light GBM is also well-known for its low computational cost in many renewable energy applications, as well as its inherent resilience to over-fitting, provided suitable regularization is added.³⁵

We built a system that combines long short-term memory (LSTM) as a neural sequence model to exploit globally optimal attribute combinations, with Light GBM as an ensemble method to efficiently process heterogeneous meteorological inputs, by leveraging the complementary advantages of high-performing modelling methods.

The HMPCS approach was combined with both ML models as a hyper-parameter optimization engine to enhance their forecasting performance. While the standard Cuckoo Search (CS) focuses solely on replacing the worst-performing cuckoos identified as such, HMPCS differs in that it maintains multiple evolving sub-populations throughout a single optimization run. This allows for an effective trade-off between exploration (i.e. global search over parameter space) and exploitation (i.e. local search around promising solutions).

So, the way the optimization process works is: Each sub-population evolves in isolation but periodically exchanges elite solutions to avoid premature convergence.

→ Lévy flight randomization to diverge the searching course of the path that can be used by candidate solutions to get out of the local minima. HMPCS encapsulates large population structures (with associated diversity) at the cost of speed, due to the need for many reproducible runs, while retaining the most optimal hyper-parameters.

In this work, we directly optimize predictive accuracy and generalization by using the negative cross-validation score of the target ML model as the fitness function.

HMPCS Parameter Settings:

Number of subpopulations: 3

Per subpopulation (N = 20):

α: Lévy flight parameter: 1.5

Maximum generations: 50

It was focused on important hyper-parameter optimization. These were:

LSTM: Learning rate, hidden layers, neurons on layer, dropout and sequence length.

Regularization limits and n_ “estimators,” max_depth, “learning rate” for Light GBM.

We propose a framework that utilizes HMPCS to automatically fine-tune LSTM models for both seen and unseen data,⁸ achieving high accuracy over manually tuned or grid-searched baselines.

To evaluate the robustness of the proposed HMPCS–ML framework, synthetic datasets were generated. Fortunately, this is relatively straightforward to accomplish in an environmentally responsible manner. That’s why CLAES can still be used for some near-real-time measurements of the Earth’s atmosphere.

The synthetic data were formed by adding controlled Gaussian noise and perturbations to actual meteorological variables|that is irradiance, temperature, wind speed. This maintains the sequential nature of the data plus variability necessary (to simulate Uncertain real world). The synthetic dataset generation can be expressed as:

Xsynthetic=Xreal+ε

where ε is Gaussian noise with zero mean and controlled variance. This is a way to assess model stability under Varying degrees of uncertainty while still being consistent with realistic photovoltaic patterns.

Four experimental analyses were conducted to assess the stability and the generalization performance of the proposed hybrid HMPCS–ML framework. Use of publicly available meteorological datasets that guarantee reprehensibility and transparency of the results. This experiment measures the performance of the proposed framework against the best existing hyper-parameter tuning. Besides Classic Methods we compared also HMPCS with Grid Search (high computational cost, due to large search area) and Particle Swarm Optimizations (PSO, fast and popular swarm intelligence technique, more suitable for low-non-adaptability in high-dimension search space). The analysis was performed using typical error factors: Mean Absolute Error (MAE), Root Mean Square Error (RMSE), Mean Absolute Percentage Error (MAPE), and Coefficient of Determination (R²). Finally, as an additional exploration analysis, convergence curves (i.e., the optimization dynamics of HMPCS in contrast to PSO & Grid Search were also; ~ 10 lines for each comparison + maps + figs).

Using this comparative framework, HMPCS also has the following advantages: Parallel populations to speed up convergence System Architecture-periodic sharing of elite solutions to avoid local minima-adaptive hyper-parameter tuning for a few critical hyper-parameters enabling Forecasting accuracy improvement for LSTM and Light GBM models. Thus, the evidence from these experiments, in addition to direct demonstration of how HMPCS outperforms the previous strategies in optimizing, indicates that the ability of HMPCS to maintain the balance between exploration and exploitation yields the most accurate predictions with the lowest amount of cost in computation.

Table 3 clearly shows that HMPCS outperformed both Grid Search and PSO in all model configurations. The LSTM + HMPCS model achieved the lowest RMSE (0.139) and highest R² (0.93), indicating superior predictive performance. This validates the use of HMPCS as a robust optimizer for fine-tuning ML-based solar forecasting models.

It can be observed from Table 4 that HMPCS gave satisfactory results compared to Grid Search and PSO in all models. The LSTM + using the test dataset, it was shown that RMSE is standardized mean square error which measure one unit root null and alternate for HMPCS model with 0.139 lowest level metric score from validation data. The proposed LSTM+HMPCS model gives the lowest RMSE (0.139) and highest R² (= 0.93) among all other optimization methods using the test dataset, illustrating superior overall forecasting accuracy compared to Grid Search method as well as Particle Swarm Optimization method³⁴ implemented in the present study. These results validate HMPCS as an effective method for enhancing predictive performance and model generalization. This justifies the use of HMPCS as a powerful optimizer to fine-tune ML-based solar forecasting models.

We adopted a cross-validation strategy over time-series to achieve sound evaluation and overcome oversaturation. Specifically, we followed a rolling-origin (walk-forward) validation approach in which the training window is gradually extended and the model evaluated on subsequent unseen segments of data.

A 5-fold time-series cross-validation scheme was adopted (with gaps between each period) so that the order of observation is kept for temporal stability. Quite unlike normal k-folds cross-validation.

Each experiment was repeated 10 independent times with fixed random seeds to ensure reproducibility and stability of the results.

The average performance metrics (MAE, RMSE, MAPE, R²) were reported with standard deviations to give a relatively solid verdict on model performance.

In all experiments, models optimized with HMPCS consistently outperformed both their standalone models and fine-tuned models with Grid Search or Particle Swarm Optimization (PSO).

With improved by approximately 23% on average across multiple independent runs, with statistically significant differences confirmed using the Wilcoxon test, exemplifying the gains through hyper-parameter tuning. Likewise, LSTM+HMPCS also showed an improved RMSE and MAE, indicating that, as only the hyper-parameters were optimized, effective hyper-parameter optimization can decrease error rates and improve the model’s capability to generalize in changing weather conditions.³⁷

We further visualized the improvements using bar charts and comparative plots. Results from these showed that both Grid Search and PSO improved upon baseline performance to a degree; however, HMPCS provided the best trade-off between accuracy and speed. The better convergence of HMPCS was due to the use of a multi-population strategy and elite solution exchange in HMPCS, as it helps to escape from local minimum and to search the parameter space more thoroughly. Overall, the results validate HMPCS as a fast, powerful and robust optimizer with measurable benefits in RMSE and robustness over the current state of the art optimization methods in the renewable energy forecasting space.

3.11.1 Statistical validation

To guarantee that the claimed improvements made by the proposed HMPCS-ML architecture are statistically real, a series of non-parametric statistical tests were carried out.

Specifically, using the Wilcoxon signed-rank check, the efficiency of HMPCS–ML algorithms was compared to those of other optimization methods in wide use, such as Grid Search and Particle Swarm Optimization (PSO), over multiple independent runs.

Each test was done ten times and the resulting performance indicators (RMSE and R²) were recorded. Where differences in performance occurred, a Wilcoxon test was applied to find out whether they were statistically important.

Most of the obtained p-values were less than 0.05, proving that this HMPCS-ML system has made significant improvements over the benchmark systems compared with it.

These findings confirm that the beneficial performance gains observed are robust and not simply due to random fluctuation.

As Table 5 documents, stated p-values confirm the new HMPCS–ML approach has performed much more significantly than previous types of approaches.

3.11.2 Ablation study

Moreover, the study design was employed to examine the respective roles of different components in the proposed framework.

Specifically, three different model configurations were contrasted:

The results show the baseline LSTM model offers reasonable forecasting performance; however, optimization via CS will increase accuracy by tuning parameters better. And conversely, proposed HMPCS approach reaches best record, indicating that multi-population strategy has significant impact on balance between exploitation and exploration, themselves Combining to ensure system doesn’t mistakenly jump into a local minimum too fast before data points suggest an altogether different picture.

These results demonstrate that it is not only due to the forecasting model that the proposed HMPCS approach, outperforms (Optimized)’s CS–ML above) shows the above performance.

As shown in Table 6, the proposed HMPCS optimization provides the best performance compared to both the baseline model and the standard CS-based optimization.

Compared with traditional optimization methods, the HMPCS consumes a little bit more time in running. Yet, this additional computational expense is compensated with a substantial gain in forecasting accuracy, robustness and convergence performance.

Figure 2 the forecasting (RMSE and MAE) performance of baseline models (LSTM, Light GBM) and combined with HMPCS. HMPCS-enabled optimization resulted in the most pronounced increases in accuracy for both LSTM and Light GBM, as shown in the results section. Especially for LSTM+HMPCS, there is a significant reduction in RMSE and MAE compared to the version without optimization, while Light GBM+HMPCS achieves a 23% reduction in RMSE. The results highlight the effectiveness of HMPCS in hyper-parameter tuning, convergence acceleration, and generalization of machine learning models for renewable energy prediction.

Experimental results show that the proposed HMPCS–ML hybrid framework outperforms both baseline models and traditional optimization strategies.

3.11.3 Experimental setup

To ensure the reproducibility and transparency of the HMPCS–ML system presented in this study, we give a detailed description of our used experiments in this section.

Datasets: The data used for our study were real-world photovoltaic datasets obtained from NASA POWER and PVWatts platforms. The dataset consists of meteorological variables such as Global Horizontal Irradiance (GHI), temperature, wind speed, and humidity.

The total dataset consists of approximately 8,760 samples collected over a one-year period. Data Handling: All input features were normalized using Min-Max Scaler. Missing values were filled in using linear interpolation.

Model setting: LSTM: Number of layers: 2, Hidden units: 64

Batch size: 32, Epochs: 100, Optimizer: Adam, Learning rate: 0.001

LightGBM: Number of leaves: 31, Learning rate: 0.05, Number of estimators: 100

HMPCS Parameters: Population size: 25, Subpopulations count: 3, Discover rate (pa): 0.25

Initial step size (a): 0.01, Maximum number of generations to run: 100

Hardware and Procedure: All the experiments in this paper were implemented using Python, TensorFlow and LightGBM libraries on a system with an Intel Core i7 processor, 16 GB of memory, and the option to use a GPU for acceleration.

Reproducibility: To help assure the stations and reliability of our experimental results, we used random seeds which were then fixed and repeated each of our different experiments 10 times in a row.

Description of Data Sets: The data set used in this study was an hourly one consisting of 8,760 samples of meteorological and photovoltaic observations that were collected over the course of a year. Input features data Global Horizontal Irradiance (GHI), temperature, wind speed and humidity.

Before incorporating data in the final study, establish a threshold value. As such, further designs and parameters of lysis methods (compart-mental, concurrent response or parametric) would be left entirely to user preference at this stage. The data split into training (70 percent), validation (15%) and testing (15%) sets while maintaining the original order of observations.

3.11.4 Evaluation metrics

Due to the various facets of model performance assessed, we applied the following evaluation metrics:

These metrics together provide an overall assessment of the prediction accuracy (using RMSE & MAE) and generalization ability (using R²) of the proposed forecasting framework. The multiple complementary measures used in the study ensure that the benefits of HMPCS are not restricted to one performance measure, but rather comprise strong, generalization, and practically essential improvements in performance across several dimensions of evaluation.

Future work may incorporate a comprehensive sensitivity analysis to further investigate the effect of HMPCS parameters on forecasting accuracy and model robustness, following established optimization methodologies reported in the literature.³⁶

Though indicating some good results this paper has several limitations. At First, the experiments were performed on just a few sets of data, little different in any actual photovoltaic routine could possibly not be simulated accurately enough. Second, when compared to other solutions with simpler computations, the computational cost of our HMPCS framework is higher. Thirdly, this performance was tested under controlled experimental conditions and additional testing in the field is necessary to deal with very large installations in industrial sites.