3.1 Data collection and Preprocessing

The crop recommendation dataset, available at Kaggle (https://www.kaggle.com/siddharthss/crop-recommendation-dataset/), contains vital agricultural parameters such as rainfall, crop yield, nitrogen (N), phosphorus (P), potassium (K) levels, temperature, humidity, pH, and precipitation. To ensure high-quality inputs, data preprocessing is performed—addressing missing values (denoted by '.'), removing duplicates, and standardizing the structure. Missing entries are replaced with distinct large negative values to help models identify them as outliers. As the dataset lacks class labels, supervised learning labels are generated based on crop yield and cultivated area: crops with an area > 0 are labeled as class 1, otherwise class 0.

Sensors are used to monitor soil moisture, pH, temperature, humidity, and rainfall, which are essential for accurate crop recommendations. Additionally, mobile cameras detect crop infections to suggest suitable fertilizers, with all data securely stored in the cloud. The dataset is split into training and testing sets, and deep learning (DL) algorithms are used to train the crop recommendation model effectively.

3.2 MSSA algorithm for feature selection

In this proposed modification of the SSA (MSSA) [31], the concept of the local best solution is introduced. The mathematical model of the MSSA is kept unchanged compared to SSA for its leader salp position update, while the followers salp positions are updated using the following equation:

where $b_j^i$ is the $i^{th}$ local best position in the $j^{th}$ dimension, $x_j^i$ is the $i^{th}$ position in the $j^{th}$ dimension, $x_j^q$ is a randomly selected position in a $j^{th}$ dimension from a set of N solution. dimension. While the best fitness values $B_i$ are assigned the corresponding outcomes $F_i$'s, the best local solutions $b_i$'s are first made equal to the first created population. As the least (or greatest) fitness function value $F_i$ and its accompanying solution $x_i$, the global optimum G and its corresponding solution g are chosen.

Inside the main loop, Eq. (1) is used to update the leader's salp position in [31], and Eq. (1) is used to calculate the new follower salp's locations. The newly created positions' feasibility is confirmed, and the associated fitness function is assessed and stored. By comparing the recently acquired best local optimums to the previously stored global optimum, the MSSA's final step finds a new global optimum. Once a predetermined maximum number of generations has been achieved, the iteration ends. In contrast to SSA, MSSA incorporates information on the local best in each individual of the population into its exploration techniques. This could be useful if the food supply becomes trapped in a local minimum and enable the finding of other food sources using the modified MSSA method. However, each neighborhood of the local best is additionally exploited using a step size that is established by comparing the associated local position $x^i$ with a random individual $x^q$. This enables the neighborhood search area surrounding the local best to be expanded or reduced.

Each solution is represented by a binary value ("1" denoting a feature that was selected and "0" denoting features that were not), and a random population of salps is produced. The food fitness of each solution (climate feature) is then calculated by MSSA. Next, the food source is selected from the solution with the lowest fitness value. As a result, it uses the K-Nearest Neighbor (KNN) based fitness goal function to choose the best answer. In the subsequent phase, MSSA uses Eq. (1) in [31] to modify the salp positions. Using climate data, the fitness value is updated at each cycle to update the optimal solution and enhance the relevant feature. The optimal solution is provided as a vector of binary values once the maximum number of iterations has been reached. On the dataset's testing segment, KNN makes use of the chosen features. It is used in conjunction with the KNN classifier to assess it on feature selection problems. Therefore, MSSA is used to identify feature combinations that use the fewest features possible while maximizing classification precision. Using training data, the fitness function is changed to improve crop prediction performance on the validation dataset.

Therefore, using climatic data as a reference, the salp with the highest fitness score is modified to improve the chosen features. Salps with the highest total fitness score related to food discovery process are superior, and optimal values for each salp (matching to climate features) are established. The MSSA method uses a small set of attributes to find feature combinations that maximize classification accuracy. In the feature space, every feature has a distinct dimension that ranges from 0 to 1.

3.3 Long Short Term Memory Network

A specific kind of Recurrent Neural Network called LSTM was created to solve the vanishing gradient problem and learn long-term dependencies. Every LSTM cell has a hidden state (ht) for short-term output and a cell state (Ct) for long-term memory. Three gates are used, namely:

The forget gate determines which data from the prior cell state should be discarded.

The input gate determines which additional data should be added to the cell state.

The input gate decides what portion of the cell state is output.

Cell state Update equation:

Hidden State (Output):

Each gate controls the cell state using a tanh function and permits or prohibits information flow using a sigmoid function. Retained old memory and newly acquired information are combined to update the cell state.

3.4 Crop recommendation using BiLSTM Network Ensemble via Adaptive Weighting method

This research employs a BiLSTM Network Ensemble via Adaptive Weighting to predict optimal crop conditions by learning from historical data. The system enables farmers to quickly receive accurate crop recommendations based on climate and crop preferences. LSTM networks are well-suited for handling sequential data due to their ability to capture long-term dependencies through memory cells and gated structures. These networks efficiently extract high-level temporal features using recurrent hidden layers, making them effective in representing evolving input sequences [21]. LSTMs operate over time-series data by using a sequence length parameter, which reflects changes in input vectors across time steps. Each memory block includes a memory cell, input gate, forget gate, and output gate. The memory cell retains temporal knowledge, while the gates manage the flow of information. At each time step t, the LSTM updates the memory cell $c_t$ and generates a hidden state $h_t$. These operations are governed by a set of equations that define the internal dynamics of modern LSTM models with forget gates [22].

where $W_{mn}$ represents the weight of the link between gate $m$ and $n$, and $b_n$ acts as a bias parameter to be learned, where $m\in x,h$ and $n\in i,f,o,c$. Furthermore, $\oplus$ represents the Hadamard product, $\sigma$ designates the normal logistic sigmoid function, and $\varphi$ represents the hyperbolic tangent function. $\sigma \left(x\right)=1/\left(1+e^{-x}\right)$; $\varphi \left(x\right)=\left(e^x-e^{-x}\right)/\left(e^x+e^{-x}\right)$. The input, forget, and output gates are indicated as $i_t$, $f_t$, and $o_t$, respectively; $c_t$ represents the internal state of the memory cell $c$ at time $t$. It's hidden layer at time $t$ is represented by the vector $h_t$, whereas $h_{t-1}$ represents the values output by each memory cell in the hidden layer at the previous time step.

This paper proposes an innovative BiLSTM ensemble recommendation approach to improve prediction accuracy. The method combines outputs from multiple BiLSTM models by dynamically adjusting their weights based on previous prediction errors and a forgetting factor. These adaptive weights enhance the ability of the system to generate accurate recommendations. Each LSTM network in the ensemble serves as a base predictor [23]. To promote diversity and capture complex data patterns, individual networks are trained with varying sequence lengths and memory cell configurations. This variation allows the ensemble to better model non-linear dependencies and complement each other's strengths, resulting in more precise and robust predictions.

Assuming a total of M LSTM models, their combined prediction for the time series, denoted as ($y^{\left(1\right)},y^{\left(2\right)},\dots ,y^{\left(N\right)}$) with N observations, can be shown as:

The recommendation output (at the kth time stamp) derived using mth LSTM model denoted by ${\hat{y}}_m^k$, while the related combining weight is $w_m$. Each weight $w_m$ corresponds to certain model's output. We have $0\le w_m\le 1$ and ${\sum }_{m=1}^M{w}_m=1$.

BiLSTM is an advanced form of LSTM that processes input sequences in both forward and backward directions. This dual processing allows BiLSTMs to capture contextual information from both past and future elements, making them highly effective for sequence modeling tasks. A BiLSTM consists of two LSTM layers: one reads the input from start to end, while the other reads from end to start. Their outputs are combined at each time step, offering a richer understanding of the sequence. BiLSTMs excel in capturing long-range dependencies and are especially useful in tasks like time series prediction, natural language processing, and speech recognition, where full context is essential for accurate predictions.

3.4.1 Adaptive weights for BiLSTM models

A distinctive weight determination method is used to accommodate the changing dynamics of the underlying time series data in a flexible manner. When the regression errors of individual estimators exhibit zero mean and are uncorrelated, their weighted aggregation achieves minimal variance by assigning weights inversely proportional to the variances of the estimators [26]. This principle forms the theoretical foundation of our weight determination solution. A novel strategy for weight determination is utilized, with the goal of dynamically capturing the evolving dynamics of the underlying time series. The computation of the combining weights is conducted recursively:

$w_m^{k=1}=w_m^k+\lambda \Delta w_m^k,\text{for}m=1,\dots ,M$

where $\lambda =0.3$, and $\Delta w_m^k$ is computed according to the inverse prediction error of the respective BiLSTM base model:

$$\Delta w_m^k=\frac{\frac{1}{{ϵ}_m^k}}{\frac{1}{{ϵ}_m^1+\frac{1}{{ϵ}_m^2+\frac{1}{{ϵ}_m^3+\dots .+\frac{1}{{ϵ}_m^k}}}}}$$

The ${ϵ}_m^k$ is related to past prediction error measured up to the $k^{th}$ time step:

$${ϵ}_m^k=\sum _{t=k-V+1}^k{\gamma }^{k-t}{e}_m^t$$

where , $1\le V\le k$, and $e_m^t$ denotes the prediction error at each time step $t$ of the $m$th LSTM model, ${ϵ}_m^k$ is computed by constructing a sliding time window comprising the last $V$ prediction errors. The introduction of the forgetting factor $\gamma$ serves to reduce the influence of older prediction errors. Through updating weights over time, we can determine these weights by examining inherent patterns in consecutive data forecasting attempts. This approach is suitable as it circumvents the need for complex optimization to ascertain adaptive weights.

Using the preceding Eq., the weights evolve over the time series encompassing $T$ time steps (where $K=1,\dots ,T$), culminating in the ultimate weights $w_m^T$ (where $m=1,\dots ,M$). Eventually, the weight designated to each model is calculated as:

$$w_m=\frac{w_m^T}{{\sum }_{n=1}^M{w}_n^T}form=1,\dots ,M$$

The weights computed satisfy the constraints such that $0\le w_m\le 1$ and ${\sum }_{m=1}^M{w}_m=1$[27].

The equations presented indicate that adaptive weight based amplification is effectively carried out within the LSTM. Therefore, it proves instrumental in generating precise crop recommendation outcomes [27]. Figure 1 shows the general architecture of AWBiLSTM model.

Figure 1 General architecture of AWBiLSTM model.

4.1 Multivariable Analysis

Soil pH, nitrogen (N), phosphorus (P), potassium (K), organic carbon, temperature, rainfall, humidity, sunshine hours, soil type index, and area ID were among the important variables that were included in the multivariable analysis for crop recommendation. Principal component analysis (PCA) was performed to reduce dimensionality and maintain 95% of the variance using the top six principal components, while multiple linear regression (MLR) was utilized to evaluate the effect of these factors on crop yield. Additionally, the analysis used the variance inflation factor (VIF) to check for multicollinearity and the analysis of variance (ANOVA) to assess the significance of the grouped variables. The AWBiLSTM ensemble model was used to improve prediction after the MSSA selected features based on the PCA results. The main conclusion of the analysis were that temperature, rainfall, and soil pH all significantly affected crop yield (p < 0.01). Furthermore, when multivariable features were used instead of raw input data, AWBiLSTM performance increased.