In the establishment of differential equations, the determination of time-varying parameters is a difficult problem, especially for equations related to life activities. Thus, we propose a new framework named BioE-PINN based on a physical information neural network that successfully obtains the time-varying parameters of differential equations. In the proposed framework, the learnable factors and scale parameters are used to implement adaptive activation functions, and hard constraints and loss function weights are skillfully added to the neural network output to speed up the training convergence and improve the accuracy of physical information neural networks. In this paper, taking the electrophysiological differential equation as an example, the characteristic parameters of ion channel and pump kinetics are determined using BioE-PINN. The results demonstrate that the numerical solution of the differential equation is calculated by the parameters predicted by BioE-PINN, the Root Mean Square Error (RMSE) is between 0.01 and 0.3, and the Pearson coefficient is above 0.87, which verifies the effectiveness and accuracy of BioE-PINN. Moreover, real measured membrane potential data in animals and plants are employed to determine the parameters of the electrophysiological equations, with RMSE 0.02-0.2 and Pearson coefficient above 0.85. In conclusion, this framework can be applied not only for differential equation parameter determination of physiological processes but also the prediction of time-varying parameters of equations in other fields.

Differential equations describing dynamical processes are widely utilized in physics [

In recent years, Physics-Informed Neural Networks (PINN) have achieved good performance in solving complex differential equations [

How to predict time-varying parameters in differential equations is a problem we need to handle at present. However, traditional parametric prediction methods are usually based on linear and non-time-varying premises, which is not easy to achieve for the time-varying parameter determination of general nonlinear systems. Although PINN allows us to obtain time-varying parameters, conventional PINN is slow to converge in dealing with inverse problems of complex differential equations. In addition, the values of membrane potential and switch rate in HH are not in the order of magnitude, requiring a long training time. In summary, existing PINN cannot predict the time-varying parameters of HH accurately. For this purpose, this paper proposes a novel time-varying parameter estimation framework for electrophysiological models based on physical information neural networks, named BioE-PINN. This framework offers a total loss function based on the Hodgkin-Huxley equation, observed data errors, and hard constraints on initial conditions. The training time can be reduced, and the prediction accuracy can be improved of the model after modifying the ion channel switch rate equation. Furthermore, the stimulation currents that evoke membrane potentials can be predicted.

The contributions of this paper are summarized as follows:

1. We propose a new PINN-based framework for time-varying parameter estimation in electrophysiological differential equations that combines physical prior knowledge of bioelectrodynamics with advanced deep learning. The loss function of a neural network is carefully designed to not only perfectly predict observations (i.e., data-driven losses), but also take into account the underlying physics (i.e., physics-based losses).

2. We employ hard constraints to accommodate differences between orders of magnitude and adaptive activation functions to accelerate the convergence of BioE-PINN.

3. We verify the effectiveness of the model with simulation experiments and real experimental data, using membrane potential to predict the conductivity and switch rate parameters of ion channels and pumps of electrophysiological equations.

The rest of this paper is organized as follows.

Parameter inference from observational data is a core problem in many fields and quite an important research direction in process modeling. Traditional parameter prediction methods are usually based on linearity and time-invariance, but the parameter determination of general nonlinear systems is not easy to achieve. The neural network has the advantages of strong approximation ability, a high degree of nonlinearity, strong self-learning, and self-adaptation, and has been widely exploited in the parameter determination of nonlinear systems.

In recent years, with the further development of deep learning and computing resources, a deep learning algorithm framework called Physics-informed Neural Network has been proposed [

In PINN, multiple loss terms appear corresponding to the residuals, initial values, and boundary conditions of the equation, and it is critical to balance these different loss terms. By reviewing the latest literature, we found that the imbalance of loss functions in PINNs can be addressed by employing meta-learning techniques. And it can be applied to discontinuous function approximation problems with different frequencies, advection equations with different initial conditions and discontinuous solutions [

In addition, researchers have developed a large number of software libraries designed for PINNs, most of which is based on Python, such as DeepXDE [

The A-PINN framework uses a multi-output neural network to simultaneously represent the main variables and equation integrals in nonlinear integrodifferential equations, solving integral discdiscretization and truncation errors. Since A-PINN does not set fixed grids and nodes, unknown parameters can be discovered without interpolation [

PINN also has applications in the parameter determination of electrophysiological equations. For example, to capture multi-channel data related to cardiac electrical activity from high-dimensional body sensor data, a modified P-DL framework based on PINN is proposed. The framework adds a balancing metric to optimize the model training process. P-DL assists medical scientists in monitoring and detecting abnormal cardiac conditions, handling the inverse ECG modeling problem, that is, predicting spatiotemporal electrodynamics in the heart from electrical potential [

The above studies show that PINN has been continuously improved since its original proposal and has been applied to differential equation solving and predicting constant parameters. The above also provides a feasible indication for applying PINN to the time-varying parameter estimation method of physiological models in this study.

In this section, we propose a physics-based neural network to estimate parameters in electrophysiological equations. Compared with traditional PINN, our proposed BioE-PINN contains hard constraints and an adaptive activation function method, which can improve the accuracy of PINN.

The core mathematical framework for current biophysical-based neural modeling was developed by Hodgkin and Huxley. It is one of the most successful mathematical models of complex biological processes that have been proposed so far [

When using PINN to predict the switch rate, it is first necessary to construct a feedforward neural network to approximate the electrical signal or rate. A feedforward neural network consists of an input layer, multiple hidden layers, and an output layer. Information is passed layer by layer in one direction. The value of any neuron in the network is calculated by the sum of the products of the weights and outputs of the previous layer, and then activated by a specific activation function. The relationship between two adjacent layers is usually described by the following equation:

This paper aims to solve the inverse problem of membrane potential

The data-driven loss function is designed to solve the inverse problem. In order to infer the parameters in the differential equation based on the membrane potential, the predicted value of the neural network needs to be close to the real membrane potential. We define the data-driven loss:

Physiologically based constraints will be implemented by minimizing the magnitudes of

To calculate the loss function during DNN training, it is necessary to obtain the derivatives of

When conventional PINN solves time-dependent ordinary differential equations or inversion of parameters, the constraints of initial conditions are mainly in the form of soft constraints, which will affect the calculation accuracy to a certain extent [

Hard constraints based on initial conditions will be achieved by minimizing the size of

Regarding the activation functions in neurons, there are currently Sigmoid, Logistic, tanh, sin, cos, and so on, as well as ReLU, and various variants based on ReLU, ELU, Leakly ReLU, PReLU, and so forth. The purpose of introducing the activation function is to make the neural network have the characteristics of expressing nonlinearity. The activation function plays an important role in the training process of the neural network, because the calculation of the gradient value of the loss function depends on the optimization of the network parameters, and the optimization learning of the network parameters depends on the derivability of the activation function.

The size of a neural network is related to its ability to express the complexity of a problem. Complex problems require a deep network, which is difficult to train. In most cases, choosing an appropriate architecture based on the experience of the rest of the scholars requires a large number of experimental approaches. This paper considers tuning the network structure for optimal performance. To this end, a learnable parameter

The learnable parameter

The learnable parameter

The learning rate has a dramatic impact when searching for a global minimum. Larger learning rates may skip the global minimum, while smaller learning rates increase computational cost. In the training process of PINN, a small learning rate is generally used, which makes the convergence slow. In order to speed up the convergence, a scaling factor

After establishing the PINN network for estimating the time-varying parameters of the electrophysiological equation, combined with the improved adaptive activation function algorithm in this paper, the prediction of the switching rate parameters in the electrophysiological equation can be realized. The algorithm is summarized as follows:

The main factors affecting the time complexity of the BioE-PINN algorithm are the number of iterations

In order to evaluate the error of predicted and observed values, the following indicators are exploited: Spearman correlation coefficient (SPCC) and Root Mean Square Error (RMSE). In addition, Relative Errors are employed to evaluate the effect of parameter prediction.

SPCC is used to measure the correlation between two variables, especially shape similarity. RMSE reflects the degree of difference between the predicted value and the real value, and a smaller RMSE means that the accuracy of the predicted value is higher.

In this section, we first use conventional PINN and BioE-PINN to predict the parameters of HH. Specifically, we obtain the observation data through simulation, determine the HH parameters according to the observation data and compare them with the real parameters. Subsequently, we conduct experiments on the inference of electrophysiological equation parameters in animals as well as plants, and solve the HH equation according to the parameters determined by BioE-PINN. To demonstrate the effectiveness of the proposed method, various comparisons are performed. In this study, we implement BioE-PINN with the PyTorch [

The first step in the simulation experiment is to generate observation data, namely the membrane potential V. Specifically, we first compute the numerical solutions of

(a) A constant current, where

(b) A step function with a long pulse, such as

(c) A multi-pulse pulsing step function, such as

(d) A sine function, such as

We briefly discuss the sensitivity of BioE-PINN to the number of neurons and layers of DNN models. The performance is found to be sensitive to the size of the DNN (neurons

Neurons |
RMSE |
RMSE |
RMSE |
RMSE |
RMSE |
RMSE |
---|---|---|---|---|---|---|

64 |
0.621 | 0.728 | 0.612 | 0.734 | 0.318 | 0.391 |

64 |
0.245 | 0.294 | 0.545 | 0.625 | 0.264 | 0.221 |

128 |
0.125 | 0.241 | 0.331 | 0.361 | 0.148 | 0.183 |

128 |
0.025 | 0.003 | 0.329 | 0.243 | 0.013 | 0.065 |

256 |
0.131 | 0.235 | 0.462 | 0.413 | 0.164 | 0.189 |

The optimal structure of BioE-PINN is selected to predict the switch rate under different stimulation currents, and the results are shown in

Stimulation type | RMSE |
RMSE |
RMSE |
RMSE |
RMSE |
RMSE |
---|---|---|---|---|---|---|

Constant current | 0.025 | 0.003 | 0.329 | 0.243 | 0.013 | 0.065 |

Long pulse current | 0.029 | 0.007 | 0.331 | 0.219 | 0.021 | 0.071 |

Short pulse current | 0.018 | 0.002 | 0.137 | 0.016 | 0.012 | 0.053 |

Sine function current | 0.012 | 0.004 | 0.192 | 0.095 | 0.032 | 0.074 |

Stimulation type | RMSE |
SPCC |
---|---|---|

Constant current | 0.183 | 0.881 |

Long pulse current | 0.291 | 0.913 |

Short pulse current | 0.116 | 0.959 |

Sine function current | 0.093 | 0.973 |

Stimulation type | SPCC |
SPCC |
---|---|---|

Constant current | 0.864 | 0.865 |

Long pulse current | 0.902 | 0.931 |

Short pulse current | 0.943 | 0.958 |

Sine function current | 0.982 | 0.972 |

Stimulation type | RMSE |
SPCC |
---|---|---|

Constant current | 0.132 | 0.941 |

Long pulse current | 0.151 | 0.978 |

Short pulse current | 0.172 | 0.969 |

Sine function current | 0.183 | 0.997 |

The squid giant axon is part of the squid’s jet propulsion control system. In 1952, after experiments on giant axons, Hodgkin and Huxley made assumptions about the dynamic characteristics of ion channels and uncovered the ion current mechanism of action potentials for the first time. Therefore, the BioE-PINN is also validated with the membrane potential data of the Squid giant axon and applied to predict the time-varying parameters and stimulation currents of the HH equation.

Specifically, we first use the

Parameter | Nominal value | Description |
---|---|---|

Membrane capacitance | ||

Maximum Na |
||

Maximum delayed rectifier K |
||

Maximum leak channel conductance | ||

Na |
||

K |
||

Leak reversal potential |

The PINN-based framework can be used to solve the HH equation and determine the parameters in the equation. This allows us to try this method for model solving of plant electrical signals. Plant electrical signaling is a rather complex process, and the ion channels involved in its formation have not been fully discovered. Therefore, it is of great significance to help biologists to discover the dynamic characteristics of plant ion channels using computational methods. At present, scientists have found potassium, calcium, chloride, and hydrogen ions in the cell membranes of higher plants. Based on previous studies [

Parameter | Nominal value | Description |
---|---|---|

Membrane capacitance | ||

Maximum delayed rectifier K |
||

Maximum Ca |
||

Maximum Cl |
||

Maximum H |
||

Maximum leak channel conductance | ||

K |
||

Ca |
||

Cl |
||

H |
||

Leak reversal potential |

The

The second column of

The current mechanism of ions involved in the formation of electrical signals in plants needs to be further studied. There are many mechanistic models that cause plant electrical activity, but each parameter of a model often comes from different plants [

To further verify the reliability of the model, we derived the conductance change curves of potassium and chloride channels from patch-clamp data extracted from the literature. Specifically, the whole-cell voltage clamp data in [

When the membrane potential changes, it can be obtained from

We bring

The above equation is applied to find the switch rate of the activation factor

In this paper, we developed a framework for identifying the time-varying parameters of physiological equations based on improved PINN, which integrates the electrophysiological equations into the loss function of the neural network and adds hard constraints to make the predicted output of the neural network conform to physical laws. The difference between our framework and traditional PINN is that we add an adaptive activation function to improve the accuracy of the model and slow down the convergence. Finally, simulation experiments prove that BioE-PINN can predict the dynamic characteristics of ion channels and pumps by using membrane potential, such as gating variables, conductivity and other parameters. At the same time, we verified the effectiveness and accuracy of the model with real electrical signal data of animals and plants. In future work, it is possible to consider more complex physiological equation modeling. In addition, how to promote BioE-PINN to different fields to obtain application value is also worth studying.

This work was supported by the National Natural Science Foundation of China under 62271488 and 61571443.

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

^{+}influx