1 Extreme Dissipation, Optimal Control, and the Least Action Principle
Tải bản đầy đủ - 0trang
44
The Common Extremalities in Biology and Physics
The Lagrange function in the elementary one-dimensional case is
L5m
x_2
2 UðxÞ;
2
ð2:2Þ
where mðx_2 =2Þ is the kinetic term and U(x) is the potential.
Then the corresponding EulerÀLagrange equation can be written as:
x€ 5 2
1 @U
:
m @x
ð2:3Þ
The Hamiltonian will be
H5
x_2
p2
1 UðxÞ 5 m 1 UðxÞ
2m
2
ð2:4Þ
and the canonical system (2.4) is
@H
5 p=m;
@p
@H
@U
52
:
p_ 5 2
@x
@x
x_ 5
ð2:5Þ
For example, in a simple one-dimensional harmonic oscillator, when the elastic
force originates from the quadratic potential
UðxÞ 5 k
x2
;
2
ð2:6Þ
the Lagrangian will be
_ 5m
Lðx; xÞ
x2
x_2
2k :
2
2
ð2:7Þ
The equation of motion is then
m x€ 1 kx 5 0 or x€ 5 2
k
x:
m
ð2:8Þ
Using the Legendre transform, we can build the Hamiltonian
Hðx; pÞ 5
p2
x2
1k
2m
2
ð2:9Þ
Some General Optimal Control Problems Useful for Biokinetics
45
and the canonical system (2.5) is
x_ 5 p=m;
p_ 5 2kx:
ð2:10Þ
The mechanical motion described by Eqs. (2.7)À(2.9), as it is well known, is
referred to as harmonic motion. The solution of the equation
rﬃﬃﬃﬃ !
rﬃﬃﬃﬃ !
k
k
xðtÞ 5 aUexp i
t 1 bUexp 2 i
t
m
m
ð2:11Þ
can be represented in the explicit form of the sum of sine and cosine functions:
rﬃﬃﬃﬃ !
rﬃﬃﬃﬃ !
k
k
t 1 bUcos
t :
x 5 aUsin
m
m
The surface plots and the lines of identical levels (contour plots) for the case
considered above in Eqs. (2.2)À(2.11) are indicated in Figure 2.1. The surface
plot of the Lagrangian is shown in Figure 2.1A with its lines of equipotential
levels (Figure 2.1B). The surface plot of the Hamiltonian, Eq. (2.9), is shown
in Figure 2.1C and phase plot of this Hamiltonian is presented in Figure 2.1D,
which are the phase trajectories. One can note the essentially different character
of the lines of identical levels from Figure 2.1B and D, where the motion of
the system is represented on the plane of the coordinate x and its derivative dx/dt.
Because of that, the Hamiltonian is the first integral, when its lines of identical
levels are also the phase trajectories of the motion. A cyclic character of this
motion is seen, which is expressed as periodic dependence of the deviation x on
the time t.
So, the curves in Figure 2.1D indicate the cyclic transformations in state and
costate variables without any attenuation or relaxation, and this is confirmed by the
trajectories of equipotential lines of the first integral or energy in its physical
interpretation:
E5
@L
x2
x_2
x_2
x_ 2 L 5 m 1 UðxÞ 5 m 1 k 5 Const:
2
2
2
@x_
ð2:12Þ
This expression is considered in mechanics as the law of conservation of energy;
therefore, dissipation in the corresponding physical system is not observed.
Let us note that in mechanics, the square-law dependence of the kinetic term on
the velocities indicates a symmetry of motion in relation to some directions in space
[1]. The explicit independence of the kinetic term from the mechanical coordinates
means the uniformity of mechanical space and, consequently, the uniformity of
kinematical laws in the different frame of references.
46
The Common Extremalities in Biology and Physics
(A)
(B) 2.00
0
1.0
2.00
L 0.
00
0.00
00
X
0.
0
.0
0
.0
–1
–1.00
–1.
1.
00
0.
X
0
.0
–1
0
00 .0
2. –2
0
00
00
00
1.
.0
2.
00
X
–0
00
2.
0
0
.0
–0
–2.0
–0
–2.00
–2.00
–
0
.0
–0
–1.
00
1.00
0
1.0
–1.00
1.00
0.00
2.00
X
(C)
(D) 2.00
1.00
00
0
X
0.0
0
0.00
1.00
2.0
0
1.00
1.00
4.0
E
00
2.
2.
00
0
.0
0.
00
–2.00
–2.00
–1.00
–2
.0
–20
.0
0
–1
–1
.0
0
.0
0
X
0.
1.00
00
X
–1.00
2.
00
1.
1.
00
00
2.
2
–0.00
2.0
1.00
0
2.00
X
Figure 2.1 (A) The surface plot of the Lagrangian (2.7) with quadratic potential. (B) The
lines of identical levels of the Lagrangian. (C) The surface plot of corresponding
Hamiltonian (2.9) with quadratic potential (2.5). (D) The lines of identical levels for the
Hamiltonian (energy) E, (k 5 1, m 5 1).
2.1.2
On Optimal Control Formulation of Mechanics
Let us consider this simple classical mechanics case (Eqs. (2.1)À(2.11)) in the OC
formulation in a more detailed way compared to that of Section 1.3, then
ðτ
ðu 2 ueq Þ2
ðx 2 xeq Þ2
2k
dt-extr:
ð2:13Þ
m
S5
2
2
t0
subject to
x_ 5 u;
xðt0 Þ 5 x0 :
ð2:14Þ
Some General Optimal Control Problems Useful for Biokinetics
47
The integrand in the functional is similar to the classical mechanics Lagrangian
in the sense of definition of signs of these parts. Then to apply the Pontryagin maximum principle [1], the optimal control Hamiltonian needs to be built
H 5 2m
ðu 2 ueq Þ2
ðx 2 xeq Þ2
1k
1 pu;
2
2
ð2:15Þ
and the Pontryagin conditions for this problem from Eqs. (2.13) and (2.14) will be
x_ 5 u; xÃ ðt0 Þ 5 x0 ;
p_ 5 2
@H
5 2kðx 2 xeq Þ;
@x
@H
5 2mðu 2 ueq Þ 1 p 5 0;
@u
ð2:16Þ
HðxÃ ; uÃ ; pÃ Þ $ HðxÃ ; u; pÃ Þ
or
x_ 5
p
1 ueq ; xÃ ðt0 Þ 5 x0 ;
m
p_ 5 2kðx 2 xeq Þ;
ð2:17Þ
HðxÃ ; uÃ ; pÃ Þ $ HðxÃ ; u; pÃ Þ
when xeq 5 0 and ueq 5 0, this system coincides with Eq. (2.10).
Because the integrand function in Eq. (2.13) and the right part in the constrained
equation (2.14) are not dependent on time explicitly, the Hamiltonian H is constant
along an optimal trajectory; in other words, it is the first integral to the problem,
Eq. (2.16), which in mechanics is interpreted as energy.
Formally ueq means that the kinetical picture is shown in the frame of reference
(one-dimensional) moving with a velocity ueq (the variable u could be considered
as a dummy variable). If we choose ueq 5 0, it is formally relevant to the picture in
the resting frame of reference. Clearly, all our consideration is nonrelativistic and
is employed just to discuss analogy to dissipative mechanics. The phase plot for
this example and trajectories are shown in Figure 2.2. One notes that qualitatively
it coincides with the example from the previous section, where it was considered in
terms of the pure variational method.
2.1.3
Dissipation in Classical Mechanics
For biological systems where kinetics is known by the irreversible character of
energy transformation processes, dissipation is normal. Therefore, one should turn
directly to those equations of mechanics that also show dissipation.
In mechanics, dissipative effects can be included by the introduction of the friction forces that are proportional to the velocity of mechanical motion [1]
48
The Common Extremalities in Biology and Physics
(A) 1
λ 0.625
(B) 4
X
0.0
0
E=0
λ 2
X
U
–1
U
0
–2
–3
–1
0
1
2
X
λ
–2
–0.375
3
(C) 2
U
1
5
(D)
10
15
Time
X
2
20
0.0
–0.375
0
0
0.625
–1
–2
–2
–3
–2
–1
0
λ1
5
10
15
Time
20
Figure 2.2 Graphical interpretation of the dynamic system (2.16). (A) The phase plane x, p
for system (2.16). (B) Kinetic plot of x, p, u parameters against dimensionless time. (C) The
phase plane λ, u. (D) Kinetical curves for different values of the first integral (energy):
xeq 5 0, ueq 5 0.
fitr 5 2
X
αik x_k :
ð2:18Þ
k
This equation can be rewritten as a derivative
fitr 5 2
@F
@x_i
ð2:19Þ
from a quadratic form
F5
11X
αik x_i x_k
2 2 i;k
ð2:20Þ
and then the dissipative forces can be added to right-hand parts of the
EulerÀLagrange equations
dL @L
@L @F
52
2
dt @xi
@xi @x_i
ð2:21Þ
Some General Optimal Control Problems Useful for Biokinetics
49
and then, for example, the equation for the one-dimensional harmonic oscillator
will be written considering the friction force:
mx€ 1 γ x_ 1 kx 5 0:
ð2:22Þ
When one can ignore m in this equation, the motion is strongly dissipative and
Eq. (2.22) can be reduced to:
γ x_ 1 kx 5 0;
ð2:23Þ
and the motion in the system or the relaxation of the system occurs as follows:
k
xðtÞ 5 x0 exp 2 t :
γ
ð2:24Þ
In chemical and biological kinetics, one can deal with similar circumstances.
Kinetic equations in the majority of cases look like those in mechanics in the case
of strong dissipation and are represented by first-order equations. Therefore, it
seems reasonable to consider the applications of the variational approach to biological kinetics through those in mechanics. However, we need to bear in mind that
the dependence of free energy on the first concentration derivatives violates the
principle of local equilibrium. Therefore, following mechanics, one can try to form
the Lagrange function in two parts: one part plays the role of a potential and the
other a kinetic role.
2.1.4
On an Alternative Way to Describe Biological and Chemical
Dissipation
Nevertheless, our consideration concerns the outline of the extreme method (related
to the least action principle) for nonconservative (chemical and biological) kinetics,
e.g., the description of systems with explicitly expressed dissipation.
Therefore, simple classical mechanics case from Eqs. (2.13) and (2.14) can be
reformulated in a form more relevant to the optimal control and dissipative kinetics
(see Section 1.3), then
S5
ðτ
t0
ðu 2 ueq Þ2
ðx 2 xeq Þ2
1k
dt-min
m
2
2
ð2:25Þ
xðt0 Þ 5 x0 ;
ð2:26Þ
subject to
x_ 5 u;
xðτÞ 5 xeq ;
where x is the extent from the equilibrium and when the terminal time τ is not a
specified. Then the integrand in the functional is the classical mechanic’s
50
The Common Extremalities in Biology and Physics
Lagrangian. Then applying the Pontryagin maximum principle, we could write the
OC Hamiltonian
H 5 2m
ðu 2 ueq Þ2
ðx 2 xeq Þ2
2k
1 pu;
2
2
ð2:27Þ
and the Pontryagin maximum principle conditions for this problem will be
x_ 5 u; xÃ ðt0 Þ 5 x0 ;
p_ 5 2
xÃ ðτÞ 5 xeq :
@H
5 kðx 2 xeq Þ;
@x
ð2:28Þ
@H
5 2mðu 2 ueq Þ 1 p 5 0;
@u
HðxÃ ; uÃ ; pÃ Þ $ HðxÃ ; u; pÃ Þ;
p#0
or
x_ 5
p
1 ueq ; xÃ ðt0 Þ 5 x0 ;
m
xÃ ðτÞ 5 xeq :
ð2:29Þ
p_ 5 kðx 2 xeq Þ;
HðxÃ ; uÃ ; pÃ Þ $ HðxÃ ; u; pÃ Þ;
p # 0:
Because the integrand function in Eq. (2.25) and the right-hand part in the constrained equation (2.26) are not dependent on t explicitly, the Hamiltonian H is a
constant along an optimal trajectory; in other words, it is a first integral to the problem (2.16).
If we choose the ueq 5 0, we imply that the control amplitude in the equilibrium
obviously tends to zero, and then the numerical solutions of this problem are shown in
Figure 2.3. One can see from this figure that the character of motion changes dramatically when the sign in the Lagrangian changes for the part that is the penalty for not
being in the equilibrium. In Eqs. (2.13) and (2.25), this penalty and control both have
a quadratic form for simplicity. As a result, the trajectories are the most simple—
harmonic in conservative case and exponential in the dissipative case. The numerical
calculations of this example (Eq. (2.29)) are shown in Figure 2.3. Figure 2.3A indicates this trajectory in the phase plane (p, x), which is shown for some values of the
Hamiltonian (21.25, 0.0, 0.35) and for optimal trajectory the Hamiltonian H 5 0.0.
One can see the exponential relaxation for the optimal trajectory designated by (0.0) in
Figure 2.3B. The phase trajectories in phase plane p, u are shown in Figure 2.3C and
correspond to Figure 2.3B. The kinetical curves in logarithmic scale are shown in
Figure 2.3D, ueq 5 0, xeq 5 1. Clearly, because the exponential relaxation of the state
variable x goes to zero, this really describes dissipation.
The comparison between these two essentially opposite processes, conservative
and dissipative, is summarized in Figure 2.4 as the phase plot of the Hamiltonian.
Some General Optimal Control Problems Useful for Biokinetics
(A)
(B)
5
X
p
3
p
0.35
3
2
–1.25
1
–1.25
1
0
2
4
p
5
X 6
0.0
10
15
20
Time
(C)
(D)
1
U
–1
–3
–5
–1
0.0
X
–1.25
0.35
0.0
–1
51
3
1
p 5
10
1
0.1
0.01
1·10–3
1·10–4
1·10–5
1·10–6
1·10–7
1·10–8
0.35
–1.25
X
0.0
p
5
10
0.0
20
15
Time
Figure 2.3 Dynamic system (2.29) interpretation. (A) Phase plane x, p for system (2.29).
(B) Kinetic plot of x, p, u parameters versus dimensionless time. (C) Phase plane p, u.
(D) Kinetical curves in logarithmic scale, ueq 5 0, xeq 5 1.
(B) 4
(A) 4
p
15
2
10
15
10
–5
p
0
–5
0
5
2
5
5
5
0
5
5
0
0
0
0
5
0
–2
–2
5
5
0
–5
–4
5
10
–2
0
2
–4
X 4
0
–5
–2
0
2
4
X
Figure 2.4 Phase plot of the first integral (Hamiltonian) for the optimal control problem.
(A) Classical harmonic oscillator solved as an optimal control problem (2.17), xeq 5 1.0,
ueq 5 1.0 and (B) dissipative optimal control problem (2.29), xeq 5 1.0, ueq 5 0.0, Eq. (2.4).
52
The Common Extremalities in Biology and Physics
Reformulating the pure variational approach, analogous to the optimal control
framework by Eqs. (2.25)À(2.29), when we need to take the Lagrange function not
in the form of a difference, but in the form of a sum of the positively definite
kinetic and positively definite potential terms [2,3], we can obtain
L5m
x_2
1 GðxÞ;
2
ð2:30Þ
and the EulerÀLagrange equation will be written as:
x€ 5
1 @G
;
m @x
ð2:31Þ
and the variational Hamiltonian will be
H5
p2
x_2
2GðxÞ 5 m 2GðxÞ 5 Const:
2m
2
ð2:32Þ
In the aforementioned harmonic-like case, when potential G(x) is a square-law
function (2.6), the equation will be written as:
x€ 5
k
x;
m
ð2:33Þ
and the canonical system will be
p
;
m
p_ 5 kx:
x_ 5
ð2:34Þ
Now it should be noted that the solution of this problem is
rﬃﬃﬃﬃ !
rﬃﬃﬃﬃ !
k
k
xðtÞ 5 x1 exp
t 1 x2 exp 2
t :
m
m
ð2:35Þ
Let us note that the Lagrange function is chosen in the form of a sum of the
positively definite kinetic term dependent only on derivatives explicitly (not on
the time t) and the positively definite potential term. It is distinguished from
mechanics and is similar to the case, taking place in another mathematical discipline, in the theory of optimal control, which has a strong relation to the variational method.
The phase trajectories for the case considered above in Eqs. (2.30)À(2.35) are indicated in Figure 2.5. The surface plot of the Lagrangian is shown in Figure 2.5A.
Some General Optimal Control Problems Useful for Biokinetics
(A)
53
(B) 2.00
00
1.00
2.
00
2.
1.00
4.00
0.00
0.00
1.00
1.00
X
L 2.00
00
00
2.
2.
0
0.
X
–2.00
–2.00
0
.0
0
.0
–1
–1
1.00
00
0
0
2.
1.
0
0.
X
–1.00
00
00
1.
–1.00
0
1.00
0.00
0 00
.0 2.
–2 –
2.0
2.00
X
(C)
(D) 2.00
1.00
2.00
.0
–0
H 0
.00
–1.
00
1.00
0
.0
–0
0
1.
1.
0
0
0.
X
00
0.
00
.
–1
0
0
1.
–
0
0
.0 2.0
–2 –
.00
–1.00
–1
2.
00
2.
0
00
0
.0
00
–0
0
0
X 0.00
–2.0
0
.0
X
–2.00
–2.00
–0
1.0
–1.00
0
0.00
1.00
2.00
X
Figure 2.5 Surface plot of the penalty function (2.30) and the corresponding first integral
(2.32) with quadratic potential similar to Eq. (2.6) at k 5 1, m 5 1. (A) The surface plot of
the penalty function. (B) The lines of identical levels of the Lagrangian. (C) The surface of
the first integral H, Eq. (2.32). (D) The lines of identical levels for the first integral H,
(2.32).
Its lines of equipotential levels are shown in Figure 2.5B. The Hamiltonian
equation (2.32) is shown in Figure 2.5C, and the phase plot of this Hamiltonian is
presented in Figure 2.5D.
One can note the essentially different character of the lines of identical levels
from Figure 2.5B and Figure 2.5D, where the motion of the system is represented
on the plane of the coordinate x and its derivative dx/dt. Because of that, the
Hamiltonian is the first integral, when its lines of identical levels are also the phase
54
The Common Extremalities in Biology and Physics
trajectories of the motion. The relaxational character of the motion can be seen,
which coincides with the optimal control results (Eqs. (2.25)À(2.29)).
2.1.5
Comparing Closely the Linear Dissipative and Conservative Models
Let us now compare these two different ways of building the variational/OC
approaches: one for conservative mechanics and another for dissipative. As one can
see from Eqs. (2.1)À(2.17), when the Lagrange function (Lagrangian) is a difference (conservative), the Hamiltonian is a positive definite (a sum of these two
parts), and vice versa, Eqs. (2.25)À(2.35).
As one can see from the biological/chemical kinetics example, everything is correct from the optimal control perspective. We can define the penalty for a control—u,
and the penalty for not being in the equilibrium state, where the state coordinate is the
magnitude of deviation of the system from the equilibrium (it can be a zero point,
x 5 0). The penalty for the control, or the penalty for the deviation of the magnitude
of the effect from the zero value, may be chosen as a quadratic. Then the problem
of minimization of the cost of the control can be reduced to minimization of the integral (2.25). Consequently, the Lagrangian/Hamiltonian can be built as in Eq. (2.27),
and it is easy to obtain the system of the EulerÀLagrange equations or canonical
system.
However, from the physical perspective, the difficulty is that the first integral,
considering formally all possible state and costate (control) spectrum of values, can
formally take negative values, as for example, in the simplest case of square-law
potential, Figure 2.5,
H5
x_2 x2
2 5 Const:
2
2
ð2:36aÞ
It is apparent that the first term in the expression of energy is relevant to the
cost for the optimal control and, consequently, expresses the “energy” of this control, or in other words, the “energy” expenditure for the implementation of the
dynamic control. When we consider this example that uses the positively defined
Lagrangian, we can obtain the phase plot, Figures 2.4B and 2.5C. Conversely, taking into account the transversality condition (which coincides with the additional
demand of the Pontryagin maximum principle that HÃ 5 0), this integral is equal to
zero at the optimal trajectory because of open-end (free end) of the OC or variational problem. Figure 2.5 illustrates the cost function (A, B) and the first integral
surface plots (C for this case). Formally, the physical real (HÃ 5 0) and nonphysical
phase trajectories of the first integral H are illustrated by Figure 2.5C. This can be
interpreted in the sense of energy conservation, as a situation, when the energy, dissipated in this elementary system by dissipative mechanisms is equal to the free
energy dissipated. Formally, therefore, the term analogous to kinetic dissipative
(dissipative function) is equal to the free energy dissipated, expressed by the potential part (thermodynamic potential).