Tải bản đầy đủ - 0 (trang)
1 Extreme Dissipation, Optimal Control, and the Least Action Principle

1 Extreme Dissipation, Optimal Control, and the Least Action Principle

Tải bản đầy đủ - 0trang

44



The Common Extremalities in Biology and Physics



The Lagrange function in the elementary one-dimensional case is

L5m



x_2

2 UðxÞ;

2



ð2:2Þ



where mðx_2 =2Þ is the kinetic term and U(x) is the potential.

Then the corresponding EulerÀLagrange equation can be written as:

x€ 5 2



1 @U

:

m @x



ð2:3Þ



The Hamiltonian will be

H5



x_2

p2

1 UðxÞ 5 m 1 UðxÞ

2m

2



ð2:4Þ



and the canonical system (2.4) is

@H

5 p=m;

@p

@H

@U

52

:

p_ 5 2

@x

@x



x_ 5



ð2:5Þ



For example, in a simple one-dimensional harmonic oscillator, when the elastic

force originates from the quadratic potential

UðxÞ 5 k



x2

;

2



ð2:6Þ



the Lagrangian will be

_ 5m

Lðx; xÞ



x2

x_2

2k :

2

2



ð2:7Þ



The equation of motion is then

m x€ 1 kx 5 0 or x€ 5 2



k

x:

m



ð2:8Þ



Using the Legendre transform, we can build the Hamiltonian

Hðx; pÞ 5



p2

x2

1k

2m

2



ð2:9Þ



Some General Optimal Control Problems Useful for Biokinetics



45



and the canonical system (2.5) is

x_ 5 p=m;

p_ 5 2kx:



ð2:10Þ



The mechanical motion described by Eqs. (2.7)À(2.9), as it is well known, is

referred to as harmonic motion. The solution of the equation

rffiffiffiffi !

rffiffiffiffi !

k

k

xðtÞ 5 aUexp i

t 1 bUexp 2 i

t

m

m



ð2:11Þ



can be represented in the explicit form of the sum of sine and cosine functions:

rffiffiffiffi !

rffiffiffiffi !

k

k

t 1 bUcos

t :

x 5 aUsin

m

m

The surface plots and the lines of identical levels (contour plots) for the case

considered above in Eqs. (2.2)À(2.11) are indicated in Figure 2.1. The surface

plot of the Lagrangian is shown in Figure 2.1A with its lines of equipotential

levels (Figure 2.1B). The surface plot of the Hamiltonian, Eq. (2.9), is shown

in Figure 2.1C and phase plot of this Hamiltonian is presented in Figure 2.1D,

which are the phase trajectories. One can note the essentially different character

of the lines of identical levels from Figure 2.1B and D, where the motion of

the system is represented on the plane of the coordinate x and its derivative dx/dt.

Because of that, the Hamiltonian is the first integral, when its lines of identical

levels are also the phase trajectories of the motion. A cyclic character of this

motion is seen, which is expressed as periodic dependence of the deviation x on

the time t.

So, the curves in Figure 2.1D indicate the cyclic transformations in state and

costate variables without any attenuation or relaxation, and this is confirmed by the

trajectories of equipotential lines of the first integral or energy in its physical

interpretation:

E5



@L

x2

x_2

x_2

x_ 2 L 5 m 1 UðxÞ 5 m 1 k 5 Const:

2

2

2

@x_



ð2:12Þ



This expression is considered in mechanics as the law of conservation of energy;

therefore, dissipation in the corresponding physical system is not observed.

Let us note that in mechanics, the square-law dependence of the kinetic term on

the velocities indicates a symmetry of motion in relation to some directions in space

[1]. The explicit independence of the kinetic term from the mechanical coordinates

means the uniformity of mechanical space and, consequently, the uniformity of

kinematical laws in the different frame of references.



46



The Common Extremalities in Biology and Physics



(A)



(B) 2.00

0



1.0



2.00

L 0.

00



0.00



00



X



0.



0

.0



0



.0



–1



–1.00



–1.



1.



00

0.

X



0



.0



–1



0

00 .0

2. –2



0



00



00



00



1.



.0



2.

00



X



–0



00



2.



0



0



.0

–0



–2.0



–0



–2.00

–2.00







0



.0



–0



–1.

00



1.00



0



1.0



–1.00



1.00



0.00



2.00



X



(C)



(D) 2.00

1.00



00



0



X



0.0

0



0.00



1.00



2.0

0



1.00



1.00



4.0

E



00



2.



2.



00



0

.0



0.



00



–2.00

–2.00



–1.00



–2



.0

–20

.0

0



–1



–1



.0



0



.0



0



X



0.



1.00



00



X



–1.00

2.



00



1.



1.



00



00



2.



2



–0.00



2.0



1.00



0



2.00



X



Figure 2.1 (A) The surface plot of the Lagrangian (2.7) with quadratic potential. (B) The

lines of identical levels of the Lagrangian. (C) The surface plot of corresponding

Hamiltonian (2.9) with quadratic potential (2.5). (D) The lines of identical levels for the

Hamiltonian (energy) E, (k 5 1, m 5 1).



2.1.2



On Optimal Control Formulation of Mechanics



Let us consider this simple classical mechanics case (Eqs. (2.1)À(2.11)) in the OC

formulation in a more detailed way compared to that of Section 1.3, then



ðτ 

ðu 2 ueq Þ2

ðx 2 xeq Þ2

2k

dt-extr:

ð2:13Þ

m

S5

2

2

t0

subject to

x_ 5 u;



xðt0 Þ 5 x0 :



ð2:14Þ



Some General Optimal Control Problems Useful for Biokinetics



47



The integrand in the functional is similar to the classical mechanics Lagrangian

in the sense of definition of signs of these parts. Then to apply the Pontryagin maximum principle [1], the optimal control Hamiltonian needs to be built

H 5 2m



ðu 2 ueq Þ2

ðx 2 xeq Þ2

1k

1 pu;

2

2



ð2:15Þ



and the Pontryagin conditions for this problem from Eqs. (2.13) and (2.14) will be

x_ 5 u; xà ðt0 Þ 5 x0 ;

p_ 5 2



@H

5 2kðx 2 xeq Þ;

@x



@H

5 2mðu 2 ueq Þ 1 p 5 0;

@u



ð2:16Þ



Hðxà ; uà ; pÃ Þ $ Hðxà ; u; pà Þ

or

x_ 5



p

1 ueq ; xà ðt0 Þ 5 x0 ;

m



p_ 5 2kðx 2 xeq Þ;



ð2:17Þ



Hðxà ; uà ; pÃ Þ $ Hðxà ; u; pà Þ

when xeq 5 0 and ueq 5 0, this system coincides with Eq. (2.10).

Because the integrand function in Eq. (2.13) and the right part in the constrained

equation (2.14) are not dependent on time explicitly, the Hamiltonian H is constant

along an optimal trajectory; in other words, it is the first integral to the problem,

Eq. (2.16), which in mechanics is interpreted as energy.

Formally ueq means that the kinetical picture is shown in the frame of reference

(one-dimensional) moving with a velocity ueq (the variable u could be considered

as a dummy variable). If we choose ueq 5 0, it is formally relevant to the picture in

the resting frame of reference. Clearly, all our consideration is nonrelativistic and

is employed just to discuss analogy to dissipative mechanics. The phase plot for

this example and trajectories are shown in Figure 2.2. One notes that qualitatively

it coincides with the example from the previous section, where it was considered in

terms of the pure variational method.



2.1.3



Dissipation in Classical Mechanics



For biological systems where kinetics is known by the irreversible character of

energy transformation processes, dissipation is normal. Therefore, one should turn

directly to those equations of mechanics that also show dissipation.

In mechanics, dissipative effects can be included by the introduction of the friction forces that are proportional to the velocity of mechanical motion [1]



48



The Common Extremalities in Biology and Physics



(A) 1

λ 0.625



(B) 4

X



0.0



0



E=0



λ 2



X



U

–1



U



0



–2

–3

–1



0



1



2



X



λ



–2



–0.375

3



(C) 2

U

1



5

(D)



10



15



Time



X

2



20



0.0

–0.375



0



0



0.625

–1

–2

–2

–3



–2



–1



0



λ1



5



10



15



Time



20



Figure 2.2 Graphical interpretation of the dynamic system (2.16). (A) The phase plane x, p

for system (2.16). (B) Kinetic plot of x, p, u parameters against dimensionless time. (C) The

phase plane λ, u. (D) Kinetical curves for different values of the first integral (energy):

xeq 5 0, ueq 5 0.



fitr 5 2



X



αik x_k :



ð2:18Þ



k



This equation can be rewritten as a derivative

fitr 5 2



@F

@x_i



ð2:19Þ



from a quadratic form

F5



11X

αik x_i x_k

2 2 i;k



ð2:20Þ



and then the dissipative forces can be added to right-hand parts of the

EulerÀLagrange equations

dL @L

@L @F

52

2

dt @xi

@xi @x_i



ð2:21Þ



Some General Optimal Control Problems Useful for Biokinetics



49



and then, for example, the equation for the one-dimensional harmonic oscillator

will be written considering the friction force:

mx€ 1 γ x_ 1 kx 5 0:



ð2:22Þ



When one can ignore m in this equation, the motion is strongly dissipative and

Eq. (2.22) can be reduced to:

γ x_ 1 kx 5 0;



ð2:23Þ



and the motion in the system or the relaxation of the system occurs as follows:





k

xðtÞ 5 x0 exp 2 t :

γ



ð2:24Þ



In chemical and biological kinetics, one can deal with similar circumstances.

Kinetic equations in the majority of cases look like those in mechanics in the case

of strong dissipation and are represented by first-order equations. Therefore, it

seems reasonable to consider the applications of the variational approach to biological kinetics through those in mechanics. However, we need to bear in mind that

the dependence of free energy on the first concentration derivatives violates the

principle of local equilibrium. Therefore, following mechanics, one can try to form

the Lagrange function in two parts: one part plays the role of a potential and the

other a kinetic role.



2.1.4



On an Alternative Way to Describe Biological and Chemical

Dissipation



Nevertheless, our consideration concerns the outline of the extreme method (related

to the least action principle) for nonconservative (chemical and biological) kinetics,

e.g., the description of systems with explicitly expressed dissipation.

Therefore, simple classical mechanics case from Eqs. (2.13) and (2.14) can be

reformulated in a form more relevant to the optimal control and dissipative kinetics

(see Section 1.3), then

S5



ðτ 

t0





ðu 2 ueq Þ2

ðx 2 xeq Þ2

1k

dt-min

m

2

2



ð2:25Þ



xðt0 Þ 5 x0 ;



ð2:26Þ



subject to

x_ 5 u;



xðτÞ 5 xeq ;



where x is the extent from the equilibrium and when the terminal time τ is not a

specified. Then the integrand in the functional is the classical mechanic’s



50



The Common Extremalities in Biology and Physics



Lagrangian. Then applying the Pontryagin maximum principle, we could write the

OC Hamiltonian

H 5 2m



ðu 2 ueq Þ2

ðx 2 xeq Þ2

2k

1 pu;

2

2



ð2:27Þ



and the Pontryagin maximum principle conditions for this problem will be

x_ 5 u; xà ðt0 Þ 5 x0 ;

p_ 5 2



xà ðτÞ 5 xeq :



@H

5 kðx 2 xeq Þ;

@x



ð2:28Þ



@H

5 2mðu 2 ueq Þ 1 p 5 0;

@u

Hðxà ; uà ; pÃ Þ $ Hðxà ; u; pà Þ;



p#0



or

x_ 5



p

1 ueq ; xà ðt0 Þ 5 x0 ;

m



xà ðτÞ 5 xeq :

ð2:29Þ



p_ 5 kðx 2 xeq Þ;

Hðxà ; uà ; pÃ Þ $ Hðxà ; u; pà Þ;



p # 0:



Because the integrand function in Eq. (2.25) and the right-hand part in the constrained equation (2.26) are not dependent on t explicitly, the Hamiltonian H is a

constant along an optimal trajectory; in other words, it is a first integral to the problem (2.16).

If we choose the ueq 5 0, we imply that the control amplitude in the equilibrium

obviously tends to zero, and then the numerical solutions of this problem are shown in

Figure 2.3. One can see from this figure that the character of motion changes dramatically when the sign in the Lagrangian changes for the part that is the penalty for not

being in the equilibrium. In Eqs. (2.13) and (2.25), this penalty and control both have

a quadratic form for simplicity. As a result, the trajectories are the most simple—

harmonic in conservative case and exponential in the dissipative case. The numerical

calculations of this example (Eq. (2.29)) are shown in Figure 2.3. Figure 2.3A indicates this trajectory in the phase plane (p, x), which is shown for some values of the

Hamiltonian (21.25, 0.0, 0.35) and for optimal trajectory the Hamiltonian H 5 0.0.

One can see the exponential relaxation for the optimal trajectory designated by (0.0) in

Figure 2.3B. The phase trajectories in phase plane p, u are shown in Figure 2.3C and

correspond to Figure 2.3B. The kinetical curves in logarithmic scale are shown in

Figure 2.3D, ueq 5 0, xeq 5 1. Clearly, because the exponential relaxation of the state

variable x goes to zero, this really describes dissipation.

The comparison between these two essentially opposite processes, conservative

and dissipative, is summarized in Figure 2.4 as the phase plot of the Hamiltonian.



Some General Optimal Control Problems Useful for Biokinetics



(A)



(B)



5



X



p



3



p



0.35



3



2



–1.25

1



–1.25

1



0



2



4



p

5



X 6



0.0

10



15



20

Time



(C)



(D)



1

U

–1



–3



–5

–1



0.0



X



–1.25



0.35



0.0

–1



51



3



1



p 5



10

1

0.1

0.01

1·10–3

1·10–4

1·10–5

1·10–6

1·10–7

1·10–8



0.35



–1.25

X



0.0



p



5



10



0.0

20



15

Time



Figure 2.3 Dynamic system (2.29) interpretation. (A) Phase plane x, p for system (2.29).

(B) Kinetic plot of x, p, u parameters versus dimensionless time. (C) Phase plane p, u.

(D) Kinetical curves in logarithmic scale, ueq 5 0, xeq 5 1.

(B) 4



(A) 4

p



15



2



10



15



10



–5



p



0



–5



0

5



2



5



5



5

0



5

5



0



0



0



0

5



0



–2



–2



5



5

0

–5



–4



5



10



–2



0



2



–4



X 4



0



–5



–2



0



2



4



X



Figure 2.4 Phase plot of the first integral (Hamiltonian) for the optimal control problem.

(A) Classical harmonic oscillator solved as an optimal control problem (2.17), xeq 5 1.0,

ueq 5 1.0 and (B) dissipative optimal control problem (2.29), xeq 5 1.0, ueq 5 0.0, Eq. (2.4).



52



The Common Extremalities in Biology and Physics



Reformulating the pure variational approach, analogous to the optimal control

framework by Eqs. (2.25)À(2.29), when we need to take the Lagrange function not

in the form of a difference, but in the form of a sum of the positively definite

kinetic and positively definite potential terms [2,3], we can obtain

L5m



x_2

1 GðxÞ;

2



ð2:30Þ



and the EulerÀLagrange equation will be written as:

x€ 5



1 @G

;

m @x



ð2:31Þ



and the variational Hamiltonian will be

H5



p2

x_2

2GðxÞ 5 m 2GðxÞ 5 Const:

2m

2



ð2:32Þ



In the aforementioned harmonic-like case, when potential G(x) is a square-law

function (2.6), the equation will be written as:

x€ 5



k

x;

m



ð2:33Þ



and the canonical system will be

p

;

m

p_ 5 kx:



x_ 5



ð2:34Þ



Now it should be noted that the solution of this problem is

rffiffiffiffi !

rffiffiffiffi !

k

k

xðtÞ 5 x1 exp

t 1 x2 exp 2

t :

m

m



ð2:35Þ



Let us note that the Lagrange function is chosen in the form of a sum of the

positively definite kinetic term dependent only on derivatives explicitly (not on

the time t) and the positively definite potential term. It is distinguished from

mechanics and is similar to the case, taking place in another mathematical discipline, in the theory of optimal control, which has a strong relation to the variational method.

The phase trajectories for the case considered above in Eqs. (2.30)À(2.35) are indicated in Figure 2.5. The surface plot of the Lagrangian is shown in Figure 2.5A.



Some General Optimal Control Problems Useful for Biokinetics



(A)



53



(B) 2.00

00



1.00



2.

00



2.



1.00



4.00



0.00



0.00



1.00



1.00



X



L 2.00

00



00



2.



2.



0

0.



X



–2.00

–2.00



0



.0



0



.0



–1



–1



1.00



00



0



0



2.



1.

0

0.



X



–1.00



00



00



1.



–1.00



0



1.00



0.00



0 00

.0 2.

–2 –



2.0



2.00



X



(C)



(D) 2.00

1.00



2.00



.0



–0



H 0

.00



–1.



00



1.00



0

.0

–0



0

1.



1.



0



0

0.



X



00



0.



00



.



–1



0



0

1.





0



0

.0 2.0

–2 –



.00



–1.00



–1



2.



00



2.



0



00



0



.0



00



–0



0



0



X 0.00



–2.0



0



.0



X



–2.00

–2.00



–0



1.0



–1.00



0



0.00



1.00



2.00



X



Figure 2.5 Surface plot of the penalty function (2.30) and the corresponding first integral

(2.32) with quadratic potential similar to Eq. (2.6) at k 5 1, m 5 1. (A) The surface plot of

the penalty function. (B) The lines of identical levels of the Lagrangian. (C) The surface of

the first integral H, Eq. (2.32). (D) The lines of identical levels for the first integral H,

(2.32).



Its lines of equipotential levels are shown in Figure 2.5B. The Hamiltonian

equation (2.32) is shown in Figure 2.5C, and the phase plot of this Hamiltonian is

presented in Figure 2.5D.

One can note the essentially different character of the lines of identical levels

from Figure 2.5B and Figure 2.5D, where the motion of the system is represented

on the plane of the coordinate x and its derivative dx/dt. Because of that, the

Hamiltonian is the first integral, when its lines of identical levels are also the phase



54



The Common Extremalities in Biology and Physics



trajectories of the motion. The relaxational character of the motion can be seen,

which coincides with the optimal control results (Eqs. (2.25)À(2.29)).



2.1.5



Comparing Closely the Linear Dissipative and Conservative Models



Let us now compare these two different ways of building the variational/OC

approaches: one for conservative mechanics and another for dissipative. As one can

see from Eqs. (2.1)À(2.17), when the Lagrange function (Lagrangian) is a difference (conservative), the Hamiltonian is a positive definite (a sum of these two

parts), and vice versa, Eqs. (2.25)À(2.35).

As one can see from the biological/chemical kinetics example, everything is correct from the optimal control perspective. We can define the penalty for a control—u,

and the penalty for not being in the equilibrium state, where the state coordinate is the

magnitude of deviation of the system from the equilibrium (it can be a zero point,

x 5 0). The penalty for the control, or the penalty for the deviation of the magnitude

of the effect from the zero value, may be chosen as a quadratic. Then the problem

of minimization of the cost of the control can be reduced to minimization of the integral (2.25). Consequently, the Lagrangian/Hamiltonian can be built as in Eq. (2.27),

and it is easy to obtain the system of the EulerÀLagrange equations or canonical

system.

However, from the physical perspective, the difficulty is that the first integral,

considering formally all possible state and costate (control) spectrum of values, can

formally take negative values, as for example, in the simplest case of square-law

potential, Figure 2.5,

H5



x_2 x2

2 5 Const:

2

2



ð2:36aÞ



It is apparent that the first term in the expression of energy is relevant to the

cost for the optimal control and, consequently, expresses the “energy” of this control, or in other words, the “energy” expenditure for the implementation of the

dynamic control. When we consider this example that uses the positively defined

Lagrangian, we can obtain the phase plot, Figures 2.4B and 2.5C. Conversely, taking into account the transversality condition (which coincides with the additional

demand of the Pontryagin maximum principle that HÃ 5 0), this integral is equal to

zero at the optimal trajectory because of open-end (free end) of the OC or variational problem. Figure 2.5 illustrates the cost function (A, B) and the first integral

surface plots (C for this case). Formally, the physical real (HÃ 5 0) and nonphysical

phase trajectories of the first integral H are illustrated by Figure 2.5C. This can be

interpreted in the sense of energy conservation, as a situation, when the energy, dissipated in this elementary system by dissipative mechanisms is equal to the free

energy dissipated. Formally, therefore, the term analogous to kinetic dissipative

(dissipative function) is equal to the free energy dissipated, expressed by the potential part (thermodynamic potential).



Tài liệu bạn tìm kiếm đã sẵn sàng tải về

1 Extreme Dissipation, Optimal Control, and the Least Action Principle

Tải bản đầy đủ ngay(0 tr)

×