Chapter 18 Linear Quadratic optimal control I
Consider the input-state-output system
\[ \dot {x}(t)=Ax(t)+Bu(t),\qquad y(t)=Cx(t)+Du(t). \]
Here \(A\in \mR ^{n\times n}\), \(B\in \mR ^{n\times m}\), \(C\in \mR ^{p\times n}\) and \(D\in \mR ^{p\times n}\).
Our objective is to find, for a given initial state \(x^0\in \mR ^n\) (i.e. such that \(x(0)=x^0\)), a control \(u\) which minimizes
\(\seteqnumber{0}{18.}{0}\)\begin{equation} \label {eq:cost} \int _0^\infty \|y(t)\|^2\,dt. \end{equation}
We will also consider this problem with the additional requirement that \(\lim _{t\to \infty }x(t)=0\); this is called the zero endpoint case.
-
Definition 18.1. Assume that \(D\) is injective. The algebraic Riccati equation associated to the input-state-output system is
\(\seteqnumber{0}{18.}{1}\)\begin{equation} \label {eq:Riccati} A^*X+XA+C^*C-(XB+C^*D)(D^*D)^{-1}(D^*C+B^*X)=0, \end{equation}
where the unknown \(X\) is a \(n\times n\) matrix.
-
Theorem 18.2. Assume that \(D\) is injective. Then the following are equivalent:
-
1. For every \(x^0\in \mR ^n\) there exists a \(u\) such that \(\int _0^\infty \|y(t)\|^2\,dt<\infty \);
-
2. The algebraic Riccati equation (18.2) has a symmetric positive semidefinite solution.
Assume that the above conditions hold. Then the following hold:
-
• There exists a smallest symmetric positive semidefinite solution \(X_0\) of the algebraic Riccati equation (18.2)
-
• For every \(x^0\in \mR ^n\) there exists a unique optimal control and this optimal control is given by the state feedback matrix
\[ F_0=-(D^*D)^{-1}(B^*X_0+D^*C), \]
as \(u(t)=F_0x(t)\)
-
• The minimimum of the cost (18.1) is given by \(\ipd {X_0x^0}{x^0}\).
-
The theorem below is about the zero endpoint case.
-
Theorem 18.4. Assume that \(D\) is injective and that \((A,B)\) is stabilizable. The following hold:
-
• There exists a largest symmetric positive semidefinite solution \(X_+\) of the algebraic Riccati equation (18.2)
-
• The infimum of the cost (18.1) amongst all \(u\) such that \(\lim _{t\to \infty }x(t)=0\) is given by \(\ipd {X_+x^0}{x^0}\).
The following two statements are equivalent
-
1. For every \(x^0\in \mR ^n\) there exists an optimal control;
-
2. The Rosenbrock matrix (18.3) is injective for all \(s\) with \(\re (s)=0\).
Assume that the above conditions hold. Then the following hold:
-
• For every \(x^0\in \mR ^n\) there exists a unique optimal control, this optimal control is given by the state feedback matrix
\[ F_+=-(D^*D)^{-1}(B^*X_++D^*C), \]
as \(u(t)=F_+x(t)\), and the closed-loop matrix \(A+BF_+\) is asymptotically stable.
-
-
Remark. Recall that a matrix \(X\in \mR ^{n\times n}\) is called symmetric positive semidefinite if \(X=X^*\) and \(\ipd {Xx}{x}\geq 0\) for all \(x\in \mR ^n\). A matrix \(X\) is symmetric positive semidefinite if and only if all of its eigenvalues are real and nonnegative. In the case \(n=2\) with
\[ X=\bbm {X_1&X_0\\X_0&X_2}, \]
we have that \(X\) is symmetric positive semidefinite if and only if \(X_1,X_2\geq 0\) and \(\det (X)\geq 0\). For symmetric positive semidefinite matrices \(X^1,X^2\in \mR ^{n\times n}\) we define that \(X^2\geq X^1\) if \(\ipd {X^2x}{x}\geq \ipd {X^1x}{x}\) for all \(x\in \mR ^n\). The notions of smallest in Theorem 18.2 and largest in Theorem 18.4 should be understood in that sense. Note that being symmetric positive semidefinite precisely means that \(X\) (is symmetric and) satisfies \(X\geq 0\) where \(0\) denotes the \(n\)-by-\(n\) zero matrix.
18.1 Examples
-
Example 18.5. Consider the first order scalar differential equation
\[ \dot {x}(t)+x(t)=u(t), \]
and the performance output
\[ y=\bbm {x\\\varepsilon u}, \]
where \(\varepsilon >0\), i.e.
\[ A=-1,\qquad B=1,\qquad C=\bbm {1\\0},\qquad D=\bbm {0\\\varepsilon }, \]
and the objective is to minimize
\[ \int _0^\infty |x(t)|^2+\varepsilon ^2|u(t)|^2\,dt, \]
with or without a stability condition.
We have that \(D\) is injective. We also have that \((A,B)\) is stabilizable (since \(A\) is stable). The Rosenbrock matrix is
\[ \bbm {s+1&-1\\1&0\\0&\varepsilon }. \]
If we delete the first row, then the resulting matrix is diagonal with nonzero entries on its diagonal and is therefore invertible. It follows that the Rosenbrock matrix is injective (in fact for all \(s\in \mC \)). Hence the conditions of Theorems 18.2 and 18.4 are satisfied.
The Riccati equation is (using that \(C^*D=0\) and \(D^*D=\varepsilon ^2\))
\[ -2X+1-\varepsilon ^{-2}X^2=0. \]
This quadratic equation has the two solutions
\[ X=\frac {-1\pm \sqrt {1+\varepsilon ^{-2}}}{\varepsilon ^{-2}} =-\varepsilon ^2\pm \varepsilon \sqrt {\varepsilon ^2+1}. \]
To obtain a positive semidefinite solution (which here just means \(X\geq 0\)) we need the plus sign, so
\[ X=-\varepsilon ^2+\varepsilon \sqrt {\varepsilon ^2+1}. \]
The optimal feedback then is (again using that \(C^*D=0\) and \(D^*D=\varepsilon ^2\)):
\[ F=-\varepsilon ^{-2}\left (-\varepsilon ^2+\varepsilon \sqrt {\varepsilon ^2+1}\right ) =1-\sqrt {1+\varepsilon ^{-2}}. \]
The optimal control then is \(u=\left (1-\sqrt {1+\varepsilon ^{-2}}\right )~x\), which gives the closed-loop system
\[ \dot {x}+\sqrt {1+\varepsilon ^{-2}}~x=0. \]
Note that since the conditions of Theorem 18.4 are satisfied and the symmetric positive semidefinite solution is unique, we obtain stability.