Example 19.1. Consider the undamped second order scalar differential equation

\[ \ddot {q}(t)+q(t)=u(t), \]

with the state \(x=\sbm {q\\\dot {q}}\) and the performance output

\[ y=\bbm {\dot {q}\\\varepsilon u}, \]

where \(\varepsilon >0\), i.e.

\[ A=\bbm {0&1\\-1&0},\quad B=\bbm {0\\1},\quad C=\bbm {0&1\\0&0},\qquad D=\bbm {0\\\varepsilon }, \]

and the objective is to minimize

\[ \int _0^\infty |\dot {q}(t)|^2+\varepsilon ^2|u(t)|^2\,dt, \]

with or without a stability condition.

We have that \(D\) is injective. We also have that \((A,B)\) is stabilizable (even controllable by Example 13.2). The Rosenbrock matrix is

\[ \bbm {s&-1&0\\1&s&-1\\0&1&0\\0&0&\varepsilon }. \]

If we delete the first row, then the resulting matrix is upper-triangular with nonzero entries on its diagonal and is therefore invertible. It follows that the Rosenbrock matrix is injective (in fact for all \(s\in \mC \)). Hence the conditions of Theorems 18.2 and 18.4 are satisfied.

The Riccati equation is (using that \(X\) is symmetric and that \(C^*D=0\) and \(D^*D=\varepsilon ^2\))

\begin{multline*} \bbm {0&-1\\1&0}\bbm {X_1&X_0\\X_0&X_2} +\bbm {X_1&X_0\\X_0&X_2}\bbm {0&1\\-1&0} +\bbm {0&0\\1&0}\bbm {0&1\\0&0} \\ -\varepsilon ^{-2}\bbm {X_1&X_0\\X_0&X_2}\bbm {0\\1}\bbm {0&1}\bbm {X_1&X_0\\X_0&X_2}=\bbm {0&0\\0&0}, \end{multline*} which is

\[ \bbm {-2X_0&X_1-X_2\\X_1-X_2&2X_0} +\bbm {0&0\\0&1} -\varepsilon ^{-2}\bbm {X_0^2&X_0X_2\\X_0X_2&X_2^2}=\bbm {0&0\\0&0}, \]

which is

\[ \bbm { -2X_0-\varepsilon ^{-2}X_0^2 &X_1-X_2-\varepsilon ^{-2}X_0X_2 \\X_1-X_2-\varepsilon ^{-2}X_0X_2 &2X_0+1-\varepsilon ^{-2}X_2^2 }=\bbm {0&0\\0&0}. \]

The top-left corner gives \(X_0=0\) or \(X_0=-2\varepsilon ^2\).

We first consider the case \(X_0=-2\varepsilon ^2\). The off-diagonal entry then gives

\[ X_1+X_2=0. \]

Since \(X_1,X_2\geq 0\) (because \(X\) is symmetric positive semidefinite), this implies \(X_1=X_2=0\). The determinant of \(X\) then equals \(-X_0^2\) which is negative which contradicts that \(X\) is symmetric positive semidefinite. Therefore this case must be excluded.

We now consider the case \(X_0=0\). Then the bottom-right corner gives \(X_2=\varepsilon \) and finally the off-diagonal entry gives \(X_1=\varepsilon \). It follows that

\[ X=\varepsilon \bbm {1&0\\0&1}. \]

The optimal feedback then is (again using that \(C^*D=0\) and \(D^*D=\varepsilon ^2\)):

\[ F=-\varepsilon ^{-2}\bbm {0&1}\varepsilon \bbm {1&0\\0&1} =\bbm {0&-\varepsilon ^{-1}}, \]

which gives the control \(u=-\varepsilon ^{-1}\dot {q}\). The closed-loop system then is (in second order form)

\[ \ddot {q}(t)+\varepsilon ^{-1}\dot {q}(t)+q(t)=0. \]

From this we see that increasing \(\varepsilon \) (i.e. putting more emphasis on the control cost) leads to a closed-loop system with a smaller damping ratio.

Since the Riccati equation has only one symmetric positive semidefinite solution and the conditions of Theorem 18.4 are satisfied, it follows that \(F\) is stabilizing.

Example 19.2. We consider a combination of two first order systems

\[ \dot {x}_1=x_1+u,\qquad \dot {x}_2=-x_2, \]

with the cost function

\[ \int _0^\infty |x_2(t)|^2+|u(t)|^2\,dt. \]

Since \(u\) doesn’t influence \(x_2\) (also not indirectly), it is clear that \(u=0\) is the minimizer if asymptotic stability is not demanded. Since \(u=0\) does not make \(x_1\) stable, if asymptotic stability is demanded, then \(u=0\) is not the minimizer. Therefore the \(X_0\) and \(F_0\) from Theorem 18.2 will be different from the \(X_+\) and \(F_+\) from Theorem 18.4.

We have

\[ A=\bbm {1&0\\0&-1},\qquad B=\bbm {1\\0},\qquad C=\bbm {0&1\\0&0},\qquad D=\bbm {0\\1}. \]

We have that \(D\) is injective, that \((A,B)\) is stabilizable (for example \(F=\bbm {-2&0}\) will do since then \(A+BF=-I\)). The Rosenbrock matrix is

\[ \bbm {s-1&0&-1\\0&s+1&0\\0&1&0\\0&0&1}. \]

If we delete the second row, then we obtain an upper-triangular matrix with \(s-1\), \(1\) and \(1\) on the diagonal, so that this matrix is invertible for all \(s\neq 1\) (in particular: for all \(s\) with real part zero). Hence the Rosenbrock matrix is injective.

The Riccati equation is (using that \(X\) is symmetric and that \(C^*D=0\) and \(D^*D=1\))

\[ \bbm {1&0\\0&-1}\bbm {X_1&X_0\\X_0&X_2} +\bbm {X_1&X_0\\X_0&X_2}\bbm {1&0\\0&-1} +\bbm {0&0\\1&0}\bbm {0&1\\0&0} -\bbm {X_1&X_0\\X_0&X_2}\bbm {1\\0}\bbm {1&0}\bbm {X_1&X_0\\X_0&X_2} =\bbm {0&0\\0&0}, \]

which is

\[ \bbm {2X_1&0\\0&-2X_2}+\bbm {0&0\\0&1}-\bbm {X_1^2&X_0X_1\\X_0X_1&X_0^2}=\bbm {0&0\\0&0}, \]

which is

\[ \bbm {2X_1-X_1^2&-X_0X_1\\-X_0X_1&-2X_2+1-X_0^2}=\bbm {0&0\\0&0}. \]

The off-diagonal entry gives that either \(X_0=0\) and \(X_1=0\).

We first consider the case \(X_1=0\). Then the determinant of \(X\) equals \(-X_0^2\) and for \(X\) to be symmetric positive semidefinite we therefore need \(X_0=0\). Hence we do not need to consider the case \(X_1=0\) separately further.

We now consider the case \(X_0=0\). Then the bottom-right corner gives \(X_2=\frac {1}{2}\) and the top left entry gives that either \(X_1=0\) or \(X_1=2\). Therefore we obtain two solutions

\[ X=\bbm {0&0\\0&\frac {1}{2}},\qquad X=\bbm {2&0\\0&\frac {1}{2}}. \]

The first of these is smaller than the second and therefore

\[ X_0=\bbm {0&0\\0&\frac {1}{2}},\qquad X_+=\bbm {2&0\\0&\frac {1}{2}}. \]

The corresponding feedback matrices are

\[ F_0=-\bbm {1&0}\bbm {0&0\\0&\frac {1}{2}}=\bbm {0&0}, \qquad F_+=-\bbm {1&0}\bbm {2&0\\0&\frac {1}{2}}=\bbm {-2&0}. \]

The closed-loop system corresponding to \(F_0\) is

\[ \dot {x}_1=x_1,\qquad \dot {x}_2=-x_2, \]

which we note is not asymptotically stable. The closed-loop system corresponding to \(F_+\) is

\[ \dot {x}_1=-x_1,\qquad \dot {x}_2=-x_2, \]

which is asymptotically stable.

Example 19.3. Consider

\[ A=0,\quad B=1,\quad C=0,\quad D=1. \]

Then \(D\) is injective and \((A,B)\) is stabilizable (for example with \(F=-1\) we have \(A+BF=-1\) which is stable). The Rosenbrock matrix is

\[ \bbm {s&-1\\0&1}. \]

For \(s=0\) this is

\[ \bbm {0&-1\\0&1}, \]

which is not injective since it is square and not invertible (its determinant equals zero). Therefore the Rosenbrock condition from Theorem 18.4 is not satisfied and therefore by that theorem (at least for some initial condition \(x^0\)) there does not exist an optimal control (the infimum is not a minimum). We check this directly. The differential equation and the cost are

\[ \dot {x}=u,\qquad \int _0^\infty |u(t)|^2\,dt. \]

Let \(\varepsilon >0\) and \(u=-\varepsilon x\). Then \(\dot {x}=-\varepsilon x\) so that \(x(t)=\e ^{-\varepsilon t}x^0\) (so that we have stability) and \(u(t)=-\varepsilon \e ^{-\varepsilon t}x^0\). Therefore the cost with this \(u\) is

\[ \int _0^\infty \varepsilon ^2\e ^{-2\varepsilon t}(x^0)^2\,dt =\frac {\varepsilon }{2}(x^0)^2. \]

For a given \(x^0\), the cost can therefore be made arbitrarily small by choosing \(\varepsilon \) small enough. Therefore the infimum of the cost equals zero (as the cost is clearly always greater than or equal to zero). If the cost is zero, then we need \(u=0\). However, for \(u=0\) we obtain \(\dot {x}=0\) which is not stable (solutions are \(x=x^0\) which do not converge to zero as \(t\to \infty \) for \(x^0\neq 0\)). Therefore, for the problem with stability, there does not exist an optimal control for \(x^0\neq 0\) (for the problem without stability \(u=0\) is the optimal control).