\headline={\tenrm\lefthead\hfill \righthead\hfill Page \folio} \def\lefthead{{\bf Math 8602, Spring 2001}} \def\righthead{{\bf Inner product spaces, and Hilbert spaces}} \def\ip#1#2{\langle #1,\,#2\rangle} \def\ipv{$\langle v,\,w\rangle$ } \def\ip#1#2{\left\langle #1,\,#2\right\rangle} \def\norm#1{\left\Vert #1\right\Vert} {\bf 1 Definition: } {\sl An {\it inner-product space with complex scalars\/} $\com,$ is a vector space $V$ with complex scalars, and a complex-valued function $\ip vw,$ called the {\it inner product,\/} defined on $V\times V,$ that has the following properties: (1) For all $v \in V,\ \ \ip vv\ge 0.$ (2) If $\ip vv = 0$ then $v = 0.$ (3) For all $v$ and $w$ in $V,$ $\ip vw=\overline{\ip wv}.$ (4) For all $v_1,\ v_2\put{and} w$ in $V,\quad\ip {v_1+v_2}w=\ip {v_1}w+\ip {v_2}w.$ (5) For all $v,\ w$ in $V,$ and all scalars $a,$ $\ip {av}w=a\ip vw.$} \gap In case the scalars are real, the axioms are the same, except that \ipv is assumed to be real-valued, so the complex conjugation is dropped in (3): $\ip vw=\ip wv.$ \gap When (3) is combined with (4) and (5) in turn, we have $(4')$ For all $v_1,\ v_2$ and $w$ in $V,\ \langle w,v_1 + v_2\rangle = \langle w,v_1\rangle + \langle w,v_2\rangle.$ $(5')$ For all $v,\ w \in V,$ and all scalars $a,\ \langle v,aw\rangle = \bar a\ip vw.$ Thus the inner product is linear in the first variable, and {\it conjugate linear\/} in the second. \gap The ``dot product'' in Euclidean space is the basic example of an inner product (the scalars are real in that case...). Note that $\app{Re} \ip vw$ is an inner product on the {\sl real\/} vector space obtained by restricting scalar multiplication to the real numbers. \gap {\bf Examples } include $\com$ itself, with $\ip zw = z\bar w;$ complex $C([0,\,1]),$ with $\ip fg := \int_0^1f(x)\overline{g(x)}\,dx;$ and $L^2(\real^n),$ with $\ip fg := \int f(x)\overline{g(x)}\,dx.$ \gap {\bf 2 Definition: } {\sl The {\it norm\/} of an element $v$ in an inner product space is denoted $\norm v,$ and is given by taking the non-negative square root of $\norm v^2 = \ip vv.$ That is, $\norm v=\sqrt{\ip vv\,}.$} \gap Simply calling $\norm v$ a ``norm'' does not make it one. To prove this is a norm, we'll use the very important {\it Schwarz inequality\/} in the proof of the triangle inequality. \gap {\bf 3 Theorem (The Schwarz Inequality): } {\sl In an inner product space $V,$ for all vectors $v,\ w,$ $$ |\ip vw|\le \norm v \norm w, $$ and equality holds if and only if one of $v$ and $w$ is a multiple of the other.} \Pf This argument uses the quadratic formula! If one of $v$ and $w$ is zero, then equality holds, and the one that is zero is a multiple of the other. So suppose that neither of $v$ and $w$ is zero. Let $z\in\com.$ Consider $\norm{v - zw}^2.$ Let us express $z$ in ``polar coordinates.'' For $\theta$ real and fixed (to be chosen later), and for $t \in R,$ put $z = t e^{i\theta},$ and let $f(t) = \norm{v - zw}^2.$ The following are typical {\bf \ e x p a n s i o n } steps: $$\eqalign{ f(t) &= \langle v - zw,v - zw\rangle\cr &= \langle v,v - zw\rangle - \langle zw,v - zw\rangle \cr &= \langle v,v\rangle - \langle v,zw\rangle - \langle zw,v\rangle + \langle zw,zw\rangle \cr &= \langle v,v\rangle - (\bar z\ip vw + z\langle w,v\rangle) + |z|^2\langle w,w\rangle \cr &= \langle v,v\rangle - 2 \app{Re } \bar z\ip vw + |z|^2\langle w,w\rangle\cr &= \norm v^2 - 2t \app{Re } e^{-i\theta}\ip vw + t^2 \norm w^2.\cr }$$ Next, choose $\theta$ so that $e^{-i\theta}\ip vw = |\ip vw| .$ With this choice of $\theta,$ we can write $$ f(t) =\norm v^2 - 2t |\ip vw| + t^2 \norm w^2. $$ Then the quadratic polynomial $f(t),$ being non-negative for all real $t,$ has no real roots, or one, repeated, root. In either case, it has non-positive discriminant. That is, $ 4 |\ip vw|^2 \le 4 \norm v^2 \norm w^2 ,$ as desired. \gap If equality holds, then $f(t)$ has zero discriminant, hence a root for the chosen value of $\theta.$ By the definition of $f,$ this means that $v = zw.$ \gap {\bf 4 Theorem: } {\sl An inner product space, with $\norm v$ as norm, is indeed a normed space.\/} \Pf By (1) and (2) in the definition of an inner product space, $\norm v$ is non-negative, and is 0 \iffi $v = 0.$ When $c$ is a scalar, $\norm {cv}^2 = \langle cv,cv\rangle = |c|^2\norm v^2.$ The triangle inequality is an application of the Schwarz inequality: $\norm{v+w}^2 = \norm v^2 + 2 \app{Re} \ip vw + \norm w^2 \le \norm v^2 + 2 \norm v \norm w + \norm w^2 = (\norm v+\norm w)^2;$ the inequality follows. \gap An inner product space is a {\it Hilbert space\/} if it is complete with respect to the norm just defined. Usually we work with Hilbert spaces, since it's handy to have limits of Cauchy sequences available. The first and third of the examples are Hilbert spaces; the second is not. Finite dimensional inner product spaces are Hilbert spaces. The {\bf parallelogram identity} is useful: $$ \put{For all} u \put{and} v \put{in} V,\put{} \norm {u+v}^2 + \norm {u-v}^2 = 2\norm u^2 + 2\norm v^2. \leqno{{\bf 5}}$$ The proof is a direct calculation, by expansion of the left-hand side. The {\bf polarization formula:} $$ \ip vw = {1\over 4}\sum_{k=0}^3, i^k\norm{v+i^kw}^2 . \leqno{{\bf 6}}$$ This is proved by expansion and simplification on the right-hand side. \gap {\bf 7 Definition:} {\sl Vectors $v$ and $w$ in an inner product space $V$ are {\it orthogonal\/} if $\ip vw = 0.$ In particular, $ 0 $ is orthogonal to every vector $v.$ Notation: $v\perp w.$} \gap {\bf An inner product space can be embedded into its dual space by a conjugate-linear isometry} \gap If $V$ is an inner product space, we let $V^*$ denote the space of continuous linear functionals on $V.$ Called the {\it dual space of\/} $V,\ \ V^*$ is a Banach space,\footnote {$^1$}{Completeness will be shown in ``deferred proofs,'' Item 5.} namely one in which Cauchy sequences converge, and the norm we will use on $V^*,$ at least at first, is $$ \norm {v^*}_* :=\sup_{\norm v\le 1} |v^*(v)|. \leqno{{\bf 8}}$$ We will use the notations \ipv for the inner product of $v$ and $w,$ and $\norm v$ for the norm, in $V,$ of $v.$ \gap {\bf A conjugate-linear embedding of $V$ into $V^*$} \gap There is always a natural embedding of a topological vector space into the dual space of its dual space. In the special case of inner product spaces, there is a natural embedding of $V$ into $V^*,$ given not by a linear mapping, but by a \sla{conjugate} linear mapping $E.$ For each $v_o\in V,$ we define $Ev_o(v) := \langle v,v_o\rangle.$ It is routine to show that $Ev_o$ is a linear functional on $V.$ \gap {\bf 9 Claim: } {\sl the linear functional $Ev_o$ belongs to $V^*$, and $\norm {Ev_o}_* =\norm {v_o}.$} \gap \Pf $|Ev_o(v)| = |\langle v,v_o\rangle |\le \norm v \norm {v_o},$ by the Schwarz inequality. Therefore $\norm {Ev_o}_* \le \norm {v_o}.$ If $v_o \ne 0,$ then $v := v_o/\norm {v_o}$ yields $Ev_o(v)=\langle v_o/\norm {v_o} , v_o\rangle = \norm {v_o} \le \norm {Ev_o}_*,$ so actually $\norm {Ev_o}_* = \norm {v_o}$ (if $v_o = 0,$ then $Ev_o =0$ (of $V^*)).$ \gap A pair of properties of the mapping $E\colon\ E(v_o + v_1) = Ev_o + Ev_1,$ and $Eav_o = \bar a Ev_o;$ that is, $E$ is {\it conjugate linear.\/} In particular, $E(v_o - v_1) = Ev_o - Ev_1.$ These properties are proved using the definition of $E.$ Therefore, \gap {\bf 10}\hskip 1.4in{$E$ is an isometry (a distance-preserving map) of $V$ onto a subset of $V^*.$} \gap {\bf The conjugate-linear embedding $E$ of $V$ into $V^*$ has dense range} \gap {\bf 11 Theorem: } {\sl $E(V)$ is dense in $V^*.$} \gap \Pf What we have to prove is that, for each $v^*$ in $V^*,$ there exists a sequence $\{v_n\}$ of elements of $V$ such that $Ev_n \to v^*$ in the norm of $V^*.$ Let $v^* \in V^*.$ If $v^* = 0,$ then $E0 = v^*,$ so we may assume that $v^* \ne 0.$ Further, we may assume that $\norm {v^*}_* = 1.$ Then, \gap {\sl there exists a sequence $\{v_n\}$ of elements of $V,$ with $\norm {v_n} = 1$ for each $n,$ such that $0 \le v^*(v_n) \to 1,$ from below, as $n \to\infty.$\/} \gap The non-negativity of $v^*(v_n)$ is a useful convenience. It is assured by multiplying some ``original $v_n\app{'s''}$ by suitable complex numbers of unit length, as in the proof of the Schwarz inequality. See {\bf Deferred proofs, Item 1} for details. \gap We will show that, as $n \to \infty,$ $Ev_n \to v^*,$ in the norm of $V^*.$ If we expect $v^*(v)$ to be the limit of $\langle v,v_n\rangle$ and if $w\perp v_n,$ we would expect $v^*(w)$ to be a smaller and smaller fraction of $\norm w,$ as $n$ increases. First we will prove a Lemma to that effect, and a useful Corollary of it. The Lemma gives an estimate for $v^*(w)$ when $w \perp v_n.$ \gap {\bf 12 Lemma: } {\sl If $v^*\in V^*,\ v\in V,$ and $\norm {v^*}_*=1=\norm v,$ then whenever $w\in V$ and $w\perp v,$ $$ |v^*(w)|\le\sqrt{1-|v^*(v)|^2\,}\,\norm w. \leqno{{\bf 13}}$$} {\bf Remark } If we knew that $v^*(w)=\langle w,\,\hat v\rangle$ for some $\hat v$ in $V,$ this would follow from the fact that the cosine of the angle between $\hat v$ and $w$ is more or less equal (in absolute value) to the {\sl sine\/} of the angle between $\hat v$ and $v$ in the \sla{real} vector space spanned by $\hat v$ and $v.$ \gap \Pf We notice that $|v^*(v)|\le 1,$ so the square root makes sense. We'll use the quadraric formula to make the estimate. Let $a$ be a complex number. Since $v^*$ has norm one and $w \perp v,$ $$ |v^*(av + w)|\le\norm {v^*}_*\norm {av + w} =1\cdot\sqrt{|a|^2 + 2 \app{Re } \langle av, w\rangle + \norm w^2 }=\sqrt{|a|^2 + \norm w^2\, }. $$ Thus $|v^*(av + w)|^2 \le |a|^2 + \norm w^2.$ \gap Here is another way to express $|v^*(av + w)|^2\colon$ $$ |v^*(av + w)|^2 = |v^*(av) + v^*(w)|^2 =|a|^2v^*(v)^2 + 2\app{Re }a v^*(v) \overline{v^*(w)} + |v^*(w)|^2. $$ Therefore, $$ |a|^2v^*(v)^2 + 2\app{Re }a v^*(v) \overline{v^*(w)} + |v^*(w)|^2 \le |a|^2 + \norm w^2. $$ Now we let $a = re^{i\fie}$ where $r$ is an \sla{arbitrary} real number and we choose $\fie,$ also real, so that $a v^*(v) \overline{v^*(w)} = r|v^*(v)||v^*(w)|.$ After we substitute this formula $a = re^{i\fie}$ into the last inequality and do some rearranging, we find that, for all real $r,$ $$ 0 \le r^2(1 - |v^*(v)|^2) - 2 r|v^*(v)||v^*(w)| + (\norm w^2 - |v^*(w)|^2). \leqno{{\bf 14}}$$ There are two cases to consider now: $|v^*(v)|^2=1$ and $|v^*(v)|^2<1.$ If $|v^*(v)|^2=1$ then {\bf 14} becomes $$ \put{\fA}r\in\real,\put{}2 r|v^*(w)|\le (\norm w^2 - |v^*(w)|^2), $$ which can only be true if $v^*(w)=0.$ Thus {\bf 13} holds in this case. Thanks here are due to [3]! \gap If $|v^*(v)|^2<1$ the discriminant of the non-negative quadratic in {\bf 14} must therefore be non-positive; that is, $$ |v^*(v)|^2|v^*(w)|^2 \le (1 - |v^*(v)|^2) (\norm w^2 - |v^*(w)|^2). $$ We add $(1 - |v^*(v)|^2)|v^*(w)|^2$ to both sides of this inequality and take square roots to complete the proof of {\bf 13.} \gap {\bf 15 Corollary: } {\sl If $v^*\in V^*,\ v\in V,\quad {\norm {v^*}}_*=1=\norm v,$ and $0\le v^*(v),$ then $\|Ev-v^*\|_*\le\del+\del^2,$ where $\del$ is non-negative and $\del^2:=1-v^*(v)^2.$} \gap \Pf For any $u\in V,$ $$ (Ev - v^*)(u) = \langle u,v\rangle- v^*(u) =\langle u,v\rangle - v^*(u - \langle u,v\rangle v) - \langle u,v\rangle v^*(v) =(1-v^*(v))\langle u,v\rangle-v^*(w), $$ where $w := u - \langle u,v\rangle v \perp v.$ By Pythagoras' Theorem, applied to $u=w+\langle u,v\rangle v,$ $\norm w\le \norm u.$ \gap Thus by {\bf Lemma 12,} with $\sqrt{1-|v^*(v)|^2\,}$ there replaced by $\del,$ so that $|v^*(w)|\le \del\norm w\le \del\norm u,$ $$| (Ev - v^*)(u) | \le | \langle u,v\rangle |(1 - v^*(v) ) + \del\norm w \le \norm u(\del^2 + \del). $$ This completes the proof (we actually got the smaller but uglier estimate $\|Ev-v^*\|_*\le\del+(1-v^*(v))\ ).$ \gap We can now quickly complete the proof of {\bf Theorem 11.} We had to show that as $n \to \infty,$ $Ev_n \to v^*$ in the norm of $V^*.$ We set $\del_n:=\sqrt{1-|v^*(v_n)|^2\,}.$ By {\bf Corollary 15,} $\norm {Ev_n - v^*}\le \del_n(1 + \del_n)\to 0$ as $n \to\infty.$ We're done! \gap {\bf Consequences of Theorem 11, and what preceded it} \gap 1. $\{Ev_n\}$ is a Cauchy sequence in $V^*$ because it converges. By the isometric property of $E,$ $\{v_n\}$ is Cauchy in $V,$ so if $V$ is already complete, and $v := \lim_{n\to\infty} v_n,$ then $v^* = Ev.$ Thus, if $V$ is a Hilbert space, $E$ is onto as well as one-to-one. This gives us the \gap {\bf 16 The Riesz Representation Theorem: } {\sl Let $H$ be a Hilbert space. If $\lam(x)$ is a continuous linear functional on $H,$ then there exists a unique $y\in H$ such that $\lam(x) = \langle x,y\rangle,$ and $\norm{\lam}_* = \norm{y}.$} \gap In other words, $E$ is an isometric one-to-one correspondence between $H$ and $H^*.$ \gap 2. If $E$ is onto, the isometric property shows that, because $V^*$ is complete, so is $V.$ \gap 3. The inner product can be ``exported'' to $V^*.$ We begin by assuming that $V$ is not complete. We let $$ \langle v^*,w^*\rangle^* :=\lim_{n\to\infty} {1\over 4}\sum_{k=0}^3 i^k\norm {w_n+i^kv_n}^2 =\lim_{n\to\infty} \langle w_n,v_n\rangle,\put{for}v^*,\ w^*\put{in}V, $$ where $Ev_n\to v^*,\ Ew_n\to w^*.$ \gap {\bf To show $\langle v^*,w^*\rangle^*$ is well-defined} \gap Suppose $E\tilde v_n\to v^*,\ \ E\tilde w_n\to w^*.$ Then $$ \norm {\tilde w_n+i^k\tilde v_n}^2 =\norm {\tilde w_n - w_n + w_n +i^k\tilde v_n}^2 =\norm {\tilde w_n - w_n}^2+ 2\app{Re } \langle \tilde w_n - w_n,\,w_n + i^k\tilde v_n\rangle + \norm {w_n + i^k\tilde v_n}^2. $$ The first 2 terms tend to zero. We repeat the calculation to replace $\tilde v_n$ by $v_n.$ Each sequence is Cauchy in $V$ because the mapping $E$ is isometric. \gap {\bf To show: the well-defined quantity $\langle v^*,w^*\rangle^*$ is an inner product} \gap It is immediate that $\langle v^*,w^*\rangle^*$ is additive in each argument. Congugate symmetry and the properties of $\langle v^*,v^*\rangle^*$ are also immediate. Since $Ev_n\to v^*,$ for any scalar $a,\ \ E(\bar a v_n)\to av^*,$ so $$ \langle av^*,w^*\rangle^* =\lim_{n\to\infty} \langle w_n,\bar a v_n\rangle = a\langle v^*,w^*\rangle^*. $$ A similar arugment shows $\langle v^*,aw^*\rangle^* = \bar a \langle v^*,w^*\rangle^*.$ The norm given by this inner product: $$ \norm {v^*}^2=\lim_{n\to\infty}\langle v_n,v_n\rangle =\lim_{n\to\infty}Ev_n(v_n) =\lim_{n\to\infty}\norm{Ev_n}^2_* $$ agrees with the standard norm $\norm {v^*}_*$ in $V^*,$ and this completes the proof that $\langle v^*,w^*\rangle^*$ is an inner product on $V^*.$ \gap If $V$ is a Hilbert space, we may use $\langle E^{-1}v^*, E^{-1}w^*\rangle $ in place of the limits used in the definition of the inner product on $V^*.$ \gap We have shown: \gap {\bf 17 Theorem: } {\sl If $V$ is an inner product space, then $V^*$ is a Hilbert space that is homeomorphic to the completion of $V$ with respect to its norm. Moreover, $\ip{Ev}{Ew}^*=\ip{v}{w}$ \fA $v$ and $w$ in $V.$\/} \gap {\bf Orthogonal decomposition and projections} \gap We can ``drop a perpendicular'' in a Hilbert space. Put another way: if $d$ is the distance from a point $y$ to a closed convex set $X$ in $H$, then the closed ball of radius $d,$ center $y,$ meets $X$ at exactly one point. With reference to that point, ``real'' angles between $y$ and points in $X$ are at least $90^\circ.$ \gap {\bf 18 Theorem: } {\sl If $X$ is a closed convex set in a Hilbert space $H,$ then for every $y$ in $H,$ there is a unique $\xi\in X$ such that $\app{Re }\langle y - \xi, x - \xi\rangle \le 0$ for all $x\in X.$ Indeed, $\xi$ is the element of $X$ closest to $y.$} \gap \Pf This classic argument exploits the parallelogram identity. Let $d := \app{dist}(y,X) = \inf_{x\in X}\norm{y-x}.$ Then there is a sequence $\{x_n\}$ in $X$ such that $d =\lim_{n\to\infty}\norm{y-x_n}.$ We define $\ep_n$ by $\ep_n^2 = \norm{y-x_n}^2 - d^2.$ Then $$ \norm{(y-x_n) + (y-x_m)}^2 + \norm{x_m-x_n}^2 = 2\norm{y-x_n}^2 + 2\norm{y-x_m}^2, $$ or (making changes on both sides of this equation) $$ 4\left\|y - {x_n + x_m\over 2}\right\|^2 + \norm{x_m-x_n}^2 = 4d^2 + 2\ep_n^2 + 2\ep_m^2. $$ Since $X$ is convex, $\norm{y - {x_n + x_m\over 2} } \ge d.$ Hence $4d^2 + \norm{x_m-x_n}^2 \le 4d^2 + 2\ep_n^2 + 2\ep_m^2.$ Thus $\{x_n\}$ is Cauchy, and so converges to an element $\xi$ of $X.$ This argument, applied with some other minimizing sequence $\{\hat x_n\}$ in place of $\{x_m\}$ and ${\hat{\ep_n}}^2$ in place of $\ep_m^2,$ shows the uniqueness of $\xi.$ \gap To verify the statement about angles in the ``real'' version of $H,$ let $x\in X.$ Then $$ d^2 \le \norm{y - x}^2 = \norm{y - \xi}^2 + 2 \app{Re } \langle y - \xi,\xi - x\rangle + \norm{\xi - x}^2. $$ Since $d=\norm{y- \xi},$ $$ 0 \le 2 \app{Re } \langle y - \xi,\xi - x\rangle + \norm{\xi - x}^2, \put{which yields:}\app{Re } \langle y - \xi,x-\xi\rangle \le {1\over 2}\norm{x-\xi}^2. $$ For $0 < r < 1,$ let $x^* := \xi + r(x-\xi)=(1-r)\xi+rx\in X,$ so $\app{Re}\langle y - \xi,x^*-\xi\rangle \le {1\over 2}\norm{x^*-\xi}^2.$ Since $x^* - \xi = r(x - \xi),$ we have $\app{Re } \langle y - \xi,x-\xi\rangle \le {r\over 2}\norm{x-\xi}^2.$ We now let $ r\to 0.$ \gap {\bf Remark } The argument just completed really took place in the {\sl real\/} vector space spanned by $x,\ y$ and $\xi,$ using the given inner product. \gap {\bf 19 Corollary: } {\sl If $X$ is a closed subspace of $H,$ then $y- \xi\perp X.$ If $\xi'\in X$ and $y - \xi' \perp X,$ then $\xi' = \xi.$} \gap \Pf Because $X$ is a subspace, we also have $x^* := \xi - r(x-\xi) \in X,$ so $\app{Re } \langle y - \xi,x - \xi\rangle \ge 0$ as well. The same is true when $r$ is replaced by $ir,$ or by $-ir.$ This yields $\app{Im } \langle y - \xi,x-\xi\rangle = 0,$ so that $y- \xi\perp X.$ \gap If $ \xi'\in X$ and $y - \xi' \perp X,$ then $$ d^2 = \norm{y - \xi}^2 = \norm{y - \xi' + \xi' - \xi}^2 = \norm{y - \xi'}^2 + \norm{\xi' - \xi}^2 \ge d^2 + \norm{\xi' - \xi}^2, $$ and this implies that $\xi' = \xi.$ \gap {\bf20 Definitions of orthogonal complement and orthogonal projection} \gap If $X$ is a closed subspace of $H,$ set $X^\perp = \{y\in H: \langle x,\,y\rangle = 0 \put{for all} x\in X\}.$ Then $X^\perp$ is a closed subspace (routine to show it), and $X^\perp \cap X = \{0\}.$ $X^\perp$ is called the {\it orthogonal complement\/} of $X.$ For $u\in H,$ let $P(u) = P_X(u)$ denote the element of $X$ closest to $u.$ Recall that $P(u)$ is unique and $P(u)$ is the only element $\xi$ of $X$ such that $u-\xi\perp X.$ Let us show that $u\ \mapsto P(u)$ is a linear map. Suppose that $a,\ b$ are scalars, and $u,\ v$ elements of $H.$ Then $$ \langle au + bv - (aP(u) + bP(v)), x\rangle = \langle au - aP(u), x\rangle + \langle bv - bP(v), x\rangle = a\langle u - P(u), x\rangle + b\langle v - P(v), x\rangle = 0 $$ for all $x\in X.$ Hence, $P(au + bv) = aP(u) + bP(v).$ Now we can express $u = P_Xu + (u - P_Xu)$ as the sum of terms $P_Xu\in X$ and $(I - P_X)u=u - P_Xu\in X^\perp.$ This implies too that $I - P_X = P_{X^\perp}.$ These are called the {\it orthogonal projections\/} onto $X$ and $X^\perp,$ respectively. It is routine to show that they are projections, e.g. that $P_X^2=P_X.$ Orthogonality shows that ${\norm u}^2 = \norm{P_Xu}^2 + \norm{u - P_Xu}^2 \ge \norm{P_Xu}^2,$ so $P_X$ is continuous, and has (routine) operator norm $1.$ The formula $I - P_X = P_{X^\perp}$ leads easily to a proof of the relation $(X^\perp)^\perp = X.$ All this can be applied to deduce such things as: Ths span of a subset $S$ of $H$ is dense in $H$ if and only if $ y\perp S$ implies $y = 0.$ \gap {\bf An additional property of these projections: self-adjointness } \gap {\bf 21 Theorem: } {\sl A linear mapping $P:H\to H$ is the projection $P_X$ onto some closed subspace $X$ of a Hilbert space $H$ \iffi $P^2=P\put{and \fA} u,\ v\in H,\ \ \langle Pu,\,v\rangle=\langle u,\,Pv\rangle.$\/} \gap \Pf First we will suppose that $P=P_X,$ where $X$ is a closed subspace of $H.$ We have already seen that $P^2=P$ in this case. To show that we can move $P$ from one side of the inner product to the other, let $u$ and $v$ be given in $H.$ Then $v=Pv+(I-P)v$ and $(I-P)v\in X^\perp,$ while $Pu\in X,$ so $\langle Pu,\,v\rangle=\langle Pu,\,Pv\rangle.$ The same argument, applied to $\langle u,\,Pv\rangle$ with the roles of $u$ and $v$ reversed, shows that $\langle u,\,Pv\rangle=\langle Pu,\,Pv\rangle.$ This completes this half of the proof. \gap Now we assume $P^2=P\put{and \fA} u,\ v\in H,\ \ \langle Pu,\,v\rangle=\langle u,\,Pv\rangle.$ Let us first show that $P$ is a bounded operator: $$ \|Pu\|^2 =\langle Pu,\,Pu\rangle=\langle u,\,P^2u\rangle =\langle u,\,Pu\rangle\le \|u\|\|Pu\| $$ by the Schwarz inequality. We can cancel $\|Pu\|$ if $\|Pu\|\ne0.$ Thus even if $\|Pu\|=0$ we have $\|Pu\|\le\|u\|$ \FA $u\in H,$ so $P$ is bounded. \gap We (naturally?) define $X$ to be the image of $P,$ namely $X:=\{x\in H:Px=x\}.$ This {\sl is\/} the image of $P$ since $P^2=P.$ \gap To show $X$ is closed, we let $x_n\in X$ and suppose $x_n\to y.$ Then $x_n=Px_n\to Py,$ so $Py=y$ and thus $X$ is closed. \gap To finish the proof it is enough to show that \fA $u\in H,\ u-Pu\perp X.$ Let $x\in X.$ Then $$ \langle u-Pu,\,x\rangle=\langle u,\,x\rangle-\langle Pu,\,x\rangle =\langle u,\,x\rangle-\langle u,\,Px\rangle=0. $$ \gap {\bf Existence and properties of an orthonormal basis } \gap {\bf 22 Definition: } A set $S$ in an inner product space is {\it orthogonal\/} if $v \perp w$ whenever $v$ and $w$ are two different elements of $S.$ If every element of an orthogonal set $S$ has norm $1,$ we say $S$ is {\it orthonormal.\/} \gap {\bf 23 Theorem: } {\sl Every non-trivial inner product space has a maximal orthonormal set.\/} \gap \Pf This argument uses one of the forms of the Axiom of Choice (called ``The Maximal Principle'' in [1, p. 33]). The collection of (non-empty) orthonormal subsets of an inner product space is non-empty, since for each non-zero $v\in V,\ \{v/\norm v\}$ is a non-empty orthonormal set. If a collection of orthonormal sets is linearly ordered by inclusion, it is routine to show that the union of them all is an orthonormal set. Hence, there is a maximal such set. \gap {\bf 24 Corollary: } {\sl Every orthonormal subset of a Hilbert space is contained in some maximal orthonormal set.\/} \gap {\bf 25 Theorem: } {\sl The span of a maximal orthonormal set in a Hilbert space is dense.\/} \gap \Pf Suppose not. Let $X$ denote the closure of the span of the maximal orthonormal set under discussion. Let $y\in H\setminus X.$ Then $0 \ne v = y-P_Xy \in X^\perp,$ so the union of the given maximal orthonormal set and $\{v/\norm v\}$ is a larger orthonormal set, contradicting maximality. \gap {\bf 26 Definition: } {\sl A maximal orthonormal set in a Hilbert space is called an {\it orthonormal basis.\/}\/} \gap An orthonormal basis is not a basis in the usual sense, unless it is finite. This is a consequence of completeness, and will be shown later, in {\bf Deferred proofs, Item 2}. One feature of orthonormal sets is: \gap {\bf 27 Theorem (Bessel's inequality): } {\sl If ${\cal O}$ is an orthonormal set in an inner product space $V,$ then for each $v\in V,$ at most countably many of the numbers $\langle v,y\rangle$ can be non-zero, and $\sum_{y\in {\cal O}}|\langle v,y\rangle |^2 \le {\norm v}^2.$} \gap \Pf Let ${\cal F}$ be a finite subset of ${\cal O}.$ Let $w =\sum_{y\in {\cal F}}\langle v,y\rangle y.$ Then, by orthonormality, $$ \norm w^2 = \norm{\sum_{y\in {\cal F}}\langle v,y\rangle y\, }^2 = \sum_{y\in {\cal F}}|\langle v,y\rangle |^2 , $$ and $y\perp v-w$ for each $y\in{\cal F}.$ Thus, $w\perp v-w,$ so $\norm v^2= \norm w^2 + \norm{v-w}^2 \ge \norm w^2,$ as claimed, at least for finite orthonormal sets. \gap It follows that there are only finitely many $y\in {\cal O}$ such that $|\langle v,y\rangle | \ge 1,\ \ |\langle v,y\rangle | \ge 1/2,\ \ |\langle v,y\rangle | \ge 1/3,$ and so on. This proves the countability assertion. We {\it define\/} $\sum_{y\in {\cal O}}|\langle v,y\rangle |^2$ as follows: $$ \sum_{y\in {\cal O}}|\langle v,y\rangle |^2 := \sup_{y\in {\cal F}\sub{\cal O},\ {\cal F} \app{\ \small{finite} }} \sum_{y\in {\cal F}}|\langle v,y\rangle |^2 . $$ Each sum on the right is bounded by $\norm v^2,$ so Bessel's Inequality holds. Now $\norm v^2 = \sum_{y\in {\cal F}}|\langle v,y\rangle |^2 + \norm{v-w}^2 ,$ where $w = \sum_{y\in {\cal F}}\langle v,y\rangle y.$ Since $\norm{v-w}^2=\dsp\inf_{\hat w\in \app{\small{ span}}\,{\cal F}}\norm{v-\hat w}^2,$ we can show that $$ \norm v^2 = \sum_{y\in {\cal O}}|\langle v,y\rangle |^2 + \inf_{w\in \app{\small{ span}}\,{\cal O}}\norm {v-w}^2 =\sum_{y\in {\cal O}}|\langle v,y\rangle |^2 + d^2, $$ where $d^2$ denotes the square of the distance from $v$ to $\overline{\app{ span}\,{\cal O}} .$ Let us prove this (in {\bf Deferred proofs, Item 3}) after we look at some applications. If ${\cal O}$ is an orthonormal basis then $d^2 = 0,$ and so, in a Hilbert space, \gap {\bf 28 Theorem (Parseval's relation): } {\sl If ${\cal O}$ is an orthonormal basis in a Hilbert space, then for all $x\in H,$ $$ \norm {x}^2 =\sum_{y\in {\cal O}}|\langle x,y\rangle |^2. $$} \gap Polarization, in $H$ and in $\com$ gives Plancherel's Theorem: \gap {\bf 29 Theorem (Plancherel's Theorem): } {\sl Suppose ${\cal O}$ is an orthonormal basis in a Hilbert space $H.$ Then, for all $x\in H,\ y\in H,$ $$ \langle x,y\rangle = \sum_{u\in {\cal O}}\langle x,u\rangle \langle u,y\rangle . $$} \gap An application of Parseval's relation: if $\langle x,y\rangle = \langle x',y\rangle$ for all $y$ in an orthonormal basis of a Hilbert space, then $x = x'$ (we replace $x$ by $x-x'$ in Parseval's relation). \gap Just as these numerical series converge, so do vector-valued series of the form $\sum_{y\in {\cal O}}c_yy,$ where ${\cal O}$ is an orthonormal set in a Hilbert space, whenever $\sum_{y\in {\cal O}}|c_yy|^2 < \infty.$ Proof that the ``sum'' is independent of the order of the terms will be part of {\bf Deferred proofs (Item 4)}. Proof that a specific (as to order) such ``sum'' exists is part of the proof of the next Theorem, in which we change our point of view, starting there with a set of coefficients as ``givens.'' \gap {\bf 30 Theorem of Fischer and Riesz: } {\sl If ${\cal O}$ is an orthonormal set in a Hilbert space $ H,$ and for each $y\in {\cal O},\ \ c_y$ is a given complex number such that $\sum_{y\in {\cal O}}|c_y|^2 < \infty,$ then there exists $x\in H$ such that $\langle x,y\rangle = c_y$ for all $y\in {\cal O}.$} \gap \Pf Since $\sum_{y\in {\cal O}}|c_y|^2 < \infty,$ the set of $y$ such that $c_y$ is not zero is countable. They can be enumerated in some way: $y_1,\ y_2,\ \dots$ Consider $x_n:= \sum_{k=1}^n c_ky_k,$ where $c_k$ denotes the cumbersome $c_{y_k}.$ If $m < n,$ then $\norm {x_n-x_m}^2 = \sum_{k=m+1}^n|c_k|^2,$ so $\{x_n\}$ is Cauchy, hence has a limit $x$ in $H$. By continuity of the inner product, for $k$ fixed $\langle x,y_k\rangle = \lim_{n\to\infty}\langle x_n,y_k\rangle = c_k.$ If $ y\in {\cal O}$ is not one of the $y_k,$ then $\langle x_n,y\rangle = 0$ for every $n,$ so $\langle x,y\rangle = 0 = c_y.$ \gap {\bf Hilbert space isomorphism} \gap Here we take up the question of when two Hilbert spaces are isommorphic in a way that preserves ``Hilbert space structure.'' The answer depends on {\bf the} cardinal number of an orthonormal basis. \gap {\bf 31 Theorem: } {\sl Two orthonormal bases in a Hilbert space have the same cardinal number.\/} \gap \Pf Let ${\cal U},\ {\cal V}$ be orthonormal bases for a Hilbert space $H.$ If one is finite so is the other and they have the same number of elements, by the replacement theorem from linear algebra. Otherwise, without loss of generality we may assume $\app{card }{\cal V} \le \app{card } {\cal U}.$ For each $v\in {\cal V},$ let $U(v) = \{u\in {\cal U}: \langle u,v\rangle \ne 0\}.$ Each $U(v)$ is nonempty, countable, and $\bigcup_{v\in {\cal V}}U(v) = {\cal U}.$ In particular, if ${\cal V}$ is countable, so is ${\cal U}.$ If not, the cardinal number of the union is at most $\app{card } {\cal V}.$ Hence $\app{card } {\cal V} \ge \app{card } {\cal U},$ so $\app{card } {\cal V} = \app{card } {\cal U},$ as desired. \gap {\bf 32 Definition: } {\sl The common cardinal number of the orthonormal bases of a Hilbert space is called the {\it Hilbert space dimension\/} of H.\/} \gap \small{I don't know how common this term is...} \gap Two Hilbert spaces are isomorphic as Hilbert spaces if there is a linear one-to-one correspondence between them that preserves inner products. These special operators are called {\it unitary operators.\/} A linear one-to-one mapping that preserves inner products is a unitary mapping from its domain onto its image. \gap {\bf 33 Problem } Show that, if $f:H_1\to H_2$ is a function that is onto and that preserves inner products (that is, \fA $u,\ v\in H_1,$ $\ip{f(u)}{f(v)}_{H_2}=\ip uv_{H_1}),$ then $f$ is linear and one-to-one, so that $f$ is unitary. \gap {\bf Theorem: } {\sl Hilbert spaces $H_1$ and $H_2$ are isomorphic as Hilbert spaces \iffi they have the same Hilbert space dimension.\/} \gap \Pf Let ${\cal O}_1,\ {\cal O}_2$ be orthonormal bases in $H_1,\ H_2$ respectively. If the Hilbert space dimensions are the same, let $\lam$ be a one-to-one correspondence between ${\cal O}_1$ and ${\cal O}_2 .$ Then $Ux := \sum_{y_1\in {\cal O}_1}\langle x,y_1\rangle \lam(y_1) $ is a unitary isomorphism. This is an application of previous theorems. Now suppose $ U: H_1 \to H_2$ is a unitary isomorphism. Then $U({\cal O}_1)$ is an orthonormal set in $H_2.$ Since $$ \overline{\app{span} U({\cal O}_1)} = \overline{U(\app{span} {\cal O}_1)} = U\left(\,\overline{\app{span} {\cal O}_1}\,\right) = H_2, $$ $U({\cal O}_1)$ is maximal, so $\app{dim }H_2 = \app{card } U({\cal O}_1) = \app{dim }H_1.$ \gap {\bf 34 Theorem: } {\sl A Hilbert space $H$ is separable \iffi it has a countable orthonormal basis.\/} \gap \Pf If $H$ has a countable orthonormal basis then $H\simeq\ell^2,$ which is separable. \gap If $H$ is separable and ${\cal U}$ is an orthonormal basis of $H$ then there is a countable dense subset $\{y_k\}_{k=1}^\infty$ of $H.$ For each element $u\in{\cal U}$ there is some positive integer $k(u)$ \st $\|u-y_{k(u)}\|<1/2.$ If ${\cal U}$ were uncountable there would exist $u_1\ne u_2$ in ${\cal U}$ \st $k(u_1)=k(u_2)=:K.$ But then $2=\|u_1-u_2\|^2\le(\|u_1-y_K\|+\|y_K-u_2\|)^2<1.$ This contradiction shows that ${\cal U}$ is countable. \gap {\bf 35 Problem } Prove that every $x\in H$ has the norm-convergent ``Fourier series'' $$ x=\sum_{y\in {\cal O}} \ip{x}{y}y $$ \wrt an orthonormal basis ${\cal O}.$ The order of summation is irrelevant when the sum is taken over those $y\in {\cal O}$ \st $\ip{x}{y}\ne 0.$ How is this related to the Theorem of Fischer and Riesz? \gap {\bf Deferred proofs} \gap {\bf Item 1 }The following appeared in the proof that $E(V)$ is dense in $V^*$ ({\bf Theorem 11}). \gap We assumed that $\norm {v^*}_* = 1.$ We want to show that there exists a sequence $\{v_n\}$ of elements of $V,$ with $\norm {v_n} = 1$ for each $n,$ such that $0 \le v^*(v_n) \to 1,$ as $n \to\infty,$ with all $v^*(v_n)\le 1.$ \gap $\norm {v^*}_* = 1$ means that there exist vectors $\tilde v_n\ne 0$ such that $\|\tilde v_n\|\le 1$ and $|v^*(\tilde v_n)|\to 1.$ We set $v_n:=e^{i\theta_n}\tilde v_n,$ where the numbers $\theta_n$ will be chosen in a moment, we have, since $\|\tilde v_n\|\le 1,$ that $$ 1\ge|v^*(v_n)|=\left|v^*({\tilde v_n\over \|\tilde v_n\|})\right| ={1\over \|\tilde v_n\|}|v^*(\tilde v_n)|\ge |v^*(\tilde v_n)|\to 1, $$ so $|v^*(v_n)|\to 1,$ by the Squeeze Principle. We now choose $\theta_n$ so that $v^*(e^{i\theta_n}\tilde v_n)=e^{i\theta_n}v^*(\tilde v_n)=|v^*(\tilde v_n)|.$ When we divide by $\|\tilde v_n\|$ we get what we wanted: $v^*(v_n)=|v^*(v_n)| \to 1.$ \gap {\bf Item 2 }An orthonormal basis is not a basis in the usual sense, unless it is finite. This is a consequence of completeness. \gap Suppose not, namely, we have an infinite orthonormal basis that is a basis in the usual sense. \gap We may select a denumerable set $\{y_n\}_{n=1}^\infty$ of members of the orthonormal basis. Then the following series (i.e. sequence of partial sums) converges in the Hilbert space to a non-zero vector $x\colon$ $$ \sum_{n=1}^\infty {y_n\over n^2}. $$ Proof that this is so is left to the reader. It involves straightforward checking that the definition of ``Cauchy sequence'' is satisfied by the partial sums. The limiting vector $x$ is non-zero because $\langle x,y_1\rangle=1.$ \gap Since our o.n. basis is a linear-algebra basis, we can also write $x=\sum_{y\in{\cal F}} c_y\,y,$ where ${\cal F}$ is a finite subset of our o.n. basis. Therefore $$ 0=\sum_{n=1}^\infty {y_n\over n^2}-\sum_{y\in{\cal F}} c_y\,y. $$ But ${\cal F}$ is finite, so for all $k$ sufficiently large we have to have $$ 0=\left\langle\sum_{n=1}^\infty {y_n\over n^2} -\sum_{y\in{\cal F}} c_y\,y,\,y_k\right\rangle =\left\langle\sum_{n=1}^\infty {y_n\over n^2} ,\,y_k\right\rangle=1/k^2, $$ which is a contradiction. \gap {\bf Remark }Completeness was really used in the last argument! Here is an example of a normed {\sl incomplete\/} space with a countable basis in the linear-algebra sense. Let $V$ be the collection of all polynomials in $d$ real variables. This means that a typical element of $V$ has the form $$ P(x)=\sum_{\alpha\ge0} p_\alpha x^\alpha, $$ where only finitely many of the coefficients $p_\alpha$ are non-zero, the quantities $\alpha$ are ``multi-indices'' belonging to $\nat^d,$ the collection of all $d\app{-tuples}$ of non-negative integers, and $x^\alpha:=x_1^{\alpha_1}\cdots x_d^{\alpha_d}.$ For example, $P(x):=|x|^2=\sum_{k=1}^d x^{2e_k}.$ \gap We define the norm of $P$ by $\|P\|^2:=\sum_{\alpha\ge0} |p_\alpha^2|.$ The set $\{x^\alpha: \alpha\in \nat^d\}$ is a basis for $V$ in the sense of linear algebra. This example is useful in applications. \gap {\bf Item 3 }We are to show that \fA $v\in H$ (and we will assume $v\ne0)$ $$ \norm v^2 = \sum_{y\in {\cal O}}|\langle v,y\rangle |^2 + \inf_{w\in \app{\small{ span}}\,{\cal O}}\norm {v-w}^2 =\sum_{y\in {\cal O}}|\langle v,y\rangle |^2 + d^2, $$ where $d^2$ denotes the square of the distance from $v$ to $\overline{\app{ span}\,{\cal O}} .$ \gap First, we know that the projection operator $P_o$ for $\overline{\app{ span}\,{\cal O}}$ is defined and continuous. We are given some $v\in H.$ Thus we know that $$ d^2=\inf_{w\in \app{\small{ span}}\,{\cal O}}\norm {v-w}^2=\norm {v-P_ov}^2. $$ For the given $v,$ we let $NZ:=\{y\in{\cal O}:\langle v,y\rangle\ne0\}.$ Then $NZ$ is countable, so we can enumerate the elements in $NZ,$ putting them into a sequence $\{y_k\}_{k=1}^\infty.$ Let us define $v_o:=\sum_{k=1}^\infty\langle v,y_k\rangle y_k.$ To show that this definition makes sense, we set $v_n:=\sum_{k=1}^n\langle v,y_k\rangle y_k$ and proceed as we did in the proof of the Theorem of Fischer and Riesz, to show that the sequence $\{v_n\}$ is Cauchy. We then set $v_o$ equal to the limit. In particular, we have $\|v_n-v_o\|\to 0.$ Now suppose that $y\in{\cal O}.$ Then $$ \langle v_o,y\rangle =\lim_{n\to\infty}\langle v_n,y\rangle =\lim_{n\to\infty} \sum_{k=1}^n\langle\langle v,y_k\rangle y_k,y\rangle =\left\{ \matrix{\langle v,y\rangle,\put{if}y\in NZ\cr \ \ 0,\quad\put{if}y\notin NZ.} \right. $$ Therefore \fA $y\in{\cal O},$ we have $\langle v-v_o,y\rangle=0.$ The same is true when $y$ is replaced by any element of $\app{ span}\,{\cal O}.$ Now let us suppose that $w\in \overline{\app{ span}\,{\cal O}}.$ Then there is a sequence $\{w_k\}$ of elements of $\app{ span}\,{\cal O}$ \st $w_k\to w.$ This gives us $$ \langle v-v_o,w\rangle=\lim_{k\to\infty}\langle v-v_o,w_k\rangle=0. $$ That is, $v-v_o\perp w$ \fA $w\in \overline{\app{ span}\,{\cal O}}.$ By the uniqueness of the projection, $v_o=P_ov.$ Therefore $$ \|v\|^2=\|v-v_o\|^2+\|v_o\|^2=\|v-P_ov\|^2 +\sum_{k=1}^\infty|\langle v,y_k\rangle |^2 =d^2+\sum_{y\in {\cal O}}|\langle v,y\rangle |^2,\put{as desired.} $$ {\bf Item 4 }Proof that the ``sum'' $\sum_{y\in {\cal O}}c_yy,$ where ${\cal O}$ is an orthonormal set in a Hilbert space, and $\sum_{y\in {\cal O}}|c_yy|^2 <\infty,$ is independent of the order of the terms. \gap As in Item 3 and as in the proof of the Theorem of Fischer and Riesz, for every enumeration of the non-zero coefficients $c_y,$ we have a well-defined element of $H$ given by a Cauchy sequence. Let us choose one enumeration as the starting one. Then every other enumeration is a rearrangement of the chosen one. Let us distinguish them by the name of the mapping $\pi:\ints^+\to\ints^+,$ one-to-one and onto, that accomplishes the rearrangement. Thus we let $c_k$ denote the coefficients of the starting element, $x_o:=\sum_{k=1}^\infty c_k\,y_k,$ and we let $x_\pi:=\sum_{n=1}^\infty c_{\pi n}\,y_{\pi n}.$ We want to show that $x_\pi=x_o$ no matter which $\pi$ is used. We can do this by showing that, \fA $\ep>0,$ $\|x_\pi-x_o\|<\ep.$ We may choose $K$ so large that $$ \sum_{k>K}|c_k|^2<\ep^2/9. $$ We can then be sure that there is $N$ so large that for each $k\le K,$ it is true that $k\in\{\pi 1,\,\dots\,\pi N\}.$ Then $$ x_o-x_\pi=\sum_{k=1}^K c_k\,y_k+R_{o,K} -\sum_{n=1}^N c_{\pi n}\,y_{\pi n}-R_{\pi,N}, $$ where the terms with $R$ denote the ``tails'' of the corresponding series. All the terms in the very first sum are cancelled by terms in the first ``negated'' sum. We can thus write $$ x_o-x_\pi=R_{o,K} -\sum_{n=1}^N [\pi n>K]c_{\pi n}\,y_{\pi n}-R_{\pi,N}. $$ Thus $\|x_o-x_\pi\|\le\|R_{o,K}\| +\|\sum_{n=1}^N [\pi n>K]c_{\pi n}\,y_{\pi n}\| +\|R_{\pi,N}\|.$ By construction, $\|R_{o,K}\|<\ep/3.$ Since we have made no use at all of rearrangement invariance, we can use Parseval's relation on the Hilbert space $\overline{\app{ span}\,{\cal O}}.$ Thus $$ \left\|\sum_{n=1}^N [\pi n>K]c_{\pi n}\,y_{\pi n}\right\|^2 =\sum_{n=1}^N [\pi n>K]|c_{\pi n}\,y_{\pi n}|^2 \le\sum_{k>K}|c_k|^2<\ep^2/9 $$ and (similarly) $\|R_{\pi,N}\|^2<\ep^2/9.$ Thus $\|x_\pi-x_o\|<\ep.$ It follows that $x_\pi=x_o,$ which is what we had to show. \gap This work allows us to regard series of the form $\sum_{y\in {\cal O}}c_yy$ as well-defined vectors in a Hilbert space, provided that $\sum_{y\in {\cal O}}|c_y|^2<\infty.$ \gap {\bf References} \gap [1] J. L. Kelley, {\it General Topology,\/} \ D. Van Nostrand, 1955. (Chapter 0, last section: Hausdorff Maximal Principle) \gap [2] K. Yosida, {\it Functional Analysis,\/} \ Springer Verlag, 1965. (Chapter I, \S5 and Chapter III) \gap [3] The Math 8802 class, Spring 2001. \bye It is straightforward to show that such correspondences are linear and continuous.