Subsection3.4.10(Optional) — Derivation of the Error Formulae

In this section we will derive the formula for the error that we gave in equation 3.4.33 — namely

\begin{align*} R_n(x) = f(x) - T_n(x) &= \frac{1}{(n+1)!}f^{(n+1)}(c)\cdot (x-a)^{n+1} \end{align*}

for some $c$ strictly between $a$ and $x\text{,}$ and where $T_n(x)$ is the $n^\mathrm{th}$ degree Taylor polynomial approximation of $f(x)$ about $x=a\text{:}$

\begin{align*} T_n(x) &= \sum_{k=0}^n \frac{1}{k!} f^{(k)}(a). \end{align*}

Recall that we have already proved a special case of this formula for the constant approximation using the Mean-Value Theorem (Theorem 2.13.5). To prove the general case we need the following generalisation  22 It is not a terribly creative name for the generalisation, but it is an accurate one. of that theorem:

Notice that setting $G(x) = x$ recovers the original Mean-Value Theorem. It turns out that this theorem is not too difficult to prove from the MVT using some sneaky algebraic manipulations:

• First we construct a new function $h(x)$ as a linear combination of $F(x)$ and $G(x)$ so that $h(a)=h(b)=0\text{.}$ Some experimentation yields
\begin{gather*} h(x)=\big[F(b)-F(a)\big]\cdot \big[G(x)-G(a)\big]- \big[G(b)-G(a)\big] \cdot \big[F(x)-F(a)\big] \end{gather*}
• Since $h(a)=h(b)=0\text{,}$ the Mean-Value theorem (actually Rolle's theorem) tells us that there is a number $c$ obeying $a \lt c \lt b$ such that $h'(c)=0\text{:}$
\begin{align*} h'(x) &= \big[F(b)-F(a)\big] \cdot G'(x) - \big[G(b)-G(a)\big] \cdot F'(x) & \text{ so}\\ 0 &= \big[F(b)-F(a)\big] \cdot G'(c) - \big[G(b)-G(a)\big] \cdot F'(c) \end{align*}
Now move the $G'(c)$ terms to one side and the $F'(c)$ terms to the other:
\begin{align*} \big[F(b)-F(a)\big] \cdot G'(c) &= \big[G(b)-G(a)\big] \cdot F'(c). \end{align*}
• Since we have $G'(x) \neq 0\text{,}$ we know that $G'(c) \neq 0\text{.}$ Further the Mean-Value theorem ensures  23 Otherwise if $G(a)=G(b)$ the MVT tells us that there is some point $c$ between $a$ and $b$ so that $G'(c)=0\text{.}$ that $G(a) \neq G(b)\text{.}$ Hence we can move terms about to get
\begin{align*} \big[F(b)-F(a)\big] &= \big[G(b)-G(a)\big] \cdot \frac{F'(c)}{G'(c)}\\ \frac{F(b)-F(a)}{G(b)-G(a)} &= \frac{F'(c)}{G'(c)} \end{align*}
as required.

Armed with the above theorem we can now move on to the proof of the Taylor remainder formula.

We begin by proving the remainder formula for $n=1\text{.}$ That is

\begin{align*} f(x) - T_1(x) &= \frac{1}{2}f''(c) \cdot(x-a)^2 \end{align*}
• Start by setting
\begin{align*} F(x) &= f(x)-T_1(x) & G(x) &= (x-a)^2 \end{align*}
Notice that, since $T_1(a)=f(a)$ and $T'_1(x) = f'(a)\text{,}$
\begin{align*} F(a) &= 0 & G(a)&=0\\ F'(x) &= f'(x)-f'(a) & G'(x) &= 2(x-a) \end{align*}
• Now apply the generalised MVT with $b=x\text{:}$ there exists a point $q$ between $a$ and $x$ such that
\begin{align*} \frac{F(x)-F(a)}{G(x)-G(a)} &= \frac{F'(q)}{G'(q)}\\ \frac{F(x)-0}{G(x) - 0} &= \frac{f'(q)-f'(a)}{2(q-a)}\\ 2 \cdot \frac{F(x)}{G(x)} &= \frac{f'(q)-f'(a)}{q-a} \end{align*}
• Consider the right-hand side of the above equation and set $g(x) = f'(x)\text{.}$ Then we have the term $\frac{g(q)-g(a)}{q-a}$ — this is exactly the form needed to apply the MVT. So now apply the standard MVT to the right-hand side of the above equation — there is some $c$ between $q$ and $a$ so that
\begin{align*} \frac{f'(q)-f'(a)}{q-a} &= \frac{g(q)-g(a)}{q-a} = g'(c) = f''(c) \end{align*}
Notice that here we have assumed that $f''(x)$ exists.
• Putting this together we have that
\begin{align*} 2 \cdot \frac{F(x)}{G(x)} &= \frac{f'(q)-f'(a)}{q-a} = f''(c)\\ 2 \frac{f(x)-T_1(x)}{(x-a)^2} &= f''(c)\\ f(x) - T_1(x) &= \frac{1}{2!} f''(c) \cdot (x-a)^2 \end{align*}
as required.

Oof! We have now proved the cases $n=1$ (and we did $n=0$ earlier).

To proceed — assume we have proved our result for $n=1,2,\cdots, k\text{.}$ We realise that we haven't done this yet, but bear with us. Using that assumption we will prove the result is true for $n=k+1\text{.}$ Once we have done that, then

• we have proved the result is true for $n=1\text{,}$ and
• we have shown if the result is true for $n=k$ then it is true for $n=k+1$

Hence it must be true for all $n \geq 1\text{.}$ This style of proof is called mathematical induction. You can think of the process as something like climbing a ladder:

• prove that you can get onto the ladder (the result is true for $n=1$), and
• if I can stand on the current rung, then I can step up to the next rung (if the result is true for $n=k$ then it is also true for $n=k+1$)

Hence I can climb as high as like.

• Let $k \gt 0$ and assume we have proved
\begin{align*} f(x) - T_k(x) &= \frac{1}{(k+1)!} f^{(k+1)}(c) \cdot (x-a)^{k+1} \end{align*}
for some $c$ between $a$ and $x\text{.}$
• Now set
\begin{align*} F(x) &= f(x) - T_{k+1}(x) & G(x) &= (x-a)^{k+1}\\ \end{align*}

and notice that, since $T_{k+1}(a)=f(a)\text{,}$

\begin{align*} F(a) &= f(a)-T_{k+1}(a)=0 & G(a) &= 0 & G'(x) &= (k+1)(x-a)^k \end{align*}
and apply the generalised MVT with $b=x\text{:}$ hence there exists a $q$ between $a$ and $x$ so that
\begin{align*} \frac{F(x)-F(a)}{G(x)-G(a)} &= \frac{F'(q)}{G'(q)} &\text{which becomes}\\ \frac{F(x)}{(x-a)^{k+1}} &= \frac{F'(q)}{(k+1)(q-a)^k} & \text{rearrange}\\ F(x) &= \frac{(x-a)^{k+1}}{(k+1)(q-a)^k} \cdot F'(q) \end{align*}
• We now examine $F'(q)\text{.}$ First carefully differentiate $F(x)\text{:}$
\begin{align*} F'(x) &= \diff{}{x} \bigg[f(x) - \bigg( f(a) + f'(a)(x-a) + \frac{1}{2} f''(a)(x-a)^2 + \cdots \\ \amp\hskip2.5in+ \frac{1}{k!}f^{(k)}(x-a)^k \bigg) \bigg]\\ &= f'(x) - \bigg( f'(a) + \frac{2}{2} f''(a)(x-a) + \frac{3}{3!} f'''(a)(x-a)^2 + \cdots \\ \amp\hskip2.5in+ \frac{k}{k!}f^{(k)}(a) (x-a)^{k-1} \bigg)\\ &= f'(x) - \bigg( f'(a) + f''(a)(x-a) + \frac{1}{2} f'''(a)(x-a)^2 +\cdots \\ \amp\hskip2.5in+ \frac{1}{(k-1)!}f^{(k)}(a)(x-a)^{k-1} \bigg) \end{align*}
Now notice that if we set $f'(x) = g(x)$ then this becomes
\begin{align*} F'(x) &= g(x) - \bigg( g(a) + g'(a)(x-a) + \frac{1}{2} g''(a)(x-a)^2 + \cdots \\ \amp\hskip2.5in+ \frac{1}{(k-1)!}g^{(k-1)}(a)(x-a)^{k-1} \bigg) \end{align*}
So $F'(x)$ is then exactly the remainder formula but for a degree $k-1$ approximation to the function $g(x) = f'(x)\text{.}$
• Hence the function $F'(q)$ is the remainder when we approximate $f'(q)$ with a degree $k-1$ Taylor polynomial. The remainder formula, equation 3.4.33, then tells us that there is a number $c$ between $a$ and $q$ so that
\begin{align*} F'(q) &= g(q) - \bigg( g(a) + g'(a)(q-a) + \frac{1}{2} g''(a)(q-a)^2 + \cdots \\ \amp\hskip2.5in + \frac{1}{(k-1)!}g^{(k-1)}(a)(q-a)^{k-1} \bigg)\\ &= \frac{1}{k!} g^{(k)}(c) (q-a)^k = \frac{1}{k!} f^{(k+1)}(c)(q-a)^k \end{align*}
Notice that here we have assumed that $f^{(k+1)}(x)$ exists.
• Now substitute this back into our equation above
\begin{align*} F(x) &= \frac{(x-a)^{k+1}}{(k+1)(q-a)^k} \cdot F'(q)\\ &= \frac{(x-a)^{k+1}}{(k+1)(q-a)^k} \cdot \frac{1}{k!} f^{(k+1)}(c)(q-a)^k\\ &= \frac{1}{(k+1)k!} \cdot f^{(k+1)}(c) \cdot \frac{(x-a)^{k+1}(q-a)^k}{(q-a)^k}\\ &= \frac{1}{(k+1)!} \cdot f^{(k+1)}(c) \cdot(x-a)^{k+1} \end{align*}
as required.

So we now know that

• if, for some $k\text{,}$ the remainder formula (with $n=k$) is true for all $k$ times differentiable functions,
• then the remainder formula is true (with $n=k+1$) for all $k+1$ times differentiable functions.

Repeatedly applying this for $k=1,2,3,4,\cdots$ (and recalling that we have shown the remainder formula is true when $n=0,1$) gives equation 3.4.33 for all $n=0,1,2,\cdots\text{.}$