Derive Numerical Differentiation Formulae

The traditional "pencil and paper" derivations for numerical differentiation formulas for and are done independently as if there was no connection among the derivations.This new approach gives a parallel development of the formulas.It requires the solution of a "linear system" that includes symbolic quantities as coefficients and constants. The power of a computer algebra system such as Mathematica is used to elegantly solve the linear system for and.The extension to proofs of higher order numerical differentiation formulas for the central, forward or backward differences is easy to accomplish.The notation convention using square brackets "[ and ]" instead of parenthesis is used to make all the formulas look compatible with the Mathematica input and output that is used.

The Three Point Central Difference Formulas

The Mathematica notation for the remainder term is.The term is removed from the series with the command Normal, and a Taylor polynomial of degree is formed.However, we must remember that the accuracy for the Taylor polynomial is.

Construct two "equations" by setting the series equal to the function at .

We have two equations in the two unknown and, and all the other quantities ,, , , and are considered as constants.Now use the Mathematica command "Solve" to solve for and.

The result is not too easy to read, so we can use the commands and to manipulate the formula.In the print statements we add""to remind us that we are using truncated infinite series.The following commands will create a nicer looking printout for the above solution.

Thus, we have derived the numerical differentiation formula forand and the first term in the series expansion for the remainder which involves, or, respectively.Since the "numerical differentiation formulae" are "truncated" infinite series, we know that the ellipses "" means that there are infinitely many more term which are not shown.

When we do not include the"", we must evaluate the lowest order derivative in the series for the remainder at the value instead of , then we can "chop off" the infinite series at the term involving or, respectively.The Mathematica commandReplaceAll is used to accomplish this task.

We have obtained the desired numerical differentiation formulas and their remainder terms.We should mention that new in version 4.1 ofMathematica is symbolic recognition of order of approximation for divided difference formulas.We can add the term to the numerical differentiation formula term, and Mathematica will conclude that it is the derivative plus .The following commands illustrate this feature of Mathematica.

Therefore, we have established the numerical differentiation formulas


and


And the corresponding formulas with the big "O" notation are


and

Comparison with the Traditional Derivations

The above derivation differs only slightly from the traditional derivations which also start with the two equations


(1)

When deriving the numerical differentiation formula for the equations (1) are subtracted and the terms involving cancel and manipulations are used to solve for and its truncation error term.

The traditional derivation of the numerical differentiation formula for starts with the same equations (1).But this time the equations are added and the terms involving cancel and manipulations are used to solve for and its truncation error term.

There is no "big deal" made about the fact that the starting place is the same.We shall see for the higher order formulas that using the same starting place will be the key to successful computer derivations of numerical differentiation formulas.

The Five Point Central Difference Formulas

Using five points , , ,, and we can give a parallel development of the numerical differentiation formulas for,, and .Start with the Taylor series for expanded in powers of.Terms must be included so that the remainder term is included.Using degree will suffice.Since the remainder term, we will use Mathematica's Normal command to remove it from the series, and form the Taylor polynomial of degree .

Use five points and set up the "equations" by setting for, which is automated by using Mathematica's Table command.Although the value, will produce which is the boolean value True, it will do no harm when solving the set of equations and we will see that the generations of higher order formulas which require more complicated sets of equations is more easily automated with the use of a table.

We can consider that these are four equations in the four unknowns , , and , and all the other quantities , , , , , , and are constants.Now use Mathematica to solve for , , and .Since this requires typing of four derivatives to be solved, we will automate this process too, by using a table to construct the "variables."

Amazingly, the algebra involved in solving the four equations will result in the cancellation of in the odd order derivatives and the term will cancel in the even order derivatives.This can be accomplished using Mathematica's Solve procedure.

The output is not too clear to read, so we shall use the formula manipulation commands and to group the numerical differentiation portion, and then use the ReplaceAll command to change and to and, then a final use of Together will clean up the numerical differentiation formula part.All this will be accomplished in the following three Mathematica commands.If the reader is curious about exactly what is happening in each step, then the semi-colons can be deleted and the results of each operation will be shown.

Notice that the formula for approximating and have truncation error terms involving so they are numerical differentiation formulas of order .But the formula for approximating and have truncation error terms involving so they are numerical differentiation formulas of order .This is one of the many surprises in the theory of numerical analysis.

Again, we can uses Mathematica's 4.1 ability to symbolic recognize the order of approximation for divided difference formulas.If we add the term or to the numerical differentiation formula then we will be surprised to learn that Mathematica will tell us that it is the derivative plus or .The following commands illustrate this point.

Therefore, we have established the numerical differentiation formulas

These formulas can be written with the big "O" notation and if desired.

Appendix. Subroutines for Generating Numerical Differentiation Formulae

The central difference formulae use an odd number of points,and an even number of equations.Since we want to include the remainder terms, we need to use series expansions of order.The remainder term in these formulas all involve even powers ofhand even and odd derivatives depending on the situation.Also, the subroutine requires a replacement of the point where the remainder term is evaluated to be instead of.

The Five Point Central Difference Formulae

These are to difficult to do by hand, so we use the computer.

Higher order formulas are easy to obtain with Mathematica.

Subroutines for Generating the Forward and Backward Difference Formulae

Once we have the idea for the central difference formula, all we need to do is adjust the set of to be forward step sizes or backward step sizes and we can generate the forward and backward differentiation formulas.

Explorations with the Higher Order Formulas

Higher order formulas are easy to obtain with Mathematica.