Newton's Method

Example of Newton’s Method

f[x_]=Cos[Exp[x]]-Sin[x^2+4]

Here's a plot, so we can see that this function does have roots.

Plot[f[x],{x,-3,3.4},PlotStyle->Thick]

This is one way to set up Newton's method in Mathematica:

new[x_]:=SetPrecision[x-f[x]/f'[x],30]

Let's choose to start at 0.5, and see which root we find:

new[0.5]

new[%]

new[%]

new[%]

new[%]

new[%]

new[%]

new[%]

f[%]

So we can quickly get 30 decimals of accuracy using Newton’s method. That is pretty impressive!

Another Way of Programming Newton’s Method

There is another way to program Newton’s method that relies on recursion relations. The output in this case is the table of values (Subscript[r, i]) where (Subscript[r, i]) is the next approximation to the root. These commands are explained in detail at the end of the document.

Clear[r]
f[x_]=Cos[Exp[x]]-Sin[x^2+4];
r[0]=0.5; (*initial guess to root*)
r[i_]:=r[i]=SetPrecision[r[i-1]-f[r[i-1]]/f'[r[i-1]],30]
Table[{i,r[i]},{i,0,10}]//TableForm
f[r[10]]

Clear[r]
f[x_]=Cos[Exp[x]]-Sin[x^2+4];
r[0]=0.25; (*initial guess to root*)
r[i_]:=r[i]=SetPrecision[r[i-1]-f[r[i-1]]/f'[r[i-1]],30]
Table[{i,r[i]},{i,0,10}]//TableForm
f[r[10]]

Investigation of How Newton’s Method Works

We can get a graphical picture of what Newton's Method is doing using the following commands.

f[x_]=Cos[Exp[x]]-Sin[x^2+4];
plotf=Plot[f[x],{x,0,2},PlotStyle->{Blue,Thick}];
new[x_]:=SetPrecision[x-f[x]/f'[x],30]

Clear[plot,line,dot]
a=0.5; (*change this value of a to change starting value*)
Do[
plot[i]=Plot[f[a]+f'[a](x-a),{x,0,2},PlotRange->{{0,2},{-2,2}},PlotStyle->{Red,Thick}];
line[i]=Graphics[Line[{{a,0},{a,f[a]}}]];
dot[i]=Graphics[Disk[{a,0},0.04]];

a=new[a];

,{i,1,5,1}]

Now evaluate the Manipulate command to get the pictures. As we vary i, we are stepping through Newton's Method.

The blue curve is the function f(x) we are trying to find the root of.
The red line is the tangent line which is used to get the next approximation.
The black dots are the sequence of approximations to the root, (Subscript[r, 0]), (Subscript[r, 1]),(Subscript[r, 2]),(Subscript[r, 3]),(Subscript[r, 4]).
As you change i, you see (Subscript[r, i-1]) and (Subscript[r, i]). So i=1 shows (Subscript[r, 0]) and (Subscript[r, 1]).

Manipulate[Show[plotf,plot[i],line[i],dot[i],dot[i+1],AspectRatio->1],{i,1,4,1}]

Change the starting value a in the above to 0.25 and see what root is found. 
What if you start at a=0.15? 
What if you start at a=0.35?

Mathematica’s Built-in Command for Newton’s Method

To implement Newton’s Method, you need a function and a starting point. This is implemented in Mathematica using the command FindRoot.

Here is the syntax for the command.

?FindRoot

Here is how to use it for example we have been working on above.

FindRoot[Cos[Exp[x]]-Sin[x^2+4],{x,0.5}]

FindRoot[Cos[Exp[x]]-Sin[x^2+4],{x,0.15}]

FindRoot[Cos[Exp[x]]-Sin[x^2+4],{x,0.35}]

Sensitivity to Initial Starting Point

Notice that the FindRoot result starting at (Subscript[x, 0])=0.35 actually found the root at x=1.02836 even though in our graphical solution above it seems to not be anywhere near the root after four steps in our animation. This is because the implementation of Newton’s method in FindRoot is slightly different than ours (uses an equivalent but different equation) and the root we find is very sensititve to the initial starting point when (Subscript[x, 0]) is near 0.35.

Clear[r]
f[x_]=Cos[Exp[x]]-Sin[x^2+4];
r[0]=0.3499; (*initial guess to root*)
r[i_]:=r[i]=SetPrecision[r[i-1]-f[r[i-1]]/f'[r[i-1]],30]
Table[{i,r[i]},{i,0,10}]//TableForm
f[r[10]]

Clear[r]
f[x_]=Cos[Exp[x]]-Sin[x^2+4];
r[0]=0.3500; (*initial guess to root*)
r[i_]:=r[i]=SetPrecision[r[i-1]-f[r[i-1]]/f'[r[i-1]],30]
Table[{i,r[i]},{i,0,10}]//TableForm
f[r[10]]

Clear[r]
f[x_]=Cos[Exp[x]]-Sin[x^2+4];
r[0]=0.3501; (*initial guess to root*)
r[i_]:=r[i]=SetPrecision[r[i-1]-f[r[i-1]]/f'[r[i-1]],30]
Table[{i,r[i]},{i,0,10}]//TableForm
f[r[10]]

Clear[r]
f[x_]=Cos[Exp[x]]-Sin[x^2+4];
r[0]=0.3502; (*initial guess to root*)
r[i_]:=r[i]=SetPrecision[r[i-1]-f[r[i-1]]/f'[r[i-1]],30]
Table[{i,r[i]},{i,0,10}]//TableForm
f[r[10]]

Starting at (Subscript[x, 0])=0.3499, 0.3500, 0.3501, 0.3502 ends up finding four different roots! Now that’s some sensitivity! This is why it is very important to try to find a good starting value close to the root you are looking for.

Detailed Explanation of Recursion Relations in Mathematica

Clear[r]

The Clear command wipes Mathematica’s memory of the results stored in r. This is very important to do when you are using recursion relations, since when you run the commands again you do not want old values to corrupt the new ones.

f[x_]=Cos[Exp[x]]-Sin[x^2+4];

This line defines the function that we are trying to find the root of.

r[0]=0.5; (*initial guess to root*)

This line contains the initial guess to the root, (Subscript[r, 0]), which we store in Mathematica as r[0].

r[i_]:=r[i]=SetPrecision[r[i-1]-f[r[i-1]]/f'[r[i-1]],30]

This line is the recursion relation. There is a lot going on here, so let’s look at each piece.

The r[i_]:= is telling Mathematica to come back here whenever it needs to determine an (Subscript[r, i]). We need the underscore to tell Mathematica that r depends on i. This is a delayed assignment command, and if we ask for r[2] Mathematica will return to this definition to work out what r[2] is.

The r[i]= tells Mathematica to save the result each time it computes a value for a specific i, so it will save in memory the values of r[1], r[2], r[3] etc. as it computes them. This saving of the values speeds up the code immensely, and is why we needed the Clear command at the beginning.

The SetPrecision[r[i-1]-f[r[i-1]]/f’[r[i-1]],30] is the implementation of Newton’s method, keeping 30 decimals of precision in the calculation.

Table[{i,r[i]},{i,0,10}]//TableForm

The Table command is actually what tells Mathematica to go ahead and figure out what r[0], r[1], r[2], r[3], … , r[10] actually are, using the previous line that defined these quantities resursively.

f[r[10]]

This line is a check, evaluating the function at the approximation to the root r[10] to see if it is close to zero.