Example of Random Search to Find Minimum

First, let's define the function for which we want to find the minimum. This is a function for which the Mathematica cannot find the minimum (at least, the NSolve below took so long that I finally gave up waiting and aborted the calculation!).

f[x_, y_]  =  (Cos[x] + Sin[x]  - Cos[x y]) 
        Exp[-x^2 - y^2]
Plot3D[f[x, y], {x, -2, 2}, {y, -3, 3}, 
      AxesLabel -> {x, y, z}, PlotPoints -> 35, 
      ColorFunction -> Hue, 
      PlotRange -> {-1, 1}]
eq1  =  D[f[x, y], y] == 0
eq2  =  D[f[x, y], x] == 0

NSolve[{eq1, eq2}, {x, y}]

Instead, let's find a rough estimate of the minimum using a random search technique

xlower = -2
xupper = +2
ylower = -2
yupper = +2

    
xmin  =  Random[Real, {xlower, xupper}]
ymin  =  Random[Real, {yupper, ylower}]
zmin  =  f[xmin, ymin]

    
n = 10000
Do[{xr  =  
          Random[Real, {xlower, xupper}], 
yr  =  
          Random[Real, {yupper, ylower}], 
zr  =  
          f[xr, yr], 
If[
          zr < zmin, 
{zmin = zr, xmin = xr, 
            ymin = yr}]}, {i, 1, n}]
Print[" " xmin, " " ymin, " " zmin]

Once we have a global estimate of the minimum, we use these values to initialize a local procedure to refine our rough estimate. Essentially, global means quickly find a rough estimate, and local means take a rough estimate and make it better. In Mathematica, you can use the built in function FindRoot to do this for you.

FindRoot[{eq1, eq2}, {x, -0.72769}, {y, -0.0105025}]

f[-0.7471, 0.0000]

There is also the function FindMinimum:

FindMinimum[f[x, y], {x, -0.72769}, {y, -0.0105025}]

Notice how the FindRoot will find a different root (the maximum), if we initialize with different numbers. Finding the global estimate is important so that we can initialize any local methods as accurately as possible.

FindRoot[{eq1, eq2}, {x, 0}, {y, 0}]

Finding an accurate answer using a global method is not an efficient use of resources. Although it certainly will work, taking a large number of random points in an interval is not the best way to proceed!

xlower = -2;
xupper = +2;
ylower = -2;
yupper = +2;

    
xmin  =  
        Random[Real, {xlower, xupper}];
ymin  =  
        Random[Real, {yupper, ylower}];
zmin  =  f[xmin, ymin];

    
n = 100000;
Do[{xr  =  
          Random[Real, {xlower, xupper}], 
yr  =  
          Random[Real, {yupper, ylower}], 
zr  =  
          f[xr, yr], 
If[
          zr < zmin, 
{zmin = zr, xmin = xr, 
            ymin = yr}]}, {i, 1, n}]
Print[" " xmin, " " ymin, " " zmin]

Here you can see how the random numbers fill space. I put two points in to begin to hold the plot at the proper width.

xlower = -2;
xupper = +2;
ylower = -2;
yupper = +2;

    
pointlist  =  {{-2, 0}, {2, 0}};

    
n = 100;
Do[{
xr  =  
          Random[Real, {xlower, xupper}], 
yr  =  
          Random[Real, {yupper, ylower}], 
pointlist  =  
          Append[pointlist, {xr, yr}], 
If[n > 20, 
          ListPlot[pointlist, Axes -> False, 
            PlotRange -> {-2, 2}, 
            Frame -> True]]
}, {i, 1, n}]

Here is an example of how random points fill a region. Pick an n value, and then rerun to see how the points change. How large should n be to cover the region? n bigger than 10 000 will take a long time to run. What does it mean to cover a region? Notice that even for large n there are still gaps in the region being covered.

xlower = -2;
xupper = +2;
ylower = -2;
yupper = +2;

    
pointlist  =  {};

    
n = 10000;
Do[{
xr  =  
          Random[Real, {xlower, xupper}], 
yr  =  
          Random[Real, {yupper, ylower}], 
pointlist  =  
          Append[pointlist, {xr, yr}]
}, {i, 1, 
        n}]
ListPlot[pointlist, Axes -> True, Ticks -> False]