Starting off with NonLinearModel Class

Here are some of the points I have been working on-

Base Class
The nonlinear regression models can be expressed as

y = f(x,parameters) + e

Here f is the nonlinear function of explanatory variable x and different parameters.

NonLinearModel, the base class for all nonlinear model needs to have a function which contains the expression of 'f' and calculates the response for a set of given data points and parameters.
The expr function in NonLinearModel is introduced for the above reason.An example using NonLinearLS class is given here.
Slightly refactored 'nonlinls.py'  is given here.

Calculation of Jacobian
The leastsq() procedure is based on the minpack/lmder.f and minpack/lmdif.f. These modules use Levenberg-Marquardt algorithm for nonlinear least squares optimization. Both the modules use forward-differences method to calculate the Jacobian. It gives the following output for the above example-

 

However, calculating the jacobian explicitly using the numdiff/approx_fprime1 module in sandbox and passing on to 'leastsq' gave different results as following.

Currently working on it to find a solution.
The reason for explicitly calculating jacobian is to replace the forward differences method by better methods like automatic differentiation or n-point numerical derivative. Jacobian values are needed to keep track and prevent any deviation from the expected descent direction.