Least Squares method relating to an equation

nicko999

New Member
I have been given 5 sets of data each at different temperatures measuring stress vs time in each set. The question is:1) Using the data above, estimate using the least squares technique, the following relationship between the tests conditions and time to failure.
to.5j = (A1/STj^A2)exp[A3/8.31Tj]
Where STj is stress at test condition j, Tj is the temp at test condition j and to.5j is the median time to failure at test condition j.
A1= A2= A3=

Any pointers in where to start in building my understanding of the least squares rule as google has been very un - enlightening.

BGM

TS Contributor
$$t_{0.5j} = \frac {A_1} {S_{T_j}^{A_2}} \exp\left\{\frac {A_3} {8.31T_j} \right\} + \epsilon$$ (is it correct? $$T_j$$ is in the numerator or denominator?)

Here I assume $$A_1, A_2, A_3$$ are the parameters you want to estimate by least square method; $$t_{0.5j}$$ is the response and $$S_{T_j}, T_j$$ are the regressors are observable data that you have collected.

So you can have the following sum of squares, for $$n$$ observations:

$$S(A_1, A_2, A_3) \triangleq \sum_{i=1}^n \left[t_{0.5j} - \frac {A_1} {S_{T_j}^{A_2}} \exp\left\{\frac {A_3} {8.31T_j} \right\} \right]^2$$

and the least square estimate $$\hat{A}_1, \hat{A}_2, \hat{A}_3$$ are the parameters minimize $$S(A_1, A_2, A_3)$$

Of course another way is the transform it into a linear equation to do the ordinary least square.

Consider
$$\ln t_{0.5j} = \ln A_1 - A_2 \ln S_{T_j} + A_3 \frac {1} {8.31T_j}$$
if you let $$y \triangleq \ln t_{0.5j}, \beta_0 \triangleq \ln A_1, \beta_1 \triangleq - A_2, \beta_3 \triangleq A_3, x_1 \triangleq \ln S_{T_j}, x_2 \triangleq \frac {1} {8.31T_j}$$, then you can see it becomes a linear equation:

$$y = \beta_0 + \beta_1x_1 + \beta_2x_2$$

nicko999

New Member
Thanks for the replies here is the piece of work I will try to understand the above method in relation to the work.
Thanks again,
Nicko

nicko999

New Member
I have used a linear trend line to give me a the offset points as I read in a book however I cant see the translation from the graph to the equation :/

BGM

TS Contributor
Sorry the symbol "triangle-equal" $$\triangleq$$ just mean "is defined to be equal to". You can just treat that as a simple equal sign. I use the symbol to emphasize that I am not deriving the relationship from the other places, but to define those variables appearing at the first time by those relationships.

nicko999

New Member
Please can some one help me im at my wits end im sitting here with three massive stats book and i still dont understand it!!!!!!!!!!

BGM

TS Contributor
Ok I have read the question you attached.

So it depends on what your teacher/text told you to do. So one way, as I mentioned above, is to do a little transformation to get a linear equation, in which standard ordinary least square will help you to get the estimates for the parameters. Another way is to do a non-linear least square. Most statistical software provides the algorithm, and the closed form solution for those ordinary least square is readily available. I do not see any problem unless those least square problem is completely new to you. So I think question 1 is done. (Hopefully it is not related to the quantile regression)

If you are really new to this, may I ask you a simple question: Do you know how to estimate the parameters in the linear regression model
$$y = \beta_0 + \beta_1 x + \epsilon$$
If yes, just slight extend to 1 more variable. If no, re-read text, and derive the estimates by doing differentiation yourself.

For the second question, it depends on the model of the distribution of the residual error. If you model as normal, you have the normal quantiles for you; if you have no assumption at all, you can do bootstrapping.

nicko999

New Member
Thanks for the quick reply here is what i have done so far not sure how to differentiate this stuff to get A1 A2 and A3

BGM

TS Contributor
Several things to point out:

1. The variables, as described by the equation, does not have an linear relationship.
So directly applying the formula
$$\hat{\beta}_1 = \frac {\displaystyle \sum_{i=1}^n x_iy_i} {\displaystyle \sum_{i=1}^n x_i^2 }$$
on the raw data won't help. As I said, if you need a close form for this, transforming the variables to obtain a linear relationship before you applying any result in linear regression.

2. Now you are conditional on (fixing) the temperature and doing the regression separately for different temperature, which is not what the question want. The question want you to do a multiple one, with 2 regressors. There are also closed form, although a little bit more complicated. Usually expressed in terms of the matrix.

nicko999

New Member
So if I plot each temperature and then take the linear line equation from each one and then use regression would that work instead?

nicko999

New Member
also this is the first time ive done any stats apart from averages and the odd spearmans rank correlation :/

BGM

TS Contributor
Ok it seems too much to cover.. anyway as posted in #2 already give you the algorithm.

You may try to do the following with your excel:
1. Transform the variable as follow: by taking log from the equation
$$y \triangleq \ln t_{0.5j}, x_1 \triangleq \ln S_{T_j}, x_2 \triangleq \frac {1} {8.31T_j}$$

2. Put the variables $$\mathbf{y}$$ as a column vector.
Create a column vector $$\mathbf{1}$$ match with your data size, along with the column vectors $$\mathbf{x}_1, \mathbf{x}_2$$. These 3 columns vectors combine to from the design matrix, $$\mathbf{X} \triangleq\begin{bmatrix} \mathbf{1} && \mathbf{x}_1 && \mathbf{x}_2 \end{bmatrix}$$

3. So for this multiple linear regression, $$y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \epsilon$$
the least square estimate is given by
$$\begin{bmatrix} \hat{\beta}_0 \\ \hat{\beta}_1 \\ \hat{\beta}_2 \end{bmatrix}= (\mathbf{X}^T\mathbf{X})^{-1}\mathbf{X}^T\mathbf{Y}$$

The transpose and matrix inverse function is readily available in Excel.
If you want to understand why this is the solution, just need to do differentiation and solve the system of linear equations.

4. Transform the obtained estimates $$\hat{\beta}_0, \hat{\beta}_1, \hat{\beta}_2$$ to $$\hat{A}_1, \hat{A}_2, \hat{A}_3$$
You just need to write $$A_1, A_2, A_3$$ as the subjects, matching the transformation in equation 1.

I think the steps are quite clear now, and thats what I can help.

Of course this step just help you to do the regression manually in excel. If you understand everything, actually many statistical software, even in excel has built-in function for regression.

nicko999

New Member
Im sorry I still cant do it thanks for your patience and help i guess its out of the question to ask you to show me how to do this example in a step by step way?
thanks for the help nicko