Method of least squares examples of problem solving. Development of a forecast using the least squares method. An example of solving a problem Solving a system of equations by the least squares method

We approximate the function by a polynomial of the 2nd degree. To do this, we calculate the coefficients of the normal system of equations:

, ,

Let us compose a normal system of least squares, which has the form:

The solution of the system is easy to find:, , .

Thus, the polynomial of the 2nd degree is found: .

Theoretical background

Back to page<Введение в вычислительную математику. Примеры>

Example 2. Finding the optimal degree of a polynomial.

Back to page<Введение в вычислительную математику. Примеры>

Example 3. Derivation of a normal system of equations for finding the parameters of an empirical dependence.

Let us derive a system of equations for determining the coefficients and functions , which performs the root-mean-square approximation of the given function with respect to points. Compose a function and write the necessary extremum condition for it:

Then the normal system will take the form:

We have obtained a linear system of equations for unknown parameters and, which is easily solved.

Theoretical background

Back to page<Введение в вычислительную математику. Примеры>

Example.

Experimental data on the values ​​of variables X and at are given in the table.

As a result of their alignment, the function

Using least square method, approximate these data with a linear dependence y=ax+b(find options a and b). Find out which of the two lines is better (in the sense of the least squares method) aligns the experimental data. Make a drawing.

The essence of the method of least squares (LSM).

The problem is to find the linear dependence coefficients for which the function of two variables a and btakes the smallest value. That is, given the data a and b the sum of the squared deviations of the experimental data from the found straight line will be the smallest. This is the whole point of the least squares method.

Thus, the solution of the example is reduced to finding the extremum of a function of two variables.

Derivation of formulas for finding coefficients.

A system of two equations with two unknowns is compiled and solved. Finding partial derivatives of functions by variables a and b, we equate these derivatives to zero.

We solve the resulting system of equations by any method (for example substitution method or Cramer's method) and obtain formulas for finding coefficients using the least squares method (LSM).

With data a and b function takes the smallest value. The proof of this fact is given below in the text at the end of the page.

That's the whole method of least squares. Formula for finding the parameter a contains the sums , , , and the parameter n is the amount of experimental data. The values ​​of these sums are recommended to be calculated separately.

Coefficient b found after calculation a.

It's time to remember the original example.

Solution.

In our example n=5. We fill in the table for the convenience of calculating the amounts that are included in the formulas of the required coefficients.

The values ​​in the fourth row of the table are obtained by multiplying the values ​​of the 2nd row by the values ​​of the 3rd row for each number i.

The values ​​in the fifth row of the table are obtained by squaring the values ​​of the 2nd row for each number i.

The values ​​of the last column of the table are the sums of the values ​​across the rows.

We use the formulas of the least squares method to find the coefficients a and b. We substitute in them the corresponding values ​​from the last column of the table:

Consequently, y=0.165x+2.184 is the desired approximating straight line.

It remains to find out which of the lines y=0.165x+2.184 or better approximates the original data, i.e. to make an estimate using the least squares method.

Estimation of the error of the method of least squares.

To do this, you need to calculate the sums of squared deviations of the original data from these lines and , a smaller value corresponds to a line that better approximates the original data in terms of the least squares method.

Since , then the line y=0.165x+2.184 approximates the original data better.

Graphic illustration of the least squares method (LSM).

Everything looks great on the charts. The red line is the found line y=0.165x+2.184, the blue line is , the pink dots are the original data.

What is it for, what are all these approximations for?

I personally use to solve data smoothing problems, interpolation and extrapolation problems (in the original example, you could be asked to find the value of the observed value y at x=3 or when x=6 according to the MNC method). But we will talk more about this later in another section of the site.

Top of page

Proof.

So that when found a and b function takes the smallest value, it is necessary that at this point the matrix of the quadratic form of the second-order differential for the function was positive definite. Let's show it.

The second order differential has the form:

That is

Therefore, the matrix of the quadratic form has the form

and the values ​​of the elements do not depend on a and b.

Let us show that the matrix is ​​positive definite. This requires that the angle minors be positive.

Angular minor of the first order . The inequality is strict, since the points do not coincide. This will be implied in what follows.

Angular minor of the second order

Let's prove that method of mathematical induction.

Conclusion: found values a and b correspond to the smallest value of the function , therefore, are the desired parameters for the least squares method.

Ever understand?
Order a Solution

Top of page

Development of a forecast using the least squares method. Problem solution example

Extrapolation - this is a method of scientific research, which is based on the dissemination of past and present trends, patterns, relationships to the future development of the object of forecasting. Extrapolation methods include moving average method, exponential smoothing method, least squares method.

Essence least squares method consists in minimizing the sum of square deviations between the observed and calculated values. The calculated values ​​are found according to the selected equation - the regression equation. The smaller the distance between the actual values ​​and the calculated ones, the more accurate the forecast based on the regression equation.

The theoretical analysis of the essence of the phenomenon under study, the change in which is displayed by a time series, serves as the basis for choosing a curve. Considerations about the nature of the growth of the levels of the series are sometimes taken into account. So, if the growth of output is expected in an arithmetic progression, then smoothing is performed in a straight line. If it turns out that the growth is exponential, then smoothing should be done according to the exponential function.

The working formula of the method of least squares : Y t+1 = a*X + b, where t + 1 is the forecast period; Уt+1 – predicted indicator; a and b are coefficients; X is a symbol of time.

Coefficients a and b are calculated according to the following formulas:

where, Uf - the actual values ​​of the series of dynamics; n is the number of levels in the time series;

The smoothing of time series by the least squares method serves to reflect the patterns of development of the phenomenon under study. In the analytic expression of a trend, time is considered as an independent variable, and the levels of the series act as a function of this independent variable.

The development of a phenomenon does not depend on how many years have passed since the starting point, but on what factors influenced its development, in what direction and with what intensity. From this it is clear that the development of a phenomenon in time appears as a result of the action of these factors.

Correctly setting the type of curve, the type of analytical dependence on time is one of the most difficult tasks of pre-predictive analysis. .

The choice of the type of function that describes the trend, the parameters of which are determined by the least squares method, is in most cases empirical, by constructing a number of functions and comparing them with each other according to the value of the root-mean-square error, calculated by the formula:

where Uf - the actual values ​​of the series of dynamics; Ur – calculated (smoothed) values ​​of the time series; n is the number of levels in the time series; p is the number of parameters defined in the formulas describing the trend (development trend).

Disadvantages of the least squares method :

• when trying to describe the economic phenomenon under study using a mathematical equation, the forecast will be accurate for a short period of time and the regression equation should be recalculated as new information becomes available;
• the complexity of the selection of the regression equation, which is solvable using standard computer programs.

An example of using the least squares method to develop a forecast

A task . There are data characterizing the level of unemployment in the region, %

• Build a forecast of the unemployment rate in the region for the months of November, December, January, using the methods: moving average, exponential smoothing, least squares.
• Calculate the errors in the resulting forecasts using each method.
• Compare the results obtained, draw conclusions.

Least squares solution

For the solution, we will compile a table in which we will make the necessary calculations:

ε = 28.63/10 = 2.86% forecast accuracy high.

Conclusion : Comparing the results obtained in the calculations moving average method , exponential smoothing and the least squares method, we can say that the average relative error in calculations by the exponential smoothing method falls within 20-50%. This means that the prediction accuracy in this case is only satisfactory.

In the first and third cases, the forecast accuracy is high, since the average relative error is less than 10%. But the moving average method made it possible to obtain more reliable results (forecast for November - 1.52%, forecast for December - 1.53%, forecast for January - 1.49%), since the average relative error when using this method is the smallest - 1 ,13%.

Least square method

Other related articles:

List of sources used

1. Scientific and methodological recommendations on the issues of diagnosing social risks and forecasting challenges, threats and social consequences. Russian State Social University. Moscow. 2010;
2. Vladimirova L.P. Forecasting and planning in market conditions: Proc. allowance. M .: Publishing House "Dashkov and Co", 2001;
3. Novikova N.V., Pozdeeva O.G. Forecasting the National Economy: Educational and Methodological Guide. Yekaterinburg: Publishing House Ural. state economy university, 2007;
4. Slutskin L.N. MBA course in business forecasting. Moscow: Alpina Business Books, 2006.

MNE Program

Enter data

Data and Approximation y = a + b x

i- number of the experimental point;
x i- the value of the fixed parameter at the point i;
y i- the value of the measured parameter at the point i;
ω i- measurement weight at point i;
y i, calc.- the difference between the measured value and the value calculated from the regression y at the point i;
S x i (x i)- error estimate x i when measuring y at the point i.

Data and Approximation y = kx

Click on the chart

User manual for the MNC online program.

In the data field, enter on each separate line the values ​​of `x` and `y` at one experimental point. Values ​​must be separated by whitespace (space or tab).

The third value can be the point weight of `w`. If the point weight is not specified, then it is equal to one. In the overwhelming majority of cases, the weights of the experimental points are unknown or not calculated; all experimental data are considered equivalent. Sometimes the weights in the studied range of values ​​are definitely not equivalent and can even be calculated theoretically. For example, in spectrophotometry, weights can be calculated using simple formulas, although basically everyone neglects this to reduce labor costs.

Data can be pasted through the clipboard from an office suite spreadsheet, such as Excel from Microsoft Office or Calc from Open Office. To do this, select the range of data to be copied in the spreadsheet, copy it to the clipboard, and paste the data into the data field on this page.

To calculate by the least squares method, at least two points are required to determine two coefficients `b` - the tangent of the angle of inclination of the straight line and `a` - the value cut off by the straight line on the `y` axis.

To estimate the error of the calculated regression coefficients, it is necessary to set the number of experimental points to more than two.

Least squares method (LSM).

The greater the number of experimental points, the more accurate the statistical estimate of the coefficients (due to the decrease in the Student's coefficient) and the closer the estimate to the estimate of the general sample.

Obtaining values ​​at each experimental point is often associated with significant labor costs, therefore, a compromise number of experiments is often carried out, which gives a digestible estimate and does not lead to excessive labor costs. As a rule, the number of experimental points for a linear least squares dependence with two coefficients is chosen in the region of 5-7 points.

A Brief Theory of Least Squares for Linear Dependence

Suppose we have a set of experimental data in the form of pairs of values ​​[`y_i`, `x_i`], where `i` is the number of one experimental measurement from 1 to `n`; `y_i` - the value of the measured value at the point `i`; `x_i` - the value of the parameter we set at the point `i`.

An example is the operation of Ohm's law. By changing the voltage (potential difference) between sections of the electrical circuit, we measure the amount of current passing through this section. Physics gives us the dependence found experimentally:

`I=U/R`,
where `I` - current strength; `R` - resistance; `U` - voltage.

In this case, `y_i` is the measured current value, and `x_i` is the voltage value.

As another example, consider the absorption of light by a solution of a substance in solution. Chemistry gives us the formula:

`A = εl C`,
where `A` is the optical density of the solution; `ε` - solute transmittance; `l` - path length when light passes through a cuvette with a solution; `C` is the concentration of the solute.

In this case, `y_i` is the measured optical density `A`, and `x_i` is the concentration of the substance that we set.

We will consider the case when the relative error in setting `x_i` is much less than the relative error in measuring `y_i`. We will also assume that all measured values ​​of `y_i` are random and normally distributed, i.e. obey the normal distribution law.

In the case of a linear dependence of `y` on `x`, we can write the theoretical dependence:
`y = a + bx`.

From a geometric point of view, the coefficient `b` denotes the tangent of the angle of inclination of the line to the `x` axis, and the coefficient `a` - the value of `y` at the point of intersection of the line with the `y` axis (for `x = 0`).

Finding the parameters of the regression line.

In an experiment, the measured values ​​of `y_i` cannot lie exactly on the theoretical line due to measurement errors, which are always inherent in real life. Therefore, a linear equation must be represented by a system of equations:
`y_i = a + b x_i + ε_i` (1),
where `ε_i` is the unknown measurement error of `y` in the `i`th experiment.

Dependence (1) is also called regression, i.e. the dependence of the two quantities on each other with statistical significance.

The task of restoring the dependence is to find the coefficients `a` and `b` from the experimental points [`y_i`, `x_i`].

To find the coefficients `a` and `b` is usually used least square method(MNK). It is a special case of the maximum likelihood principle.

Let's rewrite (1) as `ε_i = y_i - a - b x_i`.

Then the sum of squared errors will be
`Φ = sum_(i=1)^(n) ε_i^2 = sum_(i=1)^(n) (y_i - a - b x_i)^2`. (2)

The principle of the least squares method is to minimize the sum (2) with respect to the parameters `a` and `b`.

The minimum is reached when the partial derivatives of the sum (2) with respect to the coefficients `a` and `b` are equal to zero:
`frac(partial Φ)(partial a) = frac(partial sum_(i=1)^(n) (y_i - a - b x_i)^2)(partial a) = 0`
`frac(partial Φ)(partial b) = frac(partial sum_(i=1)^(n) (y_i - a - b x_i)^2)(partial b) = 0`

Expanding the derivatives, we obtain a system of two equations with two unknowns:
`sum_(i=1)^(n) (2a + 2bx_i - 2y_i) = sum_(i=1)^(n) (a + bx_i - y_i) = 0`
`sum_(i=1)^(n) (2bx_i^2 + 2ax_i - 2x_iy_i) = sum_(i=1)^(n) (bx_i^2 + ax_i - x_iy_i) = 0`

We open the brackets and transfer the sums independent of the desired coefficients to the other half, we get a system of linear equations:
`sum_(i=1)^(n) y_i = a n + b sum_(i=1)^(n) bx_i`
`sum_(i=1)^(n) x_iy_i = a sum_(i=1)^(n) x_i + b sum_(i=1)^(n) x_i^2`

Solving the resulting system, we find formulas for the coefficients `a` and `b`:

`a = frac(sum_(i=1)^(n) y_i sum_(i=1)^(n) x_i^2 - sum_(i=1)^(n) x_i sum_(i=1)^(n ) x_iy_i) (n sum_(i=1)^(n) x_i^2 — (sum_(i=1)^(n) x_i)^2)` (3.1)

`b = frac(n sum_(i=1)^(n) x_iy_i - sum_(i=1)^(n) x_i sum_(i=1)^(n) y_i) (n sum_(i=1)^ (n) x_i^2 - (sum_(i=1)^(n) x_i)^2)` (3.2)

These formulas have solutions when `n > 1` (the line can be drawn using at least 2 points) and when the determinant `D = n sum_(i=1)^(n) x_i^2 — (sum_(i= 1)^(n) x_i)^2 != 0`, i.e. when the `x_i` points in the experiment are different (i.e. when the line is not vertical).

Estimation of errors in the coefficients of the regression line

For a more accurate estimate of the error in calculating the coefficients `a` and `b`, a large number of experimental points is desirable. When `n = 2`, it is impossible to estimate the error of the coefficients, because the approximating line will uniquely pass through two points.

The error of the random variable `V` is determined error accumulation law
`S_V^2 = sum_(i=1)^p (frac(partial f)(partial z_i))^2 S_(z_i)^2`,
where `p` is the number of `z_i` parameters with `S_(z_i)` error that affect the `S_V` error;
`f` is a dependency function of `V` on `z_i`.

Let's write the law of accumulation of errors for the error of the coefficients `a` and `b`
`S_a^2 = sum_(i=1)^(n)(frac(partial a)(partial y_i))^2 S_(y_i)^2 + sum_(i=1)^(n)(frac(partial a )(partial x_i))^2 S_(x_i)^2 = S_y^2 sum_(i=1)^(n)(frac(partial a)(partial y_i))^2 `,
`S_b^2 = sum_(i=1)^(n)(frac(partial b)(partial y_i))^2 S_(y_i)^2 + sum_(i=1)^(n)(frac(partial b )(partial x_i))^2 S_(x_i)^2 = S_y^2 sum_(i=1)^(n)(frac(partial b)(partial y_i))^2 `,
because `S_(x_i)^2 = 0` (we previously made a reservation that the error of `x` is negligible).

`S_y^2 = S_(y_i)^2` - the error (variance, squared standard deviation) in the `y` dimension, assuming that the error is uniform for all `y` values.

Substituting formulas for calculating `a` and `b` into the resulting expressions, we get

`S_a^2 = S_y^2 frac(sum_(i=1)^(n) (sum_(i=1)^(n) x_i^2 - x_i sum_(i=1)^(n) x_i)^2 ) (D^2) = S_y^2 frac((n sum_(i=1)^(n) x_i^2 - (sum_(i=1)^(n) x_i)^2) sum_(i=1) ^(n) x_i^2) (D^2) = S_y^2 frac(sum_(i=1)^(n) x_i^2) (D)` (4.1)

`S_b^2 = S_y^2 frac(sum_(i=1)^(n) (n x_i - sum_(i=1)^(n) x_i)^2) (D^2) = S_y^2 frac( n (n sum_(i=1)^(n) x_i^2 - (sum_(i=1)^(n) x_i)^2)) (D^2) = S_y^2 frac(n) (D) ` (4.2)

In most real experiments, the value of `Sy` is not measured. To do this, it is necessary to carry out several parallel measurements (experiments) at one or several points of the plan, which increases the time (and possibly cost) of the experiment. Therefore, it is usually assumed that the deviation of `y` from the regression line can be considered random. The variance estimate `y` in this case is calculated by the formula.

`S_y^2 = S_(y, rest)^2 = frac(sum_(i=1)^n (y_i - a - b x_i)^2) (n-2)`.

The divisor `n-2` appears because we have reduced the number of degrees of freedom due to the calculation of two coefficients for the same sample of experimental data.

This estimate is also called the residual variance relative to the regression line `S_(y, rest)^2`.

The assessment of the significance of the coefficients is carried out according to the Student's criterion

`t_a = frac(|a|) (S_a)`, `t_b = frac(|b|) (S_b)`

If the calculated criteria `t_a`, `t_b` are less than the table criteria `t(P, n-2)`, then it is considered that the corresponding coefficient is not significantly different from zero with a given probability `P`.

To assess the quality of the description of a linear relationship, you can compare `S_(y, rest)^2` and `S_(bar y)` relative to the mean using the Fisher criterion.

`S_(bar y) = frac(sum_(i=1)^n (y_i - bar y)^2) (n-1) = frac(sum_(i=1)^n (y_i - (sum_(i= 1)^n y_i) /n)^2) (n-1)` - sample estimate of the variance of `y` relative to the mean.

To evaluate the effectiveness of the regression equation for describing the dependence, the Fisher coefficient is calculated
`F = S_(bar y) / S_(y, rest)^2`,
which is compared with the tabular Fisher coefficient `F(p, n-1, n-2)`.

If `F > F(P, n-1, n-2)`, the difference between the description of the dependence `y = f(x)` using the regression equation and the description using the mean is considered statistically significant with probability `P`. Those. the regression describes the dependence better than the spread of `y` around the mean.

Click on the chart
to add values ​​to the table

Least square method. The method of least squares means the determination of unknown parameters a, b, c, the accepted functional dependence

The method of least squares means the determination of unknown parameters a, b, c,… accepted functional dependence

y = f(x,a,b,c,…),

which would provide a minimum of the mean square (variance) of the error

, (24)

where x i , y i - set of pairs of numbers obtained from the experiment.

Since the condition for the extremum of a function of several variables is the condition that its partial derivatives are equal to zero, then the parameters a, b, c,… are determined from the system of equations:

; ; ; … (25)

It must be remembered that the least squares method is used to select parameters after the form of the function y = f(x) defined.

If from theoretical considerations it is impossible to draw any conclusions about what the empirical formula should be, then one has to be guided by visual representations, primarily a graphical representation of the observed data.

In practice, most often limited to the following types of functions:

1) linear ;

2) quadratic a .

(see picture). It is required to find the equation of a straight line

The smaller the number in absolute value, the better the straight line (2) is chosen. As a characteristic of the accuracy of the selection of a straight line (2), we can take the sum of squares

The minimum conditions for S will be

	(6)
	(7)

Equations (6) and (7) can be written in the following form:

	(8)
	(9)

From equations (8) and (9) it is easy to find a and b from the experimental values ​​x i and y i . The line (2) defined by equations (8) and (9) is called the line obtained by the least squares method (this name emphasizes that the sum of squares S has a minimum). Equations (8) and (9), from which the straight line (2) is determined, are called normal equations.

It is possible to indicate a simple and general way of compiling normal equations. Using experimental points (1) and equation (2), we can write down the system of equations for a and b

Multiply the left and right parts of each of these equations by the coefficient at the first unknown a (i.e. x 1 , x 2 , ..., x n) and add the resulting equations, resulting in the first normal equation (8).

We multiply the left and right sides of each of these equations by the coefficient of the second unknown b, i.e. by 1, and add the resulting equations, resulting in the second normal equation (9).

This method of obtaining normal equations is general: it is suitable, for example, for the function

is a constant value and it must be determined from experimental data (1).

The system of equations for k can be written:

Find the line (2) using the least squares method.

Solution. We find:

x i =21, y i =46.3, x i 2 =91, x i y i =179.1.

We write equations (8) and (9)

From here we find

Estimating the accuracy of the least squares method

Let us give an estimate of the accuracy of the method for the linear case when equation (2) takes place.

Let the experimental values ​​x i be exact, and the experimental values ​​y i have random errors with the same variance for all i.

We introduce the notation

Then the solutions of equations (8) and (9) can be represented as

	(17)
	(18)
where
	(19)
From equation (17) we find
	(20)
Similarly, from equation (18) we obtain
	(21)
because
	(22)
From equations (21) and (22) we find
	(23)

Equations (20) and (23) give an estimate of the accuracy of the coefficients determined by equations (8) and (9).

Note that the coefficients a and b are correlated. By simple transformations, we find their correlation moment.

From here we find

0.072 at x=1 and 6,

0.041 at x=3.5.

Literature

Shore. Ya. B. Statistical methods of analysis and quality control and reliability. M.: Gosenergoizdat, 1962, p. 552, pp. 92-98.

This book is intended for a wide range of engineers (research institutes, design bureaus, test sites and factories) involved in determining the quality and reliability of electronic equipment and other mass industrial products (machine building, instrument making, artillery, etc.).

The book gives an application of the methods of mathematical statistics to the processing and evaluation of test results, in which the quality and reliability of the tested products are determined. For the convenience of readers, the necessary information from mathematical statistics is given, as well as a large number of auxiliary mathematical tables that facilitate the necessary calculations.

The presentation is illustrated by a large number of examples taken from the field of radio electronics and artillery technology.

The least squares method is one of the most common and most developed due to its simplicity and efficiency of methods for estimating the parameters of linear. At the same time, some caution should be observed when using it, since the models built using it may not meet a number of requirements for the quality of their parameters and, as a result, not “well” reflect the patterns of process development.

Let us consider the procedure for estimating the parameters of a linear econometric model using the least squares method in more detail. Such a model in general form can be represented by equation (1.2):

y t = a 0 + a 1 x 1 t +...+ a n x nt + ε t .

The initial data when estimating the parameters a 0 , a 1 ,..., a n is the vector of values ​​of the dependent variable y= (y 1 , y 2 , ... , y T)" and the matrix of values ​​of independent variables

in which the first column, consisting of ones, corresponds to the coefficient of the model .

The method of least squares got its name based on the basic principle that the parameter estimates obtained on its basis should satisfy: the sum of squares of the model error should be minimal.

Examples of solving problems by the least squares method

Example 2.1. The trading enterprise has a network consisting of 12 stores, information on the activities of which is presented in Table. 2.1.

The company's management would like to know how the size of the annual depends on the sales area of ​​the store.

Table 2.1

Least squares solution. Let us designate - the annual turnover of the -th store, million rubles; - selling area of ​​the -th store, thousand m 2.

Fig.2.1. Scatterplot for Example 2.1

To determine the form of the functional relationship between the variables and construct a scatterplot (Fig. 2.1).

Based on the scatter diagram, we can conclude that the annual turnover is positively dependent on the selling area (i.e., y will increase with the growth of ). The most appropriate form of functional connection is − linear.

Information for further calculations is presented in Table. 2.2. Using the least squares method, we estimate the parameters of the linear one-factor econometric model

Table 2.2

In this way,

Therefore, with an increase in the trading area by 1 thousand m 2, other things being equal, the average annual turnover increases by 67.8871 million rubles.

Example 2.2. The management of the enterprise noticed that the annual turnover depends not only on the sales area of ​​the store (see example 2.1), but also on the average number of visitors. The relevant information is presented in table. 2.3.

Table 2.3

Solution. Denote - the average number of visitors to the th store per day, thousand people.

To determine the form of the functional relationship between the variables and construct a scatterplot (Fig. 2.2).

Based on the scatter diagram, we can conclude that the annual turnover is positively related to the average number of visitors per day (i.e., y will increase with the growth of ). The form of functional dependence is linear.

Rice. 2.2. Scatterplot for example 2.2

Table 2.4

In general, it is necessary to determine the parameters of the two-factor econometric model

y t \u003d a 0 + a 1 x 1 t + a 2 x 2 t + ε t

The information required for further calculations is presented in Table. 2.4.

Let us estimate the parameters of a linear two-factor econometric model using the least squares method.

In this way,

Evaluation of the coefficient = 61.6583 shows that, all other things being equal, with an increase in sales area by 1 thousand m 2, the annual turnover will increase by an average of 61.6583 million rubles.

• introductory lesson is free;
• A large number of experienced teachers (native and Russian-speaking);
• Courses NOT for a specific period (month, six months, year), but for a specific number of lessons (5, 10, 20, 50);
• Over 10,000 satisfied customers.
• The cost of one lesson with a Russian-speaking teacher - from 600 rubles, with a native speaker - from 1500 rubles

The essence of the least squares method is in finding the parameters of the trend model that best describes the development trend of any random phenomenon in time or space (a trend is a line that characterizes the trend of this development). The task of the least squares method (OLS) is to find not just some trend model, but to find the best or optimal model. This model will be optimal if the sum of the squared deviations between the observed actual values ​​and the corresponding calculated trend values ​​is minimal (smallest):

where is the standard deviation between the observed actual value

and the corresponding calculated trend value,

The actual (observed) value of the phenomenon under study,

Estimated value of the trend model,

The number of observations of the phenomenon under study.

MNC is rarely used on its own. As a rule, most often it is used only as a necessary technique in correlation studies. It should be remembered that the information basis of the LSM can only be a reliable statistical series, and the number of observations should not be less than 4, otherwise, the smoothing procedures of the LSM may lose their common sense.

The OLS toolkit is reduced to the following procedures:

First procedure. It turns out whether there is any tendency at all to change the resulting attribute when the selected factor-argument changes, or in other words, whether there is a connection between " at " and " X ».

Second procedure. It is determined which line (trajectory) is best able to describe or characterize this trend.

Third procedure.

Example. Suppose we have information on the average sunflower yield for the farm under study (Table 9.1).

Table 9.1

Since the level of technology in the production of sunflower in our country has not changed much over the past 10 years, it means that, most likely, the fluctuations in yield in the analyzed period depended very much on fluctuations in weather and climate conditions. Is it true?

First MNC procedure. The hypothesis about the existence of a trend in the change in sunflower yield depending on changes in weather and climate conditions over the analyzed 10 years is being tested.

In this example, for " y » it is advisable to take the yield of sunflower, and for « x » is the number of the observed year in the analyzed period. Testing the hypothesis about the existence of any relationship between " x " and " y » can be done in two ways: manually and with the help of computer programs. Of course, with the availability of computer technology, this problem is solved by itself. But, in order to better understand the OLS toolkit, it is advisable to test the hypothesis about the existence of a relationship between " x " and " y » manually, when only a pen and an ordinary calculator are at hand. In such cases, the hypothesis of the existence of a trend is best checked visually by the location of the graphic image of the analyzed time series - the correlation field:

The correlation field in our example is located around a slowly increasing line. This in itself indicates the existence of a certain trend in the change in sunflower yield. It is impossible to speak about the presence of any trend only when the correlation field looks like a circle, a circle, a strictly vertical or strictly horizontal cloud, or consists of randomly scattered points. In all other cases, it is necessary to confirm the hypothesis of the existence of a relationship between " x " and " y and continue research.

Second MNC procedure. It is determined which line (trajectory) is best able to describe or characterize the trend in sunflower yield changes for the analyzed period.

With the availability of computer technology, the selection of the optimal trend occurs automatically. With "manual" processing, the choice of the optimal function is carried out, as a rule, in a visual way - by the location of the correlation field. That is, according to the type of chart, the equation of the line is selected, which is best suited to the empirical trend (to the actual trajectory).

As you know, in nature there is a huge variety of functional dependencies, so it is extremely difficult to visually analyze even a small part of them. Fortunately, in real economic practice, most relationships can be accurately described either by a parabola, or a hyperbola, or a straight line. In this regard, with the "manual" option for selecting the best function, you can limit yourself to only these three models.

Parabola of the second order: :

It is easy to see that in our example, the trend in sunflower yield changes over the analyzed 10 years is best characterized by a straight line, so the regression equation will be a straight line equation.

Third procedure. The parameters of the regression equation that characterizes this line are calculated, or in other words, an analytical formula is determined that describes the best trend model.

Finding the values ​​of the parameters of the regression equation, in our case, the parameters and , is the core of the LSM. This process is reduced to solving a system of normal equations.

(9.2)

This system of equations is quite easily solved by the Gauss method. Recall that as a result of the solution, in our example, the values ​​of the parameters and are found. Thus, the found regression equation will have the following form:

It is widely used in econometrics in the form of a clear economic interpretation of its parameters.

Linear regression is reduced to finding an equation of the form

or

Type equation allows for given parameter values X have theoretical values ​​of the effective feature, substituting the actual values ​​of the factor into it X.

Building a linear regression comes down to estimating its parameters − a and in. Linear regression parameter estimates can be found by different methods.

The classical approach to estimating linear regression parameters is based on least squares(MNK).

LSM allows one to obtain such parameter estimates a and in, under which the sum of the squared deviations of the actual values ​​of the resultant trait (y) from calculated (theoretical) mini-minimum:

To find the minimum of a function, it is necessary to calculate the partial derivatives with respect to each of the parameters a and b and equate them to zero.

Denote by S, then:

Transforming the formula, we obtain the following system of normal equations for estimating the parameters a and in:

Solving the system of normal equations (3.5) either by the method of successive elimination of variables or by the method of determinants, we find the desired parameter estimates a and in.

Parameter in called the regression coefficient. Its value shows the average change in the result with a change in the factor by one unit.

The regression equation is always supplemented with an indicator of the tightness of the relationship. When using linear regression, the linear correlation coefficient acts as such an indicator. There are various modifications of the linear correlation coefficient formula. Some of them are listed below:

As you know, the linear correlation coefficient is within the limits: -1 ≤ ≤ 1.

To assess the quality of the selection of a linear function, the square is calculated

A linear correlation coefficient called determination coefficient . The coefficient of determination characterizes the proportion of the variance of the effective feature y, explained by regression, in the total variance of the resulting trait:

Accordingly, the value 1 - characterizes the proportion of dispersion y, caused by the influence of other factors not taken into account in the model.

Questions for self-control

1. The essence of the method of least squares?

2. How many variables provide a pairwise regression?

3. What coefficient determines the tightness of the connection between the changes?

4. Within what limits is the coefficient of determination determined?

5. Estimation of parameter b in correlation-regression analysis?

1. Christopher Dougherty. Introduction to econometrics. - M.: INFRA - M, 2001 - 402 p.

2. S.A. Borodich. Econometrics. Minsk LLC "New Knowledge" 2001.

3. R.U. Rakhmetova Short course in econometrics. Tutorial. Almaty. 2004. -78s.

4. I.I. Eliseeva. Econometrics. - M.: "Finance and statistics", 2002

5. Monthly information and analytical magazine.

Nonlinear economic models. Nonlinear regression models. Variable conversion.

Nonlinear economic models..

Variable conversion.

elasticity coefficient.

If there are non-linear relationships between economic phenomena, then they are expressed using the corresponding non-linear functions: for example, an equilateral hyperbola , parabolas of the second degree, etc.

There are two classes of non-linear regressions:

1. Regressions that are non-linear with respect to the explanatory variables included in the analysis, but linear with respect to the estimated parameters, for example:

Polynomials of various degrees - , ;

Equilateral hyperbole - ;

Semilogarithmic function - .

2. Regressions that are non-linear in the estimated parameters, for example:

Power - ;

Demonstrative -;

Exponential - .

The total sum of the squared deviations of the individual values ​​of the resulting attribute at from the average value is caused by the influence of many factors. We conditionally divide the entire set of reasons into two groups: studied factor x and other factors.

If the factor does not affect the result, then the regression line on the graph is parallel to the axis oh and

Then the entire dispersion of the resulting attribute is due to the influence of other factors and the total sum of squared deviations will coincide with the residual. If other factors do not affect the result, then u tied With X functionally, and the residual sum of squares is zero. In this case, the sum of squared deviations explained by the regression is the same as the total sum of squares.

Since not all points of the correlation field lie on the regression line, their scatter always takes place as due to the influence of the factor X, i.e. regression at on X, and caused by the action of other causes (unexplained variation). The suitability of the regression line for the forecast depends on what part of the total variation of the trait at accounts for the explained variation

Obviously, if the sum of squared deviations due to regression is greater than the residual sum of squares, then the regression equation is statistically significant and the factor X has a significant impact on the outcome. y.

, i.e. with the number of freedom of independent variation of the feature. The number of degrees of freedom is related to the number of units of the population n and the number of constants determined from it. In relation to the problem under study, the number of degrees of freedom should show how many independent deviations from P

The assessment of the significance of the regression equation as a whole is given with the help of F- Fisher's criterion. In this case, a null hypothesis is put forward that the regression coefficient is equal to zero, i.e. b= 0, and hence the factor X does not affect the result y.

The direct calculation of the F-criterion is preceded by an analysis of the variance. Central to it is the expansion of the total sum of squared deviations of the variable at from the average value at into two parts - "explained" and "unexplained":

Total sum of squared deviations;

Sum of squares of deviation explained by regression;

Residual sum of squared deviation.

Any sum of squared deviations is related to the number of degrees of freedom , i.e. with the number of freedom of independent variation of the feature. The number of degrees of freedom is related to the number of population units n and with the number of constants determined from it. In relation to the problem under study, the number of degrees of freedom should show how many independent deviations from P possible is required to form a given sum of squares.

Dispersion per degree of freedomD.

F-ratios (F-criterion):

If the null hypothesis is true, then the factor and residual variances do not differ from each other. For H 0, a refutation is necessary so that the factor variance exceeds the residual by several times. The English statistician Snedecor developed tables of critical values F-relationships at different levels of significance of the null hypothesis and a different number of degrees of freedom. Table value F-criterion is the maximum value of the ratio of variances that can occur if they diverge randomly for a given level of probability of the presence of a null hypothesis. Computed value F-relationship is recognized as reliable if o is greater than the tabular one.

In this case, the null hypothesis about the absence of a relationship of features is rejected and a conclusion is made about the significance of this relationship: F fact > F table H 0 is rejected.

If the value is less than the table F fact ‹, F table, then the probability of the null hypothesis is higher than a given level and it cannot be rejected without a serious risk of drawing the wrong conclusion about the presence of a relationship. In this case, the regression equation is considered statistically insignificant. N o does not deviate.

Standard error of the regression coefficient

To assess the significance of the regression coefficient, its value is compared with its standard error, i.e., the actual value is determined t-Student's test: which is then compared with the table value at a certain level of significance and the number of degrees of freedom ( n- 2).

Parameter Standard Error a:

The significance of the linear correlation coefficient is checked based on the magnitude of the error correlation coefficient r:

Total variance of a feature X:

Multiple Linear Regression

Model building

Multiple regression is a regression of an effective feature with two or more factors, i.e. a model of the form

Regression can give a good result in modeling if the influence of other factors affecting the object of study can be neglected. The behavior of individual economic variables cannot be controlled, that is, it is not possible to ensure the equality of all other conditions for assessing the influence of one factor under study. In this case, you should try to identify the influence of other factors by introducing them into the model, i.e. build a multiple regression equation: y = a+b 1 x 1 +b 2 +…+b p x p + .

The main goal of multiple regression is to build a model with a large number of factors, while determining the influence of each of them individually, as well as their cumulative impact on the modeled indicator. The specification of the model includes two areas of questions: the selection of factors and the choice of the type of regression equation