Readings are high and low. How to find the trend behind it?
Put the scattered observation records together and find a line that is closest to them as a whole.
General Algorithms and Numerical Computing
When shooting continuously, the position reading of the light point in the frame cannot be perfect every time. You want to know how it moves overall, rather than being guided by a single read bias. First look at the schematic records seven times, and then compare the fits of straight lines and curves.
Guess first
When recording errors, does a good fitted curve have to pass through every point?
Show answer and explanation
No need — Least squares seeks to minimize the sum of squares of the overall residuals and does not require each measured value to fall exactly on the curve.
Then connect experience and science
First select a type of curve
The same batch of records can fit both straight lines and curved lines. Choosing the curve family first will have the meaning of "best".
y = Σ cⱼ fⱼ(x)
Measure each vertical difference
Each point is at the same x position, a vertical difference from the curve. These differences may be positive or negative, and adding them directly will cancel each other out.
Residual error eᵢ = yᵢ − ŷᵢ
Minimize the sum of squares
Add up the squares of each difference. The program finds the coefficients that minimize it without the curve having to pass through every point.
SSE = Σeᵢ² → minimum
Good fit does not mean proving the reason
The correlation coefficient only measures a linear relationship. There may be clear curve patterns when r is close to zero; neither fitting nor correlation can automatically prove cause and effect.
r = Sxy / √(Sxx Syy)
Expand the calculation
ŷᵢ = Σⱼ cⱼ fⱼ(xᵢ) SSE = Σᵢ(yᵢ − ŷᵢ)² Σⱼ[Σᵢ fₖ(xᵢ)fⱼ(xᵢ)]cⱼ = Σᵢ fₖ(xᵢ)yᵢ r = Sxy/√(Sxx Syy)
The normal equation writes one line for each basis function k and calculates the coefficients simultaneously. Sxx and Syy are the mean-removed sum of squares, and Sxy is the mean-removed sum of products; r is undefined when any sum of squares is zero.
give it a try
First compare the straight line and the quadratic curve between the same set of records; then offset a reading. Cut to the "r=0 curve" to see if the absence of linear correlation really equals no pattern.
No need to memorize symbols
- x / y
- The abscissa is considered accurate, and the ordinate contains measurement errors.
- eᵢ
- The vertical residual from the i-th point to the curve
- SSE
- residual sum of squares
- r
- Pearson linear correlation coefficient; undefined for constant data
- cⱼ / fⱼ
- Undetermined coefficients/pre-selected basis functions
Algorithm, accuracy and scope of application
Chapter 4 of the original book, printed pages 35–46, key formulas (4.2), (4.3), (4.5)–(4.8). Ordinary schematic records are artificially constructed pixel coordinates, not real celestial motion measurements. Experiment 0/1 used the same seven points; Experiment 2 used the parabolic counterexample of printed page 40; Experiment 3 used the 20 four-digit decimal data of Example 4.c; Experiment 4 used the six points of Example 4.d. All models assume accurate x, unweighted vertical least squares on y, and do not handle x errors or weighted/robust fits. The units of x for the three-sine model are degrees. Using QR to solve linear coefficients has the same goal as the normal equations in the book, avoiding direct inversion; it is not about interpreting correlation as cause and effect. AA+ does not provide corresponding dedicated modules, so the original book examples and independent optimality checks are used. The entire historical comet/sunspot data analysis will not be reproduced here, nor will future celestial positions be predicted.
my understanding check
Why add the squared residuals?
Avoid positive and negative errors from canceling each other out — The direct sum of positive and negative residuals may be zero, but it does not mean that the deviation is small.
The linear correlation coefficient r = 0, can we say that there is no pattern at all?
cannot — The linear correlation of the seven points in the book is zero, but they fall exactly on a parabola.
The fit is great, has it been proven that x causes y?
No — Evidence such as physical mechanism and experimental design is also needed; mathematical correlation is not causal proof.
The best fit is only true for the selected curve family and error definition; residuals, correlation and causation should be looked at separately.
Jean Meeus · Astronomical Algorithms, 2nd ed. · No. 4 chapter
General Algorithms and Numerical Computing · All interactive courses