NIST(x):=interp(loess(X,Y,.5),X,Y,x)

where x was set as a range variable x:=0,0.1,..20

This creates a function NIST(x) that you can plot versus the range variable x

Here is the MathCAD explanation for the two built-in functions ‘interp’ & ‘loess’:

loess(vx, vy, span) Returns a vector which interp uses to find a set of second-order polynomials that best fit the neighborhood of x and y data values in vx and vy in the least-squares sense. The size of the neighborhood is controlled by span.

interp(vs, vx, vy, x) Returns the interpolated y-value corresponding to x using the output vector vs from loess.

Your new output is the result of LOESS calculation using a 7 point moving regression, using X and Y input as inputs and X input as the X output.

Please see the difference of the output:

X input Y input X output original output new output
0.55782 18.63654 0 -13.07212037 20.59304051
2.021727 103.4965 1 47.06914445 107.1603064
2.577325 150.3539 2 105.8974689 139.7673806
3.414029 190.5103 3 159.6924955 174.2630716
4.301408 208.7012 4 196.8626739 207.2333938
4.744839 213.7114 5 219.0115635 216.6616039
5.107378 228.4935 6 227.6417493 220.5444981
6.541166 233.5539 7 230.2534309 229.8606994
6.721618 234.5505 8 227.0376643 229.8347242
7.260058 223.8923 9 221.246659 229.4301269
8.133587 227.6834 10 202.9876659 226.6044626
9.122438 223.9198 11 187.8518139 220.3904231
11.92967 168.02 12 170.6711911 172.3479193
12.37977 164.9575 13 162.210816 163.8416617
13.27286 152.6111 14 160.7756941 161.8489846
14.27675 160.7874 15 159.7155843 160.3350921
15.3731 168.5557 16 161.0838629 160.1920102
15.64766 152.4266 17 194.5740586 161.0555463
18.56054 221.707 18 215.4190384 227.3399984
18.58664 222.6904 19 236.7904956 227.8985782
18.75728 243.1883

The line of code works fine in VBA, which assigns the values in a range to a 1-based array. You have to adjust the syntax to use the code in C++.

Thank you for the nice code. It helped me easily added a function to a C++ App. There is a small bug you might not notice. The assignment for xNow should be:

xNow = X(xDomain(iPoint, 1) + 1, 1)

instead of:

xNow = xDomain(iPoint, 1)

Thank you,

Public Function LOESSW(X As Variant, Y As Variant, PtWt As Variant, xDomain As Variant, nPts As Long) As Double()
  Dim i As Long
  Dim iMin As Long
  Dim iMax As Long
  Dim iPoint As Long
  Dim iMx As Long
  Dim mx As Variant
  Dim maxDist As Double
  Dim SumWts As Double, SumWtX As Double, SumWtX2 As Double, SumWtY As Double, SumWtXY As Double
  Dim Denom As Double, WLRSlope As Double, WLRIntercept As Double
  Dim xNow As Double
  Dim distance() As Double
  Dim Weight() As Double
  Dim yLoess() As Double
  Dim totalWeight As Double

  If TypeName(X) = "Range" Then
    X = X.Value
  End If

  If TypeName(Y) = "Range" Then
    Y = Y.Value
  End If

  If TypeName(PtWt) = "Range" Then
    PtWt = PtWt.Value
  End If

  If TypeName(xDomain) = "Range" Then
    xDomain = xDomain.Value
  End If

  ReDim yLoess(LBound(xDomain, 1) To UBound(xDomain, 1), 1 To 1)

  For iPoint = LBound(xDomain, 1) To UBound(xDomain, 1)

    iMin = LBound(X, 1)
    iMax = UBound(X, 1)

    xNow = xDomain(iPoint, 1)

    ReDim distance(iMin To iMax)
    ReDim Weight(iMin To iMax)

    For i = iMin To iMax
      ' populate x, y, distance
      distance(i) = Abs(X(i, 1) - xNow)
    Next

    Do
      ' find the nPts points closest to xNow
      If iMax + 1 - iMin  distance(iMax) Then
        ' remove first point
        iMin = iMin + 1
      ElseIf distance(iMin)  maxDist Then maxDist = distance(i)
      totalWeight = totalWeight + PtWt(i, 1)
    Next

    ' calculate weights using scaled distances
    For i = iMin To iMax
      Weight(i) = (PtWt(i, 1) / totalWeight) * (1 - (distance(i) / maxDist) ^ 3) ^ 3
    Next

    ' do the sums of squares
    SumWts = 0
    SumWtX = 0
    SumWtX2 = 0
    SumWtY = 0
    SumWtXY = 0
    For i = iMin To iMax
      SumWts = SumWts + Weight(i)
      SumWtX = SumWtX + X(i, 1) * Weight(i)
      SumWtX2 = SumWtX2 + (X(i, 1) ^ 2) * Weight(i)
      SumWtY = SumWtY + Y(i, 1) * Weight(i)
      SumWtXY = SumWtXY + X(i, 1) * Y(i, 1) * Weight(i)
    Next
    Denom = SumWts * SumWtX2 - SumWtX ^ 2

    ' calculate the regression coefficients, and finally the loess value
    WLRSlope = (SumWts * SumWtXY - SumWtX * SumWtY) / Denom
    WLRIntercept = (SumWtX2 * SumWtY - SumWtX * SumWtXY) / Denom
    yLoess(iPoint, 1) = WLRSlope * xNow + WLRIntercept

  Next

  LOESSW = yLoess

End Function
 

I used 150 for the number of points in the analysis. Here is a comparison of three curves, for 100, 150, and 200 points in the moving regression:

The website data is from my own site’s history several months ago. I’m not sure I still have that particular data set.

Many thanks for the link – I missed it when I first visited the Wired website. What smoothing coefficient did you use for your loess fit? And is the website data as readily available as the Playmate data?

Again, thanks in anticipation.

Andrew