8.3 局部加權回歸
8.3 Locally Weighted Regression
前一節描述的最近鄰方法可以被看作在單一的查詢點x=xq上逼近目標函數f(x)。局部加權
回歸是這種方法的推廣。它在環繞xq的局部區域內為目標函數f建立明確的逼近。局部加
權回歸使用附近的或距離加權的訓練樣例來形成這種對f的局部逼近。例如，我們可以使
用線性函數、二次函數、多層神經網路或者其他函數形式在環繞xq的鄰域內逼近目標函數
。“局部加權回歸”名稱中，之所以叫“局部”是因為目標函數的逼近僅僅根據查詢點附
近的資料，之所以叫“加權”是因為每一個訓練樣例的貢獻是由它與查詢點間的距離加權
的，之所以叫“回歸”是因為統計學習界廣泛使用這個術語來表示逼近實數值函數的問題
。
The nearest-neighbor method described in the previous chapter can be seen as
a method of approximation to the target function f(x) at a single point x = xq.
Locally weighted regression is an extension of this method. It establishes
explicit approximation to target function f in a local area around xq. Locally
weighted regression achieves local approximation to f using neighboring or
distance-weighted training examples. For example, we can use linear functions,
quadratic functions, multi-layer neural networks, or other types of functions
to approximate the target function in the vicinity of xq. Regarding the name of
"locally weighted regression," why it is called locally is because that the
target function is approximated as to only the data surrounding the query
point; it is called "weighted" because the contribution of each training
example is weighted by its distance to the query point; it is called
"regression" because because this term is widely used in statistician circles
to formulate problems about approximation of real-valued functions.
給定一個新的查詢實例xq，局部加權回歸的一般方法是建立一個逼近f hat，使f hat擬合
環繞xq的鄰域內的訓練樣例。然後用這個逼近來計算f hat (xq)的值，也就是為查詢實例
估計的目標值輸出。然後f hat的描述被刪除，因為對於每一個獨立的查詢實例都會計算
不同的局部逼近。
For a given new query example xq, the general method of locally weighted
^
regression is to construct a f for approximation and fit the training examples
^
in the neighborhood around xq. Subsequently, compute the value of f using this
approximation; that is, estimate an output value for the training example. Then
^
the description of f is removed, because a local approximation for each query
example is computed independently.
8.3.1 局部加權線性回歸
8.3.1 Locally-Weighted Linear Regression
下面，我們先考慮局部加權回歸的一種情況，即使用如下形式的線性函數來逼近xq鄰域的
目標函數f：
Below, we first consider one situation in locally-weighted regression, which is
the use of the following linear function to approximate the neighborhood of xq
in target function f:
^
f (x)=w0+w1a1(x)+...+wnan(x)
和前面一樣，ai(x)表示實例x的第i個屬性值。
回憶第4章中我們討論的梯度下降方法，在擬合以上形式的線性函數到給定的訓練集合時
，它被用來找到使誤差最小化的係數w0 ... wn。在那一章中我們感興趣的是目標函數的
全域逼近。所以當時我們推導出的權值選擇方法是使訓練集合D上的誤差平方和最小化，
即：
E = ....... （8.5）
As above, ai(x) denotes the ith attribute value of example x.
Remember the gradient descent method we discussed in chapter 4. When fitting
linear functions of the above form to a given training set, the method is used
to find coefficients w0 to wn that minimize the error. In that chapater, we
were interested in the global approximation of the target function. Thus, our
weighting method was derived by minimizing the sum of square errors on training
set D.
E = ....... （8.5）
http://tinyurl.com/ntos6ca