Importance sampling remains unbiased because it corrects for the bias introduced by using a biased (proposal) distribution. The core idea is that while we sample from a simpler, potentially biased distribution, we adjust each sample's contribution to the final estimate using weights. These weights counteract the bias, leading to an unbiased overall estimator.
Understanding the Bias Correction
The key is the weighting scheme. The reference states: "the simulation outputs are weighted to correct for the use of the biased distribution, and this ensures that the new importance sampling estimator is unbiased." This weighting is crucial. It ensures that samples from regions with low probability under the target distribution but high probability under the proposal distribution don't disproportionately influence the final estimate. Conversely, samples from regions with high probability under the target distribution but low probability under the proposal distribution have their influence increased through the weighting. This balancing act eliminates the bias.
Illustrative Example
Imagine estimating the average value of a function f(x) over a complex distribution P(x). Directly sampling from P(x) might be difficult. Importance sampling uses a simpler distribution Q(x) instead. Each sample xi drawn from Q(x) is then weighted by the ratio Wi = P(xi) / Q(xi). The final estimate becomes:
∑i Wi * f(xi) / N
where N is the number of samples. The weights Wi effectively adjust the contribution of each sample, compensating for the difference between P(x) and Q(x) and leading to an unbiased estimate.
