About the re-weighted loss

[thuxmf]

P2-weighting/guided_diffusion/gaussian_diffusion.py

Line 818 in 3ea1470

weight = _extract_into_tensor(1 / (self.p2_k + self.snr)**self.p2_gamma, t, target.shape)

I found that you use the weight to multiply on the final loss, in which the denominator is greater than 1 since self.p2_k >= 1 and self.snr > 0 . Therefore, weight is smaller than 1. I wonder how to achieve the result that the total weights of your method is greater than the baseline of DDPM when the SNR is in the interval [1e-2, 1e0]?

[jychoi118]

Above plot shows the normalized weights, where the sum of the weights is 1. The plot without normalization is shown in the appendix. Here, weights are smaller than the baseline as you expected.

[thuxmf]

Thanks！

[HaoLyou]

Why is the numerator 1 in the code, not the lambda mentioned in the paper?

P2-weighting/guided_diffusion/gaussian_diffusion.py

Line 818 in 3ea1470

weight = _extract_into_tensor(1 / (self.p2_k + self.snr)**self.p2_gamma, t, target.shape)

I found that you use the weight to multiply on the final loss, in which the denominator is greater than 1 since self.p2_k >= 1 and self.snr > 0 . Therefore, weight is smaller than 1. I wonder how to achieve the result that the total weights of your method is greater than the baseline of DDPM when the SNR is in the interval [1e-2, 1e0]?