Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

About p2-weighting #22

Open
HarperHao opened this issue Jan 3, 2024 · 3 comments
Open

About p2-weighting #22

HarperHao opened this issue Jan 3, 2024 · 3 comments

Comments

@HarperHao
Copy link

weight = _extract_into_tensor(1 / (self.p2_k + self.snr)**self.p2_gamma, t, target.shape)
issue

Why is the numerator 1 in the code, not the lambda mentioned in the equation 8?

@LinWeiJeff
Copy link

@HarperHao @jychoi118
I have the same question, why does the lambda_t equal to 1?
Do you find out the reason?

@HarperHao
Copy link
Author

@HarperHao @jychoi118 I have the same question, why does the lambda_t equal to 1? Do you find out the reason?

Sorry,Finally I modify the code to replace 1 with lambda

@HarperHao HarperHao reopened this Jun 4, 2024
@gsustar
Copy link

gsustar commented Jul 31, 2024

@HarperHao @LinWeiJeff @jychoi118
The way I understand it is as follows:

The original VLB loss $L_t$ can be rewritten as the expected value of a weighted MSE of the noise prediction and the actual noise. This is described in equation 5:

image

However, we are using $L_{simple}$, which drops this weighting factor (so the weighting factor is one). In terms of VLB this means that we are multiplying equation 5 with the inverse of the weighting factor. The authors refere to this inverse as $\lambda_t$ or $baseline$

image

later on in equation 8 the authors present a new weighting scheme that replaces $\lambda_t$ with $\lambda_t'$

image

They then state the following:

image

Note how the authors are using $L_t$ and NOT $L_{simple}$ in the highlighted text. Remember, $L_t$ still has the weighting factor attached to it and to get rid of it you need to multiply by $\lambda_t$. In the code however we are using $L_{simple}$ which is inherently already multiplied by $\lambda_t$. This means we can simply multiply $L_{simple}$ with $\frac{1}{(k + SNR(t))^\gamma}$ to get the P2 weighting scheme.

I hope my explanation was clear.
@jychoi118 Please correct me if I misinterpreted any of your claims.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants