Per Channel Bias

[hwu36]

In the 2.5 release, we added a new example to show how to do per channel bias and then relu for conv. It is very similar to the previous gemm version example.

These examples use a trick. The same layout as the A operand is used for the bias vector. However, the strides of all the dimensions are set to zeros for the bias vector. Then we can use the same iterator to load the bias vector and to store the final results. Since all the strided dimensions are zeros now for the bias vector, the iterator can only move in 1D dimension.

This trick cannot be used directly for the interleaved layout, e.g. nc32hw32 for int8 conv or n32t32 for int 8 gemm, without changing the underlying layout implementation. I think the better way is to implement a tiny iterator to load the bias vector in these cases.

[ocwins]

I think the better way is to implement a tiny iterator to load the bias vector in these cases.

Are there similar examples to implement such iterators ?

[hwu36]

Our 2.6 release would include more fused epilogue implementations. Our coming GTC talk would mention it.

If you cannot wait, you can hack your own quickly. It is just a struct that maintains a bias vector pointer and loads several channels of bias data each time. You can take a look at this one which loads/stores a matrix. In your case, you just need to load 1D data.