When&Where to transfer a tensor to cuda?

manpreet Best Answer 3 years ago

I transferred the output tensor to GPU at different positions in my code(before or after modifying its value) but got different results. What's the reason?

The failed code can be simplified as:

def Network(self):
    ........
    A = self.model(input)
    indexlist = self.indexlist
    output = torch.zeros(A.size(0))
    for i,li in enumerate(indexlist):
        if li:
            s,e = li
            output[i]+=sum(A[i,s:e])
    output = output if self.no_cuda else output.cuda(device=self.gpu,async=True)
    return output
pred = Network()
loss = F.nll_loss(pred,target)
loss.backward()

And the RuntimeError: Function torch::autograd::CopySlices returned an invalid gradient at index 1 - expected type torch.cuda.FloatTensor but got torch.FloatTensor

If I changed one line as follows, it runs normally:

def Network(self):
    ........
    A = self.model(input)
    indexlist = self.indexlist
    output = torch.zeros(A.size(0))
    output = output if self.no_cuda else output.cuda(device=self.gpu,async=True)
    for i,li in enumerate(indexlist):
        if li:
            s,e = li
            output[i]+=sum(A[i,s:e])
    return output
pred = Network()
loss = F.nll_loss(pred,target)
loss.backward()