deep-learning - Best way to handle batch during training and inference in Pytorch with GPU

Question

I am learning the best ways to manage batches and other best practices during model training and inference and I have the following question:

• If I have a batch that I move to GPU, should. I move it back to CPU after doing the training? If no, why?

  batch, label = batch.to(device), label.to(device)
  model(batch)
  # ..Training pass... 
  batch, label = batch.cpu(), label.cpu()
  

  If I cache my data in my Dataset class how can I ensure I can reuse the same batches on GPU to avoid transferring from and to CPU multiple times?

Answer 1

您不必将数据移回 cpu。GPU 上的数据分配由 PyTorch 处理。您应该使用 atorch.utils.data.DataLoader来处理从数据集中加载的数据。但是，您必须自己在 GPU 上发送输入：本质上，每次您需要推断一些输出时，您都会将批次和标签发送到 GPU 并计算结果（和损失），仅此而已。