常用代碼片段及技巧

自動選擇GPU和CPU

          
            device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
# model and tensor to device
vgg = models.vgg16().to(device)
          
        

切換當前目錄

          
            import os
try:
    os.chdir(os.path.join(os.getcwd(), '..'))
    print(os.getcwd())
except:
    pass
          
        

臨時添加環(huán)境目錄

          
            import sys
sys.path.append('引用模塊的地址')
print(sys.path)
          
        

打印模型參數(shù)

          
            from torchsummary import summary
# 1 means in_channels
summary(model, (1, 28, 28))
          
        

將tensor的列表轉(zhuǎn)換為tensor

          
            x = torch.stack(tensor_list)
          
        

內(nèi)存不夠

• Smaller batch size
• torch.cuda.empty_cache() every few minibatches
• 分布式計算
• 訓練數(shù)據(jù)和測試數(shù)據(jù)分開
• 每次用完之后刪去variable，采用 del x

debug tensor memory

resource` module is a Unix specific package as seen in https://docs.python.org/2/library/resource.html which is why it worked for you in Ubuntu, but raised an error when trying to use it in Windows.

Here is what solved it for me.

1. Downgrade to the Apache Spark 2.3.2 prebuild version
2. Install (or downgrade) jdk to version 1.8.0
   • My installed jdk was 1.9.0, which doesn't seem to be compatiable with spark 2.3.2 or 2.4.0
3. make sure that when you run java -version in cmd (command prompt), it show java version 8. If you are seeing version 9, you will need to change your system ENV PATH to ensure it points to java version 8.
4. Check this link to get help on changing the PATH if you have multiple java version installed.

          
            device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

def debug_memory():
    import collections, gc, resource, torch
    print('maxrss = {}'.format(
        resource.getrusage(resource.RUSAGE_SELF).ru_maxrss))
    tensors = collections.Counter((str(o.device), o.dtype, tuple(o.shape))
                                  for o in gc.get_objects()
                                  if torch.is_tensor(o))
    for line in sorted(tensors.items()):
        print('{}\t{}'.format(*line))
        
        
 # example
import tensor
 x = torch.tensor(3,3)
 debug_memory()
 
 y = torch.tensor(3,3)
 debug_memory()
 
 z = [torch.randn(i).long() for i in range(10)]
 debug_memory()