当我尝试在
num_workers
中指定 dataloader
时,出现此错误。如果 colab
不允许这样做或者我需要做些什么,有人可以告诉我吗?
`---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
/usr/local/lib/python3.10/dist-packages/torch/utils/data/dataloader.py in _try_get_data(self, timeout)
1132 try:
-> 1133 data = self._data_queue.get(timeout=timeout)
1134 return (True, data)
10 frames
RuntimeError: DataLoader worker (pid 13603) is killed by signal: Segmentation fault.
上述异常是导致以下异常的直接原因:
RuntimeError Traceback (most recent call last)
/usr/local/lib/python3.10/dist-packages/torch/utils/data/dataloader.py in _try_get_data(self, timeout)
1144 if len(failed_workers) > 0:
1145 pids_str = ', '.join(str(w.pid) for w in failed_workers)
-> 1146 raise RuntimeError(f'DataLoader worker (pid(s) {pids_str}) exited unexpectedly') from e
1147 if isinstance(e, queue.Empty):
1148 return (False, None)
RuntimeError: DataLoader worker (pid(s) 13603) exited unexpectedly`
` # Create data loaders with increased timeout
train_loader = DataLoader(train_dataset, batch_size=batch_size, shuffle=True,num_workers=2)
val_loader = DataLoader(val_dataset, batch_size=batch_size,num_workers=2)
test_loader = DataLoader(test_dataset, batch_size=batch_size,num_workers=2`
我想通过增加工作人员数量来更快地加载数据集。
colab
会有所帮助:
pin_memory=True