尝试在 Kaggle 笔记本上将 oobabooga textgen 作为 api 运行时出现 Cuda 错误

Question

初学者尝试一下 Autogen！当我尝试在 Kaggle 笔记本上安装 oobabooga textgen web ui 时，我不断收到有关 cuda 版本太旧的错误。该脚本适用于 google colab。

import torch
from pathlib import Path

if Path.cwd().name != 'text-generation-webui':
  print("Installing the webui...")

  !git clone https://github.com/oobabooga/text-generation-webui
  %cd text-generation-webui

  torver = torch.__version__
  print(f"TORCH: {torver}")
  is_cuda118 = '+cu118' in torver  # 2.1.0+cu118
  is_cuda117 = '+cu117' in torver  # 2.0.1+cu117

  textgen_requirements = open('requirements.txt').read().splitlines()
  if is_cuda117:
      textgen_requirements = [req.replace('+cu121', '+cu117').replace('+cu122', '+cu117').replace('torch2.1', 'torch2.0') for req in textgen_requirements]
  elif is_cuda118:
      textgen_requirements = [req.replace('+cu121', '+cu118').replace('+cu122', '+cu118') for req in textgen_requirements]
  with open('temp_requirements.txt', 'w') as file:
      file.write('\n'.join(textgen_requirements))

  !pip install -r requirements.txt --upgrade
  !pip install -r extensions/openai/requirements.txt --upgrade
  !pip install -r temp_requirements.txt --upgrade

  print("\033[1;32;1m\n --> If you see a warning about \"previously imported packages\", just ignore it.\033[0;37;0m")
  print("\033[1;32;1m\n --> There is no need to restart the runtime.\n\033[0;37;0m")

  try:
    import flash_attn
  except:
    !pip uninstall -y flash_attn

# Parameters
model_url ="typeof/dolphin-2.2.1-mistral-7b-sharded"
branch = ""
command_line_flags = "--n-gpu-layers 128 --load-in-4bit --use_double_quant"
api = False

if api:
  for param in ['--api', '--public-api']:
    if param not in command_line_flags:
      command_line_flags += f" {param}"

model_url = model_url.strip()
if model_url != "":
    if not model_url.startswith('http'):
        model_url = 'https://huggingface.co/' + model_url

    # Download the model
    url_parts = model_url.strip('/').strip().split('/')
    output_folder = f"{url_parts[-2]}_{url_parts[-1]}"
    branch = branch.strip('"\' ')
    if branch.strip() != '':
        output_folder += f"_{branch}"
        !python download-model.py {model_url} --branch {branch}
    else:
        !python download-model.py {model_url}
else:
    output_folder = ""

# Start the web UI
cmd = f"python server.py --extensions openai --share --public-api"
if output_folder != "":
    cmd += f" --model {output_folder}"
cmd += f" {command_line_flags}"
print(cmd)
!$cmd

两种环境下的 python 版本都是 3.10，我能发现的唯一区别是 Kaggle 上的 cuda 版本是 11，而 google colab 上是 12。

有谁知道发生了什么事，有没有办法解决这个问题并使服务器也能在 Kaggle 笔记本上运行。

在 Kaggle 上我不断收到此警告：

CUDA initialization: The NVIDIA driver on your system is too old (found version 11040). Please update your GPU driver by downloading and installing a new version from the URL: http://www.nvidia.com/Download/index.aspx Alternatively, go to: https://pytorch.org to install a PyTorch version that has been compiled with your version of the CUDA driver.

那么应用程序不使用 GPU。笔记本电脑内存不断不足，相同的脚本可以在 google colab 上运行，而 Kaggle 笔记本电脑的内存是两倍。

仅供参考：这可以作为 openai API 的替代方案，用于将开源 LLM 与 autogen 结合使用

Answer 1

问题是pytorch版本与cuda版本不兼容