从拥抱脸部运行meta-llama/Llama-2-7b-chat-hf时出错，我不明白我哪里出了问题

Question

我正在运行的代码是：

import torch
from llama_index.llms.huggingface import HuggingFaceLLM

llm = HuggingFaceLLM(
    context_window=4096,
    max_new_tokens=256,
    generate_kwargs={"temperature": 0.0, "do_sample": False},
    system_prompt = "You need to create proposal documents for the information that we are giving to you. This proposal has to be 5 paragraphs and answered well. If you need more information regarding the topic, you can reply with what furhter information could help",
    ## add the description of what I want it to work on
    query_wrapper_prompt = "<|USER|>{query_str}<|ASSISTANT|>",
    tokenizer_name="meta-llama/Llama-2-7b-chat-hf",
    model_name="meta-llama/Llama-2-7b-chat-hf",
    device_map="auto",
    # uncomment this if using CUDA to reduce memory usage
    model_kwargs={"torch_dtype": torch.float16 , "load_in_8bit":True}
)

我尝试运行此程序时遇到的错误是：

ImportError                               Traceback (most recent call last)
<ipython-input-34-0d2d206e16bb> in <cell line: 4>()
      2 from llama_index.llms.huggingface import HuggingFaceLLM
      3 
----> 4 llm = HuggingFaceLLM(
      5     context_window=4096,
      6     max_new_tokens=256,

3 frames
/usr/local/lib/python3.10/dist-packages/transformers/quantizers/quantizer_bnb_8bit.py in validate_environment(self, *args, **kwargs)
     60     def validate_environment(self, *args, **kwargs):
     61         if not (is_accelerate_available() and is_bitsandbytes_available()):
---> 62             raise ImportError(
     63                 "Using `bitsandbytes` 8-bit quantization requires Accelerate: `pip install accelerate` "
     64                 "and the latest version of bitsandbytes: `pip install -i https://pypi.org/simple/ bitsandbytes`"

ImportError: Using `bitsandbytes` 8-bit quantization requires Accelerate: `pip install accelerate` and the latest version of bitsandbytes: `pip install -i https://pypi.org/simple/ bitsandbytes`

---------------------------------------------------------------------------
NOTE: If your import is failing due to a missing package, you can
manually install dependencies using either !pip or !apt.

要查看安装一些常见依赖项的示例，请单击 下面的“打开示例”按钮。

我已经尝试再次使用 pip 安装库，但仍然收到错误。如果您能对此提供帮助，我将非常感激。

Answer 1

ImportError：使用

bitsandbytes

8 位量化需要 Accelerate：

pip install accelerate

和最新版本的 bitsandbytes：

pip install -i https://pypi.org/simple/ bitsandbytes

您需要 pip install Accelerator

然后 pip install -i https://pypi.org/simple/bitsandbytes