检查点的 Peft 模型导致尺寸不匹配

Question

我训练了peft模型并将其保存在huggingface中。不，我想将它与基本模型合并。 我使用了以下代码。

from peft import PeftModel, PeftConfig,AutoPeftModelForCausalLM
from transformers import AutoModelForCausalLM,pipeline,AutoTokenizer,BitsAndBytesConfig

config = PeftConfig.from_pretrained("sanduntg/ORPO_peft_llama3_8B")
base_model_name = "meta-llama/Meta-Llama-3-8B"

tokenizer = AutoTokenizer.from_pretrained(base_model_name)
print("Tokenizer vocab size:", len(tokenizer))

base_model = AutoModelForCausalLM.from_pretrained(base_model_name,device_map="auto")
base_model.resize_token_embeddings(len(tokenizer))

base_model = PeftModel.from_pretrained(model=base_model, model_id="sanduntg/ORPO_peft_llama3_8B")

base_model = base_model.merge_and_unload()
base_model.save_pretrained("merged_adapters")

但是在这个

base_model = PeftModel.from_pretrained(model=base_model, model_id="sanduntg/ORPO_peft_llama3_8B") 

步骤中出现以下错误

RuntimeError: Error(s) in loading state_dict for PeftModelForCausalLM:
    size mismatch for base_model.model.model.embed_tokens.weight: copying a param with shape torch.Size([128258, 4096]) from checkpoint, the shape in current model is torch.Size([128256, 4096]).
    size mismatch for base_model.model.lm_head.weight: copying a param with shape torch.Size([128258, 4096]) from checkpoint, the shape in current model is torch.Size([128256, 4096]).

我已经尝试了上述步骤，将大小调整为令牌嵌入。但它没有奏效。训练时我使用了以下配置

peft_config = LoraConfig(
    r=16,
    lora_alpha=32,
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM",
    target_modules=['up_proj', 'down_proj', 'gate_proj', 'k_proj', 'q_proj', 'v_proj', 'o_proj']
)

Answer 1

您的分词器和模型应使用 trl.setup_chat_format() 重新初始化：

from trl import setup_chat_format

model, tokenizer = setup_chat_format(base_model, tokenizer)

运行上述命令后，请参阅

len(tokenizer)

中的变化..