[Hugging face Error] ValueError: Cannot use chat template functions because tokenizer.chat_template is not set and no template argument was passed!

 

해당 에러가 뜨는 이유는 tokenizer.chat_template가 설정되지 않았거나 template 인자가 전달되지 않아서 발생한 문제이다. 

이 문제를 해결하려면, chat_template를 명시적으로 설정하거나 템플릿을 직접 작성해서 넘겨줘야 한다.

 

 

-- 해결 코드 --

# 기존 코드
import torch
from transformers import pipeline

model_id = "meta-llama/Llama-3.2-1B"
pipe = pipeline(
    "text-generation",
    model=model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
messages = [
    {"role": "system", "content": "You are a pirate chatbot who always responds in pirate speak!"},
    {"role": "user", "content": "Who are you?"},
]
tokenizer.apply_chat_template(chat, tokenize= False )
outputs = pipe(
    messages,
    max_new_tokens=256,
)
print(outputs[0]["generated_text"][-1])

 

# 수정 코드
import torch
from transformers import pipeline, AutoTokenizer

# Load the tokenizer and model
model_id = "meta-llama/Llama-3.2-1B"
tokenizer = AutoTokenizer.from_pretrained(model_id)

pipe = pipeline(
    "text-generation",
    model=model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

# Define the template
template = (
    "{system_message}\n\n"
    "User: {user_message}\n"
    "Assistant:"
)

messages = [
    {"role": "system", "content": "You are a pirate chatbot who always responds in pirate speak!"},
    {"role": "user", "content": "Who are you?"},
]

# Apply the template manually
chat = template.format(
    system_message=messages[0]['content'],
    user_message=messages[1]['content']
)

# Generate text
outputs = pipe(chat, max_new_tokens=256)

print(outputs[0]["generated_text"])