本笔记本提供了我们新的 gpt-3.5-turbo微调的逐步指南。我们将使用 RecipeNLG数据集执行实体提取，该数据集提供各种食谱和每个食谱提取的通用成分列表。这是命名实体识别（NER）任务的常见数据集。

我们将按照以下步骤进行：

1. 设置： 加载我们的数据集并将其过滤到一个域以进行微调。
2. 数据准备： 通过创建训练和验证示例并将它们上传到Files端点来准备您的数据以进行微调。
3. 微调： 创建您的微调模型。
4. 推理： 使用您的微调模型对新输入进行推理。

通过本文，您应该能够训练，评估和部署一个微调的gpt-3.5-turbo模型。

有关微调的更多信息，您可以参考我们的文档指南，API参考或博客文章。

设置

# 安装最新版本的openai python包
!pip install --upgrade openai

# 导入所需的模块
import json  # 用于处理 JSON 数据
import openai  # OpenAI 的 Python SDK，用于调用 OpenAI API
import os  # 用于访问操作系统的功能
import pandas as pd  # 用于处理和分析数据的库
from pprint import pprint  # 用于打印漂亮的输出

# 从环境变量中获取 OpenAI API 密钥
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY", "")

# 设置 OpenAI API 密钥
openai.api_key = OPENAI_API_KEY

微调在专注于特定领域时效果最好。确保数据集既足够专注以便模型学习，又足够通用以避免遗漏未见过的示例非常重要。考虑到这一点，我们从RecipesNLG数据集中提取了一个子集，仅包含来自www.cookbooks.com的文档。

# 导入需要使用的数据集
# 这里使用的是RecipesNLG数据集，我们已经将其清理，只包含来自www.cookbooks.com的文档
recipe_df = pd.read_csv("data/cookbook_recipes_nlg_10k.csv")

# 打印数据集的前几行，以便查看数据
recipe_df.head()

	title	ingredients	directions	link	source	NER
0	No-Bake Nut Cookies	["1 c. firmly packed brown sugar", "1/2 c. eva...	["In a heavy 2-quart saucepan, mix brown sugar...	www.cookbooks.com/Recipe-Details.aspx?id=44874	www.cookbooks.com	["brown sugar", "milk", "vanilla", "nuts", "bu...
1	Jewell Ball'S Chicken	["1 small jar chipped beef, cut up", "4 boned ...	["Place chipped beef on bottom of baking dish....	www.cookbooks.com/Recipe-Details.aspx?id=699419	www.cookbooks.com	["beef", "chicken breasts", "cream of mushroom...
2	Creamy Corn	["2 (16 oz.) pkg. frozen corn", "1 (8 oz.) pkg...	["In a slow cooker, combine all ingredients. C...	www.cookbooks.com/Recipe-Details.aspx?id=10570	www.cookbooks.com	["frozen corn", "cream cheese", "butter", "gar...
3	Chicken Funny	["1 large whole chicken", "2 (10 1/2 oz.) cans...	["Boil and debone chicken.", "Put bite size pi...	www.cookbooks.com/Recipe-Details.aspx?id=897570	www.cookbooks.com	["chicken", "chicken gravy", "cream of mushroo...
4	Reeses Cups(Candy)	["1 c. peanut butter", "3/4 c. graham cracker ...	["Combine first four ingredients and press in ...	www.cookbooks.com/Recipe-Details.aspx?id=659239	www.cookbooks.com	["peanut butter", "graham cracker crumbs", "bu...

数据准备

我们将从准备数据开始。在使用 ChatCompletion 格式进行微调时，每个训练样本都是一个简单的 messages 列表。例如，一个条目可能如下所示：

[{'role': 'system',
  'content': 'You are a helpful recipe assistant. You are to extract the generic ingredients from each of the recipes provided.'},

 {'role': 'user',
  'content': 'Title: No-Bake Nut Cookies\n\nIngredients: ["1 c. firmly packed brown sugar", "1/2 c. evaporated milk", "1/2 tsp. vanilla", "1/2 c. broken nuts (pecans)", "2 Tbsp. butter or margarine", "3 1/2 c. bite size shredded rice biscuits"]\n\nGeneric ingredients: '},

 {'role': 'assistant',
  'content': '["brown sugar", "milk", "vanilla", "nuts", "butter", "bite size shredded rice biscuits"]'}]

在训练过程中，这个对话将被分割，最后一个条目将成为模型生成的 completion，而其余的 messages 将作为提示。在构建训练样本时，请考虑到这一点——如果您的模型将在多轮对话中发挥作用，请提供代表性的例子，以免在对话开始扩展时表现不佳。

请注意，目前每个训练样本的令牌限制为4096个。超过这个长度的内容将被截断为4096个令牌。

# 定义一个空列表，用于存储训练数据
training_data = []

# 定义系统消息，作为提示信息
system_message = "You are a helpful recipe assistant. You are to extract the generic ingredients from each of the recipes provided."

# 定义一个函数，根据给定的行数据创建用户消息
def create_user_message(row):
    # 返回一个包含标题、原始食材和通用食材的字符串
    return f"""Title: {row['title']}\n\nIngredients: {row['ingredients']}\n\nGeneric ingredients: """

# 定义一个函数，根据给定的行数据准备示例对话
def prepare_example_conversation(row):
    # 创建一个空列表，用于存储对话消息
    messages = []
    
    # 添加系统消息到对话列表中
    messages.append({"role": "system", "content": system_message})

    # 创建用户消息并添加到对话列表中
    user_message = create_user_message(row)
    messages.append({"role": "user", "content": user_message})

    # 添加助手消息到对话列表中，助手消息为给定行数据中的NER字段
    messages.append({"role": "assistant", "content": row["NER"]})

    # 返回包含对话消息的字典
    return {"messages": messages}

# 打印准备好的示例对话
pprint(prepare_example_conversation(recipe_df.iloc[0]))

{'messages': [{'content': 'You are a helpful recipe assistant. You are to '
                          'extract the generic ingredients from each of the '
                          'recipes provided.',
               'role': 'system'},
              {'content': 'Title: No-Bake Nut Cookies\n'
                          '\n'
                          'Ingredients: ["1 c. firmly packed brown sugar", '
                          '"1/2 c. evaporated milk", "1/2 tsp. vanilla", "1/2 '
                          'c. broken nuts (pecans)", "2 Tbsp. butter or '
                          'margarine", "3 1/2 c. bite size shredded rice '
                          'biscuits"]\n'
                          '\n'
                          'Generic ingredients: ',
               'role': 'user'},
              {'content': '["brown sugar", "milk", "vanilla", "nuts", '
                          '"butter", "bite size shredded rice biscuits"]',
               'role': 'assistant'}]}

现在，让我们对数据集的一个子集进行此操作，以用作我们的训练数据。您可以从30-50个经过良好修剪的示例开始。随着训练集大小的增加，您应该看到性能继续呈线性扩展，但您的作业也将需要更长时间。

# 使用数据集的前100行进行训练
training_df = recipe_df.loc[0:100]

# 对training_df的每一行应用prepare_example_conversation函数
training_data = training_df.apply(prepare_example_conversation, axis=1).tolist()

# 打印前5个训练样例
for example in training_data[:5]:
    print(example)

{'messages': [{'role': 'system', 'content': 'You are a helpful recipe assistant. You are to extract the generic ingredients from each of the recipes provided.'}, {'role': 'user', 'content': 'Title: No-Bake Nut Cookies\n\nIngredients: ["1 c. firmly packed brown sugar", "1/2 c. evaporated milk", "1/2 tsp. vanilla", "1/2 c. broken nuts (pecans)", "2 Tbsp. butter or margarine", "3 1/2 c. bite size shredded rice biscuits"]\n\nGeneric ingredients: '}, {'role': 'assistant', 'content': '["brown sugar", "milk", "vanilla", "nuts", "butter", "bite size shredded rice biscuits"]'}]}
{'messages': [{'role': 'system', 'content': 'You are a helpful recipe assistant. You are to extract the generic ingredients from each of the recipes provided.'}, {'role': 'user', 'content': 'Title: Jewell Ball\'S Chicken\n\nIngredients: ["1 small jar chipped beef, cut up", "4 boned chicken breasts", "1 can cream of mushroom soup", "1 carton sour cream"]\n\nGeneric ingredients: '}, {'role': 'assistant', 'content': '["beef", "chicken breasts", "cream of mushroom soup", "sour cream"]'}]}
{'messages': [{'role': 'system', 'content': 'You are a helpful recipe assistant. You are to extract the generic ingredients from each of the recipes provided.'}, {'role': 'user', 'content': 'Title: Creamy Corn\n\nIngredients: ["2 (16 oz.) pkg. frozen corn", "1 (8 oz.) pkg. cream cheese, cubed", "1/3 c. butter, cubed", "1/2 tsp. garlic powder", "1/2 tsp. salt", "1/4 tsp. pepper"]\n\nGeneric ingredients: '}, {'role': 'assistant', 'content': '["frozen corn", "cream cheese", "butter", "garlic powder", "salt", "pepper"]'}]}
{'messages': [{'role': 'system', 'content': 'You are a helpful recipe assistant. You are to extract the generic ingredients from each of the recipes provided.'}, {'role': 'user', 'content': 'Title: Chicken Funny\n\nIngredients: ["1 large whole chicken", "2 (10 1/2 oz.) cans chicken gravy", "1 (10 1/2 oz.) can cream of mushroom soup", "1 (6 oz.) box Stove Top stuffing", "4 oz. shredded cheese"]\n\nGeneric ingredients: '}, {'role': 'assistant', 'content': '["chicken", "chicken gravy", "cream of mushroom soup", "shredded cheese"]'}]}
{'messages': [{'role': 'system', 'content': 'You are a helpful recipe assistant. You are to extract the generic ingredients from each of the recipes provided.'}, {'role': 'user', 'content': 'Title: Reeses Cups(Candy)  \n\nIngredients: ["1 c. peanut butter", "3/4 c. graham cracker crumbs", "1 c. melted butter", "1 lb. (3 1/2 c.) powdered sugar", "1 large pkg. chocolate chips"]\n\nGeneric ingredients: '}, {'role': 'assistant', 'content': '["peanut butter", "graham cracker crumbs", "butter", "powdered sugar", "chocolate chips"]'}]}

除了训练数据外，我们还可以选择性地提供验证数据，用于确保模型不会过度拟合训练集。

# 从recipe_df中选取101到200行的数据作为验证集
validation_df = recipe_df.loc[101:200]
# 对验证集中的每一行数据进行处理，生成对话样本，并将其转换为列表形式
validation_data = validation_df.apply(prepare_example_conversation, axis=1).tolist()

我们需要将数据保存为.jsonl文件，每一行都是一个训练示例对话。

# 定义一个函数write_jsonl，接受两个参数data_list和filename，返回值为None
def write_jsonl(data_list: list, filename: str) -> None:
    # 打开文件，以写入模式打开，并使用out作为文件对象
    with open(filename, "w") as out:
        # 遍历data_list中的每个元素，将其赋值给变量ddict
        for ddict in data_list:
            # 将ddict转换为JSON格式的字符串，并添加换行符
            jout = json.dumps(ddict) + "\n"
            # 将jout写入文件
            out.write(jout)

# 定义一个变量training_file_name，用于存储训练数据的文件名
training_file_name = "tmp_recipe_finetune_training.jsonl"

# 调用write_jsonl函数，将训练数据写入到training_file_name指定的文件中
write_jsonl(training_data, training_file_name)

# 定义一个变量validation_file_name，用于存储验证数据的文件名
validation_file_name = "tmp_recipe_finetune_validation.jsonl"

# 调用write_jsonl函数，将验证数据写入到validation_file_name指定的文件中
write_jsonl(validation_data, validation_file_name)

这是我们训练.jsonl文件的前5行的样子：

# 打印训练文件的前5行
!head -n 5 tmp_recipe_finetune_training.jsonl

{"messages": [{"role": "system", "content": "You are a helpful recipe assistant. You are to extract the generic ingredients from each of the recipes provided."}, {"role": "user", "content": "Title: No-Bake Nut Cookies\n\nIngredients: [\"1 c. firmly packed brown sugar\", \"1/2 c. evaporated milk\", \"1/2 tsp. vanilla\", \"1/2 c. broken nuts (pecans)\", \"2 Tbsp. butter or margarine\", \"3 1/2 c. bite size shredded rice biscuits\"]\n\nGeneric ingredients: "}, {"role": "assistant", "content": "[\"brown sugar\", \"milk\", \"vanilla\", \"nuts\", \"butter\", \"bite size shredded rice biscuits\"]"}]}
{"messages": [{"role": "system", "content": "You are a helpful recipe assistant. You are to extract the generic ingredients from each of the recipes provided."}, {"role": "user", "content": "Title: Jewell Ball'S Chicken\n\nIngredients: [\"1 small jar chipped beef, cut up\", \"4 boned chicken breasts\", \"1 can cream of mushroom soup\", \"1 carton sour cream\"]\n\nGeneric ingredients: "}, {"role": "assistant", "content": "[\"beef\", \"chicken breasts\", \"cream of mushroom soup\", \"sour cream\"]"}]}
{"messages": [{"role": "system", "content": "You are a helpful recipe assistant. You are to extract the generic ingredients from each of the recipes provided."}, {"role": "user", "content": "Title: Creamy Corn\n\nIngredients: [\"2 (16 oz.) pkg. frozen corn\", \"1 (8 oz.) pkg. cream cheese, cubed\", \"1/3 c. butter, cubed\", \"1/2 tsp. garlic powder\", \"1/2 tsp. salt\", \"1/4 tsp. pepper\"]\n\nGeneric ingredients: "}, {"role": "assistant", "content": "[\"frozen corn\", \"cream cheese\", \"butter\", \"garlic powder\", \"salt\", \"pepper\"]"}]}
{"messages": [{"role": "system", "content": "You are a helpful recipe assistant. You are to extract the generic ingredients from each of the recipes provided."}, {"role": "user", "content": "Title: Chicken Funny\n\nIngredients: [\"1 large whole chicken\", \"2 (10 1/2 oz.) cans chicken gravy\", \"1 (10 1/2 oz.) can cream of mushroom soup\", \"1 (6 oz.) box Stove Top stuffing\", \"4 oz. shredded cheese\"]\n\nGeneric ingredients: "}, {"role": "assistant", "content": "[\"chicken\", \"chicken gravy\", \"cream of mushroom soup\", \"shredded cheese\"]"}]}
{"messages": [{"role": "system", "content": "You are a helpful recipe assistant. You are to extract the generic ingredients from each of the recipes provided."}, {"role": "user", "content": "Title: Reeses Cups(Candy)  \n\nIngredients: [\"1 c. peanut butter\", \"3/4 c. graham cracker crumbs\", \"1 c. melted butter\", \"1 lb. (3 1/2 c.) powdered sugar\", \"1 large pkg. chocolate chips\"]\n\nGeneric ingredients: "}, {"role": "assistant", "content": "[\"peanut butter\", \"graham cracker crumbs\", \"butter\", \"powdered sugar\", \"chocolate chips\"]"}]}

上传文件

您现在可以将文件上传到我们的“文件”端点，以供精调模型使用。

# 打开训练文件
with open(training_file_name, "rb") as training_fd:
    # 创建一个文件对象，将训练文件上传到OpenAI平台，目的是进行微调
    training_response = openai.files.create(
        file=training_fd, purpose="fine-tune"
    )
# 获取训练文件的ID
training_file_id = training_response.id

# 打开验证文件
with open(validation_file_name, "rb") as validation_fd:
    # 创建一个文件对象，将验证文件上传到OpenAI平台，目的是进行微调
    validation_response = openai.files.create(
        file=validation_fd, purpose="fine-tune"
    )
# 获取验证文件的ID
validation_file_id = validation_response.id

# 输出训练文件ID和验证文件ID
print("Training file ID:", training_file_id)
print("Validation file ID:", validation_file_id)

Training file ID: file-PVkEstNM2WWd1OQe3Hp3tC5E
Validation file ID: file-WSdTwLYrKxNhKi1WWGjxXi87

微调

现在我们可以使用生成的文件和可选的后缀来创建我们的微调任务。响应将包含一个id，您可以使用该id来获取有关任务的更新。

注意：文件必须首先由我们的系统处理，所以您可能会收到文件未准备好的错误。在这种情况下，请稍后几分钟重试。

# 创建Fine-tuning任务

# 使用openai.fine_tuning.jobs.create()函数创建Fine-tuning任务，并将返回的结果赋值给response变量
# training_file参数指定训练文件的ID，validation_file参数指定验证文件的ID
# model参数指定使用的模型为"gpt-3.5-turbo"
# suffix参数指定任务的后缀为"recipe-ner"
response = openai.fine_tuning.jobs.create(
    training_file=training_file_id,
    validation_file=validation_file_id,
    model="gpt-3.5-turbo",
    suffix="recipe-ner",
)

# 获取任务ID

# 从response中获取任务的ID，并将其赋值给job_id变量
job_id = response.id

# 打印任务ID和状态

# 打印任务的ID，使用response.id获取任务ID
print("Job ID:", response.id)

# 打印任务的状态，使用response.status获取任务状态
print("Status:", response.status)

Job ID: ftjob-bIVrnhnZEEizSP7rqWsRwv2R
Status: validating_files

##=检查作业状态

您可以向 https://api.openai.com/v1/alpha/fine-tunes 端点发出 GET 请求，以列出您的 alpha fine-tune 作业。在这种情况下，您需要检查您从上一步获得的 ID 是否最终为 status: succeeded。

一旦完成，您可以使用 result_files 从验证集中抽样结果（如果您上传了验证集），并使用 fine_tuned_model 参数中的 ID 调用您的训练模型。

# 从openai的fine_tuning.jobs中检索出指定job_id的任务
response = openai.fine_tuning.jobs.retrieve(job_id)

# 打印出任务的ID
print("Job ID:", response.id)

# 打印出任务的状态
print("Status:", response.status)

# 打印出任务已经训练的token数量
print("Trained Tokens:", response.trained_tokens)

Job ID: ftjob-bIVrnhnZEEizSP7rqWsRwv2R
Status: running
Trained Tokens: None

我们可以使用事件端点跟踪微调的进度。您可以多次重新运行下面的单元格，直到微调准备就绪。

# 打开AI的微调任务，并获取任务的事件列表
response = openai.fine_tuning.jobs.list_events(job_id)

# 获取事件列表中的数据
events = response.data

# 将事件列表倒序排列
events.reverse()

# 遍历事件列表中的每个事件
for event in events:
    # 打印事件的消息内容
    print(event.message)

Step 131/303: training loss=0.25, validation loss=0.37
Step 141/303: training loss=0.00, validation loss=0.19
Step 151/303: training loss=0.00, validation loss=0.11
Step 161/303: training loss=0.00, validation loss=0.06
Step 171/303: training loss=0.10, validation loss=0.00
Step 181/303: training loss=0.00, validation loss=0.38
Step 191/303: training loss=0.00, validation loss=0.15
Step 201/303: training loss=0.06, validation loss=0.64
Step 211/303: training loss=0.00, validation loss=0.04
Step 221/303: training loss=0.59, validation loss=0.85
Step 231/303: training loss=0.00, validation loss=0.00
Step 241/303: training loss=0.04, validation loss=0.42
Step 251/303: training loss=0.00, validation loss=0.14
Step 261/303: training loss=0.00, validation loss=0.00
Step 271/303: training loss=0.15, validation loss=0.50
Step 281/303: training loss=0.00, validation loss=0.72
Step 291/303: training loss=0.08, validation loss=0.16
Step 301/303: training loss=0.00, validation loss=1.76
New fine-tuned model created: ft:gpt-3.5-turbo-0613:personal:recipe-ner:8PjmcwDH
The job has successfully completed

现在已经完成了，我们可以从作业中获得一个经过优化的模型ID。

# 从OpenAI的fine_tuning.jobs中导入retrieve函数
# 通过job_id检索训练任务的响应
response = openai.fine_tuning.jobs.retrieve(job_id)

# 从响应中获取fine_tuned_model_id
fine_tuned_model_id = response.fine_tuned_model

# 如果fine_tuned_model_id为空，抛出运行时错误
# 这意味着训练任务尚未完成
if fine_tuned_model_id is None: 
    raise RuntimeError("未找到经过微调的模型ID。您的任务可能尚未完成。")

# 打印fine_tuned_model_id
print("经过微调的模型ID:", fine_tuned_model_id)

Fine-tuned model ID: ft:gpt-3.5-turbo-0613:personal:recipe-ner:8PjmcwDH

推理

最后一步是使用您的微调模型进行推理。与经典的“微调”类似，您只需使用您的新微调模型名称填充“model”参数，调用“ChatCompletions”即可。

# 从recipe_df中选取第201行到第300行的数据，赋值给test_df
test_df = recipe_df.loc[201:300]

# 从test_df中选取第一行数据，赋值给test_row
test_row = test_df.iloc[0]

# 创建一个空列表test_messages
test_messages = []

# 在test_messages列表中添加一个字典，字典中包含两个键值对，分别为"role"和"content"，对应的值为"system"和system_message
test_messages.append({"role": "system", "content": system_message})

# 调用create_user_message函数，将test_row作为参数传入，返回一个用户消息，将该消息添加到test_messages列表中
test_messages.append({"role": "user", "content": create_user_message(test_row)})

# 使用pprint函数美化输出test_messages列表
pprint(test_messages)

[{'content': 'You are a helpful recipe assistant. You are to extract the '
             'generic ingredients from each of the recipes provided.',
  'role': 'system'},
 {'content': 'Title: Beef Brisket\n'
             '\n'
             'Ingredients: ["4 lb. beef brisket", "1 c. catsup", "1 c. water", '
             '"1/2 onion, minced", "2 Tbsp. cider vinegar", "1 Tbsp. prepared '
             'horseradish", "1 Tbsp. prepared mustard", "1 tsp. salt", "1/2 '
             'tsp. pepper"]\n'
             '\n'
             'Generic ingredients: ',
  'role': 'user'}]

# 导入openai库后，使用openai.chat.completions.create()函数创建一个response对象
response = openai.chat.completions.create(
    model=fine_tuned_model_id, # 指定模型ID
    messages=test_messages, # 指定测试消息
    temperature=0, # 温度参数，控制生成文本的多样性
    max_tokens=500 # 生成文本的最大长度
)

# 打印response对象中第一个choices元素的message属性的content属性
print(response.choices[0].message.content)

["beef brisket", "catsup", "water", "onion", "cider vinegar", "horseradish", "mustard", "salt", "pepper"]

结论

恭喜您，现在可以使用ChatCompletion格式来微调自己的模型了！我们期待看到您构建的内容。