基于Llama3的RAG
https://github.com/langchain-ai/langgraph/blob/main/examples/rag/langgraph_rag_agent_llama3_local.ipynb
- 展示了如何从头开始使用LangGraph和Llama3-8b构建可靠的本地代理。
- 将3种先进RAG技术（自适应RAG、纠正性RAG和自我RAG）的思想结合起来，形成LangGraph中的单一控制流程。
- 路由：自适应RAG。将问题路由到不同的检索方法。
- 后备方案：纠正性RAG。如果文档与查询不相关，则回退到网页搜索。
- 自我纠错：自我RAG。修正包含幻觉或未回答问题的答案。
Llama3微调
- LLaMA-Factory微调，已经支持LLaMA-3
https://github.com/hiyouga/LLaMA-Factory
- Colab笔记本中将Llama-3微调速度提高2倍
https://colab.research.google.com/drive/135ced7oHytdxu3N2DNe1Z0kqjyYIkDXp?usp=sharing
基于Llama3的function calling/Agent
目前官方的说法是Llama3不支持function calling
- 一种方法采用Function Calling数据微调后的Llama 3
https://huggingface.co/Trelis/Meta-Llama-3-8B-Instruct-function-calling
prompt模板：
<|begin_of_text|><|start_header_id|>function_metadata<|end_header_id|>
You have access to the following functions. Use them if required:
[
{
"type": "function",
"function": {
"name": "get_stock_price",
"description": "Get the stock price of an array of stocks",
"parameters": {
"type": "object",
"properties": {
"names": {
"type": "array",
"items": {
"type": "string"
},
"description": "An array of stocks"
}
},
"required": [
"names"
]
}
}
},
{
"type": "function",
"function": {
"name": "get_big_stocks",
"description": "Get the names of the largest N stocks by market cap",
"parameters": {
"type": "object",
"properties": {
"number": {
"type": "integer",
"description": "The number of largest stocks to get the names of, e.g. 25"
},
"region": {
"type": "string",
"description": "The region to consider, can be \"US\" or \"World\"."
}
},
"required": [
"number"
]
}
}
}
]<|eot_id|><|start_header_id|>user<|end_header_id|>
Get the names of the five largest stocks by market cap<|eot_id|><|start_header_id|>assistant<|end_header_id|>
Generated Response:
{
"name": "get_big_stocks",
"arguments": {
"number": 5,
"region": "US"
}
}<|eot_id|>
- 另外一种就是第三方项目中有一个调用示例，可以尝试下：
项目地址
https://github.com/themrzmaster/llama-cpp-python/blob/4a4b67399b9e5ee872e2b87068de75e2908d5b11/llama_cpp/llama_chat_format.py#L2866
调用示例
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="s")
response = client.chat.completions.create(
model="m",
messages = [
{
"role": "system",
"content": "You are a digital waiter Pay attention to the user requests and use the tools to help you."
},
{
"role": "user",
"content": "search for burguer and a maximum price of 10 dollars. at the sime time, look for merchant named 'Burger King'"
},
],
tools=[
{
"type": "function",
"function": {
"name": "search_merchant",
"description": "Search for merchants in the catalog based on the term",
"parameters": {
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "name to be searched for finding merchants.",
}
},
"required": ["name"],
},
},
},
{
"type": "function",
"function": {
"name": "search_item",
"description": "Search for items in the catalog based on various criteria.",
"parameters": {
"type": "object",
"properties": {
"term": {
"type": "string",
"description": "Term to be searched for finding items, with removed accents.",
},
"item_price_to": {
"type": "integer",
"description": "Maximum price the user is willing to pay for an item, if specified.",
},
"merchant_delivery_fee_to": {
"type": "integer",
"description": "Maximum delivery fee the user is willing to pay, if specified.",
},
"merchant_payment_types": {
"type": "string",
"description": "Type of payment the user prefers, if specified.",
"enum": [
"Credit Card",
"Debit Card",
"Other",
],
},
},
"required": ["term"],
},
},
}],
tool_choice="auto"
)
Llama3实操技术选型推荐
https://twitter.com/9hills/status/1781757051838050630
中文RAG选择 CommandR+;
Agent/FunctionCalling 使用 Llama3-70B 或 CommandR+;
中文文案写作用Qwen-72B;
特定任务的小参数微调base模型用 Llama3-8B 或 Mistral-7B;
大参数微调base 模型用Yi-34B;
代码生成用 Llama3-70B或deepseek-coder-33B;
特定类编程（比如nl2sql），小参数llama3-8b，大参数deepzeek-coder-33b;
生产环境：vLLM、TensorRT-LLM;
本地环境：Ollama.
推荐阅读
- • 对齐LLM偏好的直接偏好优化方法：DPO、IPO、KTO
- • 2024：ToB、Agent、多模态
- • TA们的RAG真正投产了吗？（上）
- • Agent到多模态Agent再到多模态Multi-Agents系统的发展与案例讲解（1.2万字，20+文献，27张图）
欢迎关注我的公众号“ PaperAgent ”， 每天一篇大模型（LLM）文章来锻炼我们的思维，简单的例子，不简单的方法，提升自己。