216 lines
4.9 KiB
Markdown
216 lines
4.9 KiB
Markdown
# LangSmith 集成实现计划
|
||
|
||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||
|
||
**Goal:** 为 Jarvis 后端集成 LangSmith 追踪，实现调用追踪、对话历史管理和评估支持。
|
||
|
||
**Architecture:** LangGraph 的 `compile()` 方法接受全局 Callbacks 参数，会自动将 Callback 传播到所有节点的 LLM 调用。只需在 Graph 编译时注入 `LangChainTracer`，即可覆盖 Master/Planner/Executor/Librarian/Analyst 所有 5 个节点和工具调用。
|
||
|
||
**Tech Stack:** langsmith, langchain-core, langgraph
|
||
|
||
---
|
||
|
||
## 文件变更总览
|
||
|
||
| 文件 | 职责 |
|
||
|---|---|
|
||
| `backend/pyproject.toml` | 添加 langsmith 依赖 |
|
||
| `backend/.env.example` | 新增 LangSmith 环境变量 |
|
||
| `backend/app/config.py` | 新增 3 个配置字段 |
|
||
| `backend/app/config_tracing.py` | 新建，callback 工厂函数 |
|
||
| `backend/app/agents/graph.py` | 修改 `create_agent_graph()` 支持 callbacks，合并全局 callbacks |
|
||
|
||
---
|
||
|
||
### Task 1: 添加 langsmith 依赖
|
||
|
||
**Files:**
|
||
- Modify: `backend/pyproject.toml`
|
||
|
||
- [ ] **Step 1: 添加 langsmith 依赖**
|
||
|
||
在 `dependencies` 数组中 `"langchain-ollama>=0.4.0",` 后添加：
|
||
|
||
```toml
|
||
"langchain-ollama>=0.4.0",
|
||
|
||
# 可观测性
|
||
"langsmith>=0.1.0",
|
||
```
|
||
|
||
- [ ] **Step 2: 安装依赖**
|
||
|
||
Run: `cd backend && uv sync`
|
||
|
||
---
|
||
|
||
### Task 2: 添加环境变量模板
|
||
|
||
**Files:**
|
||
- Modify: `backend/.env.example`
|
||
|
||
- [ ] **Step 1: 在文件末尾添加 LangSmith 配置节**
|
||
|
||
在 `# === 定时任务 ===` 节之前添加：
|
||
|
||
```env
|
||
# === LangSmith 可观测性 ===
|
||
# 启用 LangSmith 追踪（可选）
|
||
LANGSMITH_TRACING=false
|
||
LANGSMITH_API_KEY=your-langsmith-api-key
|
||
LANGSMITH_PROJECT=jarvis-agent
|
||
```
|
||
|
||
---
|
||
|
||
### Task 3: Config 层添加 LangSmith 配置
|
||
|
||
**Files:**
|
||
- Modify: `backend/app/config.py`
|
||
- Create: `backend/app/config_tracing.py` (callback 工厂函数)
|
||
|
||
- [ ] **Step 1: 在 Settings 类中添加 3 个配置字段**
|
||
|
||
在 `# === NAS 部署 ===` 节之前添加：
|
||
|
||
```python
|
||
# === LangSmith 可观测性 ===
|
||
LANGSMITH_TRACING: bool = False
|
||
LANGSMITH_API_KEY: str = ""
|
||
LANGSMITH_PROJECT: str = "jarvis-agent"
|
||
```
|
||
|
||
- [ ] **Step 2: 创建 callback 工厂函数**
|
||
|
||
创建新文件 `backend/app/config_tracing.py`：
|
||
|
||
```python
|
||
"""
|
||
LangSmith Tracing 配置
|
||
提供 Callback 工厂函数，用于 LangGraph 追踪
|
||
"""
|
||
|
||
from langchain_core.callbacks import LangChainTracer
|
||
|
||
from app.config import settings
|
||
|
||
|
||
def get_langsmith_callbacks() -> list:
|
||
"""
|
||
根据配置返回 LangSmith Callback 列表
|
||
未启用时返回空列表
|
||
"""
|
||
if not settings.LANGSMITH_TRACING:
|
||
return []
|
||
|
||
if not settings.LANGSMITH_API_KEY:
|
||
return []
|
||
|
||
return [
|
||
LangChainTracer(
|
||
project_name=settings.LANGSMITH_PROJECT,
|
||
)
|
||
]
|
||
```
|
||
|
||
---
|
||
|
||
### Task 4: 修改 Graph 接受 Callbacks
|
||
|
||
**Files:**
|
||
- Modify: `backend/app/agents/graph.py`
|
||
|
||
- [ ] **Step 1: 修改 create_agent_graph() 签名**
|
||
|
||
将函数签名从：
|
||
|
||
```python
|
||
def create_agent_graph():
|
||
```
|
||
|
||
改为：
|
||
|
||
```python
|
||
def create_agent_graph(callbacks: list | None = None):
|
||
```
|
||
|
||
- [ ] **Step 2: 将 callbacks 传给 compile()**
|
||
|
||
将：
|
||
|
||
```python
|
||
return graph.compile()
|
||
```
|
||
|
||
改为：
|
||
|
||
```python
|
||
return graph.compile(callbacks=callbacks)
|
||
```
|
||
|
||
- [ ] **Step 3: 修改 get_agent_graph() 注入默认 callbacks**
|
||
|
||
将：
|
||
|
||
```python
|
||
def get_agent_graph():
|
||
global _agent_graph
|
||
if _agent_graph is None:
|
||
_agent_graph = create_agent_graph()
|
||
return _agent_graph
|
||
```
|
||
|
||
改为：
|
||
|
||
```python
|
||
from app.config_tracing import get_langsmith_callbacks
|
||
|
||
|
||
def get_agent_graph(callbacks: list | None = None):
|
||
"""
|
||
获取编译好的 Agent 图（单例缓存）。
|
||
|
||
Callbacks 在首次编译时固定注入，后续调用忽略 callbacks 参数。
|
||
如需变更 Callbacks（如修改 LANGCHAIN_PROJECT），需重启服务。
|
||
|
||
Args:
|
||
callbacks: 可选的额外 Callbacks，会与全局 LangSmith Callbacks 合并
|
||
"""
|
||
global _agent_graph
|
||
if _agent_graph is None:
|
||
langsmith_callbacks = get_langsmith_callbacks()
|
||
all_callbacks = (callbacks or []) + langsmith_callbacks
|
||
_agent_graph = create_agent_graph(callbacks=all_callbacks or None)
|
||
return _agent_graph
|
||
```
|
||
|
||
---
|
||
|
||
### Task 5: 验证集成
|
||
|
||
- [ ] **Step 1: 确认依赖安装**
|
||
|
||
Run: `cd backend && uv sync`
|
||
|
||
- [ ] **Step 2: 启动服务验证无报错**
|
||
|
||
Run: `cd backend && uv run uvicorn app.main:app --reload --port 8000`
|
||
|
||
- [ ] **Step 3: 配置 .env 并测试**
|
||
|
||
在 `.env` 中添加：
|
||
|
||
```env
|
||
LANGSMITH_TRACING=true
|
||
LANGSMITH_API_KEY=your-api-key
|
||
LANGSMITH_PROJECT=jarvis-agent
|
||
```
|
||
|
||
发起一次 Agent 对话，访问 https://smith.langchain.com 确认 trace 出现。
|
||
|
||
预期在 Dashboard 中看到：
|
||
- 5 个节点（master/planner/executor/librarian/analyst）的执行记录
|
||
- 每个节点的 LLM 输入/输出
|
||
- 工具调用记录
|
||
- Token 消耗统计
|