Compare commits
4 Commits
baseline/f
...
c64fa1cd61
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
c64fa1cd61 | ||
|
|
b5c6557341 | ||
|
|
7f93ed6d09 | ||
|
|
5ec950cc9f |
65
README.md
65
README.md
@@ -103,15 +103,37 @@ GET /modelTF/model-manage
|
|||||||
GET /modelTF/dataset-manage
|
GET /modelTF/dataset-manage
|
||||||
GET /modelTF/fine-tune
|
GET /modelTF/fine-tune
|
||||||
GET /modelTF/compute/nodes
|
GET /modelTF/compute/nodes
|
||||||
|
GET /modelTF/data-convert
|
||||||
```
|
```
|
||||||
|
|
||||||
本地运行时默认 PostgreSQL 连接:
|
### 数据库配置
|
||||||
|
|
||||||
```text
|
后端通过 `backend/.env` 文件配置数据库连接(自动加载,`override=True`):
|
||||||
DATABASE_URL=postgresql+psycopg://yg_ft:change_me@localhost:15432/yg_ft
|
|
||||||
|
```env
|
||||||
|
DATABASE_URL=postgresql+psycopg://用户:密码@数据库地址:端口/库名
|
||||||
|
COMPUTE_SERVICE_TOKEN=change_me
|
||||||
```
|
```
|
||||||
|
|
||||||
本地启动前需要确保 PostgreSQL 已监听 `localhost:15432`,并已创建 `yg_ft` 数据库和 `yg_ft` 用户。后端启动后会自动创建当前运行表并写入内置管理员账号,运行数据统一写入 PostgreSQL。
|
支持远程数据库。连接池参数已针对远程库优化(`connect_timeout=30`、`max_size=20`、`max_waiting=50`)。
|
||||||
|
|
||||||
|
### 环境变量
|
||||||
|
|
||||||
|
| 变量 | 默认值 | 说明 |
|
||||||
|
|---|---|---|
|
||||||
|
| `DATABASE_URL` | `postgresql+psycopg://yg_ft:change_me@localhost:15432/yg_ft` | 数据库连接串 |
|
||||||
|
| `COMPUTE_SERVICE_TOKEN` | `""` | 算力服务认证 token,需与 compute 一致 |
|
||||||
|
| `COMPUTE_STATUS_SYNC_MODE` | `polling` | `off` 禁用轮询(远程库慢时推荐) |
|
||||||
|
| `COMPUTE_POLL_INTERVAL_SECONDS` | `3` | 轮询间隔秒数 |
|
||||||
|
| `COMPUTE_REQUEST_TIMEOUT_SECONDS` | `5` | 调 compute 的超时秒数 |
|
||||||
|
|
||||||
|
### 推荐启动命令(远程数据库)
|
||||||
|
|
||||||
|
```cmd
|
||||||
|
cd /d E:\yg_ft\backend
|
||||||
|
set COMPUTE_STATUS_SYNC_MODE=off
|
||||||
|
.\.venv\Scripts\python.exe -m uvicorn app.main:app --reload --port 17861
|
||||||
|
```
|
||||||
|
|
||||||
开发阶段内置登录账号:
|
开发阶段内置登录账号:
|
||||||
|
|
||||||
@@ -138,12 +160,29 @@ npm run dev
|
|||||||
|
|
||||||
### 方式一:Docker 启动(推荐)
|
### 方式一:Docker 启动(推荐)
|
||||||
|
|
||||||
```bash
|
**1. 构建镜像**(首次或依赖变更后):
|
||||||
|
|
||||||
|
```cmd
|
||||||
|
cd /d E:\yg_ft
|
||||||
|
docker build -f docker/compute/Dockerfile.compute -t yg-ft-compute-api:latest .
|
||||||
|
```
|
||||||
|
|
||||||
|
**2. 启动容器**:
|
||||||
|
|
||||||
|
```cmd
|
||||||
cd docker/compute
|
cd docker/compute
|
||||||
cp .env.example .env
|
cp .env.example .env
|
||||||
docker compose up -d
|
docker compose up -d
|
||||||
```
|
```
|
||||||
|
|
||||||
|
**3. 验证**:
|
||||||
|
|
||||||
|
```cmd
|
||||||
|
curl http://localhost:19100/health
|
||||||
|
```
|
||||||
|
|
||||||
|
> 注意:构建上下文必须是项目根目录 `E:\yg_ft`(`docker build` 最后的 `.`),因为 Dockerfile 需要 `COPY compute/requirements.txt`。
|
||||||
|
|
||||||
### 方式二:本地开发启动
|
### 方式二:本地开发启动
|
||||||
|
|
||||||
**Windows (cmd):**
|
**Windows (cmd):**
|
||||||
@@ -151,25 +190,35 @@ docker compose up -d
|
|||||||
```cmd
|
```cmd
|
||||||
cd /d E:\yg_ft\compute
|
cd /d E:\yg_ft\compute
|
||||||
set PYTHONPATH=E:\yg_ft
|
set PYTHONPATH=E:\yg_ft
|
||||||
.\.venv\Scripts\python.exe -m uvicorn api.main:app --reload --port 19100
|
.\.venv\Scripts\python.exe -m uvicorn api.main:app --reload --host 0.0.0.0 --port 19100
|
||||||
```
|
```
|
||||||
|
|
||||||
> `PYTHONPATH=E:\yg_ft` 是必需的,因为代码使用 `from compute.agent...` 绝对导入。
|
> `PYTHONPATH=E:\yg_ft` 是必需的,因为代码使用 `from compute.agent...` 绝对导入。
|
||||||
|
> `--host 0.0.0.0` 让其他机器可以通过 IP 访问(算力节点测试需要)。
|
||||||
|
|
||||||
**Linux / macOS:**
|
**Linux / macOS:**
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cd compute
|
cd compute
|
||||||
PYTHONPATH=.. uvicorn api.main:app --reload --port 19100
|
PYTHONPATH=.. uvicorn api.main:app --reload --host 0.0.0.0 --port 19100
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### 算力节点配置
|
||||||
|
|
||||||
|
在平台的「算力节点」页面新增节点,填入:
|
||||||
|
- **Compute API**:`http://你的IP:19100`
|
||||||
|
- **File Gateway**:`http://你的IP:19101`
|
||||||
|
|
||||||
|
本机测试用 `http://localhost:19100`。
|
||||||
|
|
||||||
### 环境变量说明
|
### 环境变量说明
|
||||||
|
|
||||||
| 变量 | 默认值 | 说明 |
|
| 变量 | 默认值 | 说明 |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| `COMPUTE_MODE` | `real` | `real` / `simulator`,仅隔离联调用 simulator |
|
| `COMPUTE_MODE` | `real` | `real` / `simulator`,仅隔离联调用 simulator |
|
||||||
| `COMPUTE_EXECUTION_MODE` | `real` | 训练执行模式 |
|
| `COMPUTE_EXECUTION_MODE` | `real` | 训练执行模式 |
|
||||||
| `COMPUTE_SERVICE_TOKEN` | `change_me` | 服务间认证 token |
|
| `COMPUTE_SERVICE_TOKEN` | `change_me` | 服务间认证 token,需与 backend 一致 |
|
||||||
|
| `COMPUTE_AUTH_ENABLED` | `true` | 是否开启 token 认证 |
|
||||||
| `MODELTF_ROUTE_PREFIX` | `/modelTF` | API 路由前缀 |
|
| `MODELTF_ROUTE_PREFIX` | `/modelTF` | API 路由前缀 |
|
||||||
|
|
||||||
应用平台通过数据库 `compute_nodes` 表中的 `api_base_url` 和 `file_gateway_url` 主动轮询算力节点状态。
|
应用平台通过数据库 `compute_nodes` 表中的 `api_base_url` 和 `file_gateway_url` 主动轮询算力节点状态。
|
||||||
|
|||||||
@@ -9,6 +9,7 @@ from app.modules.approval.router import router as approval_router
|
|||||||
from app.modules.system.router import router as system_router
|
from app.modules.system.router import router as system_router
|
||||||
from app.modules.retention.router import router as retention_router
|
from app.modules.retention.router import router as retention_router
|
||||||
from app.modules.resource.router import router as resource_router
|
from app.modules.resource.router import router as resource_router
|
||||||
|
from app.modules.data_convert.router import router as data_convert_router
|
||||||
|
|
||||||
api_router = APIRouter()
|
api_router = APIRouter()
|
||||||
api_router.include_router(health_router, tags=["health"])
|
api_router.include_router(health_router, tags=["health"])
|
||||||
@@ -20,3 +21,4 @@ api_router.include_router(project_router, tags=["project"])
|
|||||||
api_router.include_router(approval_router, tags=["approval"])
|
api_router.include_router(approval_router, tags=["approval"])
|
||||||
api_router.include_router(retention_router, tags=["retention"])
|
api_router.include_router(retention_router, tags=["retention"])
|
||||||
api_router.include_router(resource_router, tags=["resource"])
|
api_router.include_router(resource_router, tags=["resource"])
|
||||||
|
api_router.include_router(data_convert_router, tags=["data-convert"])
|
||||||
|
|||||||
@@ -378,7 +378,7 @@ class PlatformStore:
|
|||||||
# request (notably expensive against the remote PostgreSQL instance).
|
# request (notably expensive against the remote PostgreSQL instance).
|
||||||
# TCP keepalive 让操作系统持续保活连接,抵抗远程库空闲静默断连。
|
# TCP keepalive 让操作系统持续保活连接,抵抗远程库空闲静默断连。
|
||||||
pool_kwargs = {
|
pool_kwargs = {
|
||||||
"connect_timeout": 5,
|
"connect_timeout": 30,
|
||||||
"keepalives": 1,
|
"keepalives": 1,
|
||||||
"keepalives_idle": 10,
|
"keepalives_idle": 10,
|
||||||
"keepalives_interval": 5,
|
"keepalives_interval": 5,
|
||||||
@@ -388,14 +388,14 @@ class PlatformStore:
|
|||||||
conninfo=self.database_url,
|
conninfo=self.database_url,
|
||||||
kwargs=pool_kwargs,
|
kwargs=pool_kwargs,
|
||||||
min_size=2,
|
min_size=2,
|
||||||
max_size=10,
|
max_size=20,
|
||||||
# 借出前校验连接可用性,避免执行 SQL 时才发现 [BAD] 再重建。
|
# 借出前校验连接可用性,避免执行 SQL 时才发现 [BAD] 再重建。
|
||||||
check=ConnectionPool.check_connection,
|
check=ConnectionPool.check_connection,
|
||||||
# 不主动回收空闲连接(远程库约 10s 断,由 keepalive 维持),
|
# 不主动回收空闲连接(远程库约 10s 断,由 keepalive 维持),
|
||||||
# 减少无谓的重建握手。
|
# 减少无谓的重建握手。
|
||||||
max_idle=0,
|
max_idle=0,
|
||||||
# 请求最多排队等待 5s,避免雪崩时无限堆积。
|
# 请求最多排队等待,调大以适应远程库慢查询。
|
||||||
max_waiting=16,
|
max_waiting=50,
|
||||||
open=False,
|
open=False,
|
||||||
)
|
)
|
||||||
# 注意:不要在此调用 pool.wait(),它会阻塞等待 min_size 个连接就绪,
|
# 注意:不要在此调用 pool.wait(),它会阻塞等待 min_size 个连接就绪,
|
||||||
|
|||||||
3
backend/app/modules/data_convert/__init__.py
Normal file
3
backend/app/modules/data_convert/__init__.py
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
from .router import router
|
||||||
|
|
||||||
|
__all__ = ["router"]
|
||||||
310
backend/app/modules/data_convert/router.py
Normal file
310
backend/app/modules/data_convert/router.py
Normal file
@@ -0,0 +1,310 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Body, UploadFile, File
|
||||||
|
from fastapi.responses import FileResponse
|
||||||
|
|
||||||
|
from app.api.v1.endpoints.platform import ok, fail
|
||||||
|
from app.db.platform_store import get_platform_store, new_id
|
||||||
|
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/data-convert", tags=["data-convert"])
|
||||||
|
|
||||||
|
# 存储根目录
|
||||||
|
STORAGE_ROOT = Path(__file__).resolve().parents[3] / "storage" / "data-convert"
|
||||||
|
|
||||||
|
|
||||||
|
def _task_dir(task_id: str) -> Path:
|
||||||
|
return STORAGE_ROOT / task_id
|
||||||
|
|
||||||
|
|
||||||
|
def _input_dir(task_id: str) -> Path:
|
||||||
|
return _task_dir(task_id) / "input"
|
||||||
|
|
||||||
|
|
||||||
|
def _output_dir(task_id: str) -> Path:
|
||||||
|
return _task_dir(task_id) / "output"
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("")
|
||||||
|
def list_tasks(page: int = 1, page_size: int = 20) -> dict[str, Any]:
|
||||||
|
store = get_platform_store()
|
||||||
|
with store.connect() as conn:
|
||||||
|
rows = conn.execute(
|
||||||
|
"SELECT * FROM data_convert_tasks WHERE deleted_at IS NULL "
|
||||||
|
"ORDER BY create_time DESC LIMIT %s OFFSET %s",
|
||||||
|
(page_size, (page - 1) * page_size),
|
||||||
|
).fetchall()
|
||||||
|
total = conn.execute(
|
||||||
|
"SELECT COUNT(*) FROM data_convert_tasks WHERE deleted_at IS NULL"
|
||||||
|
).fetchone()[0]
|
||||||
|
return ok({"items": [dict(r) for r in rows], "total": total})
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("")
|
||||||
|
def create_task(payload: dict[str, Any] = Body(...)) -> dict[str, Any]:
|
||||||
|
name = str(payload.get("name") or "").strip()
|
||||||
|
if not name:
|
||||||
|
raise fail(400, "name is required")
|
||||||
|
task_id = new_id("dct")
|
||||||
|
output_filename = str(payload.get("output_filename") or "converted-data.jsonl").strip()
|
||||||
|
description = str(payload.get("description") or "").strip()
|
||||||
|
store = get_platform_store()
|
||||||
|
with store.connect() as conn:
|
||||||
|
conn.execute(
|
||||||
|
"INSERT INTO data_convert_tasks (id, name, description, output_filename) "
|
||||||
|
"VALUES (%s, %s, %s, %s)",
|
||||||
|
(task_id, name, description, output_filename),
|
||||||
|
)
|
||||||
|
# 创建目录
|
||||||
|
_input_dir(task_id).mkdir(parents=True, exist_ok=True)
|
||||||
|
_output_dir(task_id).mkdir(parents=True, exist_ok=True)
|
||||||
|
return ok(_get_task(task_id))
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/{task_id}")
|
||||||
|
def get_task(task_id: str) -> dict[str, Any]:
|
||||||
|
task = _get_task(task_id)
|
||||||
|
if not task:
|
||||||
|
raise fail(404, "task not found")
|
||||||
|
# 附加输入文件列表
|
||||||
|
input_dir = _input_dir(task_id)
|
||||||
|
files = []
|
||||||
|
if input_dir.exists():
|
||||||
|
for f in sorted(input_dir.iterdir()):
|
||||||
|
if f.is_file():
|
||||||
|
files.append({"name": f.name, "size": f.stat().st_size})
|
||||||
|
task["input_files"] = files
|
||||||
|
return ok(task)
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/{task_id}/source-files")
|
||||||
|
async def upload_source_files(
|
||||||
|
task_id: str,
|
||||||
|
files: list[UploadFile] = File(...),
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
task = _get_task(task_id)
|
||||||
|
if not task:
|
||||||
|
raise fail(404, "task not found")
|
||||||
|
if task["status"] not in ("pending", "uploaded"):
|
||||||
|
raise fail(400, "task is not editable")
|
||||||
|
input_dir = _input_dir(task_id)
|
||||||
|
input_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
staged = []
|
||||||
|
for upload in files:
|
||||||
|
name = Path(upload.filename or "input.json").name
|
||||||
|
if not name.lower().endswith(".json"):
|
||||||
|
raise fail(415, f"only JSON files are supported: {name}")
|
||||||
|
target = input_dir / name
|
||||||
|
content = await upload.read()
|
||||||
|
target.write_bytes(content)
|
||||||
|
staged.append({"name": name, "size": len(content)})
|
||||||
|
store = get_platform_store()
|
||||||
|
# 标记上传完成
|
||||||
|
with store.connect() as conn:
|
||||||
|
conn.execute(
|
||||||
|
"UPDATE data_convert_tasks SET status='uploaded', update_time=NOW() WHERE id=%s",
|
||||||
|
(task_id,),
|
||||||
|
)
|
||||||
|
# 自动转换并导入数据集
|
||||||
|
try:
|
||||||
|
output_dir = _output_dir(task_id)
|
||||||
|
output_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
output_path = output_dir / (task["output_filename"] or "converted-data.jsonl")
|
||||||
|
# 清空旧输出(如果重新上传)
|
||||||
|
if output_path.exists():
|
||||||
|
output_path.unlink()
|
||||||
|
input_count = 0
|
||||||
|
output_count = 0
|
||||||
|
for json_file in sorted(input_dir.iterdir()):
|
||||||
|
if not json_file.is_file() or not json_file.name.lower().endswith(".json"):
|
||||||
|
continue
|
||||||
|
input_count += 1
|
||||||
|
with open(json_file, "r", encoding="utf-8") as f:
|
||||||
|
data = json.load(f)
|
||||||
|
if isinstance(data, list):
|
||||||
|
records = data
|
||||||
|
elif isinstance(data, dict):
|
||||||
|
records = [data]
|
||||||
|
else:
|
||||||
|
raise ValueError(f"JSON must be object or array: {json_file.name}")
|
||||||
|
with open(output_path, "a", encoding="utf-8") as f:
|
||||||
|
for record in records:
|
||||||
|
f.write(json.dumps(record, ensure_ascii=False) + "\n")
|
||||||
|
output_count += 1
|
||||||
|
with store.connect() as conn:
|
||||||
|
conn.execute(
|
||||||
|
"UPDATE data_convert_tasks SET status='completed', "
|
||||||
|
"input_count=%s, output_count=%s, update_time=NOW() WHERE id=%s",
|
||||||
|
(input_count, output_count, task_id),
|
||||||
|
)
|
||||||
|
# 自动导入数据集
|
||||||
|
content = output_path.read_text(encoding="utf-8")
|
||||||
|
size_bytes = len(content.encode("utf-8"))
|
||||||
|
dataset = store.create_dataset({
|
||||||
|
"name": task["name"],
|
||||||
|
"type": "train",
|
||||||
|
"storage_type": "local",
|
||||||
|
"source": "upload",
|
||||||
|
"task_id": task_id,
|
||||||
|
"size": f"{size_bytes} B",
|
||||||
|
"count": output_count,
|
||||||
|
"description": f"由数据类型转换任务 {task_id} 自动导入",
|
||||||
|
})
|
||||||
|
dataset_id = dataset["id"]
|
||||||
|
with store.connect() as conn:
|
||||||
|
store.add_dataset_file(conn, dataset_id, task["output_filename"] or "converted-data.jsonl", content)
|
||||||
|
return ok({
|
||||||
|
"staged_files": staged,
|
||||||
|
"auto_converted": True,
|
||||||
|
"dataset_id": dataset_id,
|
||||||
|
"input_count": input_count,
|
||||||
|
"output_count": output_count,
|
||||||
|
})
|
||||||
|
except Exception as exc:
|
||||||
|
with store.connect() as conn:
|
||||||
|
conn.execute(
|
||||||
|
"UPDATE data_convert_tasks SET status='failed', error_message=%s, update_time=NOW() WHERE id=%s",
|
||||||
|
(str(exc)[:500], task_id),
|
||||||
|
)
|
||||||
|
return ok({"staged_files": staged, "auto_converted": False, "error": str(exc)[:500]})
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/{task_id}/run")
|
||||||
|
def run_convert(task_id: str) -> dict[str, Any]:
|
||||||
|
task = _get_task(task_id)
|
||||||
|
if not task:
|
||||||
|
raise fail(404, "task not found")
|
||||||
|
if task["status"] not in ("uploaded", "completed", "failed"):
|
||||||
|
raise fail(400, "please upload source files first")
|
||||||
|
# 标记运行中
|
||||||
|
store = get_platform_store()
|
||||||
|
with store.connect() as conn:
|
||||||
|
conn.execute(
|
||||||
|
"UPDATE data_convert_tasks SET status='running', error_message='', update_time=NOW() WHERE id=%s",
|
||||||
|
(task_id,),
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
input_dir = _input_dir(task_id)
|
||||||
|
output_dir = _output_dir(task_id)
|
||||||
|
output_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
output_path = output_dir / (task["output_filename"] or "converted-data.jsonl")
|
||||||
|
input_count = 0
|
||||||
|
output_count = 0
|
||||||
|
for json_file in sorted(input_dir.iterdir()):
|
||||||
|
if not json_file.is_file() or not json_file.name.lower().endswith(".json"):
|
||||||
|
continue
|
||||||
|
input_count += 1
|
||||||
|
with open(json_file, "r", encoding="utf-8") as f:
|
||||||
|
data = json.load(f)
|
||||||
|
if isinstance(data, list):
|
||||||
|
records = data
|
||||||
|
elif isinstance(data, dict):
|
||||||
|
records = [data]
|
||||||
|
else:
|
||||||
|
raise ValueError(f"JSON must be object or array: {json_file.name}")
|
||||||
|
with open(output_path, "a", encoding="utf-8") as f:
|
||||||
|
for record in records:
|
||||||
|
f.write(json.dumps(record, ensure_ascii=False) + "\n")
|
||||||
|
output_count += 1
|
||||||
|
# 更新任务状态
|
||||||
|
with store.connect() as conn:
|
||||||
|
conn.execute(
|
||||||
|
"UPDATE data_convert_tasks SET status='completed', "
|
||||||
|
"input_count=%s, output_count=%s, update_time=NOW() WHERE id=%s",
|
||||||
|
(input_count, output_count, task_id),
|
||||||
|
)
|
||||||
|
except Exception as exc:
|
||||||
|
with store.connect() as conn:
|
||||||
|
conn.execute(
|
||||||
|
"UPDATE data_convert_tasks SET status='failed', error_message=%s, update_time=NOW() WHERE id=%s",
|
||||||
|
(str(exc)[:500], task_id),
|
||||||
|
)
|
||||||
|
raise fail(500, f"convert failed: {exc}")
|
||||||
|
return ok(_get_task(task_id))
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/{task_id}/download")
|
||||||
|
def download_result(task_id: str):
|
||||||
|
task = _get_task(task_id)
|
||||||
|
if not task:
|
||||||
|
raise fail(404, "task not found")
|
||||||
|
if task["status"] != "completed":
|
||||||
|
raise fail(400, "task is not completed")
|
||||||
|
output_path = _output_dir(task_id) / (task["output_filename"] or "converted-data.jsonl")
|
||||||
|
if not output_path.exists():
|
||||||
|
raise fail(404, "output file not found")
|
||||||
|
return FileResponse(
|
||||||
|
str(output_path),
|
||||||
|
media_type="application/octet-stream",
|
||||||
|
filename=task["output_filename"] or "converted-data.jsonl",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/{task_id}/import-as-dataset")
|
||||||
|
def import_as_dataset(
|
||||||
|
task_id: str,
|
||||||
|
payload: dict[str, Any] = Body(default={}),
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""把已转换的 JSONL 文件导入为数据集管理中的上传任务记录(source='task')。"""
|
||||||
|
task = _get_task(task_id)
|
||||||
|
if not task:
|
||||||
|
raise fail(404, "task not found")
|
||||||
|
if task["status"] != "completed":
|
||||||
|
raise fail(400, "task is not completed")
|
||||||
|
output_path = _output_dir(task_id) / (task["output_filename"] or "converted-data.jsonl")
|
||||||
|
if not output_path.exists():
|
||||||
|
raise fail(404, "output file not found")
|
||||||
|
content = output_path.read_text(encoding="utf-8")
|
||||||
|
dataset_name = str(payload.get("name") or task["name"]).strip()
|
||||||
|
description = str(payload.get("description") or f"由数据类型转换任务 {task_id} 导入").strip()
|
||||||
|
size_bytes = len(content.encode("utf-8"))
|
||||||
|
store = get_platform_store()
|
||||||
|
# 用 store 提供的接口创建数据集与文件
|
||||||
|
dataset = store.create_dataset({
|
||||||
|
"name": dataset_name,
|
||||||
|
"type": "train",
|
||||||
|
"storage_type": "local",
|
||||||
|
"source": "upload",
|
||||||
|
"task_id": task_id,
|
||||||
|
"size": f"{size_bytes} B",
|
||||||
|
"count": task["output_count"],
|
||||||
|
"description": description,
|
||||||
|
})
|
||||||
|
dataset_id = dataset["id"]
|
||||||
|
with store.connect() as conn:
|
||||||
|
store.add_dataset_file(conn, dataset_id, task["output_filename"] or "converted-data.jsonl", content)
|
||||||
|
return ok({"dataset_id": dataset_id, "name": dataset_name})
|
||||||
|
|
||||||
|
|
||||||
|
@router.delete("/{task_id}")
|
||||||
|
def delete_task(task_id: str) -> dict[str, Any]:
|
||||||
|
task = _get_task(task_id)
|
||||||
|
if not task:
|
||||||
|
raise fail(404, "task not found")
|
||||||
|
store = get_platform_store()
|
||||||
|
with store.connect() as conn:
|
||||||
|
conn.execute(
|
||||||
|
"UPDATE data_convert_tasks SET deleted_at=NOW() WHERE id=%s",
|
||||||
|
(task_id,),
|
||||||
|
)
|
||||||
|
# 清理文件
|
||||||
|
import shutil
|
||||||
|
task_dir = _task_dir(task_id)
|
||||||
|
if task_dir.exists():
|
||||||
|
shutil.rmtree(task_dir, ignore_errors=True)
|
||||||
|
return ok({"deleted": task_id})
|
||||||
|
|
||||||
|
|
||||||
|
def _get_task(task_id: str) -> dict[str, Any] | None:
|
||||||
|
store = get_platform_store()
|
||||||
|
with store.connect() as conn:
|
||||||
|
row = conn.execute(
|
||||||
|
"SELECT * FROM data_convert_tasks WHERE id=%s AND deleted_at IS NULL",
|
||||||
|
(task_id,),
|
||||||
|
).fetchone()
|
||||||
|
return dict(row) if row else None
|
||||||
@@ -531,6 +531,10 @@ class LocalDataProcessStorage:
|
|||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _fsync_directory(directory: Path) -> None:
|
def _fsync_directory(directory: Path) -> None:
|
||||||
|
# Windows 不支持以 O_RDONLY 打开目录做 fsync,跳过即可。
|
||||||
|
# 数据完整性在 Linux 生产环境保障,Windows 开发环境忽略。
|
||||||
|
if os.name == "nt":
|
||||||
|
return
|
||||||
descriptor = os.open(directory, os.O_RDONLY)
|
descriptor = os.open(directory, os.O_RDONLY)
|
||||||
try:
|
try:
|
||||||
os.fsync(descriptor)
|
os.fsync(descriptor)
|
||||||
|
|||||||
5
frontend/src/api/modules/audit-visit.ts
Normal file
5
frontend/src/api/modules/audit-visit.ts
Normal file
@@ -0,0 +1,5 @@
|
|||||||
|
import { post } from '../request'
|
||||||
|
|
||||||
|
/** 记录用户访问某个业务模块(用于看板用户操作分布统计) */
|
||||||
|
export const recordModuleVisit = (module: string, detail?: string) =>
|
||||||
|
post('/system/audit/visit', { action: module, detail: detail || '' })
|
||||||
43
frontend/src/api/modules/data-convert.ts
Normal file
43
frontend/src/api/modules/data-convert.ts
Normal file
@@ -0,0 +1,43 @@
|
|||||||
|
import { del, get, post } from '../request'
|
||||||
|
|
||||||
|
export interface DataConvertTask {
|
||||||
|
id: string
|
||||||
|
name: string
|
||||||
|
description: string
|
||||||
|
status: string
|
||||||
|
output_filename: string
|
||||||
|
input_count: number
|
||||||
|
output_count: number
|
||||||
|
error_message: string
|
||||||
|
create_time: string
|
||||||
|
update_time?: string
|
||||||
|
input_files?: Array<{ name: string; size: number }>
|
||||||
|
}
|
||||||
|
|
||||||
|
export const getDataConvertTasks = (page = 1, pageSize = 20) =>
|
||||||
|
get<{ items: DataConvertTask[]; total: number }>('/data-convert', { params: { page, page_size: pageSize } })
|
||||||
|
|
||||||
|
export const getDataConvertTask = (id: string) =>
|
||||||
|
get<DataConvertTask>(`/data-convert/${id}`)
|
||||||
|
|
||||||
|
export const createDataConvertTask = (payload: Partial<DataConvertTask>) =>
|
||||||
|
post<DataConvertTask>('/data-convert', payload)
|
||||||
|
|
||||||
|
export const uploadSourceFiles = (id: string, files: File[]) => {
|
||||||
|
const formData = new FormData()
|
||||||
|
files.forEach(f => formData.append('files', f))
|
||||||
|
return post(`/data-convert/${id}/source-files`, formData)
|
||||||
|
}
|
||||||
|
|
||||||
|
export const runConvert = (id: string) =>
|
||||||
|
post<DataConvertTask>(`/data-convert/${id}/run`)
|
||||||
|
|
||||||
|
export const downloadResult = (id: string) =>
|
||||||
|
get(`/data-convert/${id}/download`, { responseType: 'blob' })
|
||||||
|
|
||||||
|
export const deleteDataConvertTask = (id: string) =>
|
||||||
|
del(`/data-convert/${id}`)
|
||||||
|
|
||||||
|
/** 把转换结果导入为数据集管理中的上传任务记录 */
|
||||||
|
export const importAsDataset = (id: string, payload: { name?: string; description?: string } = {}) =>
|
||||||
|
post(`/data-convert/${id}/import-as-dataset`, payload)
|
||||||
@@ -1,6 +1,38 @@
|
|||||||
import axios, { type AxiosInstance, type AxiosRequestConfig } from 'axios'
|
import axios, { type AxiosInstance, type AxiosRequestConfig } from 'axios'
|
||||||
import { ElMessage } from 'element-plus'
|
import { ElMessage } from 'element-plus'
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 用户操作分布:哪些模块路径算"业务操作"(用于看板统计)
|
||||||
|
* 请求命中这些路径时,会自动调用 record_visit 记录一次(同一模块 60 秒内去重)
|
||||||
|
*/
|
||||||
|
const VISIT_TRACKED_PREFIXES: Array<[string, string]> = [
|
||||||
|
['/fine-tune', 'fine-tune'],
|
||||||
|
['/model-eval', 'model-eval'],
|
||||||
|
['/model-inference', 'model-inference'],
|
||||||
|
['/model-compare', 'model-inference'],
|
||||||
|
['/data-process', 'data-process'],
|
||||||
|
['/data-convert', 'data-convert'],
|
||||||
|
['/model-manage', 'model-manage'],
|
||||||
|
['/dataset-manage', 'dataset'],
|
||||||
|
]
|
||||||
|
|
||||||
|
function trackVisit(url: string | undefined) {
|
||||||
|
if (!url) return
|
||||||
|
for (const [prefix, module] of VISIT_TRACKED_PREFIXES) {
|
||||||
|
if (url.includes(prefix)) {
|
||||||
|
const key = `visit:${module}`
|
||||||
|
const last = Number(sessionStorage.getItem(key) || 0)
|
||||||
|
if (Date.now() - last < 60000) return // 60 秒内去重
|
||||||
|
sessionStorage.setItem(key, String(Date.now()))
|
||||||
|
// fire-and-forget 调用后端记录接口
|
||||||
|
import('./modules/audit-visit').then(({ recordModuleVisit }) => {
|
||||||
|
recordModuleVisit(module, url).catch(() => { /* ignore */ })
|
||||||
|
}).catch(() => { /* ignore */ })
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* 后端统一响应格式
|
* 后端统一响应格式
|
||||||
* code === 0 表示成功,data 为业务数据
|
* code === 0 表示成功,data 为业务数据
|
||||||
|
|||||||
@@ -390,6 +390,31 @@ router.beforeEach((to, _from, next) => {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// 路由切换时记录业务模块访问(用于看板用户操作分布统计)
|
||||||
|
const ROUTE_TO_MODULE: Record<string, string> = {
|
||||||
|
'/fine-tune': 'fine-tune',
|
||||||
|
'/model-eval': 'model-eval',
|
||||||
|
'/model-inference': 'model-inference',
|
||||||
|
'/model-compare': 'model-inference',
|
||||||
|
'/data-process': 'data-process',
|
||||||
|
'/data-convert': 'data-convert',
|
||||||
|
'/model-manage': 'model-manage',
|
||||||
|
'/dataset-manage': 'dataset',
|
||||||
|
}
|
||||||
|
for (const [prefix, module] of Object.entries(ROUTE_TO_MODULE)) {
|
||||||
|
if (to.path.startsWith(prefix)) {
|
||||||
|
const key = `route-visit:${module}`
|
||||||
|
const last = Number(sessionStorage.getItem(key) || 0)
|
||||||
|
if (Date.now() - last >= 60000) {
|
||||||
|
sessionStorage.setItem(key, String(Date.now()))
|
||||||
|
import('@/api/modules/audit-visit').then(({ recordModuleVisit }) => {
|
||||||
|
recordModuleVisit(module, to.fullPath).catch(() => {})
|
||||||
|
}).catch(() => {})
|
||||||
|
}
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
next()
|
next()
|
||||||
})
|
})
|
||||||
|
|
||||||
|
|||||||
@@ -5,6 +5,7 @@ import type { PermissionCode, SystemUser } from '@/types'
|
|||||||
|
|
||||||
const USER_STORAGE_KEY = 'currentUser'
|
const USER_STORAGE_KEY = 'currentUser'
|
||||||
const SESSION_STORAGE_KEY = 'sessionId'
|
const SESSION_STORAGE_KEY = 'sessionId'
|
||||||
|
const AUTHED_FLAG_KEY = 'hasAuthed'
|
||||||
|
|
||||||
const allPermissions: PermissionCode[] = [
|
const allPermissions: PermissionCode[] = [
|
||||||
'dashboard',
|
'dashboard',
|
||||||
@@ -21,25 +22,24 @@ const allPermissions: PermissionCode[] = [
|
|||||||
'user-settings',
|
'user-settings',
|
||||||
]
|
]
|
||||||
|
|
||||||
function restoreUser(): SystemUser | null {
|
/**
|
||||||
const persisted = localStorage.getItem(USER_STORAGE_KEY)
|
* 认证 store
|
||||||
if (persisted) {
|
* - 用户信息持久化到 localStorage(刷新页面不丢失)
|
||||||
|
* - 但每次新开标签页/窗口必须重新点击登录(sessionStorage 标记本次会话已登录)
|
||||||
|
*/
|
||||||
|
export const useAuthStore = defineStore('auth', () => {
|
||||||
|
// 仅当 sessionStorage 里标记了"已登录"时,才从 localStorage 恢复用户
|
||||||
|
const hasAuthed = sessionStorage.getItem(AUTHED_FLAG_KEY) === '1'
|
||||||
|
const persistedUser = hasAuthed ? localStorage.getItem(USER_STORAGE_KEY) : null
|
||||||
|
let initialUser: SystemUser | null = null
|
||||||
|
if (persistedUser) {
|
||||||
try {
|
try {
|
||||||
return JSON.parse(persisted) as SystemUser
|
initialUser = JSON.parse(persistedUser) as SystemUser
|
||||||
} catch {
|
} catch {
|
||||||
localStorage.removeItem(USER_STORAGE_KEY)
|
localStorage.removeItem(USER_STORAGE_KEY)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
return null
|
const currentUser = ref<SystemUser | null>(initialUser)
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* 认证 store
|
|
||||||
* 登录态管理:有 currentUser 即视为已登录。
|
|
||||||
* 离开页面超时由 App.vue 的 visibilitychange 监听接管。
|
|
||||||
*/
|
|
||||||
export const useAuthStore = defineStore('auth', () => {
|
|
||||||
const currentUser = ref<SystemUser | null>(restoreUser())
|
|
||||||
const username = computed(() => currentUser.value?.username || '')
|
const username = computed(() => currentUser.value?.username || '')
|
||||||
const displayName = computed(() => currentUser.value?.display_name || username.value)
|
const displayName = computed(() => currentUser.value?.display_name || username.value)
|
||||||
const roleLabel = computed(() => {
|
const roleLabel = computed(() => {
|
||||||
@@ -54,10 +54,10 @@ export const useAuthStore = defineStore('auth', () => {
|
|||||||
async function login(user: string, password: string) {
|
async function login(user: string, password: string) {
|
||||||
const response = await loginApi(user, password)
|
const response = await loginApi(user, password)
|
||||||
currentUser.value = response.user
|
currentUser.value = response.user
|
||||||
localStorage.setItem('username', response.user.username)
|
|
||||||
localStorage.setItem(USER_STORAGE_KEY, JSON.stringify(response.user))
|
localStorage.setItem(USER_STORAGE_KEY, JSON.stringify(response.user))
|
||||||
|
sessionStorage.setItem(AUTHED_FLAG_KEY, '1')
|
||||||
if (response.session_id) {
|
if (response.session_id) {
|
||||||
localStorage.setItem(SESSION_STORAGE_KEY, response.session_id)
|
sessionStorage.setItem(SESSION_STORAGE_KEY, response.session_id)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -69,14 +69,14 @@ export const useAuthStore = defineStore('auth', () => {
|
|||||||
|
|
||||||
/** 退出 */
|
/** 退出 */
|
||||||
async function logout() {
|
async function logout() {
|
||||||
const sessionId = localStorage.getItem(SESSION_STORAGE_KEY)
|
const sessionId = sessionStorage.getItem(SESSION_STORAGE_KEY)
|
||||||
if (sessionId) {
|
if (sessionId) {
|
||||||
try { await logoutApi(sessionId) } catch { /* 静默 */ }
|
try { await logoutApi(sessionId) } catch { /* 静默 */ }
|
||||||
}
|
}
|
||||||
currentUser.value = null
|
currentUser.value = null
|
||||||
localStorage.removeItem('username')
|
|
||||||
localStorage.removeItem(USER_STORAGE_KEY)
|
localStorage.removeItem(USER_STORAGE_KEY)
|
||||||
localStorage.removeItem(SESSION_STORAGE_KEY)
|
sessionStorage.removeItem(AUTHED_FLAG_KEY)
|
||||||
|
sessionStorage.removeItem(SESSION_STORAGE_KEY)
|
||||||
}
|
}
|
||||||
|
|
||||||
return {
|
return {
|
||||||
|
|||||||
@@ -1,305 +1,165 @@
|
|||||||
<script setup lang="ts">
|
<script setup lang="ts">
|
||||||
import { reactive } from 'vue'
|
import { onMounted, ref } from 'vue'
|
||||||
import { ElMessage } from 'element-plus'
|
import { ElMessage, ElMessageBox } from 'element-plus'
|
||||||
|
import { Plus, Delete, Refresh } from '@element-plus/icons-vue'
|
||||||
|
import type { UploadRequestOptions } from 'element-plus'
|
||||||
import PageCard from '@/components/PageCard.vue'
|
import PageCard from '@/components/PageCard.vue'
|
||||||
|
import {
|
||||||
|
getDataConvertTasks,
|
||||||
|
createDataConvertTask,
|
||||||
|
uploadSourceFiles,
|
||||||
|
deleteDataConvertTask,
|
||||||
|
type DataConvertTask,
|
||||||
|
} from '@/api/modules/data-convert'
|
||||||
|
|
||||||
const settings = reactive({
|
const loading = ref(false)
|
||||||
outputName: 'converted-data',
|
const tasks = ref<DataConvertTask[]>([])
|
||||||
encoding: 'UTF-8',
|
const showCreate = ref(false)
|
||||||
})
|
const form = ref({ name: '', outputName: 'converted-data' })
|
||||||
|
|
||||||
function showPrototypeNotice() {
|
async function load() {
|
||||||
ElMessage.info('当前仅完成界面设计,转换功能将在后续接入')
|
loading.value = true
|
||||||
|
try {
|
||||||
|
const res = await getDataConvertTasks()
|
||||||
|
tasks.value = res.items || []
|
||||||
|
} finally {
|
||||||
|
loading.value = false
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
function resetSettings() {
|
async function submitCreate() {
|
||||||
Object.assign(settings, {
|
if (!form.value.name) {
|
||||||
outputName: 'converted-data',
|
ElMessage.warning('请填写任务名称')
|
||||||
encoding: 'UTF-8',
|
return
|
||||||
|
}
|
||||||
|
await createDataConvertTask({
|
||||||
|
name: form.value.name,
|
||||||
|
output_filename: form.value.outputName + '.jsonl',
|
||||||
})
|
})
|
||||||
|
ElMessage.success('任务创建成功')
|
||||||
|
showCreate.value = false
|
||||||
|
form.value = { name: '', outputName: 'converted-data' }
|
||||||
|
load()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async function customUpload(options: UploadRequestOptions) {
|
||||||
|
const taskId = options.data?.taskId as string
|
||||||
|
if (!taskId) {
|
||||||
|
ElMessage.error('任务 ID 缺失')
|
||||||
|
return
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
const res = await uploadSourceFiles(taskId, [options.file])
|
||||||
|
const data = (res as any)?.data || res
|
||||||
|
if (data?.auto_converted) {
|
||||||
|
ElMessage.success(
|
||||||
|
`上传并自动转换完成,输入 ${data.input_count} / 输出 ${data.output_count},已自动导入数据集`,
|
||||||
|
)
|
||||||
|
} else {
|
||||||
|
ElMessage.warning('上传完成,但转换失败:' + (data?.error || '未知错误'))
|
||||||
|
}
|
||||||
|
load()
|
||||||
|
} catch {
|
||||||
|
ElMessage.error('上传失败')
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function handleDelete(task: DataConvertTask) {
|
||||||
|
try {
|
||||||
|
await ElMessageBox.confirm(
|
||||||
|
`确定要删除任务「${task.name}」吗?`,
|
||||||
|
'删除确认',
|
||||||
|
{ confirmButtonText: '确定删除', cancelButtonText: '取消', type: 'warning' },
|
||||||
|
)
|
||||||
|
} catch {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
await deleteDataConvertTask(task.id)
|
||||||
|
ElMessage.success('任务已删除')
|
||||||
|
load()
|
||||||
|
}
|
||||||
|
|
||||||
|
function statusTag(status: string) {
|
||||||
|
const map: Record<string, { type: string; label: string }> = {
|
||||||
|
pending: { type: 'info', label: '待上传' },
|
||||||
|
uploaded: { type: 'warning', label: '已上传' },
|
||||||
|
running: { type: 'warning', label: '转换中' },
|
||||||
|
completed: { type: 'success', label: '已完成' },
|
||||||
|
failed: { type: 'danger', label: '失败' },
|
||||||
|
}
|
||||||
|
return map[status] || { type: 'info', label: status }
|
||||||
|
}
|
||||||
|
|
||||||
|
onMounted(load)
|
||||||
</script>
|
</script>
|
||||||
|
|
||||||
<template>
|
<template>
|
||||||
<PageCard
|
<PageCard
|
||||||
class="data-convert-page"
|
class="data-convert-page"
|
||||||
title="数据类型转换"
|
title="数据类型转换"
|
||||||
subtitle="将 JSON 文件转换为便于训练和评测使用的 JSONL 格式"
|
subtitle="将 JSON 文件转换为 JSONL 格式,转换结果自动导入到数据集管理"
|
||||||
>
|
>
|
||||||
<div class="converter-panel">
|
<div class="toolbar">
|
||||||
<div class="panel-header">
|
<el-button type="primary" :icon="Plus" @click="showCreate = true">新建转换任务</el-button>
|
||||||
<div class="tool-icon" aria-hidden="true">
|
<el-button :icon="Refresh" @click="load">刷新</el-button>
|
||||||
<i class="fa fa-exchange" />
|
|
||||||
</div>
|
|
||||||
<div>
|
|
||||||
<h3>JSON 转 JSONL</h3>
|
|
||||||
<p>每条 JSON 数据将输出为 JSONL 文件中的一行记录</p>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<el-form class="converter-form" label-position="top">
|
|
||||||
<el-form-item label="转换类型">
|
|
||||||
<div class="format-field" aria-label="JSON 转 JSONL">
|
|
||||||
<span>JSON</span>
|
|
||||||
<i class="fa fa-long-arrow-right" aria-hidden="true" />
|
|
||||||
<span>JSONL</span>
|
|
||||||
</div>
|
|
||||||
</el-form-item>
|
|
||||||
|
|
||||||
<el-form-item label="源文件" required>
|
|
||||||
<button class="upload-zone" type="button" @click="showPrototypeNotice">
|
|
||||||
<i class="fa fa-cloud-upload" aria-hidden="true" />
|
|
||||||
<span class="upload-content">
|
|
||||||
<strong>点击选择或拖拽 JSON 文件到此处</strong>
|
|
||||||
<small>仅支持 .json 格式,单文件不超过 200 MB</small>
|
|
||||||
</span>
|
|
||||||
<span class="select-button">选择文件</span>
|
|
||||||
</button>
|
|
||||||
</el-form-item>
|
|
||||||
|
|
||||||
<div class="form-row">
|
|
||||||
<el-form-item label="输出文件名">
|
|
||||||
<el-input v-model="settings.outputName">
|
|
||||||
<template #append>.jsonl</template>
|
|
||||||
</el-input>
|
|
||||||
</el-form-item>
|
|
||||||
|
|
||||||
<el-form-item label="字符编码">
|
|
||||||
<el-select v-model="settings.encoding" style="width: 100%">
|
|
||||||
<el-option label="UTF-8" value="UTF-8" />
|
|
||||||
</el-select>
|
|
||||||
</el-form-item>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div class="format-tip">
|
|
||||||
<i class="fa fa-info-circle" aria-hidden="true" />
|
|
||||||
<span>支持由 JSON 数组转换为 JSONL,每个数组元素输出为一行。</span>
|
|
||||||
</div>
|
|
||||||
</el-form>
|
|
||||||
|
|
||||||
<div class="panel-footer">
|
|
||||||
<span class="prototype-label">当前为 UI 原型,暂不执行实际转换</span>
|
|
||||||
<div class="actions">
|
|
||||||
<el-button @click="resetSettings">重置</el-button>
|
|
||||||
<el-tooltip content="转换功能将在后续开发中接入" placement="top">
|
|
||||||
<span><el-button type="primary" disabled>开始转换</el-button></span>
|
|
||||||
</el-tooltip>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
|
<el-table :data="tasks" v-loading="loading" border stripe>
|
||||||
|
<el-table-column prop="name" label="任务名称" min-width="140" />
|
||||||
|
<el-table-column label="状态" min-width="100">
|
||||||
|
<template #default="{ row }">
|
||||||
|
<el-tag :type="statusTag(row.status).type">{{ statusTag(row.status).label }}</el-tag>
|
||||||
|
</template>
|
||||||
|
</el-table-column>
|
||||||
|
<el-table-column label="数据量" min-width="120">
|
||||||
|
<template #default="{ row }">
|
||||||
|
{{ row.output_count }} 条
|
||||||
|
</template>
|
||||||
|
</el-table-column>
|
||||||
|
<el-table-column prop="output_filename" label="输出文件名" min-width="160" />
|
||||||
|
<el-table-column prop="create_time" label="创建时间" min-width="180" />
|
||||||
|
<el-table-column label="操作" width="240" fixed="right">
|
||||||
|
<template #default="{ row }">
|
||||||
|
<el-upload
|
||||||
|
v-if="row.status === 'pending'"
|
||||||
|
:auto-upload="true"
|
||||||
|
:show-file-list="false"
|
||||||
|
accept=".json"
|
||||||
|
:http-request="customUpload"
|
||||||
|
:data="{ taskId: row.id }"
|
||||||
|
style="display: inline-block; margin-right: 8px"
|
||||||
|
>
|
||||||
|
<el-button link type="primary">上传文件</el-button>
|
||||||
|
</el-upload>
|
||||||
|
<el-button link type="danger" :icon="Delete" @click="handleDelete(row)">删除</el-button>
|
||||||
|
</template>
|
||||||
|
</el-table-column>
|
||||||
|
</el-table>
|
||||||
|
|
||||||
|
<!-- 新建任务弹窗 -->
|
||||||
|
<el-dialog v-model="showCreate" title="新建转换任务" width="480px">
|
||||||
|
<el-form label-width="100px">
|
||||||
|
<el-form-item label="任务名称" required>
|
||||||
|
<el-input v-model="form.name" placeholder="请输入任务名称" />
|
||||||
|
</el-form-item>
|
||||||
|
<el-form-item label="输出文件名">
|
||||||
|
<el-input v-model="form.outputName" placeholder="converted-data">
|
||||||
|
<template #append>.jsonl</template>
|
||||||
|
</el-input>
|
||||||
|
</el-form-item>
|
||||||
|
</el-form>
|
||||||
|
<template #footer>
|
||||||
|
<el-button @click="showCreate = false">取消</el-button>
|
||||||
|
<el-button type="primary" @click="submitCreate">创建</el-button>
|
||||||
|
</template>
|
||||||
|
</el-dialog>
|
||||||
</PageCard>
|
</PageCard>
|
||||||
</template>
|
</template>
|
||||||
|
|
||||||
<style scoped lang="scss">
|
<style scoped lang="scss">
|
||||||
.converter-panel {
|
.toolbar {
|
||||||
width: 100%;
|
|
||||||
min-height: calc(100vh - 220px);
|
|
||||||
border: 1px solid #e4e7ed;
|
|
||||||
border-radius: 8px;
|
|
||||||
background: #fff;
|
|
||||||
display: flex;
|
display: flex;
|
||||||
flex-direction: column;
|
gap: 10px;
|
||||||
}
|
margin-bottom: 16px;
|
||||||
|
|
||||||
.panel-header {
|
|
||||||
min-height: 72px;
|
|
||||||
padding: 16px 20px;
|
|
||||||
border-bottom: 1px solid #ebeef5;
|
|
||||||
background: #fafafa;
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
gap: 12px;
|
|
||||||
box-sizing: border-box;
|
|
||||||
|
|
||||||
.tool-icon {
|
|
||||||
width: 38px;
|
|
||||||
height: 38px;
|
|
||||||
flex: 0 0 auto;
|
|
||||||
border-radius: 6px;
|
|
||||||
background: var(--el-color-primary-light-9);
|
|
||||||
color: var(--primary-color);
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
justify-content: center;
|
|
||||||
}
|
|
||||||
|
|
||||||
h3 {
|
|
||||||
margin: 0;
|
|
||||||
color: #303133;
|
|
||||||
font-size: 15px;
|
|
||||||
font-weight: 600;
|
|
||||||
}
|
|
||||||
|
|
||||||
p {
|
|
||||||
margin: 4px 0 0;
|
|
||||||
color: #909399;
|
|
||||||
font-size: 12px;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
.converter-form {
|
|
||||||
flex: 1;
|
|
||||||
padding: 22px 24px 6px;
|
|
||||||
|
|
||||||
:deep(.el-form-item) {
|
|
||||||
margin-bottom: 20px;
|
|
||||||
}
|
|
||||||
|
|
||||||
:deep(.el-form-item__label) {
|
|
||||||
padding-bottom: 8px;
|
|
||||||
color: #606266;
|
|
||||||
font-size: 13px;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
.format-field {
|
|
||||||
width: 100%;
|
|
||||||
min-height: 40px;
|
|
||||||
padding: 0 14px;
|
|
||||||
border: 1px solid #dcdfe6;
|
|
||||||
border-radius: 4px;
|
|
||||||
background: #f5f7fa;
|
|
||||||
color: #303133;
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
gap: 14px;
|
|
||||||
box-sizing: border-box;
|
|
||||||
font-size: 13px;
|
|
||||||
font-weight: 500;
|
|
||||||
|
|
||||||
i {
|
|
||||||
color: #909399;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
.upload-zone {
|
|
||||||
width: 100%;
|
|
||||||
min-height: 112px;
|
|
||||||
padding: 20px;
|
|
||||||
border: 1px dashed #b8c4d1;
|
|
||||||
border-radius: 6px;
|
|
||||||
background: #fafcff;
|
|
||||||
color: #606266;
|
|
||||||
cursor: pointer;
|
|
||||||
font: inherit;
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
gap: 14px;
|
|
||||||
text-align: left;
|
|
||||||
transition: border-color 0.2s ease, background 0.2s ease;
|
|
||||||
|
|
||||||
&:hover,
|
|
||||||
&:focus-visible {
|
|
||||||
border-color: var(--primary-color);
|
|
||||||
background: var(--el-color-primary-light-9);
|
|
||||||
outline: none;
|
|
||||||
}
|
|
||||||
|
|
||||||
> i {
|
|
||||||
color: var(--primary-color);
|
|
||||||
font-size: 24px;
|
|
||||||
}
|
|
||||||
|
|
||||||
.upload-content {
|
|
||||||
min-width: 0;
|
|
||||||
display: flex;
|
|
||||||
flex: 1;
|
|
||||||
flex-direction: column;
|
|
||||||
gap: 5px;
|
|
||||||
}
|
|
||||||
|
|
||||||
strong {
|
|
||||||
color: #303133;
|
|
||||||
font-size: 13px;
|
|
||||||
font-weight: 500;
|
|
||||||
}
|
|
||||||
|
|
||||||
small {
|
|
||||||
color: #909399;
|
|
||||||
font-size: 12px;
|
|
||||||
}
|
|
||||||
|
|
||||||
.select-button {
|
|
||||||
min-height: 32px;
|
|
||||||
padding: 0 14px;
|
|
||||||
border: 1px solid #dcdfe6;
|
|
||||||
border-radius: 4px;
|
|
||||||
background: #fff;
|
|
||||||
color: #606266;
|
|
||||||
display: inline-flex;
|
|
||||||
align-items: center;
|
|
||||||
white-space: nowrap;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
.form-row {
|
|
||||||
display: grid;
|
|
||||||
grid-template-columns: minmax(0, 2fr) minmax(180px, 1fr);
|
|
||||||
gap: 16px;
|
|
||||||
}
|
|
||||||
|
|
||||||
.format-tip {
|
|
||||||
min-height: 38px;
|
|
||||||
padding: 9px 12px;
|
|
||||||
border-radius: 4px;
|
|
||||||
background: var(--el-color-primary-light-9);
|
|
||||||
color: #606266;
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
gap: 8px;
|
|
||||||
box-sizing: border-box;
|
|
||||||
font-size: 12px;
|
|
||||||
|
|
||||||
i {
|
|
||||||
color: var(--primary-color);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
.panel-footer {
|
|
||||||
min-height: 64px;
|
|
||||||
padding: 12px 24px;
|
|
||||||
border-top: 1px solid #ebeef5;
|
|
||||||
background: #fafafa;
|
|
||||||
display: flex;
|
|
||||||
align-items: center;
|
|
||||||
justify-content: space-between;
|
|
||||||
gap: 20px;
|
|
||||||
box-sizing: border-box;
|
|
||||||
|
|
||||||
.prototype-label {
|
|
||||||
color: #909399;
|
|
||||||
font-size: 12px;
|
|
||||||
}
|
|
||||||
|
|
||||||
.actions {
|
|
||||||
display: flex;
|
|
||||||
gap: 10px;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
@media (max-width: 640px) {
|
|
||||||
.converter-form {
|
|
||||||
padding: 18px 16px 4px;
|
|
||||||
}
|
|
||||||
|
|
||||||
.form-row {
|
|
||||||
grid-template-columns: 1fr;
|
|
||||||
gap: 0;
|
|
||||||
}
|
|
||||||
|
|
||||||
.upload-zone {
|
|
||||||
align-items: flex-start;
|
|
||||||
flex-wrap: wrap;
|
|
||||||
|
|
||||||
.select-button {
|
|
||||||
margin-left: 38px;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
.panel-footer {
|
|
||||||
padding: 12px 16px;
|
|
||||||
align-items: flex-end;
|
|
||||||
flex-direction: column;
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
</style>
|
</style>
|
||||||
|
|||||||
380
测试脚本.md
Normal file
380
测试脚本.md
Normal file
@@ -0,0 +1,380 @@
|
|||||||
|
# YG_FT 模型微调平台测试脚本
|
||||||
|
|
||||||
|
## 环境准备
|
||||||
|
|
||||||
|
### 启动顺序
|
||||||
|
|
||||||
|
1. **数据库**:远程 PostgreSQL `www.caoxiaozhu.com:5432`(已就绪)
|
||||||
|
2. **后端**:
|
||||||
|
```cmd
|
||||||
|
cd /d E:\yg_ft\backend
|
||||||
|
set COMPUTE_STATUS_SYNC_MODE=off
|
||||||
|
.\.venv\Scripts\python.exe -m uvicorn app.main:app --reload --port 17861
|
||||||
|
```
|
||||||
|
3. **算力服务**(Docker):
|
||||||
|
```cmd
|
||||||
|
cd /d E:\yg_ft
|
||||||
|
docker build -f docker/compute/Dockerfile.compute -t yg-ft-compute-api:latest .
|
||||||
|
cd docker/compute
|
||||||
|
docker compose up -d
|
||||||
|
```
|
||||||
|
4. **前端**:
|
||||||
|
```cmd
|
||||||
|
cd /d E:\yg_ft\frontend
|
||||||
|
npm run dev
|
||||||
|
```
|
||||||
|
|
||||||
|
### 验证服务可用性
|
||||||
|
|
||||||
|
| 服务 | 验证命令 | 期望结果 |
|
||||||
|
|------|---------|---------|
|
||||||
|
| 后端 | `curl http://localhost:17861/modelTF/health` | `{"code":0,...,"data":{"cpu_percent":x,...}}` |
|
||||||
|
| 算力 | `curl http://localhost:19100/health` | `{"status":"ok","compute_host_id":"gpu-node-01"}` |
|
||||||
|
| 前端 | 浏览器打开 `http://localhost:16801` | 跳转登录页 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 一、登录与认证
|
||||||
|
|
||||||
|
### 1.1 登录
|
||||||
|
|
||||||
|
| 步骤 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 1 | 打开 `http://localhost:16801` | 显示登录页,不自动登录 |
|
||||||
|
| 2 | 输入账号 `admin`,密码 `admin123`,点击登录 | 提示"登录成功",跳转看板页 |
|
||||||
|
| 3 | F12 控制台执行 `sessionStorage.getItem('sessionId')` | 有值(非 null) |
|
||||||
|
| 4 | 浏览器 Network 查看 `/modelTF/login` 响应 | 返回 `token`、`user`、`session_id` |
|
||||||
|
|
||||||
|
### 1.2 退出登录
|
||||||
|
|
||||||
|
| 步骤 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 1 | 点击侧边栏底部"退出登录"按钮 | 跳转登录页 |
|
||||||
|
| 2 | F12 控制台执行 `sessionStorage.getItem('hasAuthed')` | 返回 null |
|
||||||
|
| 3 | 浏览器 Network 查看 `/modelTF/logout` 请求 | 状态 200 |
|
||||||
|
|
||||||
|
### 1.3 刷新页面保持登录
|
||||||
|
|
||||||
|
| 步骤 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 1 | 登录后按 F5 刷新页面 | 仍在功能页,不跳登录页 |
|
||||||
|
|
||||||
|
### 1.4 关闭标签页再打开
|
||||||
|
|
||||||
|
| 步骤 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 1 | 登录后关闭浏览器标签页 | — |
|
||||||
|
| 2 | 重新打开 `http://localhost:16801` | 跳转登录页(sessionStorage 已清空) |
|
||||||
|
|
||||||
|
### 1.5 会话超时
|
||||||
|
|
||||||
|
| 步骤 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 1 | 登录后切到其他软件,等待 5 分钟以上 | — |
|
||||||
|
| 2 | 切回浏览器 | 提示"登录已过期",跳转登录页 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 二、看板
|
||||||
|
|
||||||
|
### 2.1 服务状态
|
||||||
|
|
||||||
|
| 步骤 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 1 | 登录后进入看板页 | 显示 7 个服务:模型训练/模型评测/模型推理/模型管理/数据集管理/数据处理/数据类型转换 |
|
||||||
|
| 2 | 检查每个服务状态 | 全部显示"正常" |
|
||||||
|
| 3 | 检查实例数 | 模型管理有数量、数据集管理有数量等 |
|
||||||
|
|
||||||
|
### 2.2 运行中任务
|
||||||
|
|
||||||
|
| 步骤 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 1 | 查看顶部"运行中任务"数字 | 显示训练+评测+数据处理中 running 状态的任务总数 |
|
||||||
|
| 2 | 无运行任务时 | 显示 0 |
|
||||||
|
|
||||||
|
### 2.3 用户操作分布
|
||||||
|
|
||||||
|
| 步骤 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 1 | 查看饼图 | 4 个分类(数据处理/模型训练/模型评测/模型推理)各一色 |
|
||||||
|
| 2 | 无数据时 | 4 等分(各 25%) |
|
||||||
|
| 3 | 点击侧边栏"模型训练"菜单,回到看板 | 模型训练权重增加 |
|
||||||
|
| 4 | 点击"模型推理"菜单,回到看板 | 模型推理权重增加 |
|
||||||
|
| 5 | 浏览器控制台执行 `sessionStorage.clear()` 后重试 | 去重缓存清除,可再次记录 |
|
||||||
|
|
||||||
|
### 2.4 登录时长排行
|
||||||
|
|
||||||
|
| 步骤 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 1 | 查看柱状图 | 显示当前登录用户,时长 > 0 |
|
||||||
|
| 2 | 条形图上无文字标签 | 只在鼠标悬浮时显示 tooltip |
|
||||||
|
| 3 | Y 轴用户名清晰可读 | 字体颜色 `#1f2937`,非模糊 |
|
||||||
|
|
||||||
|
### 2.5 顶部状态栏
|
||||||
|
|
||||||
|
| 步骤 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 1 | 查看顶部 CPU/内存/磁盘 | 显示真实数值(非 0) |
|
||||||
|
| 2 | 等待 30 秒 | 数值自动刷新 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 三、租户管理
|
||||||
|
|
||||||
|
### 3.1 列表
|
||||||
|
|
||||||
|
| 步骤 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 1 | 侧边栏 → 平台治理 → 租户管理 | 显示列表(名称/用户ID/状态/创建时间) |
|
||||||
|
| 2 | 检查列名 | "编码"已改为"用户ID" |
|
||||||
|
| 3 | 检查操作列 | 有"详情"和"删除"按钮,无"配额"按钮 |
|
||||||
|
|
||||||
|
### 3.2 新建租户
|
||||||
|
|
||||||
|
| 步骤 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 1 | 点击"新建租户" | 弹窗显示:名称、用户ID、GPU数量、存储配额、最大项目数 |
|
||||||
|
| 2 | 填写名称"测试租户",GPU=2,存储=100,项目=5,点击创建 | 提示"租户创建成功",列表新增一行 |
|
||||||
|
|
||||||
|
### 3.3 租户详情与配额
|
||||||
|
|
||||||
|
| 步骤 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 1 | 点击"详情" | 跳转详情页,显示基本信息 + 配额 |
|
||||||
|
| 2 | 配额显示格式 | `GPU 2 | 存储 100GB | 项目 5` |
|
||||||
|
| 3 | 修改配额数字,点击"保存配额" | 提示"配额已保存" |
|
||||||
|
| 4 | 检查详情页无"项目空间"卡片 | 已移除 |
|
||||||
|
|
||||||
|
### 3.4 删除租户
|
||||||
|
|
||||||
|
| 步骤 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 1 | 在列表页点击"删除" | 弹出确认框 |
|
||||||
|
| 2 | 点击"确定删除" | 提示"租户已删除",列表刷新 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 四、项目空间
|
||||||
|
|
||||||
|
### 4.1 列表
|
||||||
|
|
||||||
|
| 步骤 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 1 | 侧边栏 → 项目空间 | 显示列表(项目名/编码ID/状态/描述/创建时间) |
|
||||||
|
| 2 | 检查列名 | "编码"已改为"编码ID" |
|
||||||
|
| 3 | 顶部有租户筛选下拉框 | 可按租户过滤 |
|
||||||
|
|
||||||
|
### 4.2 新建项目
|
||||||
|
|
||||||
|
| 步骤 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 1 | 点击"新建项目" | 弹窗显示:名称、编码ID(下拉)、描述 |
|
||||||
|
| 2 | 编码ID下拉选项 | 显示已有租户的编码 |
|
||||||
|
| 3 | 选择编码ID后提交 | 提示"项目创建成功" |
|
||||||
|
|
||||||
|
### 4.3 项目详情
|
||||||
|
|
||||||
|
| 步骤 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 1 | 点击"详情" | 显示基本信息 |
|
||||||
|
| 2 | 查看成员管理区域 | 显示成员列表 + 添加成员按钮 |
|
||||||
|
| 3 | 点击"资源授权(ACL)" | 弹窗显示 ACL 编辑器 |
|
||||||
|
|
||||||
|
### 4.4 ACL 授权
|
||||||
|
|
||||||
|
| 步骤 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 1 | 点击"添加授权项" | 新增一行 |
|
||||||
|
| 2 | 主体类型选"用户" | 显示用户下拉框(非文本输入) |
|
||||||
|
| 3 | 主体类型选"项目角色" | 显示角色下拉框(member/admin/viewer) |
|
||||||
|
| 4 | 勾选权限后保存 | 提示"ACL 已保存" |
|
||||||
|
|
||||||
|
### 4.5 删除项目
|
||||||
|
|
||||||
|
| 步骤 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 1 | 点击"删除" | 确认弹窗 |
|
||||||
|
| 2 | 确认删除 | 提示"项目已删除" |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 五、算力节点
|
||||||
|
|
||||||
|
### 5.1 节点列表
|
||||||
|
|
||||||
|
| 步骤 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 1 | 侧边栏 → 算力节点 | 显示节点列表 |
|
||||||
|
| 2 | 检查 gpu-node-01 | 状态为启用 |
|
||||||
|
|
||||||
|
### 5.2 测试连接
|
||||||
|
|
||||||
|
| 步骤 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 1 | 点击 gpu-node-01 的"测试"按钮 | 提示"连接成功,发现 X 张 GPU,延迟 Xms" |
|
||||||
|
| 2 | 检查 GPU Tab | 显示 GPU 资源列表 |
|
||||||
|
|
||||||
|
### 5.3 节点启停
|
||||||
|
|
||||||
|
| 步骤 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 1 | 点击"停用" | 节点状态变为停用 |
|
||||||
|
| 2 | 点击"启用" | 节点状态变为启用 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 六、用户设置
|
||||||
|
|
||||||
|
### 6.1 用户列表
|
||||||
|
|
||||||
|
| 步骤 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 1 | 侧边栏 → 系统设置 → 用户设置 | 显示用户列表 |
|
||||||
|
| 2 | admin 用户 | 不可停用、不可删除 |
|
||||||
|
|
||||||
|
### 6.2 创建用户
|
||||||
|
|
||||||
|
| 步骤 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 1 | 点击"创建用户" | 跳转创建页 |
|
||||||
|
| 2 | 填写账号/显示名/密码/角色/权限 | 表单正常 |
|
||||||
|
| 3 | 提交 | 提示创建成功,列表新增 |
|
||||||
|
|
||||||
|
### 6.3 权限分配
|
||||||
|
|
||||||
|
| 步骤 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 1 | 点击某用户"页面权限" | 弹窗显示 12 个权限 checkbox |
|
||||||
|
| 2 | 取消某权限,保存 | 该用户侧边栏不再显示对应菜单 |
|
||||||
|
|
||||||
|
### 6.4 重置密码
|
||||||
|
|
||||||
|
| 步骤 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 1 | 点击"重置密码" | 弹窗显示默认密码 `Platform@123` |
|
||||||
|
| 2 | 确认 | 提示重置成功 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 七、审批管理
|
||||||
|
|
||||||
|
### 7.1 审批模板
|
||||||
|
|
||||||
|
| 步骤 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 1 | 侧边栏 → 审批模板 | 显示模板列表 |
|
||||||
|
| 2 | 点击新建 | 弹窗显示名称 + 步骤 JSON |
|
||||||
|
| 3 | 填写并提交 | 列表新增 |
|
||||||
|
|
||||||
|
### 7.2 审批中心
|
||||||
|
|
||||||
|
| 步骤 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 1 | 侧边栏 → 审批中心 | 显示实例列表 |
|
||||||
|
| 2 | 按状态筛选 | 列表过滤正常 |
|
||||||
|
| 3 | 对 pending 实例点"通过/拒绝" | 弹窗填写意见,提交后状态变更 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 八、审计日志
|
||||||
|
|
||||||
|
| 步骤 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 1 | 侧边栏 → 审计日志 | 显示日志列表 |
|
||||||
|
| 2 | 填写筛选条件查询 | 列表过滤 |
|
||||||
|
| 3 | 点击"导出" | 下载 CSV 文件 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 九、平台性能
|
||||||
|
|
||||||
|
| 步骤 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 1 | 侧边栏 → 平台性能 | 显示 CPU/内存/磁盘/GPU 卡片 |
|
||||||
|
| 2 | 数据非 0 | 与本机实际使用率一致 |
|
||||||
|
| 3 | 开启自动刷新(3秒) | 趋势图实时更新 |
|
||||||
|
| 4 | 点击 GPU 卡片 | 弹出详情抽屉 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 十、查看日志
|
||||||
|
|
||||||
|
| 步骤 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 1 | 侧边栏 → 查看日志 | 显示系统日志 Tab |
|
||||||
|
| 2 | 选择日期和文件 | 显示日志内容 |
|
||||||
|
| 3 | 输入关键词搜索 | 高亮匹配 |
|
||||||
|
| 4 | 切换到训练日志 Tab | 显示训练日志列表 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 十一、数据流主链路
|
||||||
|
|
||||||
|
### 11.1 数据集管理
|
||||||
|
|
||||||
|
| 步骤 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 1 | 进入数据集管理 | 显示列表 |
|
||||||
|
| 2 | 新建数据集 | 创建成功 |
|
||||||
|
| 3 | 上传文件 | 文件上传成功 |
|
||||||
|
| 4 | 预览文件 | 显示内容 |
|
||||||
|
|
||||||
|
### 11.2 数据类型转换
|
||||||
|
|
||||||
|
| 步骤 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 1 | 侧边栏 → 数据类型转换 | 显示任务列表 |
|
||||||
|
| 2 | 点击"新建转换任务",填写名称,点击创建 | 列表新增一行,状态为"待上传" |
|
||||||
|
| 3 | 点击"上传文件",选择 `.json` 文件 | 上传成功后自动转换为 JSONL 并导入数据集 |
|
||||||
|
| 4 | 检查任务状态 | 变为"已完成",显示输入/输出数 |
|
||||||
|
| 5 | 进入数据集管理 → 上传任务 Tab | 可看到自动导入的数据集 |
|
||||||
|
| 6 | 上传非 `.json` 文件 | 提示"仅支持 .json 文件" |
|
||||||
|
| 7 | 点击"删除" | 确认后任务删除 |
|
||||||
|
|
||||||
|
### 11.3 数据处理
|
||||||
|
|
||||||
|
| 步骤 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 1 | 进入数据处理 | 显示任务列表 |
|
||||||
|
| 2 | 创建数据处理任务 | 创建成功 |
|
||||||
|
| 3 | 启动任务 | 状态变为 running |
|
||||||
|
| 4 | 回到看板 | "运行中任务"数字 +1 |
|
||||||
|
|
||||||
|
### 11.4 模型训练
|
||||||
|
|
||||||
|
| 步骤 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 1 | 进入模型训练 | 显示训练任务列表 |
|
||||||
|
| 2 | 创建训练任务 | 选择数据集 + 模型 + GPU |
|
||||||
|
| 3 | 启动训练 | 状态变为 running |
|
||||||
|
| 4 | 查看训练日志 | 显示实时日志 |
|
||||||
|
| 5 | 训练完成 | 产出模型 |
|
||||||
|
|
||||||
|
### 11.5 模型评测
|
||||||
|
|
||||||
|
| 步骤 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 1 | 进入模型评测 | 显示评测任务列表 |
|
||||||
|
| 2 | 创建评测任务 | 选择模型 + 数据集 |
|
||||||
|
| 3 | 启动评测 | 状态变为 running |
|
||||||
|
| 4 | 评测完成 | 显示指标结果 |
|
||||||
|
|
||||||
|
### 11.6 模型推理
|
||||||
|
|
||||||
|
| 步骤 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 1 | 进入模型推理 | 显示推理会话列表 |
|
||||||
|
| 2 | 创建推理任务 | 选择模型 + 加载 |
|
||||||
|
| 3 | 开始对话 | 返回模型回复 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 十二、异常场景
|
||||||
|
|
||||||
|
| 场景 | 操作 | 预期结果 |
|
||||||
|
|------|------|---------|
|
||||||
|
| 数据库断开 | 停止远程数据库连接 | 所有接口返回 500 |
|
||||||
|
| 算力服务停掉 | `docker compose down` | 算力测试返回失败 |
|
||||||
|
| 未登录访问 | 直接访问 `/dashboard` | 跳转登录页 |
|
||||||
|
| 权限不足 | 普通用户访问用户设置 | 跳转 403 页面 |
|
||||||
|
| Token 过期 | 关闭标签页再打开 | 跳转登录页(sessionStorage 清空) |
|
||||||
Reference in New Issue
Block a user