feat: 更新后端平台模块、Compute引擎、前端组件及构建产物
- 更新 backend 平台 API、platform_store、compute_gateway sync - 更新 compute agent/engine/adapter 及 API - 更新 Docker 部署配置(app/compute) - 新增 frontend/src/utils/ 工具模块 - 新增 scripts/ops_diagnostics.py 运维诊断脚本 - 新增 docs/2026-07-23-development-summary.md 开发总结 - 重构 frontend/dist 构建产物(新 hash) - 更新前端多个视图组件及 API 模块 Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -701,7 +701,9 @@ YG_FT/
|
||||
职责:
|
||||
|
||||
- 生成训练 YAML 或 CLI。
|
||||
- 生成 dataset_info。
|
||||
- 根据平台数据集 ID 生成 LLaMA-Factory dataset key 和 `{dataset_dir}/dataset_info.json`。
|
||||
- 单文件数据集使用一个 dataset key,多文件数据集拆分为多个 key 并通过逗号组合传入 `--dataset`。
|
||||
- 正式启动前由应用侧同步当前数据集文件到目标算力节点,确保训练使用最新激活版本。
|
||||
- 设置 `CUDA_VISIBLE_DEVICES`。
|
||||
- 启动训练。
|
||||
- 解析日志。
|
||||
@@ -1116,3 +1118,10 @@ YG_FT/
|
||||
| P1 | 数据处理、评测、推理、模型服务治理 | 形成训练前后闭环 |
|
||||
| P1 | 存储管理、checkpoint、审计中心 | 企业治理和运维收口 |
|
||||
| P2 | 训练引擎管理、外部 API、OIDC/LDAP、成本核算、人工评测 | 扩展能力,第一版可预留 |
|
||||
## P1/P2/P3 Development Status
|
||||
|
||||
| Priority | Scope | Implemented content | Remaining production hardening |
|
||||
| --- | --- | --- | --- |
|
||||
| P1 | Training runtime assets | Compute scans real `checkpoint-*` directories; backend persists checkpoints to `fine_tune_checkpoints`; backend parses log metrics into `fine_tune_metrics`; completed training auto-registers a trained model with its real output directory. | Add full TensorBoard event parsing and retention/cleanup execution policies. |
|
||||
| P2 | Model merge/export | `/modelTF/model-manage/merge` submits `llamafactory-cli export` through Compute; merge/export jobs are recorded in `compute_jobs` and `model_export_jobs`; trained model artifacts are recorded in `model_artifacts`; base->trained and merge/export relations are recorded in `model_lineage`; APIs expose artifacts, lineage and export jobs. | Add frontend polling/status panels for export jobs and richer artifact checksum generation during long-running export. |
|
||||
| P3 | Compute ops | `compute_jobs` and `gpu_allocations` are updated from runtime polling; training start uses `scheduler_locks` to serialize node selection and GPU pre-allocation; terminal task states release GPU allocations; resource replica drift check and repair-pending APIs are available; `scripts/ops_diagnostics.py` checks backend, PostgreSQL, Compute health, GPU discovery and Compute jobs. | Add dedicated scheduler worker, automatic replica repair execution from authoritative source metadata, and lock timeout alarms. |
|
||||
|
||||
Reference in New Issue
Block a user