Skip to content

feat:add batch processing note#6626

Open
gaga0714 wants to merge 9 commits intolabring:mainfrom
gaga0714:main
Open

feat:add batch processing note#6626
gaga0714 wants to merge 9 commits intolabring:mainfrom
gaga0714:main

Conversation

@gaga0714
Copy link
Copy Markdown
Contributor

No description provided.

@github-actions
Copy link
Copy Markdown

Docs Preview Deployed!

🔗 👀 Click here to visit preview

registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt-docs-pr:2e05fcf90dcc42a2cf2ce414bbeab21466454851

@github-actions
Copy link
Copy Markdown

Build Successful - Preview sandbox Image for this PR:

registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt-pr:sandbox_2e05fcf90dcc42a2cf2ce414bbeab21466454851

@github-actions
Copy link
Copy Markdown

Build Successful - Preview fastgpt Image for this PR:

registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt-pr:fastgpt_2e05fcf90dcc42a2cf2ce414bbeab21466454851

@github-actions
Copy link
Copy Markdown

Build Successful - Preview mcp_server Image for this PR:

registry.cn-hangzhou.aliyuncs.com/fastgpt/fastgpt-pr:mcp_server_2e05fcf90dcc42a2cf2ce414bbeab21466454851

Copy link
Copy Markdown
Collaborator

@c121914yu c121914yu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📊 PR 代码审查总结

PR #6626: feat: add batch processing node
作者: @gaga0714
变更: +585 / -23 行,23 个文件


✅ 优点

  1. 并发架构设计合理: 使用 worker-pool 模式(cursor 递增 + Promise.all)实现并发控制,在 JS 单线程模型下无竞争问题
  2. 结果顺序保证: 通过 orderedRawResult[index]orderedSuccessResult.sort() 保证输出顺序与输入一致,符合用户预期
  3. 三态状态输出: success / failed / partial_success 的设计让下游节点可以做精细化分支处理
  4. 向后兼容: 旧 loop 节点标记 abandon: true 而非删除,存量工作流不受影响
  5. 运行时环境变量限制: WORKFLOW_BATCH_MAX_CONCURRENCYWORKFLOW_BATCH_MAX_RETRY 防止用户输入绕过服务端限制

⚠️ 问题汇总(详见下方行级评论)

🟡 建议改进

  1. assertBatchChildNodes 应抛出 Error 对象而非 reject 字符串
  2. batchInput 在响应弹窗中复用了 loop_input 的 i18n key,显示文字不准确
  3. variableUpdate 被拖入批处理节点时提示信息不准确
  4. NodeLoopEnd 的 intro 回退逻辑有潜在问题
  5. 两个 useEffect 调用 onChangeNode 需注意无限循环风险

🧪 测试建议

  • 验证并发数为 1 时的串行执行行为
  • 验证重试逻辑:第 1 次失败、第 2 次成功的场景
  • 验证 partial_success 状态下,下游节点能正确接收 batchRawResult
  • 验证拖入禁止节点类型时的提示信息正确性(尤其是 variableUpdate
  • 验证旧 loop 节点的存量工作流不受影响

💬 总体评价

  • 代码质量: ⭐⭐⭐⭐☆ (4/5)
  • 安全性: ⭐⭐⭐⭐⭐ (5/5)
  • 性能: ⭐⭐⭐⭐⭐ (5/5)
  • 可维护性: ⭐⭐⭐⭐☆ (4/5)

🚀 审查结论

整体实现质量较高,并发设计合理,建议修复以下几个小问题后合并。


📍 详细问题说明

[1] packages/service/core/workflow/dispatch/batch/runBatch.ts L309

🟡 建议改进: assertBatchChildNodes 使用 return Promise.reject(string) 而非 Error 对象。

字符串 reject 不携带堆栈信息,调试困难。建议改为:

throw new Error('Batch child workflow does not allow loop/batch/interactive/variable-update nodes');

[2] projects/app/src/components/core/chat/components/WholeResponseModal.tsx L641

🟡 建议改进: batchInput 复用了 loop_input 的 i18n key,响应弹窗中会显示「循环输入」而非「批量输入」。建议新增 batch_input i18n key。


[3] projects/app/src/pageComponents/app/detail/WorkflowComponents/Flow/hooks/useWorkflow.tsx L791

🟡 建议改进: variableUpdate 节点被拖入批处理节点时,触发的是 can_not_loop 提示而非专属批处理提示。isInteractiveInBatch 只包含 userSelectformInput,不包含 variableUpdate,导致提示语不准确。


[4] projects/app/src/pageComponents/app/detail/WorkflowComponents/Flow/nodes/Loop/NodeLoopEnd.tsx L959

🟡 建议改进: intro 回退值直接使用了原始 i18n key 字符串 'workflow:loop_end_intro',需确认 NodeCard 是否会自动处理翻译,否则 UI 会直接显示 key 字符串。


[5] projects/app/src/pageComponents/app/detail/WorkflowComponents/Flow/nodes/Loop/NodeLoop.tsx L838-L935

🟢 注意潜在风险: 两个 useEffect 都调用 onChangeNode 并依赖 inputs,若 onChangeNode 触发后导致 inputs 引用变化,可能引发无限渲染。已有条件检查(input.value === nextValue)可部分防止,但建议增加单元测试验证稳定性。

Copy link
Copy Markdown
Collaborator

@c121914yu c121914yu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

代码结构调整下,loop 目录下增加 2 个 子目录:
v1: 旧版的
v2:新的循环

新增 batch 目录,不要跟在 loop 里

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

不要复用原来的 loop 节点,新写一个,看有没有能提取的通用内容

avatarLinear: 'core/workflow/template/loopLinear',
colorSchema: 'violetDeep',
name: i18nT('workflow:loop'),
name: i18nT('workflow:loop_deprecated'),
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个不用改,反正不能加新的了

renderTypeList: [FlowNodeInputTypeEnum.reference],
valueType: WorkflowIOValueTypeEnum.any,
label: '',
label: i18nT('workflow:loop_end_intro'),
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个也不用改?

runtimeNodes
});

const maxLength = process.env.WORKFLOW_MAX_LOOP_TIMES
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

直接用解析好的 env.xxx,里面也能设置默认值。

}
};

const workers = Array.from({ length: Math.min(concurrency, loopInputArray.length) }).map(
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

有一个 batchRun 的批量执行方法可以直接用

return Promise.reject(`Input array length cannot be greater than ${maxLength}`);
}

if (loopInputArray.length === 0) {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

不用考虑,底下数组是空的话,应该就要不允许,可以适配。

const currentNode = getNodeById(handleParams?.nodeId);
if (templateNode.flowNodeType === FlowNodeTypeEnum.loop && !!currentNode?.parentNodeId) {
if (
[FlowNodeTypeEnum.loop, FlowNodeTypeEnum.batch].includes(templateNode.flowNodeType) &&
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

封装一个方法,传入 type,识别是否是这种 parentChild 的节点类型

return basicNodeTemplates
.filter((item) => {
// hide deprecated templates from add panel
if (item.abandon) {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

basicNodeTemplates 里去掉 loop 就好了,这里不用 filter


if (parentNode) {
if (unSupportedTypes.includes(node.type as FlowNodeTypeEnum)) {
const parentType = parentNode.type as FlowNodeTypeEnum;
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

加点备注~

const node = getNodeById(nodeId);
if (!node) return 'bottom';
return node.flowNodeType === FlowNodeTypeEnum.loop ? 'top' : 'bottom';
return [FlowNodeTypeEnum.loop, FlowNodeTypeEnum.batch].includes(node.flowNodeType)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里也记得,改成封装的方法

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants