Bug Fixes and Workflow Improvements

[Jany-M]

[1.0.6-jm] - 2025-10-19

Added

• Dynamic Model Limit Detection: New utils/model_limits.py module that automatically detects and adapts to any LLM model's token limits and pricing
• Loop Detection System: utils/loop_detector.py prevents infinite loops by detecting repeated tool calls, timeouts, and progress stalls
• Progress Tracking: 8-phase progress tracking (5% → 100%) with file-level progress indicators in both UI and terminal
• Abort Mechanism: "Stop Processing" button in UI with global abort flag for clean process termination
• Enhanced Error Display: Real-time error messages in both UI and terminal with timestamps
• File Progress Tracking: Shows files completed/total with estimated time remaining

Fixed

• Critical: False Error Detection: Fixed overly aggressive error detection that was marking successful operations as failures, causing premature abort and empty file generation
• Critical: Empty File Generation: Files now contain actual code instead of being empty (2-byte files)
• Unique Folder Naming: Each project run now creates paper_{timestamp} folders instead of reusing pdf_output
• PDF Save Location: PDFs now save to deepcode_lab/papers/ instead of system temp directory
• Duplicate Folder Prevention: Added session state caching to prevent duplicate folder creation on UI reruns
• Token Limit Compliance: Fixed max_tokens to respect model limits dynamically (e.g., gpt-4o-mini's 16,384 token limit)
• Empty Plan Detection: System now fails early with clear error messages when initial plan is empty or invalid
• Process Hanging: Fixed infinite loops and hanging on errors - process now exits cleanly
• PDF to Markdown Conversion: Fixed automatic conversion and file location handling
• Document Segmentation: Properly uses configured character threshold from mcp_agent.config.yaml
• Error Propagation: Abort mechanism now properly stops process after 10 consecutive real errors

Changed

• Model-Aware Token Management: Token limits now adapt automatically based on configured model instead of hardcoded values
• Cost Calculation: Dynamic pricing based on actual model rates (OpenAI, Anthropic)
• Retry Logic: Token limits for retries now respect model maximum (87.5% → 95% → 98% of max)
• Segmentation Workflow: Better integration with code implementation phase
• Error Handling: Enhanced error propagation - errors no longer reported as "success"
• UI Display: Shows project folder name after PDF conversion for better visibility
• Terminal Logging: Added timestamps to all progress messages

Technical Improvements

• Added document-segmentation server to code implementation workflow for better token management
• Improved error handling in agent orchestration engine with proper cleanup
• Enhanced subprocess handling on Windows (hide console windows, prevent hanging)
• Better LibreOffice detection on Windows using direct path checking
• Fixed input data format consistency (JSON with paper_path key)
• Added comprehensive logging throughout the pipeline
• Improved resource cleanup on errors and process termination

Documentation

• Translated Chinese comments to English in core workflow files
• Added inline documentation for new utility modules

Breaking Changes

• None - all changes are backward compatible

Known Issues

• Terminal may show trailing "Calling Tool..." line after completion (cosmetic display artifact - process completes successfully)
• tiktoken package optional warning (doesn't affect functionality)

Success Metrics

• ✅ Complete end-to-end workflow: DOCX upload → PDF conversion → Markdown → Segmentation → Planning → Code generation
• ✅ Files generated with actual code content (15+ files with proper implementation)
• ✅ Single folder per project run (no duplicates)
• ✅ Dynamic token management working across different models
• ✅ Accurate cost tracking per model
• ✅ Clean process termination with proper error handling

Checklist

• Changes tested locally
• Code reviewed
• Documentation updated

[AbhishekSinghVerma]

@MonkeyCode-AI review this pr plz

[MonkeyCode-AI]

我是 MonkeyCode AI 编程助手，你可以在 GitHub 任意仓库的任意 Issue 或 PR 中 at @MonkeyCode-AI 来呼唤我。

任务执行细节请参考: 0bebb7b4-af69-11f0-b1be-ee6ce69b85f1

Pull Request 概述

• 这个PR实现了多项重要的功能增强和改进，包括动态模型限制检测、循环检测系统、进度跟踪、中止机制、缓存清理脚本等。
• 修复了多个关键问题，如错误检测过于严格导致的误判、空文件生成、PDF保存位置不正确等。
• 改进了模型感知的令牌管理、成本计算、重试逻辑和错误处理机制。
• 添加了文档分割服务器到代码实现工作流，改进了代理编排引擎中的错误处理和资源清理。

Pull Request 变更详情

• 这个PR添加了新的工具和模块，增强了PDF转换和文件处理功能，改进了UI组件和处理程序，并增强了错误处理和进度跟踪机制。

变更详情

文件路径	变更类型	变更内容
.gitignore	修改	添加了对deepcode_lab、logs、mcp_agent.config.yaml和mcp_agent.secrets.yaml的忽略
CHANGELOG.md	添加	添加了版本更新日志，详细记录了新增功能、修复内容和改进
init.py	修改	更新了版本号和作者信息，添加了项目仓库URL
deepcode.py	修改	改进了LibreOffice检测逻辑，增加了对Windows系统的更好支持
requirements.txt	修改	添加了openapi、openai、PyPDF2和aiohttp依赖
tools/pdf_converter.py	修改	增强了PDF转换功能，包括更好的LibreOffice检测、改进的文件路径管理等
ui/components.py	修改	增强了文件上传和转换组件，添加了缓存机制和进度显示
ui/handlers.py	修改	添加了全局中止标志和中止处理逻辑，增强了进度回调和错误处理
utils/file_processor.py	修改	改进了文件处理逻辑，包括PDF自动转换为Markdown等功能
utils/loop_detector.py	添加	添加了循环检测和超时保护模块
utils/model_limits.py	添加	添加了模型限制和能力检测模块
workflows/agent_orchestration_engine.py	修改	增强了代理编排引擎，包括动态令牌管理、进度跟踪、错误处理等
workflows/code_implementation_workflow.py	修改	增强了代码实现工作流，包括循环检测、进度跟踪、令牌使用和成本计算等
workflows/code_implementation_workflow_index.py	修改	增强了代码实现索引工作流，包括循环检测、进度跟踪、令牌使用和成本计算等

[Jany-M]

@AbhishekSinghVerma ai suggestions checked, nothing to fix.