feat(agent): add ProfilingHook for opt-in iteration timing#3204
feat(agent): add ProfilingHook for opt-in iteration timing#3204mohamed-elkholy95 wants to merge 3 commits intoHKUDS:nightlyfrom
Conversation
Add an AgentHook that logs wall-clock time per LLM iteration and tool batch names at DEBUG level. Enable via NANOBOT_PROFILING=1. The hook is injected at AgentLoop construction time (runtime check, not import-time) so the env var can be set after module import. Also defensively copy the hooks list to prevent caller mutation. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
chengyongru
left a comment
There was a problem hiding this comment.
Review: feat(agent): add ProfilingHook for opt-in iteration timing
结论: Request Changes
架构方向正确——用 hook 做性能分析是干净的做法。以下几点需要修改:
1. import 位置不当(关键问题)
loop.py 中将 import 放在了 __init__ 方法体内:
from nanobot.agent.profiling import ProfilingHook, is_profiling_enabled这是为了避免循环 import 还是有其他原因?如果没有循环依赖问题,应该放到模块顶部。即使有循环依赖问题,也应该用一个 lazy import 函数或 importlib 方案来处理,而不是在 __init__ 里直接 import。
2. list(hooks or []) 的防御性拷贝
self._extra_hooks: list[AgentHook] = list(hooks or [])
这行额外加了一个 list() 拷贝,原代码是 hooks or []。这个改动是合理的(防止外部修改传入的 list),但和 profiling 功能无关。如果需要这个改动,应该单独提交或在 PR 描述中说明。
3. is_profiling_enabled() 只在 __init__ 时检查一次
NANOBOT_PROFILING 环境变量只在 AgentLoop 初始化时读取。如果用户在运行时修改了这个环境变量,不会生效。PR 描述里提到了 "not import-time — so the env var can be set after module import",但实际上它只是从 import-time 变成了 init-time,在 long-running process 中仍然无法动态启用/禁用。
这个设计选择本身可以接受(大多数环境变量都是启动时读取),但描述不够准确。
4. 测试不足
test_after_iteration_runs_without_error和test_before_execute_tools_runs_without_error只验证了"不抛异常",没有验证 hook 的实际行为(比如日志输出内容、时间记录是否合理)- 建议增加一个测试验证
after_iteration实际记录了有意义的时间差(通过 mocktime.perf_counter)
5. 没有在 PR 描述中说明 before_execute_tools 的目的
ProfilingHook 实现了 before_execute_tools,但这个方法只打印 "executing tools: [...]" 的 debug 日志。这个信息和 after_iteration 的日志有重叠(after_iteration 已经打印了 tool_names)。建议说明为什么需要两个日志点,或者考虑移除 before_execute_tools。
Nits
self._iter_t0: float = 0.0— 用0.0作为初始值是合理的,但如果after_iteration在before_iteration之前被调用,会打印一个巨大的负数。虽然不太可能发生,但加个 guard 更安全。
Review: feat(agent): add ProfilingHook for opt-in iteration timingVerdict: Request Changes The architectural direction is correct — using a hook for profiling is a clean approach. A few issues to address: 1. Import placement (important)In from nanobot.agent.profiling import ProfilingHook, is_profiling_enabledIs this to avoid circular imports, or is there another reason? If there's no circular dependency, it should be at the module top level. Even if there is, consider a lazy-import function or 2. Defensive copy unrelated to profilingself._extra_hooks: list[AgentHook] = list(hooks or [])The extra 3.
|
- Move profiling import to module top level (no circular dependency) - Revert unrelated defensive list() copy of hooks - Remove redundant before_execute_tools (overlaps with after_iteration) - Guard _iter_t0 with None sentinel to prevent bogus timing - Fix docstring: clarify env var is read at init-time, not runtime - Add tests that verify actual timing behavior and tool name logging Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Summary
ProfilingHook(nanobot/agent/profiling.py) — anAgentHookthat logs wall-clock time per LLM iteration and tool names at DEBUG levelNANOBOT_PROFILING=1environment variableAgentLoop.__init__time — must be set before loop creation, not dynamically togglable at runtimeChanges after review
list()copy of hooksbefore_execute_tools— redundant withafter_iterationwhich already logs tool names_iter_t0sentinel from0.0toNonewith guard inafter_iterationperf_counterand log output assertionsTest plan
ruff checkclean🤖 Generated with Claude Code