refactor(forge): Clean forge

[kcze]

User description

Background

Should be merged after:

• refactor(agent, forge): Move library code from autogpt to forge #7106
• Move autogpt and forge to repo root #7151

Follow-up after Component-based Agents

Changes 🏗️

TL;DR: remove unused forge code and improve structure of forge.

• Put all Agent Protocol stuff together in forge.agent_protocol
  • ... including forge.agent_protocol.database (was forge.db)
• Remove duplicate/unused parts from forge
  • forge.actions, containing old commands; replaced by forge.components from autogpt
  • forge/agent.py (the old one, ForgeAgent)
  • forge/app.py, which was used to serve and run the ForgeAgent
  • forge/db.py (ForgeDatabase), which was used for ForgeAgent
  • forge/llm.py, which has been replaced by new forge.llm module which was ported from autogpt.core.resource.model_providers
  • forge.memory, which is not in use and not being maintained
  • forge.sdk, much of which was moved into other modules and the rest is deprecated
  • AccessDeniedError: unused
  • forge_log.py: replaced with logging
  • validate_yaml_file: not needed
  • ai_settings_file and associated loading logic and env var AI_SETTINGS_FILE: unused
  • prompt_settings_file and associated loading logic and env var PROMPT_SETTINGS_FILE: default directives are now provided by the SystemComponent
    • request_user_double_check, which was only used in AIDirectives.load
  • TypingConsoleHandler: not used

PR Quality Scorecard ✨

• Have you used the PR description template?   +2 pts
• Is your pull request atomic, focusing on a single change?   +5 pts
• Have you linked the GitHub issue(s) that this PR addresses?   +5 pts
• Have you documented your changes clearly and comprehensively?   +5 pts
• Have you changed or added a feature?   -4 pts
  • Have you added/updated corresponding documentation?   +4 pts
  • Have you added/updated corresponding integration tests?   +5 pts
• Have you changed the behavior of AutoGPT?   -5 pts
  • Have you also run agbenchmark to verify that these changes do not regress performance?   +10 pts

---

PR Type

enhancement, bug fix, tests

---

Description

• Refactored the forge module to improve structure and remove unused code.
• Consolidated all Agent Protocol related code into forge.agent_protocol.
• Removed duplicate and deprecated modules such as forge.actions, forge/agent.py, forge/app.py, forge/db.py, forge/llm.py, forge/memory, forge/sdk, AccessDeniedError, forge_log.py, and validate_yaml_file.
• Updated import paths across various modules to reflect the new structure.
• Replaced custom logging with standard logging module.
• Removed YAML validation logic from multiple files.
• Updated and removed related test cases.

---

Changes walkthrough 📝

	Relevant files
Enhancement	13 files agent_protocol_server.py Refactor imports and remove unused validation in agent_protocol_server. autogpts/autogpt/autogpt/app/agent_protocol_server.py Updated imports to reflect new module structure. Removed unused import validate_yaml_file. +9/-9      configurator.py Remove YAML validation and update imports in `configurator`. autogpts/autogpt/autogpt/app/configurator.py Removed YAML file validation logic. Updated import statements. +2/-21    main.py Update import paths and remove redundant import in `main`. autogpts/autogpt/autogpt/app/main.py Updated import paths for AgentDB. Removed redundant import. +2/-2      utils.py Remove YAML validation function in `utils`.                            autogpts/autogpt/autogpt/app/utils.py Removed validate_yaml_file import and related function. +1/-5      __main__.py Replace custom logger with standard logging in `__main__.py`. autogpts/forge/forge/main.py Replaced custom logger with standard logging. +3/-5      api_router.py Update imports and replace custom logger in `api_router`. autogpts/forge/forge/agent_protocol/api_router.py Updated import paths. Replaced custom logger with standard logging. +25/-25  db.py Update imports and replace custom logger in `db`.                autogpts/forge/forge/agent_protocol/database/db.py Updated import paths. Replaced custom logger with standard logging. +51/-47  task.py Refactor models and rename `Status` to `StepStatus`.          autogpts/forge/forge/agent_protocol/models/task.py Moved Artifact and Pagination models to separate files. Renamed Status to StepStatus. +4/-60    middlewares.py Simplify import statements in `middlewares`.                          autogpts/forge/forge/agent_protocol/middlewares.py Simplified import statements. +2/-7      parameter.py Replace `dataclasses` with `pydantic.BaseModel` in `parameter`. autogpts/forge/forge/command/parameter.py Replaced dataclasses with pydantic.BaseModel. +2/-3      code_executor.py Add `CodeExecutionError` class in `code_executor`.              autogpts/forge/forge/components/code_executor/code_executor.py Added CodeExecutionError class. +4/-1      ai_directives.py Remove YAML validation logic in `ai_directives`.                  autogpts/forge/forge/config/ai_directives.py Removed YAML validation logic. +0/-10    ai_profile.py Remove YAML file handling methods in `ai_profile`.              autogpts/forge/forge/config/ai_profile.py Removed methods for loading and saving AI profiles from YAML files. +0/-50
Tests	2 files test_ai_profile.py Remove `AIProfile` related test cases.                                      autogpts/autogpt/tests/unit/test_ai_profile.py Removed test cases related to AIProfile class. +0/-71    test_utils.py Remove `validate_yaml_file` test cases.                                    autogpts/autogpt/tests/unit/test_utils.py Removed test cases for validate_yaml_file. +0/-36

---

💡 PR-Agent usage:
Comment /help on the PR to get a list of all available PR-Agent tools and their descriptions

[github-actions]

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

[netlify]

✅ Deploy Preview for auto-gpt-docs canceled.

Name	Link
🔨 Latest commit	e401d14
🔍 Latest deploy log	https://app.netlify.com/sites/auto-gpt-docs/deploys/664ce4823cb63d0008c87775

[codiumai-pr-agent-pro]

PR-Agent was enabled for this repository. To continue using it, please link your git user with your CodiumAI identity here.

PR Description updated to latest commit (1999bfb)

• Copy walkthrough table to "Files Changed" Tab

[codiumai-pr-agent-pro]

PR-Agent was enabled for this repository. To continue using it, please link your git user with your CodiumAI identity here.

PR Review 🔍

⏱️ Estimated effort to review [1-5]	4, due to the extensive changes across multiple files, including refactoring, removal of unused code, and reorganization of modules. The PR touches on various aspects of the system, from database interactions to API routing, which requires a thorough understanding of the overall architecture and the specific functionalities being modified.
🧪 Relevant tests	No
⚡ Possible issues	Possible Bug: The removal of extensive test cases for AI profiles and utilities without corresponding new tests for the refactored or newly added functionalities might leave parts of the codebase untested, potentially leading to undetected bugs.
	Performance Concern: The restructuring and renaming of modules and methods could lead to confusion and errors if not properly documented and communicated to all team members, potentially affecting the development velocity and increasing the risk of bugs in future changes.
🔒 Security concerns	No

[codiumai-pr-agent-pro]

PR-Agent was enabled for this repository. To continue using it, please link your git user with your CodiumAI identity here.

PR Code Suggestions ✨

Category	Suggestion	Score
Possible bug	Add a check to ensure task_obj is not None before accessing its attributes Consider adding a check to ensure that task_obj is not None before accessing its attributes in the convert_to_task function. This will prevent potential AttributeError exceptions if task_obj is unexpectedly None. autogpts/forge/forge/agent_protocol/database/db.py [92-93] +if task_obj is None: + logger.error("task_obj is None in convert_to_task") + raise ValueError("task_obj cannot be None") logger.debug(f"Converting TaskModel to Task for task_id: {task_obj.task_id}") task_artifacts = [convert_to_artifact(artifact) for artifact in task_obj.artifacts] Suggestion importance[1-10]: 8 Why: This suggestion correctly identifies a potential bug where task_obj could be None, which would lead to an AttributeError when trying to access its attributes. Adding a check enhances robustness and prevents runtime errors.	8
Security	Sanitize the task_request before logging to prevent exposure of sensitive information To avoid potential security issues, consider sanitizing the task_request before logging it. This will prevent any sensitive information from being exposed in the logs. autogpts/forge/forge/agent_protocol/api_router.py [106] -logger.exception(f"Error whilst trying to create a task: {task_request}") +sanitized_task_request = sanitize_task_request(task_request) +logger.exception(f"Error whilst trying to create a task: {sanitized_task_request}") Suggestion importance[1-10]: 8 Why: This suggestion correctly identifies a security risk related to logging sensitive data and proposes a practical solution to sanitize the data before logging.	8
Possible issue	Add exception handling for the write_file method in the create_artifact function The write_file method in the create_artifact function should handle potential exceptions to ensure robustness. autogpts/forge/forge/agent/agent.py [189] -await self.workspace.write_file(file_path, data) +try: + await self.workspace.write_file(file_path, data) +except Exception as e: + logger.error(f"Failed to write file {file_path}: {e}") + raise Suggestion importance[1-10]: 8 Why: Proper exception handling for file operations is crucial to prevent crashes and ensure robust error management in file I/O operations.	8
	Add exception handling for the read_file method in the get_artifact function The read_file method in the get_artifact function should handle potential exceptions to ensure robustness. autogpts/forge/forge/agent/agent.py [211] -retrieved_artifact = self.workspace.read_file(file_path) +try: + retrieved_artifact = self.workspace.read_file(file_path) +except Exception as e: + logger.error(f"Failed to read file {file_path}: {e}") + raise Suggestion importance[1-10]: 8 Why: Similar to writing, reading operations can fail due to various reasons, and handling these exceptions is essential for robustness and error transparency.	8

[Pwuts]

With this usage gone, iirc we can remove request_user_double_check from forge. Maybe it's still used in autogpt but then it should be defined there.

[kcze]

When I remove request_user_double_check, then user_friendly_output is also not used, so removing 🧹
Also moved print_attribute and speak to logging/utils.py, so I could remove logging/helpers.py.

[Pwuts]

if user_friendly_output is not used, then the typing_console_handler and user_friendly_output_logger also are not used afaik :)