-
Notifications
You must be signed in to change notification settings - Fork 240
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Log locally with tensorboard when wandb is disabled. #154
base: main
Are you sure you want to change the base?
Log locally with tensorboard when wandb is disabled. #154
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, we should probably add option for tensorboard logging as it is quite used.
Anyone knows what is the usage for these kind of logging/viz tools in the community?
Is wandb more popular than tensorboard? What are the alternatives?
lerobot/scripts/train.py
Outdated
@@ -349,8 +349,7 @@ def evaluate_and_checkpoint_if_needed(step): | |||
start_seed=cfg.seed, | |||
) | |||
log_eval_info(logger, eval_info["aggregated"], step, cfg, offline_dataset, is_offline) | |||
if cfg.wandb.enable: | |||
logger.log_video(eval_info["video_paths"][0], step, mode="eval") | |||
logger.log_video(eval_info["video_paths"][0], step, mode="eval") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why not add a cfg.tensorboard.enable
argument to support both?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good idea! Will do.
Both are quite popular and could be preferable in different scenarios I think. A lot of people prefer Tensorboard (including myself) because of how little overhead there is to spin up and start using it, especially when you just want to visualize loss curves and maybe some input/output samples. Even some teams who don't mind managing some infra use it (my previous ML team had Tensorboard instances hosted on K8s backed by S3 storage for artifacts). Wandb of course is a managed platform with more sophisticated workflows around tracking, hosting, sharing results, etc., but comes at a cost (although the free plan is pretty generous for small-scale experimentation). In any case, think it would be great to have Tensorboard support! |
What this does
This PR adds support for locally logging metrics and videos using tensorboard when wandb is disabled.
How it was tested
How to checkout & try? (for the reviewer)
Run with wandb.enable=false and short eval / log freqs. e.g.
python lerobot/scripts/train.py policy=diffusion env=pusht env.task=PushT-v0 dataset_repo_id=lerobot/pusht training.log_freq=10 training.eval_freq=100 wandb.enable=false tensorboard.enable=true
This change is