experiment-tracking-swanlab
Tracks ML experiments with SwanLab, an open-source tool covering swanlab.init, config and metric logging, scalar charts, and media logging (images, audio, text, GIFs, point clouds, molecules), in cloud, local (mode="local" with swanlab[dashboard]), or self-hosted setups. Includes integrations for PyTorch, Transformers, PyTorch Lightning, and Fastai. Use when logging metrics and hyperparameters for training runs, comparing runs across seeds, checkpoints, or hyperparameters, running an offline or self-hosted dashboard instead of managed SaaS, logging media alongside scalars, or adding tracking callbacks to a Transformers, Lightning, or Fastai training loop. Not for other trackers such as Weights & Biases or MLflow.
Install this skill
Works with Claude Code, Codex, Cursor, Gemini CLI, GitHub Copilot, OpenCode, and 70+ other harnesses.
npx skills add KalarisLabs/research-agent-skills --skill experiment-tracking-swanlabuvx research-agent-skills install experiment-tracking-swanlabDocumentation & Instructions
SwanLab: Open-Source Experiment Tracking
When to Use This Skill
Use SwanLab when you need to:
- Track ML experiments with metrics, configs, tags, and descriptions
- Visualize training with scalar charts and logged media
- Compare runs across seeds, checkpoints, and hyperparameters
- Work locally or self-hosted instead of depending on managed SaaS
- Integrate with PyTorch, Transformers, PyTorch Lightning, or Fastai
Deployment: Cloud, local, or self-hosted | Media: images, audio, text, GIFs, point clouds, molecules | Integrations: PyTorch, Transformers, PyTorch Lightning, Fastai
Installation
# Install SwanLab plus the media dependencies used in this skill
pip install "swanlab>=0.7.11" "pillow>=9.0.0" "soundfile>=0.12.0"
# Add local dashboard support for mode="local" and swanlab watch
pip install "swanlab[dashboard]>=0.7.11"
# Optional framework integrations
pip install transformers pytorch-lightning fastai
# Login for cloud or self-hosted usage
swanlab login
pillow and soundfile are the media dependencies used by the Image and Audio examples in this skill. swanlab[dashboard] adds the local dashboard dependency required by mode="local" and swanlab watch.
Quick Start
Basic Experiment Tracking
import swanlab
run = swanlab.init(
project="my-project",
experiment_name="baseline",
config={
"learning_rate": 1e-3,
"epochs": 10,
"batch_size": 32,
"model": "resnet18",
},
)
for epoch in range(run.config.epochs):
train_loss = train_epoch()
val_loss = validate()
swanlab.log(
{
"train/loss": train_loss,
"val/loss": val_loss,
"epoch": epoch,
}
)
run.finish()
With PyTorch
import torch
import torch.nn as nn
import torch.optim as optim
import swanlab
run = swanlab.init(
project="pytorch-demo",
experiment_name="mnist-mlp",
config={
"learning_rate": 1e-3,
"batch_size": 64,
"epochs": 10,
"hidden_size": 128,
},
)
model = nn.Sequential(
nn.Flatten(),
nn.Linear(28 * 28, run.config.hidden_size),
nn.ReLU(),
nn.Linear(run.config.hidden_size, 10),
)
optimizer = optim.Adam(model.parameters(), lr=run.config.learning_rate)
criterion = nn.CrossEntropyLoss()
for epoch in range(run.config.epochs):
model.train()
for batch_idx, (data, target) in enumerate(train_loader):
optimizer.zero_grad()
logits = model(data)
loss = criterion(logits, target)
loss.backward()
optimizer.step()
if batch_idx % 100 == 0:
swanlab.log(
{
"train/loss": loss.item(),
"train/epoch": epoch,
"train/batch": batch_idx,
}
)
run.finish()
Core Concepts
1. Projects and Experiments
Project: Collection of related experiments
Experiment: Single execution of a training or evaluation workflow
import swanlab
run = swanlab.init(
project="image-classification",
experiment_name="resnet18-seed42",
description="Baseline run on ImageNet subset",
tags=["baseline", "resnet18"],
config={
"model": "resnet18",
"seed": 42,
"batch_size": 64,
"learning_rate": 3e-4,
},
)
print(run.id)
print(run.config.learning_rate)
2. Configuration Tracking
config = {
"model": "resnet18",
"seed": 42,
"batch_size": 64,
"learning_rate": 3e-4,
"epochs": 20,
}
run = swanlab.init(project="my-project", config=config)
learning_rate = run.config.learning_rate
batch_size = run.config.batch_size
3. Metric Logging
# Log scalars
swanlab.log({"loss": 0.42, "accuracy": 0.91})
# Log multiple metrics
swanlab.log(
{
"train/loss": train_loss,
"train/accuracy": train_acc,
"val/loss": val_loss,
"val/accuracy": val_acc,
"lr": current_lr,
"epoch": epoch,
}
)
# Log with custom step
swanlab.log({"loss": loss}, step=global_step)
4. Media and Chart Logging
import numpy as np
import swanlab
# Image
image = np.random.randint(0, 255, (224, 224, 3), dtype=np.uint8)
swanlab.log({"examples/image": swanlab.Image(image, caption="Augmented sample")})
# Audio
wave = np.sin(np.linspace(0, 8 * np.pi, 16000)).astype("float32")
swanlab.log({"examples/audio": swanlab.Audio(wave, sample_rate=16000)})
# Text
swanlab.log({"examples/text": swanlab.Text("Training notes for this run.")})
# GIF video
swanlab.log({"examples/video": swanlab.Video("predictions.gif", caption="Validation rollout")})
# Point cloud
points = np.random.rand(128, 3).astype("float32")
swanlab.log({"examples/point_cloud": swanlab.Object3D(points, caption="Point cloud sample")})
# Molecule
swanlab.log({"examples/molecule": swanlab.Molecule.from_smiles("CCO", caption="Ethanol")})
# Custom chart with swanlab.echarts
line = swanlab.echarts.Line()
line.add_xaxis(["epoch-1", "epoch-2", "epoch-3"])
line.add_yaxis("train/loss", [0.92, 0.61, 0.44])
line.set_global_opts(
title_opts=swanlab.echarts.options.TitleOpts(title="Training Loss")
)
swanlab.log({"charts/loss_curve": line})
See references/visualization.md for more chart and media patterns.
5. Local and Self-Hosted Workflows
import os
import swanlab
# Self-hosted or cloud login
swanlab.login(
api_key=os.environ["SWANLAB_API_KEY"],
host="http://your-server:5092",
)
# Local-only logging
run = swanlab.init(
project="offline-demo",
mode="local",
logdir="./swanlog",
)
swanlab.log({"loss": 0.35, "epoch": 1})
run.finish()
# View local logs
swanlab watch -l ./swanlog
# Sync local logs later
swanlab sync ./swanlog
Integration Examples
HuggingFace Transformers
from transformers import Trainer, TrainingArguments
training_args = TrainingArguments(
output_dir="./results",
per_device_train_batch_size=8,
evaluation_strategy="epoch",
logging_steps=50,
report_to="swanlab",
run_name="bert-finetune",
)
trainer = Trainer(
model=model,
args=training_args,
train_dataset=train_dataset,
eval_dataset=eval_dataset,
)
trainer.train()
See references/integrations.md for callback-based setups and additional framework patterns.
PyTorch Lightning
import pytorch_lightning as pl
from swanlab.integration.pytorch_lightning import SwanLabLogger
swanlab_logger = SwanLabLogger(
project="lightning-demo",
experiment_name="mnist-classifier",
config={"batch_size": 64, "max_epochs": 10},
)
trainer = pl.Trainer(
logger=swanlab_logger,
max_epochs=10,
accelerator="auto",
)
trainer.fit(model, train_loader, val_loader)
Fastai
from fastai.vision.all import accuracy, resnet34, vision_learner
from swanlab.integration.fastai import SwanLabCallback
learn = vision_learner(dls, resnet34, metrics=accuracy)
learn.fit(
5,
cbs=[
SwanLabCallback(
project="fastai-demo",
experiment_name="pets-classification",
config={"arch": "resnet34", "epochs": 5},
)
],
)
See references/integrations.md for fuller framework examples.
Best Practices
1. Use Stable Metric Names
# Good: grouped metric namespaces
swanlab.log({
"train/loss": train_loss,
"train/accuracy": train_acc,
"val/loss": val_loss,
"val/accuracy": val_acc,
})
# Avoid mixing flat and grouped names for the same metric family
2. Initialize Early and Capture Config Once
run = swanlab.init(
project="image-classification",
experiment_name="resnet18-baseline",
config={
"model": "resnet18",
"learning_rate": 3e-4,
"batch_size": 64,
"seed": 42,
},
)
3. Save Checkpoints Locally
import torch
import swanlab
checkpoint_path = "checkpoints/best.pth"
torch.save(model.state_dict(), checkpoint_path)
swanlab.log(
{
"best/val_accuracy": best_val_accuracy,
"artifacts/checkpoint_path": swanlab.Text(checkpoint_path),
}
)
4. Use Local Mode for Offline-First Workflows
run = swanlab.init(project="offline-demo", mode="local", logdir="./swanlog")
# ... training code ...
run.finish()
# Inspect later with: swanlab watch -l ./swanlog
5. Keep Advanced Patterns in References
- Use references/visualization.md for advanced chart and media patterns
- Use references/integrations.md for callback-based and framework-specific integration details
Resources
See Also
- references/integrations.md - Framework-specific examples
- references/visualization.md - Charts and media logging patterns
Agent operating procedure
- Check the environment. Confirm hardware, framework and server versions, model format, and expected load.
- Pin down the inputs. Confirm formats, identifiers and parameters from the data or the user. Ask rather than guess any value that changes the result.
- Run a small version first. Serve or log a single request or run end to end before scaling.
- Execute the full task using the instructions and references above.
- Validate the result. Measure latency, throughput and output correctness against a reference; check resource usage and costs.
- Report. State what was run (versions, commands, parameters), what was checked, and what is still uncertain.
| If this happens | Do this |
|---|---|
| The server fails to start or OOMs | Check model size versus memory, quantization and parallelism settings. |
| A function, flag or endpoint in these instructions is missing in the installed version | Check the installed version's own documentation (help(), --help, official docs), adapt, and tell the user. Never invent an API. |
| A required input, identifier or parameter is ambiguous | Ask the user, or state the assumption explicitly before running. |
Integrity rules
- Never fabricate results, parameters, identifiers, citations or statistics. If something cannot be run or verified, say so plainly.
- Do not expose services or credentials publicly; confirm cloud costs before provisioning.
- Treat version-specific details here as possibly outdated: confirm them against the official documentation for the installed version.
- Ask before actions that cost money, consume shared GPUs or cloud quota, touch personal or patient data, or cannot be undone.
Trigger Prompts & Evaluation Corpus
These evaluation prompts test that the agent correctly selects experiment-tracking-swanlab for relevant tasks and leaves it out for near misses.
Should trigger (3)
- should trigger
“How do I log loss and accuracy per epoch to SwanLab?”
expects Shows swanlab.init with project/experiment name, then swanlab.log({"loss":..., "accuracy":...}) inside the epoch loop, using stable metric names and ending/finishing the run. Notes that the swanlab package must be installed.
- should trigger
“I'm fine-tuning bert-base-uncased on SST-2 with a HuggingFace Trainer in train_sst2.py. I want the loss curves and my learning rate / batch size / seed recorded so I can compare three seeds side by side. The cluster has no internet, so it has to work offline. How should I set this up?”
expects Uses the SwanLab Transformers callback (or report_to integration) passed to Trainer, calls swanlab.init with config containing lr, batch size and seed, and uses mode="local" with swanlab[dashboard] installed for offline viewing (swanlab watch). Explains comparing the three seed runs in the dashboard, and says to verify API details against the installed version's docs.
- should trigger
“My lab doesn't want training data going to a third-party SaaS. I need something open-source that I can host on our own server to see training curves and also look at the generated images and audio samples from my GAN next to the loss plots. What should I use and how do I wire it into my PyTorch loop?”
expects Recommends SwanLab in self-hosted or local mode, shows swanlab.init and swanlab.log with swanlab.Image and swanlab.Audio alongside scalar metrics in the PyTorch loop, mentions installing pillow/soundfile, and does not send data to a managed cloud without confirming.
Near misses (2)
- near miss
“Add wandb.init and wandb.log to my training script, and set up a hyperparameter sweep with a W&B sweep config.”
expects Should not use the SwanLab skill; this is Weights & Biases work including sweeps, so the weights-and-biases skill applies.
- near miss
“Write a PyTorch Lightning LightningModule and Trainer with ModelCheckpoint and EarlyStopping for my image classifier, using DDP on 4 GPUs. No experiment tracking needed.”
expects Should not use the SwanLab skill; this is about structuring Lightning training code, callbacks and multi-GPU strategies, which belongs to the pytorch-lightning skill.