todo.md

To Do

A list of everything I need to do for a release.

Code

• Guh

Preprint

• Experiments
• Writing

Release

• Interactive demos
• Module docstrings
• Class docstrings
• Method docstrings
• Function docstrings

Preprint - Experiments

• DINOv2 vs CLIP
• Image classification control
• Image segmentation control

Experiments - Understanding DINOv2 vs CLIP

• Compute ImageNet-1K train activations for DINOv2 ViT-B/14
• Compute ImageNet-1K train activations for CLIP ViT-B/16
• Train SAE on patch-level activations of ImageNet-1K train from DINOv2 ViT-B/14
• Train SAE on patch-level activations of ImageNet-1K train from CLIP ViT-B/16
• Visualize features for DINOv2
• Visualize features for CLIP
• Find something neat.

Experiments - Image Classification Control

• Train SAE on [CLS] activations of ImageNet-1K train from CLIP ViT-B/16
• Compute Caltech-101 train activations for CLIP ViT-B/16
• Compute Caltech-101 test activations for CLIP ViT-B/16
• Train linear probe for Caltech-101 classification
• Calculate 99th percentile of feature activation for each feature.
• Develop interactive Marimo dashboard
• Find something neat.
• Calculate logit relationship

Experiments - Image Segmentation Control

• Train SAE on patch-level activations of ImageNet-1K train from DINOv2 ViT-B/14
• Compute ADE20K train activations for DINOv2 ViT-B/14
• Compute ADE20K validation activations for DINOv2 ViT-B/14
• Train linear probe for ADE20K semantic segmentation (checkpoints/contrib/semseg/lr_0_001__wd_0_001/model_step8000.pt)
• What percentage of patches meet the 90% threshold?
• Develop interactive Marimo dashboard
• Find something neat.
• Quantitative results