Skip to content

Commit

Permalink
Add demo figure
Browse files Browse the repository at this point in the history
Signed-off-by: heyufan1995 <[email protected]>
  • Loading branch information
heyufan1995 committed Jul 24, 2024
1 parent 3aaa4e4 commit adba474
Show file tree
Hide file tree
Showing 5 changed files with 31 additions and 2 deletions.
33 changes: 31 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,15 +21,44 @@ The **VISTA3D** is a foundation model trained systematically on 11,454 volumes e
### Out-of box automatic segmentation
For supported 127 classes, the model can perform highly accurate out-of-box segmentation. The fully automated process adopts a patch-based sliding-window inference and only requires a class prompt.
Compared to supervised segmentation models trained on each dataset separately, VISTA3D showed comparable out-of-box performances and strong generalizability ('VISTA3D auto' in Table.1).
<!-- <div align="center"> <img src="" width="800"/> </div> -->
<!-- <div align="center"> <img src="assets/imgs/everything.gif" width="800"/> </div> -->
<div align="center">
<figure>
<img
src="assets/imgs/everything.gif">
<figcaption> NIM Demo supports "Segment Everything" </figcaption>
</figure>
</div>



### Interactive editing
The interactive segmentation is based on user-provided clicks. Each click point will impact a local 3D patch. User can either effectively refine the automatic results with clicks ('VISTA3D auto+point' in Table.1) or simply provide a click without specifying the target class ('VISTA3D point' in Table.1) .
<!-- <div align="center"> <img src="" width="800"/> </div> -->

<div align="center">
<figure>
<img
src="assets/imgs/liver.gif">
<figcaption> Specify a supported class and edit the automatic results </figcaption>
</figure>
</div>
<div align="center">
<figure>
<img
src="assets/imgs/unspecified.gif">
<figcaption> Interactive supported class segmentation without specifying class </figcaption>
</figure>
</div>
### Zero-shot interactive segmentation
VISTA3D is built to produce visually plausible segmentations on previously unseen classes.
This capability makes the model even more flexible and accelerates practical segmentation data curation processes.
<div align="center">
<figure>
<img
src="assets/imgs/zeroshot.gif">
<figcaption> Add a new unseen class and do annotation </figcaption>
</figure>
</div>

### Fine-tuning
VISTA3D checkpoint showed improvements when finetuning in few-shot settings. Once a few annotated examples are provided, user can start finetune with the VISTA3D checkpoint.
Expand Down
Binary file added assets/imgs/everything.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/imgs/liver.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/imgs/unspecified.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/imgs/zeroshot.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit adba474

Please sign in to comment.