update readme and names

.gitignore

@@ -1,5 +1,4 @@
 **/*.pyc
-features/**/*.csv
 rasters/**/*.aux.xml
 .DS_Store
 .vscode/settings.json

README.md

@@ -6,13 +6,13 @@ By Joey and [Fancy](https://github.com/Fanchengyan) from [Cryosphere Lab](https:
 
 Geo SAM is a QGIS plugin tool that aims to help people segment, delineate or label landforms efficiently when using large-size geospatial raster images. [Segment Anything Model](https://segment-anything.com/) (SAM) is a foundation AI model with the superpower, but the model size is huge, and using it to process images can take a long time, even with a modern GPU. Our tool uses the strategies of encoding image features in advance and trimming the SAM model. The interactive segmentation process can be run in real-time on a laptop by only using a CPU, making it a convenient and efficient tool for dealing with satellite images.
 
-The tool currently only supports preprocessed images (whose features have been generated in advance using a separate program, as the included demo image). We are now building another tool for encoding image features inside QGIS, which will soon be available. So stay tuned.
+The Geo SAM plugin includes two separate tools, the encoder tool for preprocessing (encoding) images and segmentation tool for interactively segmenting landforms. The encoder tool is designed to generate and save the image features using the SAM image encoder, and the encoding process only runs once per image. The segmentation tool can only be used to segment preprocessed images (whose features have been generated in advance using the encoder tool, as the included demo image).
 
 ## Installation
 
 ### Install QGIS
 
-You are suggested to install the latest version of [QGIS](https://www.qgis.org/en/site/forusers/download.html) since the plugin has only been tested on the versions newer than QGIS 3.30 (at least ver. 3.28 is recommended).
+You are suggested to install the latest version of [QGIS](https://www.qgis.org/en/site/forusers/download.html) since the plugin has only been tested on the versions newer than QGIS 3.30.
 
 ### Install Library Dependencies
 
@@ -28,15 +28,21 @@ You are suggested to install the latest version of [QGIS](https://www.qgis.org/e
 Open the `OSGeo4W Shell` application from the Start menu, which is a dedicated shell for the QGIS. Then run the following command to install the libraries.
 
 ```bash
-pip3 install torch==1.13.1 torchvision==0.14.1
+pip3 install torch torchvision
 pip3 install torchgeo
 pip3 install segment-anything
-pip3 install rasterio==1.3.7
+```
+
+Our encoder tool now supports using CUDA GPU to accelerate the encoding process. If your PC has dedicated CUDA GPUs, you can install CUDA library first and then install the gpu-version pytorch using following command (using CUDA version 11.7 as an example):
+
+```bash
+# add `--force-reinstall` if you installed the cpu version before.
+pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117
 ```
 
 #### For Mac or Linux Users
 
-Open your own terminal application, and change the directory to the QGIS Python environment.
+Open your own terminal application, and change the directory to where the QGIS Python binary file locates.
 
 ```bash
 # Mac
@@ -57,10 +63,9 @@ Then install the libraries.
 
 ```bash
 # add ./ to avoid using your default Python in the system
-./pip3 install torch==1.13.1 torchvision==0.14.1
+./pip3 install torch torchvision
 ./pip3 install torchgeo
 ./pip3 install segment-anything
-./pip3 install rasterio==1.3.7
 ```
 
 For Linux users, if `pip3` is not found in `/usr/bin`, try the following commands:
@@ -70,10 +75,11 @@ sudo apt-get update
 sudo apt-get install python3-pip
 ```
 
+For Linux users, if your computer got available CUDA GPUs and with CUDA library installed, the above commands should have helped you installed the gpu-version pytorch. You can reach to pytorch official website for more information.
 
 ### Install the GeoSAM Plugin
 
-Download the [plugin zip file](https://github.com/coolzhao/Geo-SAM/releases/latest), unzip it, and put the `Geo-SAM` folder (please remove the version suffix of the folder to avoid potential path issues, be aware of undesired nested folders after unzipping) into the QGIS plugin folder, then restart QGIS if it's open already.
+Download the [plugin zip file](https://github.com/coolzhao/Geo-SAM/archive/refs/heads/dev.zip), unzip it, and put the `Geo-SAM` folder (please remove the version suffix of the folder to avoid potential path issues, be aware of undesired nested folders after unzipping) into the QGIS plugin folder, then restart QGIS if it's open already.
 
 #### How to Locate the QGIS Plugin folder
 
@@ -98,21 +104,21 @@ After restarting QGIS, you may go to the `Plugins` menu, select `Manage and Inst
 
 #### Find the Geo SAM Tool
 
-After activating the Geo SAM plugin, you may find the tool under the `Plugins` menu,
+After activating the Geo SAM plugin, you may find the Geo SAM tool under the `Plugins` menu,
 
 <p align="center">
   <img src="assets/Plugin_menu_geo_sam.png" width="350" title="Plugin menu">
 </p>
 
-Or somewhere on the toolbar near the Python Plugin.
+You may also find a new toolbar including two icons.
 
 <p align="center">
-  <img src="assets/Toolbar_geo_sam.png" width="350" title="Plugin toolbar">
+  <img src="assets/Toolbar_geo_sam.png" width="200" title="Plugin toolbar">
 </p>
 
-## Use the GeoSAM Tool
+## Use the Geo SAM Segmentation Tool
 
-Click the toolbar icon to open the widget of the tool. You will be shown a demo raster image with thaw slump and small pond landforms for you to try the tool. With a single click on the map, a segmentation result will be generated.
+Click the segmentation tool icon to open the interactive segmentation widget. You will be shown a demo raster image with thaw slump and small pond landforms for you to try the tool. With a single click on the map, a segmentation result will be generated.
 
 <!-- ![try geo sam](assets/try_geo_sam.png) -->
 
@@ -130,11 +136,11 @@ A user interface will be shown below.
 
 ### Add Points
 
-Click the buttons to select between the `Foreground` and `Background` points. Use `Foreground` points to add areas you desire, and use `Background` points to remove areas you don't want.
+Click the buttons to select between the `Foreground(FG)` and `Background(BG)` points. Use `FG` points to add areas you desire, and use `BG` points to remove areas you don't want.
 
 ### Add Bounding Box (BBox)
 
-Click the `Rectangle` button to activate the BBox tool to draw a rectangle on the map for segmenting a subject.
+Click the `BBox` button to activate the BBox tool to draw a rectangle on the map for segmenting a subject.
 The BBox tool can be used together with adding points or independently.
 
 ### Save Current Results
@@ -145,51 +151,55 @@ After adding points and a rectangle for segmenting a subject, you can save the s
 
 You can use the `Clear` button to clear the added points and rectangles.
 
-### Undo the last Prompt
+### Undo the Last Prompt
 
-You can use the `Undo` button to undo the last points or rectangle Prompt.
+You can use the `Undo` button to undo the last point or rectangle Prompt.
 
-### Disable the Tool
+### Enable/Disable the Tool
 
 You can uncheck the `Enable` button to temporally disable the tool and navigate on the map.
 
 ### Load Selected Image Features
 
-The plugin is initialized with features for demo purposes, and you can use the `Feature Folder` selection tool to select the folder that includes the image features you need.
+The plugin is initialized with features for demo purposes, and you can use the `Feature Folder` selection button to select the folder that includes the image features you need.
 
 <p align="center">
-  <img src="assets/Select_feature_folder.png" width="300" title="Select feature folder">
+  <img src="assets/Select_feature_folder.png" width="250" title="Select feature folder">
 </p>
 
-After selecting the feature folder, you may press the `Load` button to load the features, and it may take several seconds when you load the folder for the first time. Remember to add the corresponding raster image to the QGIS project.
-
-<p align="center">
-  <img src="assets/Load_image_feature.png" width="300" title="Load feature folder">
-</p>
+Then, press the `Load` button to load the selected image features. Remember to add the corresponding raster image to the QGIS project.
 
-## Shortcuts
+### Shortcuts
 
-- `Tab`: loop between 3 prompt types (the cursor will also change to the corresponding types):
-  - Foreground Point
-  - Background Point
-  - Rectangle/BBox
+- `Tab`: loop between 3 prompt types (the cursor will also change to the corresponding types)
 - `C`: clear all prompts in canvas [same as `Clear` button]
 - `Z`: undo the last prompt in canvas [same as `Undo` button]
 - `S`: save SAM output features into polygon [same as `Save` button]
-- `Ctrl+Z` or `command+Z`: undo the last saved features of SAM output
+- `Ctrl+Z` or `command+Z`: undo the last saved segmentation results
 
-
-## Tips for Using GeoSAM Tool
+### Tips for Using Geo-SAM Segmentation Tool
 
 - Deal with only **One Subject** each time
 - Use **Background Points** to exclude unwanted parts
 - Use **Bounding Box (BBox)** to limit the segment polygon boundary
 - The **BBox** should cover the entire subject
 - Remember to press the `Save` button after the segmentation of the chosen subject
 
-## In Progress
+## Use the Geo SAM Encoder Tool
+
+If you want to try your own images, you can use the Encoder Tool to preprocess the images. You need to download the SAM checkpoints in advance using the following links.
+
+- `vit_h`: [ViT-H SAM model.](https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth)
+- `vit_l`: [ViT-L SAM model.](https://dl.fbaipublicfiles.com/segment_anything/sam_vit_l_0b3195.pth)
+- `vit_b`: [ViT-B SAM model.](https://dl.fbaipublicfiles.com/segment_anything/sam_vit_b_01ec64.pth)
+
+The following animation shows how to use the encoder tool.
+
+<p align="center">
+  <img src="assets/encoder_demo.gif" width="600" title="Try Geo SAM">
+</p>
 
-- Image encoder module
+After processing the image, by default, the generated features will automatically be loaded in the segmentation tool for you to start labeling. Or you can choose to load the features manually afterwards.
 
 ## Future Works
 

features/beiluhe_google_img_201211_clip/beiluhe_google_img_201211_clip.csv

@@ -0,0 +1,13 @@
+,id,minx,maxx,miny,maxy,mint,maxt,filepath,crs,res
+0,0,471407.97,472431.9,3883365.2,3884389.0,0.0,9.223372e+18,sam_features_vit_h_beiluhe_google_img_201211_utm_new_export_pyramid_clip_471407.97_3883365.16_472431.91_3884389.10.tif,EPSG:32646,15.999032848232673
+1,1,471919.94,472943.88,3883186.2,3884210.0,0.0,9.223372e+18,sam_features_vit_h_beiluhe_google_img_201211_utm_new_export_pyramid_clip_471919.94_3883186.17_472943.88_3884210.11.tif,EPSG:32646,15.999032848232673
+2,2,472307.9,473331.84,3883186.2,3884210.0,0.0,9.223372e+18,sam_features_vit_h_beiluhe_google_img_201211_utm_new_export_pyramid_clip_472307.92_3883186.17_473331.85_3884210.11.tif,EPSG:32646,15.999032848232673
+3,3,471407.97,472431.9,3882674.2,3883698.2,0.0,9.223372e+18,sam_features_vit_h_beiluhe_google_img_201211_utm_new_export_pyramid_clip_471407.97_3882674.20_472431.91_3883698.14.tif,EPSG:32646,15.999032848232673
+4,4,471407.97,472431.9,3882162.2,3883186.2,0.0,9.223372e+18,sam_features_vit_h_beiluhe_google_img_201211_utm_new_export_pyramid_clip_471407.97_3882162.24_472431.91_3883186.17.tif,EPSG:32646,15.999032848232673
+5,5,472307.9,473331.84,3882162.2,3883186.2,0.0,9.223372e+18,sam_features_vit_h_beiluhe_google_img_201211_utm_new_export_pyramid_clip_472307.92_3882162.24_473331.85_3883186.17.tif,EPSG:32646,15.999032848232673
+6,6,471919.94,472943.88,3882162.2,3883186.2,0.0,9.223372e+18,sam_features_vit_h_beiluhe_google_img_201211_utm_new_export_pyramid_clip_471919.94_3882162.24_472943.88_3883186.17.tif,EPSG:32646,15.999032848232673
+7,7,471407.97,472431.9,3883186.2,3884210.0,0.0,9.223372e+18,sam_features_vit_h_beiluhe_google_img_201211_utm_new_export_pyramid_clip_471407.97_3883186.17_472431.91_3884210.11.tif,EPSG:32646,15.999032848232673
+8,8,472307.9,473331.84,3883365.2,3884389.0,0.0,9.223372e+18,sam_features_vit_h_beiluhe_google_img_201211_utm_new_export_pyramid_clip_472307.92_3883365.16_473331.85_3884389.10.tif,EPSG:32646,15.999032848232673
+9,9,471919.94,472943.88,3883365.2,3884389.0,0.0,9.223372e+18,sam_features_vit_h_beiluhe_google_img_201211_utm_new_export_pyramid_clip_471919.94_3883365.16_472943.88_3884389.10.tif,EPSG:32646,15.999032848232673
+10,10,471919.94,472943.88,3882674.2,3883698.2,0.0,9.223372e+18,sam_features_vit_h_beiluhe_google_img_201211_utm_new_export_pyramid_clip_471919.94_3882674.20_472943.88_3883698.14.tif,EPSG:32646,15.999032848232673
+11,11,472307.9,473331.84,3882674.2,3883698.2,0.0,9.223372e+18,sam_features_vit_h_beiluhe_google_img_201211_utm_new_export_pyramid_clip_472307.92_3882674.20_473331.85_3883698.14.tif,EPSG:32646,15.999032848232673

geo_sam_tool.py

@@ -51,34 +51,34 @@ def initProcessing(self):
     def initGui(self):
         self.initProcessing()
 
-        self.toolbar: QToolBar = self.iface.addToolBar('Geo-SAM Toolbar')
+        self.toolbar: QToolBar = self.iface.addToolBar('Geo SAM Toolbar')
         self.toolbar.setObjectName('mGeoSamToolbar')
-        self.toolbar.setToolTip('Geo-SAM Toolbar')
+        self.toolbar.setToolTip('Geo SAM Toolbar')
 
         self.actionSamTool = QAction(
             QIcon_GeoSAMTool,
-            "Geo-SAM Tool",
+            "Geo SAM Segmentation Tool",
             self.iface.mainWindow()
         )
 
         self.actionSamEncoder = QAction(
             QIcon_GeoSAMEncoder,
-            "Geo-SAM Encoder",
+            "Geo SAM Encoding Tool",
             self.iface.mainWindow()
         )
         self.actionSamTool.setObjectName("mActionGeoSamTool")
         self.actionSamTool.setToolTip(
-            "Geo-SAM Tool: Use it to label landforms")
+            "Geo SAM Segmentation Tool: Use it to label landforms")
         self.actionSamTool.triggered.connect(self.create_widget_selector)
 
         self.actionSamEncoder.setObjectName("mActionGeoSamEncoder")
         self.actionSamEncoder.setToolTip(
-            "Geo-SAM Encoder: Use it to encode/preprocess image before labeling")
+            "Geo SAM Encoding Tool: Use it to encode/preprocess image before labeling")
         self.actionSamEncoder.triggered.connect(self.encodeImage)
         # QgsMessageLog.logMessage(
         #     f"Geo-SAM action name {self.action.objectName()}", 'Geo SAM', level=Qgis.Info)
-        self.iface.addPluginToMenu('Geo-SAM', self.actionSamTool)
-        self.iface.addPluginToMenu('Geo-SAM', self.actionSamEncoder)
+        self.iface.addPluginToMenu('Geo SAM Tools', self.actionSamTool)
+        self.iface.addPluginToMenu('Geo SAM Tools', self.actionSamEncoder)
         # self.iface.addToolBarIcon(self.action)
         self.toolbar.addAction(self.actionSamTool)
         self.toolbar.addAction(self.actionSamEncoder)