🔖 Release 1.6.0 #895
Merged
🔖 Release 1.6.0 #895
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Filter logger output to `stdout` instead of `stderr` --------- Co-authored-by: John Pocock <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
- Remove tox which is not required on GitHub Actions Co-authored-by: John Pocock <[email protected]>
- 🚨 Use logger instead of print in `slide_info.py`. - 🔥 Remove slide-info.py from wsicore - 🎨 Update cli/slide_info to use wsi.slide_info instead of slide_info.py Co-authored-by: Adam Shephard <[email protected]>
- Move `WSIReader` and `WSIMeta` import to top-level This allows importing `WSIReader` and `WSIMeta` from `wsicore`. Improves usage of the library by importing modules as `from tiatoolbox.wsicore import WSIReader` and `from tiatoolbox.wsicore import WSIMeta` --------- Co-authored-by: Adam Shephard <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
# Changes - Change docstring examples to use concrete classes. - Add custom warning when attempting to instantiate the abstract base class. - Replace occurrences of `store.add` with `store.append` in examples. - Fix bug with `DictionaryStore.bquery` and `geometry=None`, i.e. only a where predicate given. # Documentation The live build is at https://tia-toolbox.readthedocs.io/en/docs-ann-store-eg/ Annotation storage module: https://tia-toolbox.readthedocs.io/en/docs-ann-store-eg/_autosummary/tiatoolbox.annotation.storage.html#module-tiatoolbox.annotation.storage Co-authored-by: Dima Blaginin <[email protected]> Co-authored-by: Shan E Ahmed Raza <[email protected]>
This PR fixes the dicom slide thunbnail issue (maybe). the issue seems to arise from a difference in how the wsidicom reader treats levels, in that it labels them by their relative magnification (in power of 2). so for example the levels in the wsidicom reader are [0, 2, 4] if the level downsamples are [1, 4, 16] whereas in our WSIReader level is just an index [0, 1, 2], and the level_downsamples are separate. My knowledge of DICOM is non-existent, so it may be that the fix is wrong - hopefully john can have a look at it. It fixed the thumbnail issue for me though. ---------------------------------------- Co-authored-by: Mark Eastwood <[email protected]> Co-authored-by: John Pocock <[email protected]> Co-authored-by: Shan Raza <[email protected]>
This docker fixes the bug with Ubuntu OS docker images.
- Improve installation instructions --------- Co-authored-by: John Pocock <[email protected]> Co-authored-by: Mostafa Jahanifar <[email protected]>
- Move the following models to top-level import - HoVerNet - HoVerNetPlus - IDaRS - MapDe - MicroNet - NuClick - SCCNN ------------------------------------------------------- Co-authored-by: Shan Raza <[email protected]> Co-authored-by: Adam Shephard <[email protected]>
- [x] Handle missing version keys and null values. - [x] Add test for missing value - [x] Add test for non-numeric values e.g. "0.5-dev" [see current spec doc](https://ngff.openmicroscopy.org/latest/#omero-md) - [x] Warn if version < or > 0.4 - [x] Warn for inconsistent versions --------- Co-authored-by: Shan E Ahmed Raza <[email protected]>
- Use `pyproject.toml` for `bdist_wheel` configuration - Remove `aliases` Co-authored-by: John Pocock <[email protected]>
- Use pyproject.toml for pytest Configuration Co-authored-by: John Pocock <[email protected]>
* 🔧 Use pyproject.toml for isort configuration - Use pyproject.toml for isort configuration --------- Co-authored-by: John Pocock <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
- Use `pyproject.toml` for `pytest` Configuration
This PR Changes `is_ngff` to return True when above the max supported version with a warning logged. Co-authored-by: Shan Raza <[email protected]>
This fixed potential bug with wsidicom update. The current code gets the base level as `dataset = self.wsi.base_level.datasets[0]` The WsiDicom object has recently been changed so that this base_level is no longer a property of WsiDicom object, it is instead now only in `self.wsi.levels.base_level.datasets[0]` (before the update it was in both). See [here](imi-bigpicture/wsidicom@9067121#diff-c193bbcec87f294abe8cc7a11e0ff209eea7d49407ff301734499e8956477501L98-L101), line 98-101 in wsidicom.py diff.
- Add `joblib` as dependency
* API Change for `PatchExtractor.filter_coordinates`. Instead of `coordinate_resolution`, `coordinate_units`, & `mask_resolution`, it takes `slide_shape` and a callable function `func` to validate the coordinates as input. New implementation is about ~200 times fast. WSIPatchDataset uses `filter_coordinates` method of PatchExtractor, instead of the faster `filter_coordinates_fast`. This PR updates it to use the faster method, and also includes a few other small improvements like allowing the min_mask_ratio argument to be used in the WSIPatchDataset in the same way it is available for patch extractor, so that behaviour between PatchExtractor and WSIPatchDataset is consistent. Changes in this PR: 1. WSIPatchDataset now uses filter_coordinates_fast, consistent with PatchExtractor. 2. filter_coordinates_fast now uses the relative scale between the wsi dimensions at the requested patch resolution, and the mask dimensions, as the scale factor. This is units-agnostic and gives consistent results regardless of mode. Also doesn't make an even lower-res thumbnail of an already low-res mask as it did previously, which was unnecessary and could have had some wierd effects with masks which were already very low-res. 3. The documented post_proc argument now exists and will be used if given. Changed the documentation to reflect its functionality (ie its just called as is on the patch before it is returned) 4. Allows min_mask_ratio to be used in WSIPatchDataset, consistent with usage in PatchExtractor. Rough performance comparison: ~0.6 sec using filter_coordinates_fast ~4 min using filter_coordinates consistent with what was reported in original PR which added filter_coordinates_fast to PatchExtractor. The PR also replicates the ability to provide a custom 'filter func' from filter_coordinates into filter_coordinates_fast, so in theory we can now get rid of filter_coordinates as it does nothing filter_coordinates_fast doesn't do.
- Pin Pandas Version to `>=2.0.0` - `df.append` is deprecated in favour of `pd.concat`. For more details see https://pandas.pydata.org/docs/whatsnew/v2.0.0.html#removal-of-prior-version-deprecations-changes ----------- Co-authored-by: Mark Eastwood <[email protected]>
- Use `logger` Instead Of `warnings` for `patchextraction.py`
- Add tests for lazy import
#590) - Use `logger` Instead Of `warnings` for `pyramid.py` and `storage.py` - Catches duplicate errors and prints them only once. Co-authored-by: John Pocock <[email protected]>
Code is held under a BSD-3-Clause license, but model weights are dependent on the data it is trained on. Before, we generalised this and set everything under a non-commercial creative commons license, but this is not reasonable for models trained on permissive dataset, such as [PCam](https://github.com/basveeling/pcam). This PCam specified a license per model in accordance with the original dataset license. Documentation available at this link https://tia-toolbox.readthedocs.io/en/latest/index.html --------- Co-authored-by: Simon Graham <[email protected]> Co-authored-by: Shan Raza <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
- Fix docstring and license badge
Add engine for the multi-task segmentor to allow simultaneous instance segmentation of nuclei and the semantic segmentation of regions. This replaces commit #216 due to history error in old PR. TODO ==== - [x] Update unit testing - [x] Add argument to store instance segmentation outputs as .dat files and semantic segmentation outputs as .npy - [x] Update Notebook --------- Co-authored-by: John Pocock <[email protected]> Co-authored-by: Shan E Ahmed Raza <[email protected]>
- Use `logger` Instead Of `warnings` for `wsireader.py` --------- Co-authored-by: Mark Eastwood <[email protected]>
- Add a script to update notebook URLs to match the current git ref. This can be run with `make`. This will pass in all files in `examples`, uses the current branch name as the 'to' ref, and "develop" as the 'from' ref. ```bash make update-notebook-urls ``` You can also run the script manually. If run manually, you can specify the target ref and input files. ```bash python pre-commit/notebook_urls.py --from develop --to dev-notebook-urls-hook examples/* ``` --------- Co-authored-by: Shan E Ahmed Raza <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
- Move requirements*.txt to requirements folder --------- Co-authored-by: Mostafa Jahanifar <[email protected]>
* [pre-commit.ci] pre-commit autoupdate updates: - [github.com/pre-commit/pre-commit-hooks: v4.6.0 → v5.0.0](pre-commit/pre-commit-hooks@v4.6.0...v5.0.0) - [github.com/astral-sh/ruff-pre-commit: v0.6.5 → v0.6.9](astral-sh/ruff-pre-commit@v0.6.5...v0.6.9) * 🐛 Fix pre-commit yaml * 🧑💻 Change to monthly schedule. * [pre-commit.ci] pre-commit autoupdate updates: - [github.com/executablebooks/mdformat: 0.7.17 → 0.7.18](hukkin/mdformat@0.7.17...0.7.18) - [github.com/pre-commit/pre-commit-hooks: v4.6.0 → v5.0.0](pre-commit/pre-commit-hooks@v4.6.0...v5.0.0) - [github.com/astral-sh/ruff-pre-commit: v0.6.5 → v0.6.9](astral-sh/ruff-pre-commit@v0.6.5...v0.6.9) * 🔀 Merge online and local copies * 📌 Pin dependencies * 📌 Pin `ruff` version --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Shan E Ahmed Raza <[email protected]>
- Adds the ability to get patches filtered by class or label using AnnotationStore.
- Allows the user to provide a path to an annotation store as the mask.
- By default the mask will then be built out of all the annotations, but the user can also provide any filter that works in the annotation store, and have only those annotations make the mask.
For example, if the user have some pathologist annotations with some regions, and they would only want to extract patches which have some overlap with regions labelled 'tumor' or any other class, they would just need to provide the appropriate string ("props['label'] == 'tumor'" for example).
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
- Adds a warning to the `VahadaneExtractor` to inform users about the algorithm's instability due to changes in the dictionary learning algorithm in `scikit-learn versions > 0.23.0 (see issue #382)`. - The docstrings are updated accordingly to reflect this warning. - No other functionality is altered. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
…Path`, `H-optimus-0` (#856) - Integrates pre-trained foundation models from other labs into tiatoolbox.models.architecture.vanilla.py. Currently, the `_get_architecture()` function allows the use of models from `torchvision.models`. A new function `_get_timm_architecture()` has been incorporated to support foundation models which are available from `timm` with weights on HuggingFace Hub. - All the models from `timm` that used require users to sign the licence agreement with the authors. - The users can add new models by modifying `_get_timm_architecture()` --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Shan E Ahmed Raza <[email protected]>
* 🧑💻 pre-commit autoupdate updates: - [github.com/astral-sh/ruff-pre-commit: v0.7.0 → v0.7.2](astral-sh/ruff-pre-commit@v0.7.0...v0.7.2) * 📌 Update `ruff` version. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Shan E Ahmed Raza <[email protected]>
- Integrates PyTorch 2.0's [torch.compile](https://pytorch.org/docs/stable/generated/torch.compile.html) functionality to demonstrate performance improvements in torch code. This PR focuses on adding `torch.compile` to `PatchPredictor`. **Notes:** - According to the [documentation](https://pytorch.org/tutorials/intermediate/torch_compile_tutorial.html), noticeable performance can be achieved when using modern NVIDIA GPUs (H100, A100, or V100) **TODO:** - [x] Resolve compilation errors related to using `torch.compile` in running models - [x] Initial config - [x] Add to patch predictor - [x] Add to registration - [x] Add to segmentation - [x] Test on custom models - [x] Test on `torch.compile` compatible GPUs --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Shan E Ahmed Raza <[email protected]> Co-authored-by: Jiaqi-Lv <[email protected]>
- Add changes from New engine design #578. This will not only simplify the PR but also keep the main repo up to date. - Refactor `model_to` to `model_abc` - Instead of `on_gpu` use `device` as an input in line with `PyTorch`. - `infer_batch` uses `device` as an input instead of `on_gpu`
- Fix Misleading error message #881
- `torch.compile` is not currently compatible with Windows. See pytorch/pytorch#122094
<!--pre-commit.ci start--> updates: - [github.com/executablebooks/mdformat: 0.7.18 → 0.7.19](hukkin/mdformat@0.7.18...0.7.19) --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Shan E Ahmed Raza <[email protected]>
- Add Example Notebook to explain how to use Foundation Models from `timm` module in TIAToolbox. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Shan E Ahmed Raza <[email protected]>
- Fix `benchmarks/annotation_store.ipynb` errors
- Merge branch 'master' into pre-release
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #895 +/- ##
==========================================
- Coverage 99.91% 99.90% -0.01%
==========================================
Files 69 69
Lines 8518 8715 +197
Branches 1637 1149 -488
==========================================
+ Hits 8511 8707 +196
- Misses 0 3 +3
+ Partials 7 5 -2 ☔ View full report in Codecov by Sentry. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
TIAToolbox v1.6.0 (2024-12-12)
Major Updates and Feature Improvements
timmAPI (🆕 Integrate Foundation Models Available VIAtimm:UNI,Prov-GigaPath,H-optimus-0#856, contributed by @GeorgeBatch)TimmBackbonefor running additional PyTorch Image Models.UNI,Prov-GigaPath, andH-optimus-0.timmadded as a dependency.torch.compile(⚡️Addtorch.compileFunctionality #716)torch.compile.WSIReader(🚸 Handle MultidimensionalFeaturesinWSIReader#742)cli.visualizeto ImprovecliResponse #795)API Changes
has_gpuwithdevicefor specifying GPU or CPU usage, aligning with PyTorch'sModel.to()functionality.POWERfrom sqlite-dist Calculations #769)POWERwith explicit multiplication.Bug Fixes and Other Changes
TIFFWSIReaderread_bound#777)read_boundto use adjusted bounds.WSIReader(🐛 Fix Code Complexity inwsireader#814).WSIReader(🚸 Add Support forignore_is_tiled_tiffinWSIReader.open()#807, contributed by @GeorgeBatch)HoVer-NetClass Output Information indocs#751)cffconvert(🐛 FixCITATION.cffto Make it Compatible withcffconvert#869, contributed by @Alon-Alexander)bokeh_appfor compatibility withbokeh>=3.5.0.sizetoradiusforbokeh>3.4.0compatibility (👽 Changesizetoradiusforbokeh 3.4Compatibility #796).json_extractConstruction #772)VahadaneExtractorAlgorithm Instability #871)scikit-learn>0.23.0dictionary learning (Vahadane stain-norm fails #382).PatchExtractorError Message #883)WSIReader(🐛 Fix Immutable Output fromWSIReader#850)Development-Related Changes
utils,tools,data,annotation, andcli/common.blackwithruff-format.jinja2.Ubuntu 24.04.micromambasetup.codecovReporting #811)Full Changelog: v1.5.1...v1.6.0