Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

load_tile_map: Add the new parameter 'crs' to set the CRS of the returned dataarray #3554

Merged
merged 10 commits into from
Dec 3, 2024
60 changes: 41 additions & 19 deletions pygmt/datasets/tile_map.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,25 +3,29 @@
:class:`xarray.DataArray`.
"""

import contextlib
from collections.abc import Sequence
from typing import Literal

from packaging.version import Version

try:
import contextily
from rasterio.crs import CRS
seisman marked this conversation as resolved.
Show resolved Hide resolved
from xyzservices import TileProvider

_HAS_CONTEXTILY = True
except ImportError:
CRS = None
Comment on lines +13 to +18
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe put the rasterio imports in the rioxarray block below instead of here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rasterio is a known dependency of contextily (xref: https://github.com/geopandas/contextily/blob/main/pyproject.toml), so it makes more sense to assume that they're installed together.

Copy link
Member

@weiji14 weiji14 Dec 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rasterio is also a dependency of rioxarray 😆. I would argue that rasterio is more tightly coupled to rioxarray, and since rasterio/rioxarray are both used for their projection system capabilities, it would make more sense to put those together. But ok too if you want to keep it here.

TileProvider = None
_HAS_CONTEXTILY = False

with contextlib.suppress(ImportError):
# rioxarray is needed to register the rio accessor
try:
import rioxarray # noqa: F401

_HAS_RIOXARRAY = True
except ImportError:
_HAS_RIOXARRAY = False

import numpy as np
import xarray as xr

Expand All @@ -33,6 +37,7 @@
zoom: int | Literal["auto"] = "auto",
source: TileProvider | str | None = None,
lonlat: bool = True,
crs: str | CRS = "EPSG:3857",
wait: int = 0,
max_retries: int = 2,
zoom_adjust: int | None = None,
Expand All @@ -42,7 +47,8 @@

The tiles that compose the map are merged and georeferenced into an
:class:`xarray.DataArray` image with 3 bands (RGB). Note that the returned image is
in a Spherical Mercator (EPSG:3857) coordinate reference system.
in a Spherical Mercator (EPSG:3857) coordinate reference system (CRS) by default,
but can be customized using the ``crs`` parameter.

Parameters
----------
Expand Down Expand Up @@ -80,6 +86,10 @@
lonlat
If ``False``, coordinates in ``region`` are assumed to be Spherical Mercator as
opposed to longitude/latitude.
crs
Coordinate reference system (CRS) of the returned :class:`xarray.DataArray`
image. Default is ``"EPSG:3857"`` (i.e., Spherical Mercator). The CRS can be in
either string or :class:`rasterio.crs.CRS` format.
wait
If the tile API is rate-limited, the number of seconds to wait between a failed
request and the next try.
Expand Down Expand Up @@ -128,6 +138,8 @@
... raster.rio.crs.to_string()
'EPSG:3857'
"""
_default_crs = "EPSG:3857" # The default CRS returned from contextily
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At https://contextily.readthedocs.io/en/latest/reference.html#contextily.bounds2img, there is this sentence:

IMPORTANT: tiles are assumed to be in the Spherical Mercator projection (EPSG:3857), unless the crs keyword is specified.

While tilemaps are usually served in EPSG:3857, I know that there are some tilemaps that can be served in a different projection system by default. I'm hesitant to use this default of EPSG:3857 here, unless we can be confident that contextily only supports EPSG:3857 tiles.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at the contexily source codes, I think contexitly always assumes that CRS is EPSG:3857, at least when writing the image to raster files

https://github.com/geopandas/contextily/blob/eafeb3c993ab43356b65e7a29b9dd6df3ac76f00/contextily/tile.py#L358

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is for the warp_tiles function. I don't see anything about EPSG:3857 in the bounds2img function (though it's calling several functions and I haven't checked too closely).

Non-Web Mercator (EPSG:3857) XYZ tiles are not common, but they do exist, e.g. Openlayers allows configuring the projection system of the returned tiles:

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at the contexily source codes, I think contexitly always assumes that CRS is EPSG:3857, at least when writing the image to raster files

I meant this line:

https://github.com/geopandas/contextily/blob/eafeb3c993ab43356b65e7a29b9dd6df3ac76f00/contextily/tile.py#L175

In bounds2raster, it first calls bounds2img to get the image and extent (can be any CRS), but when writing to a raster image, bounds2raster assumes the image is EPSG:3857.

Looking at the xyzservices (https://github.com/geopandas/xyzservices/blob/c847f312d2e67a0a45ceae119fdcdf7cf60858b6/provider_sources/xyzservices-providers.json#L50), I can also see some providers that has the crs attribute, but it's unclear how the crs attribute is handled in contexitly.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried using xyzservices.providers.MapTiler.Basic4326 which comes in EPSG:4326:

import contextily
import xyzservices
import matplotlib.pyplot as plt

source = xyzservices.providers.MapTiler.Basic4326(
    key="BwTZdryxoypHc4BnCZ4"  # add an 'o' to the end
)
img, bounds = contextily.bounds2img(w=0, s=-80, e=150, n=80, ll=True, source=source)
# img.shape # (4096, 2048, 4)
plt.imshow(img[:, :, :3])

produces

temp

The extent doesn't seem to be correct (it should be plotting from Greenwich to Asia), possibly a bug in contextily hardcoding to EPSG:3857. But it does show that non-Web Mercator XYZ tiles are indeed possible.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found geopandas/contextily#119, and had a closer look at the bounds2img code, specifically these lines - https://github.com/geopandas/contextily/blob/00ca0318033e69e29a164ce571d9c932a057ecc6/contextily/tile.py#L270-L272. It seems like contextily is using mercantile which only supports Spherical Mercator (EPSG:3857), see mapbox/mercantile#109 (comment). Unless contextily switches to something like https://github.com/developmentseed/morecantile in the future, then it would only 'work' with EPSG:3857.

Copy link
Member Author

@seisman seisman Dec 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The extent doesn't seem to be correct (it should be plotting from Greenwich to Asia), possibly a bug in contextily hardcoding to EPSG:3857.

https://github.com/geopandas/contextily/blob/eafeb3c993ab43356b65e7a29b9dd6df3ac76f00/contextily/tile.py#L256

When ll=True, bounds2img calls the private _sm2ll function to convert bounds in Spherical Mercator coordinates to lon/lat. I believe "EPSG:3857" is hardcoded/assumed in many places in the contextily package.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about checking source.crs (an attribute from an instance of the xyzservices.TileProvider class), and if it is present, use that as the default crs. If not, fallback to EPSG:3857.

Copy link
Member Author

@seisman seisman Dec 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. Done in 6733e19.

_default_crs = getattr(source, "crs", "EPSG:3857")

Please note that this line of code works for TileProvide, str and None. str and None don't have crs, so EPSG:3857 is returned.

Edit: renamed _default_crs to _source_crs in c1d55b0.

Please note that, currently, the returned xarray.DataArray is still "EPSG:3857" even if the source CRS is not (i.e. we don't the reprojection). This may not be ideal.


if not _HAS_CONTEXTILY:
msg = (
"Package `contextily` is required to be installed to use this function. "
Expand All @@ -136,28 +148,34 @@
)
raise ImportError(msg)

contextily_kwargs = {}
if crs != _default_crs and not _HAS_RIOXARRAY:
msg = (
f"Package `rioxarray` is required if CRS is not '{_default_crs}'. "
"Please use `python -m pip install rioxarray` or "
"`mamba install -c conda-forge rioxarray` to install the package."
)
raise ImportError(msg)

# Keyword arguments for contextily.bounds2img
contextily_kwargs = {
"zoom": zoom,
"source": source,
"ll": lonlat,
"wait": wait,
"max_retries": max_retries,
}
if zoom_adjust is not None:
contextily_kwargs["zoom_adjust"] = zoom_adjust
if Version(contextily.__version__) < Version("1.5.0"):
msg = (
"The `zoom_adjust` parameter requires `contextily>=1.5.0` to work. "
"Please upgrade contextily, or manually set the `zoom` level instead."
)
raise TypeError(msg)
raise ValueError(msg)
contextily_kwargs["zoom_adjust"] = zoom_adjust

Check warning on line 174 in pygmt/datasets/tile_map.py

View check run for this annotation

Codecov / codecov/patch

pygmt/datasets/tile_map.py#L173-L174

Added lines #L173 - L174 were not covered by tests

west, east, south, north = region
image, extent = contextily.bounds2img(
w=west,
s=south,
e=east,
n=north,
zoom=zoom,
source=source,
ll=lonlat,
wait=wait,
max_retries=max_retries,
**contextily_kwargs,
w=west, s=south, e=east, n=north, **contextily_kwargs
)

# Turn RGBA img from channel-last to channel-first and get 3-band RGB only
Expand All @@ -176,8 +194,12 @@
dims=("band", "y", "x"),
)

# If rioxarray is installed, set the coordinate reference system
# If rioxarray is installed, set the coordinate reference system.
seisman marked this conversation as resolved.
Show resolved Hide resolved
if hasattr(dataarray, "rio"):
dataarray = dataarray.rio.write_crs(input_crs="EPSG:3857")
dataarray = dataarray.rio.write_crs(input_crs=_default_crs)

# Reproject raster image from EPSG:3857 to specified CRS, using rioxarray.
if crs != _default_crs:
dataarray = dataarray.rio.reproject(dst_crs=crs)
seisman marked this conversation as resolved.
Show resolved Hide resolved

return dataarray
28 changes: 3 additions & 25 deletions pygmt/src/tilemap.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,9 @@
from pygmt.helpers import build_arg_list, fmt_docstring, kwargs_to_strings, use_alias

try:
import rioxarray # noqa: F401
from xyzservices import TileProvider

_HAS_RIOXARRAY = True
except ImportError:
TileProvider = None
_HAS_RIOXARRAY = False


@fmt_docstring
Expand Down Expand Up @@ -111,39 +107,21 @@ def tilemap(

kwargs : dict
Extra keyword arguments to pass to :meth:`pygmt.Figure.grdimage`.

Raises
------
ImportError
If ``rioxarray`` is not installed. Follow
:doc:`install instructions for rioxarray <rioxarray:installation>`, (e.g. via
``python -m pip install rioxarray``) before using this function.
"""
kwargs = self._preprocess(**kwargs)

if not _HAS_RIOXARRAY:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Figure.tilemap method no longer requires rioxarray explicitly. Instead, load_tile_map will raise the error.

msg = (
"Package `rioxarray` is required to be installed to use this function. "
"Please use `python -m pip install rioxarray` or "
"`mamba install -c conda-forge rioxarray` to install the package."
)
raise ImportError(msg)

raster = load_tile_map(
region=region,
zoom=zoom,
source=source,
lonlat=lonlat,
crs="OGC:CRS84" if lonlat is True else "EPSG:3857",
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If lonlat=True, set the CRS to let load_tile_map do the raster reprojection for us.

wait=wait,
max_retries=max_retries,
zoom_adjust=zoom_adjust,
)

# Reproject raster from Spherical Mercator (EPSG:3857) to lonlat (OGC:CRS84) if
# bounding box region was provided in lonlat
if lonlat and raster.rio.crs == "EPSG:3857":
raster = raster.rio.reproject(dst_crs="OGC:CRS84")
raster.gmt.gtype = 1 # set to geographic type
if lonlat:
raster.gmt.gtype = 1 # Set to geographic type

# Only set region if no_clip is None or False, so that plot is clipped to exact
# bounding box region
Expand Down
53 changes: 51 additions & 2 deletions pygmt/tests/test_tilemap.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,24 @@
Test Figure.tilemap.
"""

import importlib
from unittest.mock import patch

import pytest
from pygmt import Figure

contextily = pytest.importorskip("contextily")
rioxarray = pytest.importorskip("rioxarray")
try:
import contextily

_HAS_CONTEXTILY = True
except ImportError:
_HAS_CONTEXTILY = False

_HAS_RIOXARRAY = bool(importlib.util.find_spec("rioxarray"))


@pytest.mark.mpl_image_compare
@pytest.mark.skipif(not _HAS_CONTEXTILY, reason="contextily is not installed")
def test_tilemap_web_mercator():
"""
Create a tilemap plot in Spherical Mercator projection (EPSG:3857).
Expand All @@ -27,6 +37,10 @@ def test_tilemap_web_mercator():

@pytest.mark.benchmark
@pytest.mark.mpl_image_compare
@pytest.mark.skipif(
not (_HAS_CONTEXTILY and _HAS_RIOXARRAY),
reason="contextily and rioxarray are not installed",
)
def test_tilemap_ogc_wgs84():
"""
Create a tilemap plot using longitude/latitude coordinates (OGC:WGS84), centred on
Expand All @@ -45,6 +59,10 @@ def test_tilemap_ogc_wgs84():

@pytest.mark.mpl_image_compare
@pytest.mark.parametrize("no_clip", [False, True])
@pytest.mark.skipif(
not (_HAS_CONTEXTILY and _HAS_RIOXARRAY),
reason="contextily and rioxarray are not installed",
)
def test_tilemap_no_clip(no_clip):
"""
Create a tilemap plot clipped to the Southern Hemisphere when no_clip is False, but
Expand All @@ -60,3 +78,34 @@ def test_tilemap_no_clip(no_clip):
no_clip=no_clip,
)
return fig


@pytest.mark.skipif(_HAS_CONTEXTILY, reason="contextily is installed.")
def test_tilemap_no_contextily():
"""
Raise an ImportError when contextily is not installed.
"""
fig = Figure()
with pytest.raises(ImportError, match="Package `contextily` is required"):
fig.tilemap(
region=[-20000000.0, 20000000.0, -20000000.0, 20000000.0],
zoom=0,
lonlat=False,
frame="afg",
)


@pytest.mark.skipif(_HAS_RIOXARRAY, reason="rioxarray is installed.")
def test_tilemap_no_rioxarray():
"""
Raise an ImportError when rioxarray is not installed and contextily is installed.
"""
fig = Figure()
# In our CI, contextily and rioxarray are installed together, so we will see the
# error about contextily, not rioxarray. Here we mock contextily as installed, to
# make sure that we see the rioxarray error message when rioxarray is not installed.
with patch("pygmt.datasets.tile_map._HAS_CONTEXTILY", True):
with pytest.raises(ImportError, match="Package `rioxarray` is required"):
fig.tilemap(
region=[-180.0, 180.0, -90, 90], zoom=0, lonlat=True, frame="afg"
)
Loading