Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

**BREAKING** pygmt.grdcut: Refactor to store output in virtualfiles for grids #3115

Open
wants to merge 103 commits into
base: main
Choose a base branch
from

Conversation

seisman
Copy link
Member

@seisman seisman commented Mar 17, 2024

This PR applies virtualfiles in pygmt.grdcut, so that this function doesn't have to write to temporary files (#2730).

grdcut is on of the complicated module that it works for grid, image and cube. We have to know the kind of the input before calling the Session.virtualfile_out method.

It turns out it's not trivial to determine the data kind of an input file. We have some options:

  1. Based on file extensions (e.g., .nc is a grid, .tif is a image)
  2. Based on magic bytes (xref: https://en.wikipedia.org/wiki/List_of_file_signatures)
  3. Try reading as a grid/image/cube. If fail, then try the 2nd one.
  4. Add a new kind parameter (default to "grid") and users should specify the kind parameter.

Option 1-2 may not always work; option 3 also dosn't always work (see #3115 (comment)).

This PR currently adopts option 4. Need to note option 4 meaning BREAKING:

  • previously, grdcut("@earth_day_01d") works; now must use grdcut("@earth_day_01d", kind="image")
  • previously, grdcut works 1/3/4-band images; now it only supports 3-band images. [We can support 1/4-band images later]

seisman and others added 2 commits March 20, 2024 11:00
pygmt/src/grdcut.py Outdated Show resolved Hide resolved
pygmt/clib/session.py Outdated Show resolved Hide resolved
pygmt/src/grdcut.py Outdated Show resolved Hide resolved
@seisman seisman changed the title WIP: Initial try to support raster images using temporary files WIP: pygmt.grdcut: Support both grids and images Mar 27, 2024
@seisman seisman force-pushed the datatypes/gmtgrid branch 3 times, most recently from d8f1160 to d0517e0 Compare April 1, 2024 07:11
@seisman seisman force-pushed the datatypes/gmtgrid branch from d0517e0 to 4ba67df Compare April 1, 2024 11:32
@seisman seisman changed the base branch from datatypes/gmtgrid to main April 1, 2024 11:32
@seisman seisman added the enhancement Improving an existing feature label Apr 19, 2024
weiji14 added a commit that referenced this pull request Jun 18, 2024
Special case `earth_day` to be loaded as GMT_IMAGE rather than GMT_GRID. Need to use `GMT_OUT|GMT_IS_REFERENCE` in virtualfile_out to avoid segfault, xref #3115.
weiji14 added 7 commits June 18, 2024 21:16
Extra metadata from the _GMT_GRID_HEADER struct.
Reorder the dimensions to follow Channel, Height, Width (CHW) convention. Also added doctest checking output DataArray object and the image's x and y coordinates.
Get the registration and gtype info from the grid header and apply it to the GMT accessor attributes.
Trying to match some of the doctests in _GMT_GRID.
@seisman seisman added this to the 0.14.0 milestone Dec 9, 2024
@seisman seisman added the needs review This PR has higher priority and needs review. label Dec 9, 2024
@seisman seisman marked this pull request as ready for review December 9, 2024 11:02
@seisman seisman changed the title pygmt.grdcut: Refactor to store output in virtualfiles for grids **BREAKING** pygmt.grdcut: Refactor to store output in virtualfiles for grids Dec 10, 2024
@seisman seisman mentioned this pull request Dec 19, 2024
49 tasks
@seisman seisman mentioned this pull request Dec 19, 2024
7 tasks
@seisman
Copy link
Member Author

seisman commented Dec 24, 2024

I think it's better to not introduce last-minute breaking changes, so bump this PR to v0.15.0.

@seisman seisman removed this from the 0.14.0 milestone Dec 24, 2024
@seisman seisman removed the needs review This PR has higher priority and needs review. label Dec 27, 2024
@seisman seisman modified the milestone: 0.15.0 Dec 27, 2024
@seisman seisman added this to the 0.15.0 milestone Jan 5, 2025
@seisman seisman added the needs review This PR has higher priority and needs review. label Jan 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Improving an existing feature needs review This PR has higher priority and needs review.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants