Data ingest request from Beiler 2024'

[kelle]

JWST spectra of Late T and Y dwarfs. @AstroSamuel, could you please provide more details, including source names.

Source/survey/catalogue name(s)

SIMBAD/CDS/Vizier link to data source(s), if available

What data needs to be added?

Reference(s) paper for the data

[AstroSamuel]

Source Names:
CWISEP J104756.81+545741.6
CWISEP J144606.62-231717.8
SDSS J162414.37+002915.6
SDSSp J134646.45-003150.4
ULAS J102940.52+093514.6
WISEA J215949.54-480855.2
WISEPA J201824.96-742325.9
WISEA J235402.79+024014.1
WISE J024714.52+372523.5
WISE J035934.06-540154.6
WISE J043052.92+463331.6
WISE J053516.80-750024.9
WISE J073444.02-715744.0
WISE J082507.35+280548.5
WISE J120604.38+840110.6
WISE J150115.92-400418.4
WISE J210200.15-442919.5
WISE J220905.73+271143.9
WISEPA J031325.96+780744.2
WISEPA J041022.71+150248.5
WISEPA J154151.66-225025.2
WISEPA J195905.66-333833.7
WISEPC J140518.40+553421.4
WISEPC J205628.90+145953.3

Link to Raw Data:
https://mast.stsci.edu/portal/Mashup/Clients/Mast/Portal.html?searchQuery=%7B%22service%22:%22DOIOBS%22,%22inputText%22:%2210.17909/dwnb-jv28%22%7D
Link to Data Zip File:
https://astrosamuel.github.io/#repo

For WISEPA J041022.71+150248.5 : MIRI 1000W, 1500W, 1800W, and 2100W photometry
For WISEPA J201824.96-742325.9: MIRI LRS and MIRI 1000W, 1500W, 1800W, and 2100W photometry
For all other objects: JWST NIRSpec PRISM, MIRI LRS, and MIRI 1000W, 1500W, 1800W, and 2100W photometry

Reference Paper: https://arxiv.org/pdf/2407.08518
\bibitem[Beiler et al.(2024)]{2024ApJ...973..107B} Beiler, S.~A., Cushing, M.~C., Kirkpatrick, J.~D., et al.\ 2024, \apj, 973, 107. doi:10.3847/1538-4357/ad6301

[kelle]

Hi Sam, thanks for this!

What we really need is this information in tabular form. Here's a template: https://docs.google.com/spreadsheets/d/1EOaIV_NPnow8jnRpi_086zwP8vXh2efQcYcnPdqmq-E/edit

In the spectrum permalink column, you could put the file name.

[kelle]

We also prefer to ingest spectra from different instruments and obs dates individually and not stitched together. Is it feasible to get the spectra organized that way?

[AstroSamuel]

Yes, I can do that easily.
Is there a different method for uploading photometry?

[kelle]

I haven't made a template for photometry yet but will do so in the next week or so! (#520)

[kelle]

Also, with spectra, typically, the hardest part is their file format. We ingest spectra that are readable with astropy.specutils. Typically, these are FITS files in the tabular-fits format. What format are your spectra in?

[AstroSamuel]

Currently .txt files, but I was planning to fits-ify them for this

[kelle]

I have some documentation on this, but would like to have more. Right now, we do have scripts (in the astrodb_utils package) and notebook to help with making good FITS headers. https://astrodb-utils.readthedocs.io/en/latest/pages/using_existing_db/ingesting/ingesting_spectra.html

Once you get the header made and the data into a Spectrum object. Put the header into Spectrum1D.meta['header'] and then spectrum.write(path, format='tabular-fits'). Happy to help with this via Slack.

[kelle]

Related issue with the sketch of the workflow for writing spectra: astrodbtoolkit/astrodb_utils#111

[AstroSamuel]

Here's the table of information
BeilerSIMPLEIngets.xlsx
And here's a zipped file with all the fits files. I couldn't get check_spectrum_plottable to run, but was able to import as Spectrum1D and plot.
SIMPLEspectra.zip

[kelle]

busy bee! Would you mind sharing the script or code snippets that you used to make the FITS files?

And please consider opening a issue about spectrum_plottable in astrodb_utils.

[AstroSamuel]

Sure!
Target and Sample are two classes I made to help with my workflow, but here they're mostly just used to call the name and spectra of each object.

import numpy as np
from astropy.io import fits
from TargetClassJansky import Target
from SampleClass import Sample

from specutils import Spectrum1D
from astrodb_utils.fits import add_missing_keywords, add_observation_date, add_wavelength_keywords, check_header
import astropy.units as u
from astropy.nddata import StdDevUncertainty
import matplotlib.pyplot as plt

# Import Objects
FULL = Sample.build_from_params(fname="AllViableObjectsNames.csv")
FULL.sortByRA()

# Create NIRSpec fits Files
for obj in FULL.Objects:
    # Calculate Ra and Dec
    coord = obj.name.split("J")
    if "-" in obj.name:
        coord = coord[1].split("-")
        sign = -1
    else:
        coord = coord[1].split("+")
        sign = 1
    ra = float(coord[0][:2])*15+float(coord[0][2:4])*15/60+(float(coord[0][4:])*15/3600)
    dec = sign*(float(coord[1][:2])+float(coord[1][2:4])/60+(float(coord[1][4:])/3600))

    # Create Header
    header = add_missing_keywords()
    header.set("OBJECT", f"{obj.name}")
    header.set("RA_TARG", f"{ra:3.4f}")
    header.set("DEC_TARG", f"{dec:3.4f}")
    # header.set("DATE-OBS", "<value>")
    header.set("INSTRUME", "NIRSPEC/PRISM")
    header.set("TELESCOP", "JWST")
    # header.set("TELAPSE", "<value>")
    header.set("APERTURE", "0.2")
    header.set("AUTHOR", "Beiler, Samuel")
    header.set("TITLE", "Precise Bolometric Luminosities and Effective Temperatures of 23 late-T and Y dwarfs Obtained with JWST")
    header.set("VOREF", "https://arxiv.org/pdf/2407.08518")
    header.set("VOPUB", "ApJ")
    header.set("CONTRIB1", "Beiler, Samuel")
    header.set("SPEC_VAL", f"{np.nanmean(obj.nirspecUncut[0])*10000:3.0f}")
    header.set("SPEC_BW", f"{(obj.nirspecUncut[0][-1]-obj.nirspecUncut[0][0])*10000:3.0f}")
    header.set("SPECBAND", "NIR")

    # Create Spectrum
    spec = Spectrum1D(spectral_axis=obj.nirspecUncut[0]*u.um, flux=obj.nirspecUncut[1]
                      * u.Jy, uncertainty=StdDevUncertainty(obj.nirspecUncut[2]))
    spec.meta["header"] = header
    spec.write(f"SIMPLEspectra/{obj.shortName}NIR.fits", format="tabular-fits", overwrite=True)

    # Test File
    spectrum = Spectrum1D.read(f"SIMPLEspectra/{obj.shortName}MIRI.fits", format="tabular-fits")
    wave: np.ndarray = spectrum.spectral_axis.to(u.micron).value
    flux: np.ndarray = spectrum.flux.value
    nan_check: np.ndarray = ~np.isnan(flux) & ~np.isnan(wave)
    wave = wave[nan_check]
    flux = flux[nan_check]
    plt.plot(wave, flux)
    plt.show()

# Create MIRI fits Files
for obj in FULL.Objects:
    # Calculate Ra and Dec
    coord = obj.name.split("J")
    if "-" in obj.name:
        coord = coord[1].split("-")
        sign = -1
    else:
        coord = coord[1].split("+")
        sign = 1
    ra = float(coord[0][:2])*15+float(coord[0][2:4])*15/60+(float(coord[0][4:])*15/3600)
    dec = sign*(float(coord[1][:2])+float(coord[1][2:4])/60+(float(coord[1][4:])/3600))

    # Create Header
    header = add_missing_keywords()
    header.set("OBJECT", f"{obj.name}")
    header.set("RA_TARG", f"{ra:3.4f}")
    header.set("DEC_TARG", f"{dec:3.4f}")
    # header.set("DATE-OBS", "<value>")
    header.set("INSTRUME", "NIRSPEC/PRISM")
    header.set("TELESCOP", "JWST")
    # header.set("TELAPSE", "<value>")
    header.set("APERTURE", "0.2")
    header.set("AUTHOR", "Beiler, Samuel")
    header.set("TITLE", "Precise Bolometric Luminosities and Effective Temperatures of 23 late-T and Y dwarfs Obtained with JWST")
    header.set("VOREF", "https://arxiv.org/pdf/2407.08518")
    header.set("VOPUB", "ApJ")
    header.set("CONTRIB1", "Beiler, Samuel")
    header.set("SPEC_VAL", f"{np.nanmean(obj.mirispecUncut[0])*10000:3.0f}")
    header.set("SPEC_BW", f"{(obj.mirispecUncut[0][-1]-obj.mirispecUncut[0][0])*10000:3.0f}")
    header.set("SPECBAND", "MIR")

    # Create Spectrum
    spec = Spectrum1D(spectral_axis=obj.mirispecUncut[0]*u.um, flux=obj.mirispecUncut[1]
                      * u.Jy, uncertainty=StdDevUncertainty(obj.mirispecUncut[2]))
    spec.meta["header"] = header
    spec.write(f"SIMPLEspectra/{obj.shortName}MIRI.fits", format="tabular-fits", overwrite=True)

    # Test File
    spectrum = Spectrum1D.read(f"SIMPLEspectra/{obj.shortName}MIRI.fits", format="tabular-fits")
    wave: np.ndarray = spectrum.spectral_axis.to(u.micron).value
    flux: np.ndarray = spectrum.flux.value
    nan_check: np.ndarray = ~np.isnan(flux) & ~np.isnan(wave)
    wave = wave[nan_check]
    flux = flux[nan_check]
    plt.plot(wave, flux)
    plt.show()

[AstroSamuel]

And for the check_spectrum_plottable, the issue was with importing, so it may have just been I was calling to the wrong place. I tried both "from astrodb_utils.fits import" and "from astrodb_utils import"