frb.surveys - Survey Data Access

Overview

Access and query photometric/spectroscopic survey data (PanSTARRS, SDSS, DECaLS, WISE, 2MASS, DES, DESI, GALEX, etc.).

Submodules

frb.surveys.dlsurvey - Base Survey Class

DataLab survey class. Gets data from any survey available through the NOAO datalab-client.

class frb.surveys.dlsurvey.DL_Survey(coord, radius, **kwargs)

Bases: SurveyCoord

A survey class for all databases hosted by NOIR’s DataLab. Inherits from SurveyCoord

__init__(coord, radius, **kwargs)

get_catalog(query=None, query_fields=None, print_query=False, timeout=120, photomdict=None)

Get catalog sources around the given coordinates within self.radius.

Parameters:

• query (str, optional) – SQL query to generate the catalog

• query_fields (list, optional) – Over-ride list of items to query

• print_query (bool) – Print the SQL query generated

Returns:

Catalog of sources obtained from the SQL query.

Return type:

astropy.table.Table

get_image(imsize, band, timeout=120, verbose=False)

Get images from the catalog if available

for a given fov and band.

Parameters:

• imsize (Quantity) – FOV for the desired image

• band (str) – Band for the image (e.g. ‘r’)

• timeout (int, optional) – Time to wait in seconds before timing out

• verbose (bool, optional)

Returns:

Image header data unit

Return type:

HDU

get_cutout(imsize, band=None)

Get cutout (and header)

Parameters:

• imsize (Quantity) – e.g 10*units.arcsec

• band (str) – e.g. ‘r’

Returns:

cutout image, cutout image header

Return type:

ndarray, Header

frb.surveys.catalog_utils - Catalog Utilities

Methods related to fussing with a catalog

frb.surveys.catalog_utils.clean_heasarc(catalog)

Insure RA/DEC are ra/dec in the Table

Table is modified in place

Parameters:

catalog (astropy.table.Table) – Catalog generated by astroquery

frb.surveys.catalog_utils.clean_cat(catalog, pdict, fill_mask=None)

Convert table column names intrinsic to the slurped catalog with the FRB survey desired values

Parameters:

• catalog (astropy.table.Table) – Catalog generated by astroquery

• pdict (dict) – Defines the original key and desired key

• fill_mask (int or float, optional) – Fill masked items with this value

Returns:

modified catalog

Return type:

astropy.table.Table

frb.surveys.catalog_utils.sort_by_separation(catalog, coord, radec=('ra', 'dec'), add_sep=True)

Sort an input catalog by separation from input coordinate

Parameters:

• catalog (astropy.table.Table) – Table of sources

• coord (astropy.coordinates.SkyCoord) – Reference coordinate for sorting

• radec (tuple) – Defines catalog columns holding RA, DEC (in deg)

• add_sep (bool, optional) – Add a ‘separation’ column with units of arcmin

Returns:

Sorted catalog

Return type:

astropy.table.Table

frb.surveys.catalog_utils.match_ids(IDs, match_IDs, require_in_match=True)

Match input IDs to another array of IDs (usually in a table) Return the rows aligned with input IDs

Parameters:

• IDs (ndarray) – ID values to match

• match_IDs (ndarray) – ID values to match to

• require_in_match (bool, optional) – Require that each of the input IDs occurs within the match_IDs

Returns:

Rows in match_IDs that match to IDs, aligned -1 if there is no match

Return type:

ndarray

frb.surveys.catalog_utils.summarize_catalog(frbc, catalog, summary_radius, photom_column, magnitude)

Generate simple text describing the sources from an input catalog within a given radius

Parameters:

• frbc – FRB Candidate object

• catalog (astropy.table.Table) – Catalog table

• summary_radius (Angle) – Radius to summarize on

• photom_column (str) – Column specifying which flux to work on

• magnitude (bool) – Is the flux a magnitude?

Returns:

List of comments on the catalog

Return type:

list

frb.surveys.catalog_utils.xmatch_catalogs(cat1: Table, cat2: Table, dist: Quantity = <Quantity 5. arcsec>, RACol1: str = 'ra', DecCol1: str = 'dec', RACol2: str = 'ra', DecCol2: str = 'dec', distcol1: str = None, distcol2: str = None, return_match_idx: bool = False) → tuple

Cross matches two astronomical catalogs and returns the matched tables. :param cat1: astropy Tables

Two tables with sky coordinates to be matched.

Parameters:

• cat2 – astropy Tables Two tables with sky coordinates to be matched.

• dist – astropy Quantity, optional Maximum separation for a valid match. 5 arcsec by default. Can be length units if distcol1 and distcol2 are provided.

• RACol1 – str, optional Names of columns in cat1 and cat2 respectively that contain RA in degrees.

• RACol2 – str, optional Names of columns in cat1 and cat2 respectively that contain RA in degrees.

• DecCol1 – str, optional Names of columns in cat1 and cat2 respectively that contain Dec in degrees.

• DecCol2 – str, optional Names of columns in cat1 and cat2 respectively that contain Dec in degrees.

• distcol1 – str, optional Names of columns in cat1 and cat2 respectively that contain the radial distance as floats in Mpc. If None, 2D cross-matches are returned.

• distcol2 – str, optional Names of columns in cat1 and cat2 respectively that contain the radial distance as floats in Mpc. If None, 2D cross-matches are returned.

• return_match_idx – bool, optional Return the indices of the matched entries with with the distance instead?

Returns:

astropy Table

Tables of matched rows from cat1 and cat2.

idx, d2d (if return_match_idx): ndarrays

Indices of matched entries from table 2 and an array of separations to go with.

Return type:

match1, match2

frb.surveys.catalog_utils.mag_from_flux(flux, flux_err=None)

Get the AB magnitude from a flux

Parameters:

• flux (Quantity) – Flux

• flux_err (Quantity) – Error in flux (optional)

Returns:

mag, mag_err – AB magnitude and its error (if flux_err is given) AB magnitude and None (if flux_err is None)

Return type:

float, float

frb.surveys.catalog_utils.convert_mags_to_flux(photometry_table, fluxunits='mJy', exact_mag_err=False)

Takes a table of photometric measurements in mags and converts it to flux units.

..todo.. NEED TO ADD DOCS ON VISTA, ETC..

Parameters:

• photometry_table (astropy.table.Table) – A table containing photometric data from a catlog.

• fluxunits (str, optional) – Flux units to convert the magnitudes to, as parsed by astropy.units. Default is mJy.

• exact_mag_err (bool, optional) – Use if you know that the mag errors were estimated exactly as opposed to the first-order approximation that is usually quoted.

Returns:

astropy Table

photometry_table but the magnitudes are converted to fluxes. For upper limits, the flux is the 3sigma value and the error is set to -99.

Return type:

fluxtable

frb.surveys.catalog_utils.remove_duplicates(tab: Table, idcol: str) → Table

In an astropy table if there are duplicate entries, remove the duplicates. Generally, these will be duplicate objects (i.e. multiple observations of same object ID or the same entry repeated multiple times from cross-matching.)

Parameters:

• tab (Table) – A table of entries.

• idcol (str) – A column name that has unique ids for each table entry.

Returns:

A table with only the unique ids.

Return type:

unique_tab (Table)

frb.surveys.catalog_utils.xmatch_and_merge_cats(tab1: Table, tab2: Table, tol: Quantity = <Quantity 1. arcsec>, table_names: tuple = ('1', '2'), **kwargs) → Table

Given two source catalogs, cross-match and merge them. This function ensures there is a unique match between tables as opposed to the default join_skycoord behavior which matches multiple objects on the right table to a source on the left. The two tables must contain the columns ‘ra’ and ‘dec’ (case-sensitive). :param tab1: Photometry catalogs. Must contain columns named

ra and dec.

Parameters:

• tab2 (Table) – Photometry catalogs. Must contain columns named ra and dec.

• tol (Quantity[Angle], optional) – Maximum separation for cross-matching.

• table_names (tuple of str, optional) – Names of the two tables for naming unique columns in the merged table.

• kwargs – Additional keyword arguments to be passed onto xmatch_catalogs

Returns:

Merged catalog.

Return type:

merged_table (Table)

frb.surveys.sdss - SDSS

Methods related to SDSS/BOSS queries

class frb.surveys.sdss.SDSS_Survey(coord, radius, **kwargs)

Bases: SurveyCoord

Class to handle queries on the SDSS database

Parameters:

• coord (SkyCoord) – Coordiante for surveying around

• radius (Angle) – Search radius around the coordinate

__init__(coord, radius, **kwargs)

get_catalog(photoobj_fields=None, timeout=120, print_query=False)

Query SDSS for all objects within a given radius of the input coordinates.

Merges photometry with photo-z

TODO – Expand to include spectroscopy TODO – Consider grabbing all of the photometry fields

Parameters:

• coord – astropy.coordiantes.SkyCoord

• radius – Angle, optional Search radius

• photoobj_fields – list Fields for querying

• timeout – float, optional Default value - 120 s.

• print_query – bool, optional Print the SQL query for the photo-z values

Returns:

astropy.table.Table

Contains all measurements retieved WARNING :: The SDSS photometry table frequently has multiple entries for a given source, with unique objid values

Return type:

catalog

get_cutout(imsize, scale=0.396127)

Grab a cutout from SDSS

Parameters:

imsize (Quantity) – Size of image desired

Returns:

self.cutout and a None to match the image header (not provided by SDSS)

Return type:

PNG image, None

frb.surveys.sdss.get_url(coord, imsize=30.0, scale=0.396127, grid=False, label=False, invert=False)

Generate the SDSS URL for an image retrieval

Parameters:

• coord (astropy.coordiantes.SkyCoord) – Center of image

• imsize – float, optional Image size (rectangular) in arcsec and without units

• scale (float, optional)

• grid (bool, optional)

• label (bool, optional)

• invert (bool, optional)

Returns:

URL for the image

Return type:

str

frb.surveys.sdss.trim_down_catalog(catalog, keep_photoz=False, cut_within=<Quantity 1.5 arcsec>)

Cut down a catalog to keep only 1 source within cut_within

Parameters:

• catalog (astropy.table.Table) – Input source catalog

• keep_photoz (bool, optional)

• cut_within (Angle or Quantity) – Cut radius

Returns:

Catalog trimmed down

Return type:

astropy.table.Table

frb.surveys.des - DES

DES Survey

class frb.surveys.des.DES_Survey(coord, radius, **kwargs)

Bases: DL_Survey

Class to handle queries on the DECaL survey

Child of DL_Survey which uses datalab to access NOAO

Parameters:

• coord (SkyCoord) – Coordiante for surveying around

• radius (Angle) – Search radius around the coordinate

__init__(coord, radius, **kwargs)

get_catalog(query=None, query_fields=None, print_query=False, **kwargs)

Grab a catalog of sources around the input coordinate to the search radius

Parameters:

• query – Not used

• query_fields (list, optional) – Over-ride list of items to query

• print_query (bool) – Print the SQL query generated

• grab_wise (bool) – Attempt to grab WISE data too. This is not recommended.

Returns:

Catalog of sources returned. Includes WISE photometry for matched sources.

Return type:

astropy.table.Table

frb.surveys.decals - DECaLS

DECaLS

class frb.surveys.decals.DECaL_Survey(coord, radius, **kwargs)

Bases: DL_Survey

Class to handle queries on the DECaL survey

Child of DL_Survey which uses datalab to access NOAO

Parameters:

• coord (SkyCoord) – Coordinate for surveying around

• radius (Angle) – Search radius around the coordinate

__init__(coord, radius, **kwargs)

get_catalog(query=None, query_fields=None, print_query=False, exclude_stars=False, **kwargs)

Grab a catalog of sources around the input coordinate to the search radius

Parameters:

• query – SQL query

• query_fields (list, optional) – Over-ride list of items to query

• exclude_gaia (bool,optional) – If the field ‘type’ is present and is ‘PSF’, remove those objects from the output catalog.

• print_query (bool) – Print the SQL query generated

Returns:

Catalog of sources returned

Can be empty

Return type:

astropy.table.Table

frb.surveys.wise - WISE

WISE Survey

class frb.surveys.wise.WISE_Survey(coord, radius, **kwargs)

Bases: SurveyCoord

Class to handle queries on the WISE survey

Child of DL_Survey which uses datalab to access NOAO

Parameters:

• coord (SkyCoord) – Coordiante for surveying around

• radius (Angle) – Search radius around the coordinate

__init__(coord, radius, **kwargs)

get_catalog(query=None, query_fields=['source_id', 'ra', 'dec', 'tmass_key', 'w1mag', 'w1sigm', 'w2mag', 'w2sigm', 'w3mag', 'w3sigm', 'w4mag', 'w4sigm'], print_query=False, system='AB')

Grab a catalog of sources around the input coordinate to the search radius

Parameters:

• query – Not used

• query_fields (list, optional) – Over-ride list of items to query

• print_query (bool) – Print the SQL query generated

• system (str) – Magnitude system [‘AB’, ‘Vega’]

Returns:

Catalog of sources returned. Includes WISE photometry for matched sources.

Magnitudes are in AB by default

Return type:

astropy.table.Table

get_cutout(imsize, band, timeout=120)

Download an image from IRSA :param imsize: Size of the cutout in angular units. :type imsize: Quantity :param band: One of “W1”, “W2”, “W3” or “W4” :type band: str :param timeout: Number of seconds to wait to hear a response from

the IRSA SIA server.

Returns:

Fits HDU with image

Return type:

imghdu(fits.HDU)

frb.surveys.panstarrs - PanSTARRS

Slurp data from Pan-STARRS catalog using the MAST API. A lot of this code has been directly taken from http://ps1images.stsci.edu/ps1_dr2_api.html

class frb.surveys.panstarrs.Pan_STARRS_Survey(coord, radius, **kwargs)

Bases: SurveyCoord

A class to access all the catalogs hosted on the MAST database. Inherits from SurveyCoord. This is a super class not meant for use by itself and instead meant to instantiate specific children classes like PAN-STARRS_Survey

__init__(coord, radius, **kwargs)

get_catalog(query_fields=None, release='dr2', table='stack', print_query=False, use_psf=False, photoz=True)

Query a catalog in the MAST Pan-STARRS database for photometry.

Parameters:

• query_fields – list, optional A list of query fields to get in addition to the default fields.

• release – str, optional “dr1” or “dr2” (default: “dr2”). Data release version.

• table – str, optional “mean”,”stack” or “detection” (default: “stack”). The data table to search within.

• use_psf – bool, optional If True, use PSFmag instead of KronMag

• photoz – bool, optional If True, also download photometric redshifts using the Mast CasJobs API.

Returns:

astropy.table.Table

Contains all query results

Return type:

catalog

get_cutout(imsize=<Quantity 30. arcsec>, filt='irg', output_size=None)

Grab a color cutout (PNG) from Pan-STARRS

Parameters:

• imsize (Quantity) – Angular size of image desired

• filt (str) – A string with the three filters to be used

• output_size (int) – Output image size in pixels. Defaults to the original cutout size.

Returns:

PNG image, None (None for the header).

get_image(imsize=<Quantity 30. arcsec>, filt='i', timeout=120)

Grab a fits image from Pan-STARRS in a specific band.

Parameters:

• imsize (Quantity) – Angular size of the image desired

• filt (str) – One of ‘g’,’r’,’i’,’z’,’y’ (default: ‘i’)

• timeout (int) – Number of seconds to timout the query (default: 120 s)

Returns:

fits header data unit for the downloaded image

Return type:

hdu