The Monte Carlo code that adds fake galaxiesto images from the Legacy Survey
Classes
BuildStamp(tim[, seed, camera, gain, exptime]) |
Does the drawing of simulated sources on a single exposure |
SimDecals([dataset, survey_dir, metacat, …]) |
Top level object that specifying which data to run through pipeline |
SimDecalsCosmos(**kwargs) |
Filters the CCDs to just those in the cosmos realizations |
SimImage(survey, t) |
Adds simulated sources to a single exposure |
SimImageCosmos(survey, t) |
Filters the CCDs to just those in the cosmos realizations |
Functions
build_simcat([Samp, brickwcs, meta]) |
Creates the simulated source catalog for a given brick (not CCD). |
create_ith_simcat([d]) |
Write ‘simcat’ and ‘skipped_ids’ tables for a given sample of sources |
create_metadata([kwargs]) |
fits_table with configuration-like params for the simulated sources |
do_ith_cleanup([d]) |
Moves all obiwan+legacypipe outputs to a new directory stucture |
do_one_chunk([d]) |
Runs the legacypipe/Tractor pipeline on images with simulated sources |
dobash(cmd) |
|
flag_nearest_neighbors(Samp[, radius_in_deg]) |
Returns Sample indices to keep (have > dist separations) and indices to skip |
get_checkpoint_fn(outdir, brick, rowstart) |
|
get_e1_e2(q, beta) |
Given minor to major axis ratio (q) and postion angle (beta), Returns e1,e2 tuple |
get_ellip(q) |
Given minor to major axis ratio (q) Returns ellipticity |
get_fnsuffix(**kwargs) |
|
get_parser() |
return parser object, tells it what options to look for options can come from a list of strings or command line |
get_runbrick_setup(**kwargs) |
Convert runbrick.py cmd line options into **kwargs for run_brick() |
get_sample(objtype, brick, randoms_db[, …]) |
Gets all simulated randoms for a brick from PSQl db, and applies all relevant cuts |
get_skip_ids(decals_sim_dir, brickname, objtype) |
|
get_srcimg_invvar(stamp_ivar, img_ivar) |
stamp_ivar, img_ivar – galsim Image objects |
imshow_stamp(stamp[, fn, galsimobj]) |
|
ivar_for_galaxy(gal, nano2e) |
Adds gaussian noise to perfect source |
ivar_to_var(ivar[, nano2e, camera]) |
|
main([args]) |
Main routine which parses the optional inputs. |
noise_for_galaxy(gal, nano2e) |
Returns numpy array of noise in Img count units for gal in image cnt units |
plot_radial_profs(fn, profs) |
|
ptime(text, t0) |
Timer |
read_dict(fn) |
|
saturation_e(camera) |
|
write_dict(fn, d) |
d – dictionary |
obiwan.kenobi.BuildStamp(tim, seed=0, camera=None, gain=None, exptime=None)[source]¶Does the drawing of simulated sources on a single exposure
| Parameters: |
|
|---|
band¶g,r,z
camera¶‘decam’, ‘mosaic’, ‘90prime’
gsparams¶galsim object that configures how accurate simulated source will be
gsdeviate¶galsim object that configures its random number generator
wcs¶WCS from tim
psf¶psf from tim
galsim_wcs¶wcs repackaged into galsim compatible object
zpscale¶conversion factor ‘nanomaggies’ to ‘Image units used by Legacypipe’, which are ADU/sec for DECam and e/sec for Bass,MzLS
nano2e¶conversion factor ‘nanomaggies’ to ‘e-‘
obiwan.kenobi.SimDecals(dataset=None, survey_dir=None, metacat=None, simcat=None, output_dir=None, add_sim_noise=False, seed=0, image_eq_model=False, **kwargs)[source]¶Top level object that specifying which data to run through pipeline
| Parameters: |
|
|---|
DR¶see above
metacat¶fits_table configuration-like params for the simulated sources
simcat¶fits_table simulated source catalog for a given brick (not CCD).
output_dir¶legacypipe’s outdir
add_sim_noise¶add Poisson noise from the simulated source to the image
image_eq_model¶referred to as ‘testA’ wherever add a simulated source, replace both image and invvar of the image with that of the simulated source only
add_hashcode(fn, hashcode)¶Callback to be called in the write_output routine.
bricks_touching_radec_box(bricks, ralo, rahi, declo, dechi)¶Returns an index vector of the bricks that touch the given RA,Dec box.
ccd_cuts(ccds)¶Returns a bitmask of reasons the given ccds would be cut (excluded from legacypipe processing). These bits are defined in LegacySurveyData.ccd_cut_bits.
This just delegates to the LegacySurveyImage subclasses based on camera.
ccds_touching_wcs(wcs, **kwargs)¶Returns a table of the CCDs touching the given wcs region.
drop_cache()¶Clears all cached data contained in this object. Useful for pickling / multiprocessing.
find_ccds(expnum=None, ccdname=None, camera=None)¶Returns a table of CCDs matching the given expnum (exposure number, integer), ccdname (string), and camera (string), if given.
find_file(filetype, brick=None, brickpre=None, band='%(band)s', output=False)¶Returns the filename of a Legacy Survey file.
brick : string, brick name such as “0001p000”
output: True if we are about to write this file; will use self.outdir as the base directory rather than self.survey_dir.
Returns: path to the specified file (whether or not it exists).
get_annotated_ccds()¶Returns the annotated table of CCDs.
get_brick(brickid)¶Returns a brick (as one row in a table) by brickid (integer).
get_brick_by_name(brickname)¶Returns a brick (as one row in a table) by name (string).
get_bricks()¶Returns a table of bricks. The caller owns the table.
For read-only purposes, see get_bricks_readonly(), which uses a cached version.
get_bricks_readonly()¶Returns a read-only (shared) copy of the table of bricks.
get_calib_dir()¶Returns the directory containing calibration data.
get_ccds()¶Returns the table of CCDs.
get_ccds_readonly()¶Returns a shared copy of the table of CCDs.
get_image_dir()¶Returns the directory containing image data.
get_se_dir()¶Returns the directory containing SourceExtractor config files, used during calibration.
get_survey_dir()¶Returns the base LEGACY_SURVEY_DIR directory.
photometric_ccds(ccds)¶Returns an index array for the members of the table “ccds” that are photometric.
read_intermediate_catalog(brick, **kwargs)¶Reads the intermediate tractor catalog for the given brickname.
kwargs: passed to self.find_file()
Returns (T, hdr, primhdr)
tims_touching_wcs(targetwcs, mp, bands=None, **kwargs)¶Creates tractor.Image objects for CCDs touching the given targetwcs region.
mp: multiprocessing object
kwargs are passed to LegacySurveyImage.get_tractor_image() and may include:
write_output(filetype, hashsum=True, **kwargs)¶Returns a context manager for writing an output file; use like:
For FITS output, out.fits is a fitsio.FITS object. The file contents will actually be written in memory, and then a sha256sum computed before the file contents are written out to the real disk file. The ‘out.fn’ member variable is NOT set.
Does the following on entry: - calls self.find_file() to determine which filename to write to - ensures the output directory exists - appends a “.tmp” to the filename
Does the following on exit: - moves the “.tmp” to the final filename (to make it atomic) - computes the sha256sum
obiwan.kenobi.SimDecalsCosmos(**kwargs)[source]¶Filters the CCDs to just those in the cosmos realizations
Call just like SimDecals except with additional Argument ‘subset’
| Parameters: | **kwargs – SimDecals args + ‘subset’ |
|---|
add_hashcode(fn, hashcode)¶Callback to be called in the write_output routine.
bricks_touching_radec_box(bricks, ralo, rahi, declo, dechi)¶Returns an index vector of the bricks that touch the given RA,Dec box.
ccd_cuts(ccds)¶Returns a bitmask of reasons the given ccds would be cut (excluded from legacypipe processing). These bits are defined in LegacySurveyData.ccd_cut_bits.
This just delegates to the LegacySurveyImage subclasses based on camera.
ccds_touching_wcs(wcs, **kwargs)¶Returns a table of the CCDs touching the given wcs region.
drop_cache()¶Clears all cached data contained in this object. Useful for pickling / multiprocessing.
filter_ccd_kd_files(fns)¶see legacypipe/runs.py
filter_ccds_files(fns)¶see legacypipe/runs.py
find_ccds(expnum=None, ccdname=None, camera=None)¶Returns a table of CCDs matching the given expnum (exposure number, integer), ccdname (string), and camera (string), if given.
find_file(filetype, brick=None, brickpre=None, band='%(band)s', output=False)¶Returns the filename of a Legacy Survey file.
brick : string, brick name such as “0001p000”
output: True if we are about to write this file; will use self.outdir as the base directory rather than self.survey_dir.
Returns: path to the specified file (whether or not it exists).
get_annotated_ccds()¶Returns the annotated table of CCDs.
get_brick(brickid)¶Returns a brick (as one row in a table) by brickid (integer).
get_brick_by_name(brickname)¶Returns a brick (as one row in a table) by name (string).
get_bricks()¶Returns a table of bricks. The caller owns the table.
For read-only purposes, see get_bricks_readonly(), which uses a cached version.
get_bricks_readonly()¶Returns a read-only (shared) copy of the table of bricks.
get_calib_dir()¶Returns the directory containing calibration data.
get_ccds()¶Returns the table of CCDs.
get_ccds_readonly()¶Returns a shared copy of the table of CCDs.
get_image_dir()¶Returns the directory containing image data.
get_image_object(t)¶Returns a DecamImage or similar object for one row of the CCDs table.
get_se_dir()¶Returns the directory containing SourceExtractor config files, used during calibration.
get_survey_dir()¶Returns the base LEGACY_SURVEY_DIR directory.
photometric_ccds(ccds)¶Returns an index array for the members of the table “ccds” that are photometric.
read_intermediate_catalog(brick, **kwargs)¶Reads the intermediate tractor catalog for the given brickname.
kwargs: passed to self.find_file()
Returns (T, hdr, primhdr)
tims_touching_wcs(targetwcs, mp, bands=None, **kwargs)¶Creates tractor.Image objects for CCDs touching the given targetwcs region.
mp: multiprocessing object
kwargs are passed to LegacySurveyImage.get_tractor_image() and may include:
write_output(filetype, hashsum=True, **kwargs)¶Returns a context manager for writing an output file; use like:
For FITS output, out.fits is a fitsio.FITS object. The file contents will actually be written in memory, and then a sha256sum computed before the file contents are written out to the real disk file. The ‘out.fn’ member variable is NOT set.
Does the following on entry: - calls self.find_file() to determine which filename to write to - ensures the output directory exists - appends a “.tmp” to the filename
Does the following on exit: - moves the “.tmp” to the final filename (to make it atomic) - computes the sha256sum
obiwan.kenobi.SimImage(survey, t)[source]¶Adds simulated sources to a single exposure
| Parameters: |
|
|---|
inherits¶DecamImage
t¶as used by DecamImage a single row fits_table for a specific CCD
bad_astrometry(survey, ccds)¶IDL zeropoints have large rarms,decrms,phrms for some CP images that look fine. Legacy zeropoints is okay for majority of these cases. False alarm? Bug in IDL zeropoints? Doing the most conservative thing and dropping these ccds. see email: “3/30/2017: [decam-chatter 5155] Clue to zero-point errors in dr4”
bad_exposures(survey, ccds)¶Returns an index array for the members of the table ‘ccds’ that are good exposures (NOT flagged) in the bad_expid file.
ccdname_hdu_mismatch(survey, ccds)¶Mosaic + Bok, ccdname and hdu number must match. If not, IDL zeropoints files has duplicated zeropoint info from one of the other four ccds.
Returns a boolean array, True for CCDs with this problem.
check_psf(psffn)¶Returns True if the PsfEx file is ok.
get_cacheable_filename_variables()¶These are names of self.X variables that are filenames that could be cached.
get_good_image_slice(extent, get_extent=False)¶extent = None or extent = [x0,x1,y0,y1]
If get_extent = True, returns the new [x0,x1,y0,y1] extent.
Returns a new pair of slices, or extent if the whole image is good.
get_good_image_subregion()¶Returns x0,x1,y0,y1 of the good region of this chip, or None if no cut should be applied to that edge; returns (None,None,None,None) if the whole chip is good.
This cut is applied in addition to any masking in the mask or invvar map.
get_image_extent(wcs=None, slc=None, radecpoly=None)¶Returns x0,x1,y0,y1,slc
get_image_shape()¶Returns image shape H,W.
get_sky_sig1(splinesky=False)¶Returns the per-pixel noise estimate, which (for historical reasons) is stored in the sky model. NOTE that this is in image pixel counts, NOT calibrated nanomaggies.
get_tractor_image(**kwargs)[source]¶Returns a tractor.Image (“tim”) object for this image.
Options describing a subimage to return:
Options determining the PSF model to use:
Options determining the sky model to use:
Options determining the units of the image:
photometric_ccds(survey, ccds)¶Returns an index array for the members of the table ‘ccds’ that are photometric.
Slightly revised recipe by DJS in Re: [decam-data 828] 2015-07-31:
read_dq(header=False, **kwargs)¶Reads the Data Quality (DQ) mask image.
read_image(**kwargs)¶Reads the image file from disk.
The image is read from FITS file self.imgfn HDU self.hdu.
| Parameters: |
|
|---|---|
| Returns: |
|
read_image_header(**kwargs)¶Reads the FITS image header from self.imgfn HDU self.hdu.
| Returns: | header – The FITS header |
|---|---|
| Return type: | fitsio header |
read_image_primary_header(**kwargs)¶Reads the FITS primary (HDU 0) header from self.imgfn.
| Returns: | primary_header – The FITS header |
|---|---|
| Return type: | fitsio header |
read_invvar(clip=True, clipThresh=0.2, **kwargs)¶Reads the inverse-variance (weight) map image.
read_primary_header(fn)¶Reads the FITS primary header (HDU 0) from the given filename. This is just a faster version of fitsio.read_header(fn).
read_sky_model(splinesky=False, slc=None, **kwargs)¶Reads the sky model, returning a Tractor Sky object.
run_calibs(psfex=True, sky=True, se=False, funpack=False, fcopy=False, use_mask=True, force=False, just_check=False, git_version=None, splinesky=False)¶Run calibration pre-processing steps.
| Parameters: | just_check (boolean) – If True, returns True if calibs need to be run. |
|---|
shape¶Returns the full shape of the image, (H,W).
obiwan.kenobi.SimImageCosmos(survey, t)[source]¶Filters the CCDs to just those in the cosmos realizations
Call just like SimDecals except with additional Argument ‘subset’
| Parameters: | **kwargs – SimDecals args + ‘subset’ |
|---|
bad_astrometry(survey, ccds)¶IDL zeropoints have large rarms,decrms,phrms for some CP images that look fine. Legacy zeropoints is okay for majority of these cases. False alarm? Bug in IDL zeropoints? Doing the most conservative thing and dropping these ccds. see email: “3/30/2017: [decam-chatter 5155] Clue to zero-point errors in dr4”
bad_exposures(survey, ccds)¶Returns an index array for the members of the table ‘ccds’ that are good exposures (NOT flagged) in the bad_expid file.
ccdname_hdu_mismatch(survey, ccds)¶Mosaic + Bok, ccdname and hdu number must match. If not, IDL zeropoints files has duplicated zeropoint info from one of the other four ccds.
Returns a boolean array, True for CCDs with this problem.
check_psf(psffn)¶Returns True if the PsfEx file is ok.
get_cacheable_filename_variables()¶These are names of self.X variables that are filenames that could be cached.
get_good_image_slice(extent, get_extent=False)¶extent = None or extent = [x0,x1,y0,y1]
If get_extent = True, returns the new [x0,x1,y0,y1] extent.
Returns a new pair of slices, or extent if the whole image is good.
get_good_image_subregion()¶Returns x0,x1,y0,y1 of the good region of this chip, or None if no cut should be applied to that edge; returns (None,None,None,None) if the whole chip is good.
This cut is applied in addition to any masking in the mask or invvar map.
get_image_extent(wcs=None, slc=None, radecpoly=None)¶Returns x0,x1,y0,y1,slc
get_image_shape()¶Returns image shape H,W.
get_sky_sig1(splinesky=False)¶Returns the per-pixel noise estimate, which (for historical reasons) is stored in the sky model. NOTE that this is in image pixel counts, NOT calibrated nanomaggies.
get_tractor_image(**kwargs)¶Returns a tractor.Image (“tim”) object for this image.
Options describing a subimage to return:
Options determining the PSF model to use:
Options determining the sky model to use:
Options determining the units of the image:
photometric_ccds(survey, ccds)¶Returns an index array for the members of the table ‘ccds’ that are photometric.
Slightly revised recipe by DJS in Re: [decam-data 828] 2015-07-31:
read_dq(header=False, **kwargs)¶Reads the Data Quality (DQ) mask image.
read_image(**kwargs)¶Reads the image file from disk.
The image is read from FITS file self.imgfn HDU self.hdu.
| Parameters: |
|
|---|---|
| Returns: |
|
read_image_header(**kwargs)¶Reads the FITS image header from self.imgfn HDU self.hdu.
| Returns: | header – The FITS header |
|---|---|
| Return type: | fitsio header |
read_image_primary_header(**kwargs)¶Reads the FITS primary (HDU 0) header from self.imgfn.
| Returns: | primary_header – The FITS header |
|---|---|
| Return type: | fitsio header |
read_invvar(clip=True, clipThresh=0.2, **kwargs)¶Reads the inverse-variance (weight) map image.
read_primary_header(fn)¶Reads the FITS primary header (HDU 0) from the given filename. This is just a faster version of fitsio.read_header(fn).
read_sky_model(splinesky=False, slc=None, **kwargs)¶Reads the sky model, returning a Tractor Sky object.
run_calibs(psfex=True, sky=True, se=False, funpack=False, fcopy=False, use_mask=True, force=False, just_check=False, git_version=None, splinesky=False)¶Run calibration pre-processing steps.
| Parameters: | just_check (boolean) – If True, returns True if calibs need to be run. |
|---|
shape¶Returns the full shape of the image, (H,W).
obiwan.kenobi.build_simcat(Samp=None, brickwcs=None, meta=None)[source]¶Creates the simulated source catalog for a given brick (not CCD).
| Parameters: |
|
|---|---|
| Returns: | tuple of cat: skipping_ids: |
obiwan.kenobi.create_ith_simcat(d=None)[source]¶Write ‘simcat’ and ‘skipped_ids’ tables for a given sample of sources
| Parameters: | d – {‘Samp’: fits_table for the properties of sources in the brick ‘brickwcs’: WCS object for the brick ‘metacat’: fits_table with configuration params for the simulated sources } |
|---|---|
| Returns: | Nothing, saves the ‘simcat’ and ‘skipped_ids’ tables Adds ‘simcat’ table to dict ‘d’ |
obiwan.kenobi.create_metadata(kwargs=None)[source]¶fits_table with configuration-like params for the simulated sources
TODO: Should metacat table have a rowstart column? TODO: One metacat table per brick, instead of one per rs* directory?
| Parameters: | kwargs – configuration-like params for the simulated sources {‘brickname’: which chunk of sky ‘objtype’: star,elg,lrg,qso ‘nobj’: number of simulated sources for this run } |
|---|---|
| Returns: | Nothing writes the ‘metacat’ fits_table to disk and stores it in the kwargs input arg |
obiwan.kenobi.do_ith_cleanup(d=None)[source]¶Moves all obiwan+legacypipe outputs to a new directory stucture
| Parameters: | d – dict with keys brickname, simcat_dir |
|---|
obiwan.kenobi.do_one_chunk(d=None)[source]¶Runs the legacypipe/Tractor pipeline on images with simulated sources
| Parameters: | d – {‘args’: obiwan.kenobi.py cmd line argparse.Namespace object ‘brickname’: chunk of sky ‘metacat’: fits_table configuration params for the simulated sources ‘simcat’: fits_table simulated source catalog for a given brick (not CCD). |
|---|
Note
runb_brick() is ‘main’ for the legacypipe/Tractor pipeline
| Returns: | Nothing, but this func end ups writing out all the obiwan results |
|---|
obiwan.kenobi.flag_nearest_neighbors(Samp, radius_in_deg=0.001388888888888889)[source]¶Returns Sample indices to keep (have > dist separations) and indices to skip
| Returns: | keep,skip: indices of Samp to keep and skip |
|---|---|
| Return type: | tuple |
obiwan.kenobi.get_e1_e2(q, beta)[source]¶Given minor to major axis ratio (q) and postion angle (beta), Returns e1,e2 tuple
obiwan.kenobi.get_parser()[source]¶return parser object, tells it what options to look for options can come from a list of strings or command line
obiwan.kenobi.get_runbrick_setup(**kwargs)[source]¶Convert runbrick.py cmd line options into **kwargs for run_brick()
| Parameters: | **kwargs – dict of the cmd line options to obiwan.kenobi.py |
|---|---|
| Returns: |
|
obiwan.kenobi.get_sample(objtype, brick, randoms_db, minid=None, randoms_from_fits='', do_skipids='no', outdir=None, dont_sort_sampleid=False)[source]¶Gets all simulated randoms for a brick from PSQl db, and applies all relevant cuts
| Parameters: |
|
|---|---|
| Returns: | sample fits_table, seed |
| Return type: | tupe |
obiwan.kenobi.get_srcimg_invvar(stamp_ivar, img_ivar)[source]¶stamp_ivar, img_ivar – galsim Image objects
obiwan.kenobi.ivar_for_galaxy(gal, nano2e)[source]¶Adds gaussian noise to perfect source
| Parameters: |
|
|---|---|
| Returns: | nanomags |
| Return type: | galsim.Image() of invvar for the source, UNITS |