Reads 1000s per brick tables and merges them
mpi4py cannot ‘gather’ lists of fits_tables, so each rank must write out its own fits_table, then a later serial job must be used to merge the tables from each rank
Classes
MergeTable(derived_dir, savefn, **kwargs) |
Base class for merging the MPI rank derived filed tables |
RandomsTable(derived_dir, savefn) |
Merges the randoms tables (psql db, input, tractor measurements) |
SummaryTable(derived_dir, savefn) |
In addition to merging over brick tables, compute avg quantities per brick |
Functions
dir_for_mpi(derived_dir) |
|||
dir_for_serial(derived_dir) |
|||
fits_table_cols(randoms_fn) |
|||
main_mpi([doWhat, bricks, nproc, derived_dir]) |
|
||
main_serial([doWhat, derived_dir, …]) |
merges the rank tables that are stored in merge_tmp/ | ||
randoms_subset_count_rsdirs_per_brick(…) |
for Hui, count number of rsdirs per brick to get which of the randoms subsets bricks are done/ |
obiwan.runmanager.merge_tables.MergeTable(derived_dir, savefn, **kwargs)[source]¶Base class for merging the MPI rank derived filed tables
obiwan.runmanager.merge_tables.RandomsTable(derived_dir, savefn)[source]¶Merges the randoms tables (psql db, input, tractor measurements)
obiwan.runmanager.merge_tables.SummaryTable(derived_dir, savefn)[source]¶In addition to merging over brick tables, compute avg quantities per brick
derived table “randoms.fits” must exist. Joins the brick summary quantities from a data release with a similar set from the randoms.fits table. Each brick’s table has one row and all tables get merged to make the eatmap plots
obiwan.runmanager.merge_tables.main_mpi(doWhat=None, bricks=[], nproc=1, derived_dir=None)[source]¶| Parameters: |
|
|---|