draco.core.misc

Miscellaneous pipeline tasks with no where better to go.

Tasks should be proactively moved out of here when there is a thematically appropriate module, or enough related tasks end up in here such that they can all be moved out into their own module.

Classes

AccumulateList()	Accumulate the inputs into a list and return as a group.
ApplyGain()	Apply a set of gains to a timestream or sidereal stack.
CheckMPIEnvironment()	Check that the current MPI environment can communicate across all nodes.
CombineBeamTransfers()	Combine beam transfer matrices for frequency sub-bands.
DebugInfo()	Output some useful debug info.
MakeCopy()	Make a copy of the passed container.
PassOn()	Unconditionally forward a tasks input.
WaitUntil()	Wait until the the requires before forwarding inputs.

class draco.core.misc.AccumulateList

Bases: MPILoggedTask

Accumulate the inputs into a list and return as a group.

If group_size is None, return when the task finishes. Otherwise, return every time group_size inputs have been accumulated.

group_size

If this is set, this task will return the list of accumulated data whenever it reaches this length. If not set, wait until no more input is received and then return everything.

Initialize pipeline task.

May be overridden with no arguments. Will be called after any config.Property attributes are set and after ‘input’ and ‘requires’ keys are set up.

finish()

Remove the internal reference.

Prevents the items from hanging around after the task finishes.

next(input_)

Append an input to the list of inputs.

class draco.core.misc.ApplyGain

Bases: SingleTask

Apply a set of gains to a timestream or sidereal stack.

inverse

Apply the gains directly, or their inverse.

Type:

bool, optional

update_weight

Scale the weight array with the updated gains.

Type:

bool, optional

smoothing_length

Smooth the gain timestream across the given number of seconds. Not supported (ignored) for Sidereal Streams.

Type:

float, optional

Initialize pipeline task.

May be overridden with no arguments. Will be called after any config.Property attributes are set and after ‘input’ and ‘requires’ keys are set up.

process(tstream, gain)

Apply gains to the given timestream.

Smoothing the gains is not supported for SiderealStreams.

Parameters:

• tstream (TimeStream like or SiderealStream) – Time stream to apply gains to. The gains are applied in place.

• gain (StaticGainData, GainData, SiderealGainData, CommonModeGainData) – or CommonModeSiderealGainData. Gains to apply.

Returns:

tstream – The timestream with the gains applied.

Return type:

TimeStream or SiderealStream

class draco.core.misc.CheckMPIEnvironment

Bases: MPILoggedTask

Check that the current MPI environment can communicate across all nodes.

Initialize pipeline task.

May be overridden with no arguments. Will be called after any config.Property attributes are set and after ‘input’ and ‘requires’ keys are set up.

setup()

Send random messages between all ranks.

Tests to ensure that all messages are received within a specified amount of time, and that the messages received are the same as those sent (i.e. nothing was corrupted).

class draco.core.misc.CombineBeamTransfers

Bases: SingleTask

Combine beam transfer matrices for frequency sub-bands.

Here are the recommended steps for generating BTMs in sub-bands (useful if the BTMs over the full band would take too much walltime for a single batch job): 1. Create a telescope manager for the desired full set of BTMs (by running drift-makeproducts with a config file that doesn’t generate any BTMs.) Make note of the lmax and mmax computed by this telescope manager 2. Generate separate BTMs for sub-bands which cover the desired full band. In each separate config file, set force_lmax and force_mmax to the values for the full telescope, to ensure that the combined BTMs will exactly match what would be generated with a single job for the full band. 3. Run this task.

partial_btm_configs

List of config files for partial BTMs.

Type:

list

overwrite

Whether to overwrite existing m files. Default: False.

Type:

bool

pol_pair_copy

If specified, only copy BTM elements with polarization in the list. (For example, to only copy co-pol elements for CHIME: [“XX”, “YY”].) Default: None.

Type:

list

Initialize pipeline task.

May be overridden with no arguments. Will be called after any config.Property attributes are set and after ‘input’ and ‘requires’ keys are set up.

process(bt_full)

Combine beam transfer matrices and write to new set of files.

Parameters:

bt_full (ProductManager or BeamTransfer) – Beam transfer manager for desired output beam transfer matrices.

class draco.core.misc.DebugInfo

Bases: MPILoggedTask, SetMPILogging

Output some useful debug info.

Initialize pipeline task.

May be overridden with no arguments. Will be called after any config.Property attributes are set and after ‘input’ and ‘requires’ keys are set up.

class draco.core.misc.MakeCopy

Bases: SingleTask

Make a copy of the passed container.

Initialize pipeline task.

May be overridden with no arguments. Will be called after any config.Property attributes are set and after ‘input’ and ‘requires’ keys are set up.

process(data)

Return a copy of the given container.

Parameters:

data (containers.ContainerBase) – The container to copy.

class draco.core.misc.PassOn

Bases: MPILoggedTask

Unconditionally forward a tasks input.

While this seems like a pointless no-op it’s useful for connecting tasks in complex topologies.

Initialize pipeline task.

May be overridden with no arguments. Will be called after any config.Property attributes are set and after ‘input’ and ‘requires’ keys are set up.

next(input_)

Immediately forward any input.

class draco.core.misc.WaitUntil

Bases: MPILoggedTask

Wait until the the requires before forwarding inputs.

This simple synchronization task will forward on whatever inputs it gets, however, it won’t do this until it receives any requirement to it’s setup method. This allows certain parts of the pipeline to be delayed until a piece of data further up has been generated.

Initialize pipeline task.

May be overridden with no arguments. Will be called after any config.Property attributes are set and after ‘input’ and ‘requires’ keys are set up.

next(input_)

Immediately forward any input.

setup(input_)

Accept, but don’t save any input.