=====================
Overview of Driftscan
=====================

driftscan is a package for the analysis of data from transit radio
interferometers using the m-mode formalism which is described in 
`arXiv:1302.0327`_ and `arXiv:1401.2095`_.

Given a design of a telescope, this package can:

* Generate a set of products used to analyse data from it and simulate
  timestreams.
* Construct a filter which can be used to extract cosmological 21 cm emission
  from astrophysical foregrounds, such as our galaxy and radio point sources.
* Estimate the 21cm power spectrum using an optimal quadratic estimator

There are essentially two separate parts to running driftscan: generating the
analysis products, and running the pipeline. We describe how these work below.

.. _`arXiv:1302.0327`: http://arxiv.org/abs/1302.0327
.. _`arXiv:1401.2095`: http://arxiv.org/abs/1401.2095

Generating the Analysis Products
================================



Describing a Telescope
----------------------


.. sidebar:: Code tips

  These are specified by implementing the
  :class:`~driftscan.core.telescope.TransitTelescope` class. Look for helpers
  such as :class:`~driftscan.core.telescope.SimplePolarisedTelescope`.

The first step in running driftscan is to give a model for the telescope. This
consists of:

* A description of the primary beam of each feed. This is a two component
  vector at every at every point in the sky to describe the electric field
  response of the beam.
* The locations of each feed which are assumed to be co-planar and located at
  a specified latitude.
* A model of the instrument noise. The noise is assumed to be stationary and
  Gaussian and so is uniquely described by its power spectrum.



Beam Transfer Matrices
----------------------

Now the fun can begin. The next step is to generate the Beam Transfer matrices
for each m-mode. This is conceptually straightforward:


.. sidebar:: Code tips

  The Beam Transfer matrices are managed by the class
  :py:class:`~drift.core.beamtransfer.BeamTransfer`, and are generated by the method
  :py:meth:`~drift.core.beamtransfer.BeamTransfer.generate`

  The normal Beam matrices are accessed by
  :py:meth:`~drift.core.beamtransfer.BeamTransfer.beam_m`, and their inverse
  by :py:meth:`~drift.core.beamtransfer.BeamTransfer.invbeam_m`.


1. Make sky maps of the polarised response for each feed pair at all observed
   frequencies.
2. Take the spherical harmonic transform of each polarised set of maps.
3. Transpose to group by the m, of each spherical harmonic transform. We must
   also conjugate the negative `m` modes to group them with the positive `m`
   modes.

In practice this can be numerically challenging due to the shear quantity of
frequencies and feed pairs present. This step is ``MPI`` parallelised and
proceeds by distributing subsets of the responses to geneate and transform
across many nodes, and performing an in memory transpose across all these
nodes to group by m. It then processes the next subset, and repeats until we
have generated the complete set.

As much of the information measured by an interferometer is redundant (in that
it tells us nothing new about the sky), we generate a new set of transfer
matrices that map to only the useful subset of the data. This is the next step
of the analysis (described in detail in `arXiv:1401.2095`_) and is done by
taking repeated singular value decompositions of each m-mode. We generate
three matrices for each m-mode:

* :math:`\mathbf{U}` which maps the measured data into the SVD basis.
* :math:`\tilde{\mathbf{B}}` which describes how the SVD modes relate to the sky
* :math:`\tilde{\mathbf{B}}^+` the pseudo-inverse or map-making matrix,

As each m-mode and each frequency is independent this can be trivially
parallelised. This step also generates the pseudo-inverse matrices used for
imaging.



Karhunen Loeve Transform
------------------------




Running the Pipeline
====================



Meh 2.