HIGH SPEED PHOTOMETRY

(with large CCDs)

Roger Smith

Created: 2001 May 29
Last updated: 2001  July 13

CONTENT

SCIENTIFIC GOALS

Differential photometry of object and reference stars is required on sub second time scales, with accurate absolute time tagging of each exposure to allow correlation with concurrent observations elsewhere.  The initial application is the optical observation of aperiodic phenomena such as accretion events in X-ray binaries which last a minute or less requiring fine sampling to resolve the details of the process.  The time tagging allows correlation with X ray data from satellites such as Chandra.   Differential photometry is required to compensate for variable sky conditions.   Sky pixels are measured either side of both object and reference stars.
 

HOW HSP MODE WORKS

Economics dictate that the existing large format CCDs have to be used without hardware modification, and with as little software support possible.  Instrument changes are to be avoided.

Since the mechanical shutters on the available instruments are not fast enough, exposures must be defined by vertical clocking.  The shutter is left open for the entire high speed photometry sequence.  The implications are discussed later.

Regions of interest are stacked so that a burst of high speed exposures appear to be a single conventional image to the downstream software.  The main drawback of this "software free" approach is that the real time display shows a fixed piece of the image (if it works at all) instead of a continuously updated movie or scrolling display.

The available CCDs are so large that the vertical charge transfer rate limits conventional sample rates to less than 1 Hz, and seriously degrades the duty cycle (fraction of the time spent exposing).   This is mainly because the distributed resistance and capacitance of the CCD electrodes (in the image area) require SITe 2K CCDs to be clocked at 1 ms per parallel transfer.  If a region of interest is placed at the center, then there is an overhead of 1 sec just for the parallel shifts required to move it to the serial register.   If the region is placed at the corner, then a similar delay is encountered when erasing the trailing charge that would otherwise contaminate the next exposure.

If pixel size and aspect ratio are held constant, then the maximum vertical transfer rate is inversely proportional to the number of pixels.  i.e.  a 2048x2048 CCD must be parallel-clocked four times slower than a 1024x1024 CCD.   For this reason 512x512 or smaller CCDs are usually preferred in this kind of application, but this is not an option.

Instead, the following steps have been taken to achieve much higher frame rates:

These measures were all implemented by modifying the readout waveforms, thus satisfying the requirement to minimize programmer involvement.

Exposures must start at absolute times with millisecond accuracy.  The Arcon software is too slow and suffers from variations in messaging times to achieve this.  Even if the start of an exposure sequence could be perfectly timed, the sequencer clock is only accurate to 100 ppm and could thus drift a millisecond in 10 seconds.  Exposure sequences will typically last much longer than this.  To deal with these problems the Arcon's waveform sequencer is able to respond to an external strobe with only 1 clock cycle (40 ns) timing jitter.  The absolute timing is thus limited by the accuracy of the external strobe.   In this application the strobes are generated with less than 7 us absolute error by the "Datum bc635" card in the VME based Telescope Control (TCS) systems. ( The spare TCS crate is used when observing at the 0.9m.)  This Datum bc635 gets its time via the IRIG-B signal generated by the GPS clock.

Marco Bonati has provided a command in Arcon to setup absolute timing of the exposures, and to report this in the headers.  Rolando Cantarutti provided the underlying TCS software to setup the Datum bc635 card to generate the strobes at the absolute times specified by the user.

The time for any given exposure can be calculated post facto from the line number, the exposure start time and exposure period which are recorded in the image header.
 

OBSERVING GUIDE

The usual steps for observing in HSP mode are:

Configuring for high speed photometry

The detector parameter menu, invoked in the Arcon Acquisition window with the setdetector command, should show many additional entries with descriptions prefixed with the letters "HSP".   These parameters are only active when the HSP parameter is set to yes.  Otherwise the CCD will be read normally.  Some parameters are have similar functions in both modes such as amplifiers and HSP_amplifiers to allow rapid switching back and forth, between normal and HSP modes.  Those prefixed with HSP-1 apply to single HSP channel readout and those prefixed with HSP-2  apply to dual channel HSP readout.

If the HSP menu is not seen then (Observer Support should) follow this recipe:

Single readout or dual ?

The amplifiers parameter should be set to quad (or at least dual) to speed up non-HSP exposures.

HSP_Amps should normally be set to single, since this allows the two ROI centers (X_near and X_far) to be tuned independently.  Once you get the stars close to where you want them on the CCD it is a lot easier to take a test frame then tweak the ROI centers than to jog the telescope to single pixel accuracy, especially since the real time display is not useful in HSP mode.  Provided that the number of pixels read per line is small and the separation of the regions is not significantly greater than half the CCD width then there is little penalty from using only one amplifier.   In fact for closely spaced regions, single amp readout is faster than dual.  Single amplifier data is also slightly easier to process since the gain is the same for both ROI's.

Set HSP_Amps to dual when the regions are widely spaced or contain enough pixels for the pixel read time to dominate, ....but only if the exposure rate is higher than can be achieved with single amp readout.  The reason to avoid dual amp readout is that it imposes a requirement for the regions be placed symmetrically about the centerline of the CCD.  This makes it harder to line up the two stars with the regions!   Their location is specified by the center-to-center distance (X_sep) rather than by two positions.

Which amplifier is best ?

Any single amplifier, the upper pair or the lower pair can be used.  However, it should be noted that the ROI locations are specified in pixels from the corner of the image area nearest the amplifier in use.  By contrast the coordinate system for the image on disk is always referenced to the lower left corner of the image area no matter which amplifier is used.   So if the lower left amplifier is selected then these coordinates are identical, and the first exposure is the one nearest the first line.  The position of the ROI nearest the output amplifier, X_near is to the left of X_far. But if the upper right amplifier is selected, the first exposure is found at the highest line number and the ROI position X_near is to the right of X_far.  Clearly selecting the lower left amp avoids a simple coordinate transformation.

Positioning the stars for fastest readout

The object and reference star are identified in a full frame image, by taking an exposure with HSP=no in the setdetector menu.  Choose amplifier=quad to speed up readout.  To find and print the coordinates of the stars, display in ximtool, invoke IRAF's ximtool, position the cursor over star and use the "," keystroke command.
You must specicify the ROI coordinates relative to the lower left corner.  Remember to allow for any prescan or overscan columns when measureing the coordinate positions.  A convenient to do this is to quadproc the images prior to measuring the star positions.
Calculate rotation angle and adjust the rotator (by hand) to place the stars parallel to the serial register.  Take a full frame image to check alignment and to check you went the right way!

To minimize read time, offset the telescope (by command to TCS, not with guide buttons) to place the stars as close as possible to the serial register without losing light.  Allow for guiding errors and seeing motion.  At the same time place one star as close as possible to the output amplifier, allowing space for the surrounding sky pixels.

Note that the CCD coordinates have been rotated with respect to RA and DEC !  Use your calculator to convert the rectangular CCD coordinates to polar, add (or subtract?) the rotation angle, convert back to rectangular and apply the plate scale to get arcseconds.   An IRAF script to do this and help get the signs right in the early hours of the morning would be nice but none exists at this point.

Take a conventional image.   Readout can be sped up appreciably at this point by reducing ysize.  ystart only needs to be changed if using the upper serial register.   Do the stars lie at the correct location  ...close to the serial register and the output amplifier?   If so you are ready to begin high speed photometry.

There are two HSP readout modes......

2D ROI mode        (Ysize = N * Ybin)

The 2D ROI mode is quite conventional, except that multiple exposures are stacked into a single rectangular frame.  This mode allows conventional pixel by pixel flat fielding to be performed and is thus more accurate than collapsed ROI mode, but readout times are 5-10 times longer since the over head of the serial skips accrues to each line rather than once per exposure, and of course there are more pixels read out.  Serial and parallel binning factors may be selected independently.   It is parallel binning that provides the greatest speed advantage.

Use 2D ROI mode is generally chosen when exposure times exceed the 2D ROI read time, since this mode allows the most precise flat fielding.

Even if  "Collapsed ROI Mode" is to be used,  2D ROI mode is still useful for a preliminary test to confirm that the ROI size and position are adequate to contain guiding and seeing motions.

In the example at left, X_bin = Y_bin = 1, and the first exposure is at the bottom.  The first exposure should be discarded due to slightly different timing.   It can be seen that the rotation is not quite right and that Y_size needs to be increased slightly.  All rotation adjustments should be made before placing the objects near the serial register since rotation will include a translation component unless they are on the optical axis.
 


Collapsed ROI mode   (Ysize = Ybin    )

Collapsed ROI mode produces only one line of output per exposure.  It is invoked by setting Y_bin equal to Y_size.  Collapsed ROI mode allows faster sampling, higher duty cycle, and helps overcome read noise (by binning). Charge smearing due to the lack of a shutter is also irrelevant.  but the flat field correction is inferior and variations in the vertical position of the object within the ROI cannot be gauged.

X_bin is set independently.   In most cases the sample rate will be fast enough that the noise is dominated by read noise limited rather than shot noise.   In this case it is advantageous to bin in X as well as Y so that the signal is contained in a single pixel.   Sky noise can be beaten down by reading additional sky pixels with little speed penalty since in Collapse ROI mode the total parallel shift time is usually much greater than the time to read a heavily binned line.

In the example at left X_bin = 1 to show the extent of the motions induced by drive errors and telescope shake.  (The reference star is not shown.)

Gain override

Since the time spend reading pixels is insignificant for Collapsed ROI mode and in most cases for 2D ROI mode as well, then the gain is forced to a high value when the HSP is enabled, to achieve lowest noise.   When normal readout is selected the gain is switched back to whatever value was indicated by the gain parameter in the setdetector menu.

Taking a high speed exposure sequence

Invoke the setdetector menu in the dark blue IRAF window called Arcon Acquisition and check that the following parameters have been set correctly: The arcon command "HSP" prompts for number of images to take, sample period, image title and root name for the images, then invokes an exposure sequence which gets stacked into a single IRAF image.   (Note that the number exposures per image is defined in the setdetector menu, and should not be confused with number of images.)  The HSP makes sure that the Arcon is waiting for the first strobe timing strobe before requesting that the TCS generate the necessary pulse train slaved to the GPS clock.   The HSP command logs the requested start time and period in the image headers.

Charge smearing

The shutter remains open for the entire high speed exposure sequence including the readout.  In 2D ROI mode this adds a vertically smeared component which is uniform (insofar as the source is constant) and only appears on lines further from the serial register than the star.  The smeared charge is first accumulated during the reverse clocking to erase the region.  The conventional image is then accumulated during the exposure delay.  Then a second and somewhat stronger smeared component i saccumulated during the readout since this proceeds more slowly than the reverse clocking.  The relative contributions can be deduced from the messages printed during compilations. [We should record this timing info in the header]  See the engineering notes below for more details of the readout algorithm.    As one approaches maximum exposure rate, the smeared component will be dominate since the image will spend little time stationary.

In Collapsed ROI mode the smearing is irrelevant since the lines are added together.

Image motions due to telescope shake, guiding errors and seeing will move the star back and forth across the edge of the region as it sweeps by at beginning and end of the exposure modulating the amount of light collected.   To minimize this error the vertical shift rate is pushed to the limit when HSP mode is enabled on the basis that vertical charge transfer efficiency and full well are not critical in this mode due to the small number of vertical shifts and brevity of the exposures.  This error becomes less significant when the exposure period is increased relative to the minimum.  As such Collapsed ROI mode is less susceptible to this kind of error.

When exactly does the exposure begin and end ?  ....Duty Cycle ?

The times logged in the header describe the moments at which each readout starts, encoded as time of first start and interval between starts.   The time spent shifting charge is equal to the "Minimum Period" printed in the Arcon Acquisition Window at the end of the waveform compilation. i.e. whenever HSP mode is selected or any of its parameters are changed.

For Collapsed ROI mode it doesn't matter whether the light is collected during charge shifting or not and the duty cycle is very high, usually greater than 94%.  The only dead time is the time while performing the extra reverse line shifts, defined by Y_back, since these are the only lines which are not binned into the serial register.   For 2D ROI's the effective duty cycle depends on how much of the smeared charge is included in the photometry.

Serial register sensitivity

The serial register is not masked off on the T2K3 CCD at the 0.9m telescope nor on the T2K6 used at the 1.5m.  It behaves like another line at the edge of the CCD, albeit one which is twice as high.   If the stars are placed too close to the serial register some horizontal smearing may be seen as a result of the charge collected from the wings of the star while the line is being read out.  The details depend on ROI position and serial shift rates.  Except when pushing the sample rate limits this effect is unlikely to be discernible.

In 2D mode the serial register is flushed prior to the fist line transfer so that the serial register sensitivity doesn't cause a brightening of the first line.   In Collapsed ROI mode, the serial register simply forms an extension of the ROI.

Flat fielding issues

Flat fields will have a much higher average flux per pixel.  As a result it is likely that Y-back will need to be increased to avoid charge blooming into the region of interest when taking flat fields.  This should not affect the flat field calibration.  [Yet to be verified !]

Collapsed ROI's offer many advantages but suffer from a fundamental problem with flat fielding since there is no way to tell if the image moves within the binned pixel.  If sufficient signal is available to overcome read noise one can reduce X_bin so that the flat fielding can be performed in at least one dimension.  The speed penalty for this is minor, but the noise penalty could be serious.  Apart from avoiding flat fielding anomalies such as dust spots, it should be possible to mitigate this problem by defocusing the telescope to produce a rectangular profile to average together more pixels.   [This needs to be tested !]
 


 
 

ENGINEERING NOTES

The waveforms for the Arcon3.3/T2K3 and Arcon3.6/T2K6 have been modified to support HSP mode.  These modifications have been made in a copy of the usual waveform directory, /xp/run/waveforms/Arcon3.3_HSP  (or /xp/run/waveforms/Arcon3.6_HSP).   One can return to the original version in case of doubt by changing the link, xp/run/waveforms/Arcon3.3 to point to xp/run/waveforms/Arcon3.3_normal instead.  This should not normally be necessary.

Under normal circumstances it should be sufficient to change the detpars.par link (in the above waveform directory) to point to detpars.HSP.par instead of detpar.normal.par.   It is important to set HSP=no before hiding this parameter by changing the detpars.par link back to normal.

The following files were modified in the /xp/run/waveforms/Arcon3.3_HSP directory:
 

detpars.HSP.par Contains additional parameters for HSP.
detpars.normal.par Equals old detpars.par
detpars.par Now a link to one of the above.
Obs.wdef Select  read.HSP.wdef  or  read.wdef
Change order of include files.
config.wdef Add HSP parameters
Force dwell time to maximum for HSP.
constants.wdef Faster ParallelShift for HSP than normal. 
shift.wdef ReverseParallelShift waveforms for MPP and non-MPP.
ReverseSerialShift waveform. 

and in the /xp/run/waveforms/Arcon3.3_HSP directory:
 

read.HSP.wdef read1pixel.wdef  only for binfactpor in use
readNpixels  for binfactors > 6
LineRead  ...variations for single/dual readout
HSP_Expose
uReadout
shutter.wdef Change Obs_start/end to Open/CloseShutter

 

For single amplifier, the readout algorithm is as follows.....

#define  X_COUNT        (Xsize/Xbin)
#define  Y_COUNT        MIN(1,Ysize/Ybin)
#define  X_SIZE         (X_COUNT*Xbin)                /* Actual X size */
#define  Y_SIZE         (Y_COUNT*Ybin)                /* Actual Y size */
#define  X_SKIP1        (NumPreskips+Xnear-X_SIZE/2-1)
#define  X_SKIP2        MAX(0,Xfar-Xnear-X_SIZE-Xbin)
                        /* X_SKIP2 is reduced by Xbin since the pixel waveform
                         * does read-then-bin rather than bin-then-read. */
#define  X_ERASE        MAX(0,NumColumns-(3*X_SIZE)/2-X_SKIP1-X_SKIP2-Xnear+3)
                        /* = overscan_posn - shifts_done - near_edge_of_ROI + 3 */
#define  X_ERASE_REV    (NumPreskips+X_SKIP1+X_SIZE+X_SKIP2+X_SIZE+50)
 

ASSIGN   X_COUNT          -> Xcount;
ASSIGN   Y_COUNT          -> Ycount;
ASSIGN   Y_SIZE+Yback     -> Yerase;
ASSIGN   X_SKIP1        -> Xskip1;
ASSIGN   X_SKIP2        -> Xskip2;
ASSIGN   X_ERASE        -> Xerase;
ASSIGN   X_ERASE_REV    -> Xerase_rev;
ASSIGN   NumColumns+10  -> SerFlush;

NEXUS "LPic ccdposition ACEB001 1 1", 2*X_COUNT, Exposures*Y_COUNT;
NEXUS "LPic roi set 1 1",             2*X_COUNT, Exposures*Y_COUNT;
 

/**************************************************************************/
 CHAIN LineRead                          ABORT=NEVER
/**************************************************************************/
{ Ybin          * ParallelShift         /* Y-bin entire ROI */

#if Y_COUNT==1                          /* Collapsed ROI mode only */
  Yerase        * ReverseParallelShift  /* Erase ROI plus a bit */
#endif

#if X_SKIP1 > 0
  Xskip1        * SerialSkip            /* Skip to ROI start */
#endif

  Xbin          * SerialBin             /* Bin super-pixel after skips*/
#if Xbin > 6
  Xcount         * readNpixels          /* Sky/object/sky super-pixels */
#else
  Xcount         * read1pixels
#endif

#if X_SKIP2 > 0
  Xskip2        * SerialSkip
#endif

  Xbin          * SerialBin             /* Bin super-pixel after skips */
#if Xbin > 6
  Xcount        * readNpixels           /* Sky/object/sky super-pixels */
#else
  Xcount        * read1pixels
#endif

#if X_ERASE < X_ERASE_REV
#if X_ERASE > 0
  Xerase        * SerialSkip
  PRINT "   Xerase   =", Xerase,      " forward serial skips after line read";
#endif
#else
  Xerase_rev    * ReverseSerialSkip
  PRINT "   Xerase   =", Xerase_rev,  " reverse serial shifts after line read";
#endif
 };
 

/**************************************************************************/
 CHAIN HSP_Expose                       ABORT=NEVER
/**************************************************************************/
{ 1             * Wait_for_sync         /* Wait for GPS strobe */
#if Y_COUNT==1                          /* Collapsed ROI mode */
  1             * LineRead
#else                                   /* 2D ROI mode */
  SerFlush      * SerialSkip            /* Dump light detected by ser reg */
  Ycount        * LineRead
  Yerase        * ReverseParallelShift  /* Erase ROI plus a bit */
#endif
 };

 PRINT "Minimum sample period =", END/(1 ms), " ms,   Duty cycle =",
                          (END-Yback*ReverseParallelShiftT)*100/END, " %";
 
 
 

/**************************************************************************/
 CHAIN uReadout                         ABORT=BOTH
/**************************************************************************/
{
#if HSP_Shutter
  1             * OpenShutter
#endif
  1             * bReadout
  Exposures     * HSP_Expose
  1             * eReadout
#if HSP_Shutter
  1             * CloseShutter
#endif
 };
   NEXUS_NAME IS uReadout;

               .....see /xp/run/waveforms/Arcon3.3 for the dual readout case.
 

KNOWN PROBLEMS

APPLICABILITY TO SEEING MONITOR

A "Differential Image Motion Monitor" is an instrument which measures atmospheric turbulence, from which the "seeing" is inferred.  To discriminate between atmospheric effects and telescope vibration or drive errors, the same star is imaged through a pair of sub apertures separated by 10 to 20 cm.  A prism placed over one sub aperture ensures that the two images do not overlap.  Telescope vibration and guiding errors displace the two images identically whereas atmospheric turbulence produces both correlated and uncorrelated motions.  Since the spatial spectrum of the turbulence is known, a measurement on a single baseline (aperture separation) is used to estimate overall seeing amplitude.

The DIMM's in use at ESO and CTIO use a 10 um pixel 240x320 SBIG CCD camera.  The small CCD helps but the frame rate achieved with conventional readout waveforms is still only 3-4 Hz, and only 10% of the time is spent exposing.

Implementing the Collapsed ROI algorithm with X_bin = 1 would greatly increase duty cycle to nearly 100% without modifying the overlying software.  Shorter exposure times would also become an option.
 
 

APPLICATION TO SODIUM LAYER MEASUREMENTS

Experiments are in progress to measure the seasonal and nightly variations in the Sodium density profile in the upper atmosphere in preparation for designing Adaptive Optics using Laser Guide Stars.   To date (July 2001) a series of images have been taken of the laser plume using a region of interest in the middle of the CCD.  The one dimensional profile is extracted from each image and then stored as a line in a an image in which one axis is time.

A much greater sample rate and duty cycle could be acheived by rotating and offsetting to place the plume parallel to the serial register, then using the Collapse ROI HSP mode to read it out.  This would eliminate the profile extraction step too.