1. Introduction
This is a preliminary guide to using the IRAF based user interface with CTIO's new CCD controller, Arcon, to obtain spectroscopic data with the Bench Mounted Echelle at the 1.5-m telescope. It should be read in conjunction with the user's manual for this instrument.
Both the hardware and software for Arcon are still under development and are consequently a little buggy. This manual is pretty buggy too. The mountain Observer Support staff, have, as yet, little experience with the system and thus will be less able to help with problems than is usual, and will often have to refer to the "experts" in La Serena. Please bear with us during this transition phase. But please also be diligent about reporting any problems and bugs you may encounter and please feel free to make any suggestions or criticisms you may have.
The IRAF based user interface allows observing commands to be sent to Arcon from within the IRAF cl. This results in a single uniform user interface for data taking and data reduction and allows the Arcon user to employ features of the cl such as the parameter mechanism and the history editor. It also allows advanced users to write cl scripts which freely mix data acquisition and data reduction operations. It is nonetheless simple enough, that users need very little knowledge of IRAF in order to obtain their data. As far as possible the user interface is modeled on the ICE software in use at KPNO; the present manual also owes more than a little to the KPNO ICE manual. However, users should be aware that the two systems are NOT identical, so that parameter lists may differ and data acquisition scripts written for ICE will require some modification.
2 Logging in and Logging Out
2.1 Starting From Scratch
In the console room of the 1.5-m telescope you will find two Sun Computers. One of these, a SPARC-station 10/41, is the Arcon data acquisition computer. You must log on to this machine in order to use Arcon and, in general, will also want to use it for all your data reduction. The other machine is an older VME-bus based Sun 4/330; currently peripherals such as the 9-track and exabyte tape drives and laser printer are attached to this second Sun. If more than one observer is present at the telescope, or if you find a single Sun screen to be too restrictive, you may wish to use this second machine for your reductions. However, if you do so we recommend you treat it as an "intelligent terminal" from which you log on to the data acquisition machine. Accessing images across the network imposes a very stiff penalty in the form of increased I/O overheads when reducing your data.
When you first log on to the data acquisition computer using your visitor account, you will be asked which windowing system you prefer -- both Sunview and Openwindows are available -- as follows(1):
- What windowing system do you want?
0 -- No windowing system
1 -- SunView
2 -- OpenWindows
Enter selection (1): 1
Pick whichever you are most comfortable with, or select Sun View if you know neither (in this case you may want to refer to Appendix C). The system remembers your answer, so you will only be asked to make this choice the first time you log on to a given machine. You will then be asked if you want to start up the data acquisition program, or just to reduce data as follows:
- What type of IRAF setup do you want?
1 -- IRAF for data reduction
2 -- IRAF with Arcon
Enter selection (1): 2
If you select "2", as in the example, your chosen window system will start up and several windows will open so that your screen should look something like Figure 1; the function of these various windows will be explained in Section 2.2. An automatic start-up procedure will also be run which initialises the controller hardware and software. This takes some time; when it finishes the message "Array Controller ready for user commands ..." will appear along with a beep. Wait until you see this message and hear the beep before continuing.
At present there are a few additional steps which you must carry out manually by typing the following in the specified windows (as a reminder, the necessary commands are shown in the parentheses in the title bars of most of these windows):
In the window labeled Arcon STATUS: type astro1_up
In the very small green window: type countdown
The IRAF cl will automatically start up in the blue window labeled "IRAF, Acquisition (arcon......)". You must first load the arcon package, and then the specific package for the instrument you are using --bme. To do so proceed as follows:
cl> arcon
astronomer. contributed. fpccd. pfccd.
bme. csccd. laboratory. quad.
cfccd. dtools. nfccd.
ar> bme
Connecting to the Nexus...
abort detpars@ movie pflat stop
bias disconnect object preview tchange
comp gainchange observe resume teloffset
connect instrpars@ obspars@ sflat telpars@
dark more pause shutter zero
bm>
Loading the instrument package automatically establishes the connection to the controller, as indicated by the "Connecting to the Nexus...." message hidden amongst the package menus. Exiting from the package with bye or (usually inadvertently!) ctrl-z breaks the connection.
You are now ready to start taking data. However, before you begin doing so in earnest, you should check that everything is set up and working properly: (i) Check you know where your home directory is (show home); (ii) Check where your imdir is (show imdir), that it realy exists (cd imdir) and that you have the necessary privileges (copy home$login.cl imdir$junk; delete imdir$junk); (iii) check there is plenty of disk space (disks); (iv) carry out the basic tests of the operation of the detector and controller described in the instrument manual. If there are problems with any of this seek help from observer support.
2.2 A Guided Tour of the Windows
Once you have completed the initialization procedure, your screen should appear similar to Figure 1. The key windows for taking data with Arcon, identified by the names given in their title bars, are as follows:
Arcon CONSOLE -- As its name implies this window serves as a console for Arcon. While you are observing you will see many messages appear here, only some of which will be repeated in the IRAF acquisition window. This window takes up a lot of screen space so you will probably prefer to close it. But don't quit from it. In the event that something goes wrong, the diagnostic messages appearing in this window may tell you (or at least us) what happened.
Arcon STATUS -- This very important window gives several lines of information summarizing the status of the controller, the instrument, and any ongoing exposures (see Figure 2). The first line shows what the controller is currently doing. if you have just brought the system up, it should read "CONTINUOUSLY_ERASING" indicating that the CCD is idle and is continuously running the erase cycle; if it doesn't, chances are something went wrong in the initialization and you should seek help. During exposures this line should read "INTEGRATING", and should change to "READING" as the CCD is read out. Other messages which may occur will be described later as appropriate. Just below the status line are counters showing the number of seconds left in the current exposure, the number of exposures left in the current sequence, and, during read out, the number of buffers of data successfully transferred to the Sun. Also shown are parameters of the current exposure such as the title, picture name etc. Finally the bottom three lines of the status window show, during readout, the values of various statistics of the image. These numbers are generated on the fly by the real time display program (see Section 5) and are used in setting the display look up table. You, however, may find this information useful when adjusting exposure times during sequences of sky flats, etc.
COUNTDOWN --This is the very small window with the very large font. It provides a copy of the exposure time counter for the visually handicapped.
IRAF Acquisition -- This blue colored window is one of two running the IRAF cl. We recommend you type all data taking commands here, so that your data taking and data reduction activities are well separated and don't interfere with one another.
IRAF Reductions -- This window (colored a dirty brick red) is the other IRAF window. We suggest that normal IRAF commands, used to examine or reduce your data are entered in this window.
IRAF Display Window -- depending on your choice of window environment there will be either an
"SAO Image" (Open Windows) or "Imtool" (Sun View) window which will be used when displaying your images from IRAF.
2.3 Shutting Down and Logging Out
Before you log out, for whatever reason, you must first stop the various processes, related to the controller running on the Sun. If you don't do this they will continue running and cause all sorts of entertainment for you or whoever else next tries to bring up the system. To do so first, in the blue "IRAF, Acquisition" window, break the connection to the controller by either typing disconnect or by exiting from the instrument package by typing bye. Then in the "Arcon CONSOLE" window type:
arsh> stop
ctioa1% kleenex
This last command should clean-up all the processes related to Arcon; unfortunately it doesn't always. To see if it worked type:
% NexUp
No match.
No match.
No match.
If you get "No match." three times in a row, as above, all is well. If instead you see something like
ctioa1% NexUp
root 2227 0.0 3.0 2664 844 p6 S 17:32 0:05 muxnex
arcon 2229 0.0 0.6 172 160 p2 S 17:32 0:00 arsh -c 0 -e 20
arcon 2228 0.0 0.0 144 0 p2 IW 17:32 0:00 arsh -c 0 -e 20
/dev/nexc0
Nomatch.
-rw-rw-rw- 1 arcon 0 Jul 13 17:32 /tmp/xpim2229.1
-rw-rw-rw- 1 arcon 0 Jul 13 17:33 /tmp/xpim2245.1
then there are some leftover processes which you must kill by hand as follows,
ctioa1% kill -9 2227 2229 2228
ctioa1% rm /tmp/xpim*
In the first command the list of numbers after the -9 are the pid's of the leftover processes as displayed by NexUp. The second command performs, the not strictly necessary, step of deleting any temporary files left behind by the system(2).
Now you can exit from the windowing system and log out completely, by moving the mouse to a blank area of the screen, then hold down the right mouse button, and select "exit" from the menu which will appear.
2.4 Warm Starts
Rather more often than we would like, currently about once a night, something or other happens which causes the system to hang requiring that the software be reloaded.
Before doing so it is worth testing to see if the problem is confined to the IRAF interface layer, by proceeding as follows:
bm> flpr
bm> disconnect
Disconnecting from Nexus ....
bm> connect
Connecting to Nexus ....
bm> flpr
This re-establishes and initializes the connection to the Arcon and, for good measure, flushes out any brain-damaged executables locked into the IRAF process cache. Having done this, test to see if the problem has gone away by taking a "zero" frame.
If this fails, or if the status window or real time display have stopped functioning, then the problem is probably in Arcon itself and it is best to reload the software from scratch. This takes very little time. First follow the steps in Section 2.3 for shutting down the system, but do not perform the very last step of exiting from the window system. Instead restart the Arcon software as follows:
In Arcon CONSOLE type arcon_visitor, then wait until you see the message "Array Controller ready for user commands ..." and hear a beep before continuing.
In Arcon STATUS type astro1_up or simply "!!"
In green font window type countdown or simply "!!"
In the blue IRAF window type connect or reload the instrument package (bme)
2.5 If All Else Fails
Just once in a while a problem will occur which just refuses to go away even when you reload the Arcon software. This may be due to a hardware failure. However, it may also be that the Sun has got irremediably confused, in which case the rather drastic step of rebooting it may be called for. Should this prove necessary contact observer support. They can show you how to reboot the Sun safely, cleanly, and without having to know the superuser password.
3. Taking Data
3.1 Observe - the only command you really need to know
All data taking can be done by using a single command: observe. This command takes one or more ccd exposures, as in the following example(3):
bm> observe
Exposure type (|zero|dark|object|comp|pflat|sflat|) (zero): object
Number of exposures to take (1:) (1):
Exposure time (0.:) (0.): 1000
Title of picture (bias 16/07/93): HH80/81
bm> observe sequence 1 "object" 1073 "HH80/81" 1000.
4 amplifiers in total
Observation finished...
You will be prompted for all the information required which includes:
exposure type: can be "zero" (sometimes referred to as bias), "dark", "object", "comp", "pflat", or "sflat". Note that when selecting from a list of options like this you may enter any unique abbreviation.
number of exposures to take: a sequence of this number of pictures, all having the same parameters, will be taken.
exposure time: is in seconds, and will not be requested in the case of exposures of type "zero" for which it is 0.0 by definition.
picture title will be included as the title in the IRAF image header.
Note that in each parameter query you will be supplied with a default value, which you can accept by simply hitting <cr>; these values are just the previous entries.
As soon as you enter the title, the CCD will be prepared, and then the exposure will begin. The first line in the status window will change from "CONTINUOUSLY_ERASING" to "INTEGRATING" and the status window will also show parameters of the exposure such as the picture title. A counter in the status window, and more legibly the countdown window will begin displaying the time remaining in the exposure. At present these counters are not exactly synchronized with the internal clock in the controller so for very long exposures they may indicate that a few seconds remain when the exposure is in fact complete.
Note that the observe command terminates as soon as the exposure starts and you can enter other commands in the IRAF acquisition window. While you could type any IRAF command you like, we suggest you keep this window free for entering the special exposure control commands described in section 3.2
When the exposure finishes the CCD will be read out. The first line in the status window will change to "READING" and the "buffers read" counter will indicate the number of buffers of data successfully transferred to the Sun. The data is initially written in the controllers internal format to a spool file on /tmp, but it is automatically converted into an IRAF image within a few seconds of the exposure finishing. The name of this IRAF image is derived from the exposure type by appending a running number (see section 4.1 for how to adjust this number) which is automatically incremented after each exposure. The image header will be in the current directory (at the time the observe command was issued) and the pixel file will be located in your imdir.
During readout the image will also be displayed on the real time display (see section 5). This occurs independently from and in parallel with the transfer of the data to disk on the Sun. You need not wait for the real time display to finish before starting another exposure.
If you requested that observe take only a single exposure, the message "observation finished ....." will appear in the IRAF interface window as soon as the readout is complete; things are then ready for you to start another exposure. If, instead, you requested a sequence of several pictures, the next exposure will start automatically. You may immediately examine or process the resulting image even though the sequence is not complete. Note that the "pictures remaining" counter in the status window shows how many exposures remain in the sequence. Once the final picture has been readout the message "sequence finished ......" will appear in the IRAF interface window. Should you miss the end of sequence or end of exposure message, note that the CCD is idle and things are ready for you to initiate new exposures, whenever the top line of the status display reads "CONTINUOUSLY_ERASING".
3.2 Exposure Control Commands
The following commands can be used to modify an ongoing exposure:
pause - Pause the exposure e.g. while waiting out passing clouds.
resume - Resume a paused exposure.
tchange - Change exposure time. You will be prompted for the amount by which to change the exposure which may be positive or negative. If used during a sequence the duration of the present exposure and all subsequent exposures is changed.
stop - Stop the exposure early, read out the CCD and save the data to disk. If used during a sequence, the sequence is also terminated.
abort - Abort the exposure. The CCD is not read out and any data collected during the exposure is irrevocably lost. If used during a sequence, the sequence is also terminated.
Note that, currently, abort does not work properly if issued while the CCD is being read out; the exposure is terminated correctly, but the CCD controller hangs up and has to be re-initialized. If you want to abort a sequence of zero's or short exposures, use the stop command. You then will have to wait while the current exposure finishes reading out, but this is less time than required to re-initialize the controller.
3.3 Other Commands For Taking Data
In addition to observe, there are specific commands to take one or more pictures of each type:
dark - Take one or more exposures of type dark
object - Take one or more exposures of type object
comp - Take one or more exposures of type comp
pflat - Take one or more exposures of type pflat
sflat - Take one or more exposures of type sky flat
zero - Take one or more exposures of type bias
Except, of course, for the exposure type these commands take the same parameters (and prompt for them in the same order) as does observe. Apart from saving you entering that one extra parameter, use of these commands allows one to set default parameter values, and also select which parameters are prompted for according to picture type.
Another useful command is:
more - Take one or more exposures exactly like the previous one
The more command is slightly unusual in the way it prompts for parameters (it is patterned after commands like directory and help). If you type
cf> more
you will not be prompted for the number of exposures (as one might expect) but rather a single exposure will be taken (which more often than not is what you actually wanted to do). Conversely
cf> more 10
will take ten more exposures.
3.4 And Two That Take No Data - Preview and Movie
Two commands which may be useful when setting up the instrument, for instance, for centering the wavelength region of interest on the detector and for establishing rough focus, are:
preview - Take a CCD exposure and display it on the real time display but does not write any data to disk.
movie - Loop continuously taking and displaying preview exposures until terminated by stop.
Both commands prompt for a single parameter the exposure time; since the readout time in quad mode is about 12 seconds for the Tek1024 CCD some degree of real-time feedback can be obtained using movie with exposure times of a few seconds. In the near future preview and movie will optionally use special waveforms which reduce the readout time at the expense of increased noise &/or reduced spatial resolution.
4 The Parameter Files
As with ICE many of the nitty-gritty details of taking your data are hidden from your immediate view in four parameter files:
obspars This contains several parameters which you, the astronomer, can use to tailor the behavior of observe to your liking.
instrpars This parameter file contains information relating to the instrument being used. Currently the BME is not under computer control and so this parameter file is not used.
detpars One day soon this parameter file will control the fundamentals of how the CCD is readout -binning, gain, regions of interest, etc. Right now Arcon does not support any of these features, so this parameter file is unused.
telpars Except at the Curtis Schmidt all CTIO telescopes are run by a control program which is interrogated by Arcon, at the start of each exposure, in order to obtain information such as the time, telescope coordinates, etc. for inclusion in the image header. This parameter file currently does nothing and has only been retained to maintain compatibility with ICE.
You should review these parameter files, and may want to change some values, at the start of your run, but will probably leave them alone thereafter.
These parameter files can be listed by using the lpar command, eg.,
cf> lpar obspars
and may be edited using the parameter editor, epar, or by simply typing the name of the parameter set e.g.,
cf> epar obspars -OR-
cf> obspars
To change a value, in either case, move the cursor up and down with the arrow keys until you are on the correct line and then simply type the new value followed by <cr>. When done editing the parameter file type cntrl-z
4.1 obspars
The parameter file obspars contains the following parameters:
I R A F
Image Reduction and Analysis Facility
PACKAGE =
bme
TASK =
obspars
ccdtype = zero Exposure type
npics = 1 Number of exposures to take
picture = 1 Picture number of first exposure
exposure = 0. Exposure time
title = Title of picture
(autopic = yes) Generate picture number automatically ?
(mode = ql)
($nargs = 0)
These parameters are used for all exposures. It is not necessary to set the values of most of them, since they are prompted for as needed. The values appearing in obspars are simply the values entered the last time observe was run. The parameter autopicnum determines whether observe will prompt you for the running picture number, picture, which forms part of the name of your images on disk. The value of picture is always incremented after each exposure. If autopicnum=yes (the default) the automatically derived value will always be used and you will not be prompted. If autopicnum=no you will be prompted for a new value of picture for every exposure, the automatically derived value being supplied as the default. In either case you can reset the sequence by just changing the value of picture in obspars. Note that picture will get out of step if you abort an exposure or sequence; the value used will be the one which would have been appropriate if the exposure or sequence had completed normally.
4.2 instrpars
This parameter set is not used at this time.
4.3 detpars
This parameter set is not used at this time.
4.3.1 Setting the Gain
That said, it is possible to change the slope time of the "double-correlated-sampler" -- usually erroneously referred to (by astronomers) as the gain -- which determines the number of e-/ADU, the readout noise, and the readout time. Currently, however, this is not done via entries in detpars, but rather by running the program gainchange (at least at CTIO, programmers do what astronomers tell them to do, even when its wrong), as follows:
cf> gainchange
Gain setting (0 for list) (0): 2
*** Regenerating waveforms ***
csh /pxp/run/macro/wdl Tek1K_1 -I..
WDL revision 2.18
..........
*** Suspending the sequencer ***
*** Reloading new waveforms ***
You will be prompted for the gain setting, which must currently be one of a pre-defined list of values. The CCD readout waveforms are then edited, recompiled, and down loaded into the controller. This all takes about 12s. To get the list of acceptable gain settings, and also to find out the corresponding number of e-/ADU, readout noise, and readout time, enter a gain setting of 0, as follows:
as> gainchange
Gain setting (0 for list) (0):
dcsT Delay RON ADU/e- readout_time (quad)
(us)(clk) (e-) (seconds)
LL LR UL UR LL LR UL UR
---- ---- -------------- -------------- -------------------
1: 11 2
2: 15 4
3: 20 2
4: 39 4
5: 80 2
^
* *** Select gain setting from the first column ***
For advice on what gain is right for you, refer to the appropriate users manual, or consult observer support.
4.4 telpars
This parameter set is not used at this time.
5. The Real Time Display
Arcon includes a real-time display which automatically shows each picture as it is being read out on a separate Sun-style monitor next to the data Acquisition computer. This occurs independently from, and in parallel with, the transfer of the data to disk on the Sun and does not slow down this process. You need not wait for the real time display to finish before starting another exposure. The real time display offers a number of convenient features:
Display of the picture begins substantially before all the data has been transferred to the Sun and converted into an IRAF image.
Various picture statistics are accumulated on the fly and are used to optimally map the 16-bit CCD data into the 256 grey-levels shown by the display. These statistics are also shown in the status window (see Section x.x) and may be useful when estimating exposure levels for sky flats.
Saturated pixels (data value 65535) are shown in red.
Quad readout pictures are automatically overscan subtracted and trimmed for the display.
The wavelength directions are labeled on the display monitor. You will soon be able to specify any rotation and flipping necessary to have the display match the spectral line atlas.
These features mean that you can always see the last picture taken to verify that the picture looks reasonable and that no important features are saturated.
The display itself takes place in two stages. A first fast pass keeps up with the readout but the data is shown at slightly reduced spatial resolution. A second, slower full resolution pass is then performed once the entire picture is available. Since the ideal mapping from 16 to 8 bits can't be known until the readout is finished the second pass may modify the look up table unless you specify otherwise (see Section 5.2).
5.1 Changing the look of the displayed picture
You can change the way the mapping from 16 to 8 bits is performed and also whether the picture is shown in normal, reversed or false color mode. You can also show the pixels above and below the mapped range in green and blue, respectively. The commands to control these settings are temporarily contained in the contributed package (a sub-package of arcon):
lut - Change the look up table.
Parameters:
video = "reverse" (normal|reverse|falsecolor)
Show stars as white ("normal"), black ("reverse") or use false-color ("falsecolor").
colors = "nocolor" (nocolor|3color)
Disable ("nocolor") or enable ("3color") the use of colors to mark pixels outside the mapping range. When enabled pixels below the range are shown in blue and those above the range are shown in green. Note this does not affect the marking of saturated pixels which are always shown in red.
map - Defines the way the 16-bit CCD data is to be mapped to the 8-bit display. The default parameter values (algorithm = "mode", low = 0.2, high = 2.0) work well for normal star fields. Try low = 0.2 and high = 1.3 to bring out nebulosity.
Parameters:
algorithm = "mode" (mode|stdev|minmax|constant|show)
The mapping algorithm to use. Options are:
mode - Map range is specified number of standard deviations below mode and above mean.
stdev - Like mode, except bottom of range is specified number of standard deviations below mean.
minmax - Map range has specified percentages of pixels above picture minimum below picture maximum. (e.g. 0.5 and 1 would have 0.5% of the pixels below the map range and 1% above the map range.)
constant - Set map range to specified values.
show - Display current picture with specified values, but do not change the map parameters for subsequent picture displays.
low = 0.2
Low value for mapping algorithm. This is expressed as a number of standard deviations for the mode and stdev algorithms, the percentage of pixels below the bound for minmax and the absolute level in ADU's for constant and show.
high = 2
High value for mapping algorithm. This is expressed as a number of standard deviations for the mode and stdev algorithms, the percentage of pixels above the bound for minmax and the absolute level in ADU's for constant and show.
5.2Controlling when display and remapping take place
The 16-to-8-bit mapping is performed based on image statistics accumulated as the picture is read out. However, it is necessary to wait until enough of the picture data is available, so that these statistics are meaningful, before using them for the mapping. Until then the mapping from the previous exposure is used. This works well when sequential exposures are of the same type and comparable duration. In the future the system will be made smarter so that the preliminary mapping used is based on information from the previous exposure of the same type scaled according to exposure time.
The remap task (also temporarily in the contributed package) controls how soon remapping is performed:
remap - Set remapping options for real time display.
Parameters:
delaydisplay = no
If set to yes, do not start displaying until percent of the picture has been read. This prevents starting to display pictures with the mapping from the previous picture, but delays any feedback on the current picture.
percent = 43
The percentage of the picture which must be read out before the mapping for the first display pass is changed from that used for the previous picture.
With the default value of percent repainting of the screen with the new look-up table will finish before the CCD read-out completes. A smaller value will result in earlier re-mapping, but an increased risk that incomplete sampling of the field will result in a poor choice of look-up table. In some cases such as when an isolated bright comparison line falls near the center of a quad readout detector, the correct mapping cannot be known until the end of the readout. When the new look-up table is poorly chosen, the display may be repainted three times; once at the start using the look-up table from the previous exposure, then again based on statistics accumulated from the top and bottom edges of the image, and finally when the readout is complete based on statistics from the entire picture. This final remapping is done at the same time that the image is displayed at full spatial resolution. Some people find this repeated repainting of the screen confusing. If you decide you do not want any remapping, set percent = 100. and delaydisplay = yes. Display of the picture will then not start until the read-out is complete, at which time you will see a fast, medium spatial-resolution display, followed by a second slower pass at full resolution, but there will be no change in the mapping.
6. Data Reduction
Most of the process of reducing CCD data obtained with Arcon is identical to that used with CTIO's older controllers and can be performed using standard IRAF tasks, in particular those in the CCDRED package. However, Arcon data does differ in a few details:
Arcon produces 16-bit data of type "ushort" (unsigned short integer) with valid data ranging from 0 to 65535. IRAF only provides limited support for this data type. In any case almost any arithmetic operation performed on such an image requires that the result be stored either as a 32 bit signed long integer, or as a real number, to avoid wrap around or loss of precision. The principle impact of this is that your images will double in size as soon as you process them. Section 6.1 describes how to write your raw images to tape without having them double in size.
There are some differences between the image headers produced by the old and new controllers. Section 6.2 describes how to set up the instrument and subsets files used by CCDRED when reducing Arcon data.
with Arcon it is possible to be read out the CCD using more than one amplifier, in order to reduce the readout time. Data taken in this way (which is the default) requires some special reduction step which are described in Section 6.3.
Consequently, this section will concentrate on the few special procedures required in the case of Arcon data, and only briefly outline the normal steps. You will find a complete set of IRAF documentation in each dome; the locally written cookbooks in a single red binder, the standard IRAF documentation in several blue binders. Observer support personnel and the Data Reduction Specialists can also provide assistance.
6.1 Writing the Data to Tape
Before you begin to reduce your data you will, of course, want to save your raw data on tape. Both 9-track and exabyte drives are available at each dome. Currently these peripherals are both attached to the secondary VME-bus based SUN, rather than the data acquisition computer.
Arcon produces 16-bit data of type ushort ("unsigned short integer") with valid data ranging from 0 to 65535. Thus the unprocessed frame from a 1K CCD requires about 2.2 Mbytes. If you wish to write this data to a FITS file using the standard IRAF wfits defaults, a 4.4 Mbyte file of 32 bit "long integer" data will be written. This is because standard FITS does not support the "ushort" data type and so, to avoid loss of precision, the IRAF default is to write the data, without scaling and as "long integers". To circumvent doubling your image size with 2.2 Mbyte of zeros, use,
da> wfits bscale=1 bzero=32768 bitpix=16 autoscale- scale+
and to read this back use,
da> rfits datatype=ushort
You should, in general, set ccdred so that your reduced images are of type "real". These reduced images can be written using the wfits defaults.
6.2 Setting up ccdred, and the Instrument and Subset Files.
Whether you use ccdred directly to reduce single readout data, or indirectly by using the quad package, you should first check that its parameters are set as required. Pay particular attention to the following:
(pixelty = real) Output and calculation pixel datatypes
As already noted Arcon produces images of type "ushort" (unsigned 16-bit integer). Although ccdred can read such images, the output images it generates are either of type "short" (signed 16-bit integer) or "real", and all its calculations are done in one of these types. Thus you must set pixeltype as shown to avoid wrap-around.
(backup = B) Backup directory or prefix
Normally ccdred operates "in place", the raw input image being overwritten by the reduced image (unless an error is detected during the processing step). However, if you set backup to some string (e.g. backup = "B") ccdred will make a backup copy of each image using the value of backup as a prefix for the image name (e.g. the backup copy of obj101 will be called Bobj101). If you instead set backup to the logical or physical name of a directory (which must already exist), then the backup image will be place in that directory. To disable the creation of backup image set backup = "".
(instrum = bme$ccd.dat) CCD instrument file
(ssfile = subsets) Subset translation file
The instrument file tells ccdred how to interpret the information contained in the image header in order to decide what processing steps are appropriate for each image (type help instruments in IRAF to learn more about this file). A suitable template instrument file for use with the BME can be found in bme$ccd.dat.
7.2 Reducing Multi-Readout Images -- quadproc
Many of CTIO's CCD's have more than one, typically four, working amplifiers. A major advance achieved with Arcon is the ability to read out the CCD using more than one of these amplifiers in parallel, leading to substantially faster read-out. Once properly reduced, such data is indistinguishable from that obtained when reading out through only a single amplifier; that's our definition of working where amplifiers are concerned. However, raw multi-readout data does look decidedly strange as shown in Figure ?(4). Firstly, each read-out will typically have a slightly different, zero level, gain, and readout noise, and may differ slightly in its departures from perfect linearity. As a result both zero frames and uniformly illuminated exposures will show a characteristic chequer board pattern, the sections of the data read through each amplifier having different levels. Secondly, there will be a separate overscan strip, used to monitor the zero level, for each readout. When four amplifiers are used, referred to as quad-readout, the four overscan regions form a vertical stripe down the center of the raw image. A CCD read out through two amplifiers can either have its two overscan strips, one above the other, at one side of the picture (parallel split) or can have them, running side by side down the center of the picture (serial split).
Because of this peculiar image layout, the initial reduction steps -- overscan correcting and trimming the images --must be performed using a special task quadproc. This splits a raw quad image into four images one for each quadrant and sets trimsec, biassec etc. in the header of these as required. It then uses the standard IRAF routine ccdproc to overscan correct and trim each quadrant image. Finally it glues the four pieces back together again and tidies up the header. The overscan subtraction performed by quadproc corrects the quadrant-to-quadrant differences in the bias level, so that no quadrant structure should be visible in processed zero images. However, you will still see the chequer board structure in flatfield and object exposures (unless the sky level is zero) because of gain difference between the amplifiers.
Quadproc itself has only one parameter, the list of images to be processed; this can be a single image name a filename template (e.g. *.imh) or an @file. Also, like ccdproc, quadproc will silently ignore any images in the input list which have already been processed, so it is generally safe to run it repeatedly with a match-all template, in order to process newly acquired exposures as they are delivered by the controller.
Several parameters of ccdproc are referenced and must be properly set before running quadproc:
(readaxis = "line") Read out axis (column|line)
This should be set to "line" for all CCD's currently in use at CTIO.
(biassec = "[1025:1060,1:1028]") Overscan strip image section
(trimsec = "image") Trim data section
The parameter biassec specifies the region of the image to be employed to determine the zero level, while trimsec specifies the portion of the image which is to be retained after trimming. If either is set to the special value "image", then its value will be taken from the corresponding keywords trimsec and biassec recorded in the image header. In the case of trimsec we recommend that you do this. The value of biassec in the header uses the entire overscan strip without allowing any margin between the data section and the bias section. Because Arcon uses a DC-coupled pre-amplifier the transition between data and overscan is very sharp indeed. Nonetheless, we recommend that you do skip the first few pixels of the overscan strip. To decide this issue for yourself, use implot to plot the average of several lines from a high exposure level image such as a flat field. Expand the transition region between data and overscan and decide how many pixels of the overscan are contaminated. Unfortunately, the way in which an explicit value for biassec must be set is somewhat non-intuitive. Currently, the values recorded in the image header are those that would be appropriate had the detector been read out using a single amplifier. Quadproc calculates the sections to use for each quadrant based on such "single readout" sections. Use imhead to determine the value stored in the image header. If it is, for instance, "[1025:1060,1:1028]" then setting biassec = "[1028:1060,1:1028]" would leave a margin of 3 pixels (1028 - 1025). Note that the overscan strip for each quadrant is only half as wide as that for a single readout. Thus in the example a 15 pixel (36 / 2 - 3) wide strip is used for each quadrant.
The remaining parameters used by quadproc control the way in which the zero correction is determined:
(interactive = no) Fit overscan interactively?
(function = "spline3") Fitting function
(order = 10) Number of polynomial terms or spline pieces
(sample = "*") Sample points to fit
(naverage = 1) Number of sample points to combine
(niterate = 1) Number of rejection iterations
(low_reject = 3.) Low sigma rejection factor
(high_reject = 3.) High sigma rejection factor
(grow = 0.) Rejection growing radius
This process consists of collapsing biassec to one dimension by averaging across its width and then fitting a function to the resulting data. The fitted function is then subtracted from the data line-by -line. For details of the fitting algorithm and general advice on how to set these parameters consult the help pages for ccdproc and the CTIO cookbook "ccdman -- Reduction Manual for CCD Direct Imaging". Specific advice for the case of the Tek10242 CCD's, which require special treatment, is given in Section 6.3.
Note that the overscan fit can be done interactively. While you are unlikely to want to do this for all of your images, it is a good idea to process a representative sample of object and calibration frames interactively in order to ensure that the parameters have been properly chosen, before going on to blindly process the bulk of your data. When doing this with quadproc you will be presented with four fits per image, one for each quadrant. Make sure that ccdred is set to keep a backup copy of the raw images (e.g. ccdred.backup = "B") or that you have otherwise safeguarded your raw data before performing such tests.
Sequences of calibration exposures can combined using the standard ccdred task combine (or the related image type specific scripts zerocombine, flatcombine, etc.). Follow the instructions for using these tasks given in the cookbook. Sequences of zeros and darks can be combined and then quadproc can be run on the final combined image. The same is true for dome flats, provided there is no significant change in the brightness of the bulbs illuminating the white spot. Sky flats (and sometimes dome flats), for which the signal level changes appreciably during the course of the sequence are scaled to a common signal level during the combining process to facilitate the rejection of cosmic rays. In this case each image in the sequence must be separately overscan subtracted with quadproc before running combine. WARNING: Do not do this by setting process=yes in flatcombine; this would run ccdproc instead of quadproc trashing your calibration data. The statsec parameter in combine which specifies the section of the image to be used when determining the scale factor to be applied to each image should be set so that the statistics box falls entirely in one quadrant of the detector (e.g. statsec = "[200:400,200:400]" for a 10242 CCD).
You must next subtract your combined and processed zero frame from your combined and processed darks, dome flats and sky flats using ccdproc.
There are several alternate flat-fielding strategies: use dome flats alone; use sky flats alone; use dome flats plus an illumination correction derived from the sky flats. In addition your sky flats may be exposures of the twilit sky, of "star-less" high latitude fields, or may be derived by combining program object exposures of a large number of different fields. Consult the instrument manual for a discussion of the pros and cons of each method, and the cookbook for the appropriate recipe once you have chosen the flavour of flatfield you want.
With the complete set of reduced calibration frames in hand you are ready to perform the remaining reduction steps -- zero subtraction, dark subtraction (optional) and flatfielding -- on your program frames using ccdred. Follow the instructions for doing this given in the cookbook. Warning: Be very careful, that all the images you ask ccdred to process, and all the calibration images that will be used, have already been processed by quadproc; the task ccdlist will show you what processing steps have been performed on each image. In general it is a good idea to prepare an "@list" containing the names of the images you want to process at the outset, and use this same list with first quadproc and later ccdproc (i.e. quadproc @list; ccdproc @list).
A couple of features (a.k.a. bugs) in quadproc are worth knowing about. Firstly, if quadproc fails, or you abort it with <ctrl-c>, it may leave behind the temporary images created for each quadrant. Those corresponding to, for instance, obj100.imh will be called obj100.11.imh, obj100.12.imh obj100.21.imh and obj100.22.imh. You must delete these images (imdelete obj100.??.imh) before you can rerun quadproc to process obj100. If ccdred is set to make backup copies of the images during processing there may also be backup images for each quadrant e.g. Bobj100.11.imh, etc. These also should be deleted before you continue.
Secondly, ccdproc may decide, for a variety of reasons that it is inappropriate to process a particular image and will silently skip over it in the input list. In general this occurs because there is no instrument file, or because this file is in error. A number of checks have been built into quadproc so that it too will skip images which ccdproc will ignore, however, there are undoubtedly some cases that can still slip through the net. If this happens, the image is first split into its four quadrants, ccdproc is then called but does nothing, and the image is glued back together exactly as it was before. However, quadproc also adds some extra keywords to the header. As a result once you find and fix the problem causing ccdproc to skip your image, you will find that it now refuses to process your image complaining about inconsistent image sections. If you are keeping backup copies of your raw images, just delete the "processed" version and rename the backup copy; otherwise use hedit to delete the ccdsection keyword mistakenly added by quadproc as follows:
qu> hedit obj100 ccdsection delete+ add- ver-
7.3 Overscan Subtracting Data from the Tektronix 10242 CCD's
The Tekronix 10242 CCD's in use at CTIO are sufficiently old that they were fabricated without a metalization layer to prevent light falling on parts of the chip outside the imaging area. Thus these regions are light sensitive and charge up during the exposure. This produces a slight increase in the bias level which decays to the nominal value during the course of the readout. This manifests itself as a rapid exponential decrease of the zero level over the first several lines of data read out. The amplitude of the effect depends on the level of illumination and each of the four readouts responds somewhat differently. It is most noticeable in flat fields (where the peak offset of the bias level approaches 0.1% of the signal) and is barely detectable in normally exposed direct images; also beware of bright objects at the edges of the imaging area, especially in the corners of the detector. This problem is not new with Arcon. However, because of the much faster readout, especially in quad mode, the excess signal is spread over several lines of data, rather than being confined to the first one or two.
This effect is adequately corrected by proper overscan subtraction, but because of the rapid decay of the zero offset, a very high order fitting function is required -- we suggest setting function = spline3 and order = 10 - 20 in ccdproc. We stress the importance, in this case, of interactively inspecting the overscan fits for a representative sample of data and, especially, flatfields and if necessaria adjusting the fitting parameters.
Appendix A: Software Summary - The Only Page You Really Need to read.
Loading packages etc.
arcon - Load main Arcon package
bme - Load instrument specific package for Bench Mounted Echelle Spectroscopy
connect - Make connection to detector controller (automatically done by bme)
disconnect - Break connection to detector controller (automatically done on bye from bme)
Data taking commands
observe - Take one or more exposures prompting for type
dark - Take one or more dark exposures
object - Take one or more object exposures
comp - Take one or more comparison exposures
pflat - Take one or more projector (quartz lamp) flat exposures
sflat - Take one or more sky flat exposures
zero - Take one or more zero exposures
more - Take more exposures of the previous type
preview - Take an exposure preview frame. Data is shown on real time display but not saved to disk.
movie - Continuously take preview exposures until stopped with abort
Exposure control commands
abort - Stop exposure and do not readout detector
stop - Stop exposure and readout detector
tchange - Change exposure time
pause - Pause current exposure
resume - resume paused exposure
Parameter Sets
obspars - Observing parameters
detpars - Detector parameters (currently unused)
telpars - Telescope parameters (currently unused)
instrpars - Instrument parameters (currently unused)
Appendix B: Self Help with Problems - A Check List for the Wee Hours of the Morning.
Appendix C: A Survival Guide to Sunwindows and Openwindows for the Uninitiated.
1. In examples like this, text entered by the user will be shown in bold type in order to distinguish it from prompts and other output from the program.
2. 2 Refer to section ?.? before doing this. Those temporary files may contain your missing data!
3. 3The text shown in italics in this example is debug output from the controller. We are working very hard to make this annoying drivel disappear. Note especially the IRAF "bm>" prompt when the observe command terminates cunningly hidden by the command echo. All commands generate similar gibberish interspersed with the interactive prompts but this will not be shown in subsequent examples.
4. Note that these oddities are not apparent when the data is viewed on the real time display, because this automatically trims the overscan and applies an offset to each quadrant to crudely equalize the DC levels.