*** Equations Behind the Variance of e-/DN

[MYPIXEL at aol.com]

Posted to CCD-world:
-+-+-+-
*************jj

Roger,

I read your message below about e-/DN uncertainty. It definitely prompted me
to get out old calculations when photon transfer was invented many years ago.
I include the derivation here for your review (it is difficult to write
statistical equations via e-mail so bare with them). It does theoretically
appear that the uncertainty in e-/DN is a function of signal level.  See last
equation on this.

Sorry for the confusion and forgetting what was originally derived (hope it is
right). Please let me know if you come up with a different equation.

Thanks 

Jim

***********jj

K (e-/DN) = S / Z  

     where S is the signal (DN) and Z is the variance in signal (DN).


The variance of K is found using the "propagation of errors" formula (from
advanced calculus), i.e., 

Y = { dK/dS }^2  x (variance in S) + { dK/dZ }^2  x (variance in Z)

     where Y is the variance of K. The variance of Z is given by,

Z / Np

     where Np are the number of pixels sampled.

Substituting this equation and performing the required differentiation yields,

Y =  [1 / Z]^2 x Z + [-S / Z^2] ^2  x Z / Np

Simplifying

Y = 1 / Z +  (S^2 / Z^4 ) x Z / Np

or

Y = 1 / Z ( 1 + K^2 / Np)  





In a message dated 98-01-25 13:58:26 EST, roger at ctios1.ctio.noao.edu writes:

<< Subj:	 Re: e- / DN  Uncertainty
 Date:	98-01-25 13:58:26 EST
 From:	roger at ctios1.ctio.noao.edu (roger smith x294)
 Reply-to:	CCD-world at cfht.hawaii.edu
 To:	CCD-world at cfht.hawaii.edu
 
 Posted to CCD-world: -+-+-+-
 Paola and Jim,
 
 My experience supports Jim's formula for the error in the
 estimate of e-/ADU.  We developed an IRAF task last year which
 acquires the data and plots variance versus mean for thousands of 
 evenly distributed points in only a few minutes.  With so many points
 the linear growth in the scatter with increasing signal is very
 vivid, making it clear that the slope (ADU/e-) cannot be
 estimated any better at higher signal levels.  It is also clear
 that with more points (or more pixels per point) the estimate of
 the slope is indeed more accurate.
 
 Signal level only ceases to be relevant once the shot noise is
 greater than the read noise.  If one is using only a *single*
 point to estimate the slope (not recommended), then shot noise
 must be very much greater than read noise.   
 
 -------
 
 So, why am I measuring variance curves with thousands of points
 anyway?   I wanted away of estimating where the variance curve
 becomes non-linear which was both accurate and fast.  Such a
 curve allows me to select the linear range when estimating the e-
 /ADU, but more importantly I find that variance curves are the
 most robust measure of saturation level.  This may be the
 only accurate method when high contrast test targets cannot be
 projected onto the CCD, such as during the daytime on the
 telescope.
 
 We mount LEDs and a diffusing screen to provide flat illumination
 of the CCD to within a few percent, then flash the LEDs for about
 1 ms before each parallel shift to produce a linear increase in
 intensity with line number and minimal intensity variation along
 each line.  To eliminate photo-electrons detected by the serial
 register, the serial register is flushed between the flashing the
 LED and the parallel shift.  An nominally identical pair of such
 images is acquired.
 
 The variance of the difference of corresponding lines is then
 plotted against the sum of their means, excluding columns which
 are contaminated by known defects.  Since both the variance and
 mean are doubled by this calculation, the slope is still ADU/e-.
 The subtraction of similar frames eliminates the contribution of
 pixel to pixel sensitivity variation from the statistics.  Such
 variations have already been averaged out to a large extent by
 accumulating the signal at many positions on the CCD: the greater
 the signal, the greater is the amount of averaging. 
 
 A 20 point boxcar average makes even small deviations from linear
 easy to see in spite of the scatter in the individual variance
 data: remember that scatter in variance goes down linearly with
 number of points averaged (not as the square root).
 
 Roger
 -+-+-+-
 For information about CCD-world, send email to owner-CCD-
world at cfht.hawaii.edu.
 
 
-+-+-+-
For information about CCD-world, send email to owner-CCD-world at cfht.hawaii.edu.