5. IRIS Level 1 Data

The IRIS Level 1 Data have a minimal amount of calibration and processing. They are not generally recommended to the novice user, but expert users may have the need to perform a particular calibration and work with the data before it goes through the usual processing pipeline.

5.1. Searching and Downloading

The IRIS Level 1 data is hosted at Stanford’s Joint Scientific Operations Center (JSOC). If one needs to get it for further analysis, it can be downloaded in the following ways:

  1. Using JSOC’s data export web interface at http://jsoc.stanford.edu/ajax/lookdata.html
  2. From SSWIDL, by making data queries to the Stanford/LMSAL Joint Scientific Operations Center (JSOC) from the interactive prompt. These steps result in IRIS header and data being loaded directly into memory.
  3. Using SSWIDL commands to recover a list of FITS files from the JSOC which can then be downloaded by the user and manipulated.
  4. Directly from LMSAL’s website, although no search interface currently exists (need to download files manually). This is also the only way to download the special NRT series (near real time), which are quickly processed as soon as the telemetry is received (and as a consequence do not have the most accurate calibration or position keywords).

If you are gathering your IRIS Level 1 FITS files from the internet sites then please go directly to Reading Level 1 Data in IDL, the following sections provide example IRIS/SSW commands to collect data.

5.1.1. JSOC / SolarSoft Level 1 Data Access

IRIS data hosted at the Stanford/LMSAL JSOC can be accessed through the SSW routine ssw_jsoc_time2data:

IDL> ssw_jsoc_time2data, start_time, end_time, drms, ds=ds, /jsoc2

where start_time and end_time are strings in a format recognizable by anytim.pro (see above), drms is a vector of structures containing the metadata of the observations in that time period, ds is a string specifying the JSOC data series to be queried, and the jsoc2 keyword is activated to allow your IDL session direct access to the JSOC. For example, iris.lev1_prelim02 is one of the pre-release iris data series and so this particular example may not be available to the general user. The routine iris_jsoc_ds will return all of the available IRIS data series in the JSOC:

IDL> res = iris_jsoc_ds(/refresh, /jsoc2)
IDL> more, res

To first retrieve the metadata for the IRIS Level 1 images for the throughput test sequence taken on 2013 August 20, extracting the times from one of the means above:

IDL> t0 = '2013-08-20 15:05:07' & t1 = '2013-08-20T15:22:13'
IDL> ssw_jsoc_time2data, t0, t1, drms, ds='iris.lev1'
IDL> umodes = ssw_uniq_modes(drms,'instrume,exptime,SUMSPAT,SUMSPTRL,img_path', $
       mc=mc)
IDL> more, umodes + string(mc)
FUV     15.00   1           1   FUV                     60
NUV     15.00   1           1   NUV                     60
SJI     15.00   1           1   SJI_1330                15
SJI     15.00   1           1   SJI_1400                14
SJI     15.00   1           1   SJI_2796                15
SJI     15.00   1           1   SJI_2832                4

Note that it is also possible to form this query based on the frame (or image) serial number (FSN) rather than time using the /fsn keyword in the call to ssw_jsoc_time2data.

Warning

The code below does not work and needs to be updated. Gives an error “There are no files in this RecordSet”.

For example, a subset of the above data containing only the FUV 1400Å slit-jaw images could be returned as follows:

IDL> ss = struct_where(drms,search=['img_path=SJI_14*', 'exptime=1.~25.'])
IDL> ssw_jsoc_time2data, drms[SS].FSN, dummy, index, data, ds='iris.lev1', $
 /FSN, /uncomp_delete,/use_shared, /noshell

Note

By setting the FSN keyword and inputing a vector of FSNs, the second parameter (normally the end of the time interval) is ignored and the full image data are retrieved for the specified images:

IDL> help, index, data
INDEX          STRUCT           = -> <Anonymous> Array[14]
DATA            INT             = Array[2072, 1096, 14]

The Level 1 data and index arrays should now to be passed to iris_prep so that the appropriate calibration steps can be performed, so skip to the iris_prep section of this guide.

5.1.2. Obtaining a List of Level 1 Files

Once you know the start and end times of the particular IRIS observations you are interested in:

IDL> fl = iris_time2files(t0, t1, drms, /url)

will return a complete URL list for the files. Once you have the list of IRIS FITS files that you need, then they can be downloaded to your current directory within your IDL session using the following command:

IDL> sock_copy, fl, out_dir = './'

5.2. Reading Level 1 Data in IDL

If you downloaded your IRIS FITS files using the method in the previous section IRIS FITS files can be read into memory of your current IDL session using the simple command:

IDL> read_iris, <irisfits>, index ,data ,/noshell, /use_shared

where:

  • <irisfits> - is a list of IRIS FITS files
  • index - is an array of structures containing the header information for each FITS file
  • data - is the output data array
  • noshell and use_shared are designed to minimize memory load and their use is recommended.

The keywords to the read_iris call are clearly outlined in the SSW code.

Example: Find an list of IRIS Level 1 FITS file URLs for the NUV observations in our previous example and read the data into memory:

IDL> t0 = '2013-08-20 15:05:07' & t1 = '2013-08-20 15:22:13'
IDL> nuv_fl = iris_time2files(t0,t1,drms,/nuv,/url)
IDL> help, nuv_fl & more,[nuv_fl[0:1],'...',last_nelem(nuv_fl,2)]
NUV_FL          STRING    = Array[60]
http://www.lmsal.com/solarsoft//irisa/data/level1/2013/08/20/H1500/iris20130820_15051506_nuv.fits
http://www.lmsal.com/solarsoft//irisa/data/level1/2013/08/20/H1500/iris20130820_15053110_nuv.fits
...
http://www.lmsal.com/solarsoft//irisa/data/level1/2013/08/20/H1500/iris20130820_15214783_nuv.fits
http://www.lmsal.com/solarsoft//irisa/data/level1/2013/08/20/H1500/iris20130820_15220403_nuv.fits
; download the files to your local directory:
IDL> sock_copy, nuv_fl, out_dir = './'

Read the first 5 FITS files into index, and data arrays:

IDL> local_nuv_files = file_search('./*_nuv.fits')
IDL> read_iris, local_nuv_files[0:4], index, data, /uncomp_delete, /use_shared
IDL> help,index,data
INDEX     STRUCT = ->  <Anonymous> Array[5]
DATA        INT         = Array[2072, 1096, 5]

Now that you have the data in memory we have to apply corrections to the detector images using the iris_prep routine - this will create “Level 1.5” IRIS data and we again refer the reader to the discussion above about IRIS data levels.

5.3. Creating Level 1.5 Data with iris_prep

We can create Level 1.5 FITS files for a set of Level 1 FITS files. iris_prep, the Level 1.5 FITS generator performs the following steps:

  • Dark/pedestal removal and flat-fielding
  • Replacement of spikes/bad pixels
  • Geometric correction (rotation, translation, distortion, platescale)
  • Wavelength correction (dispersion/shift)
  • Update header keywords to reflect the above actions

Missing header of iris_prep

The interested (power-)user can find extensive information on iris_prep and the action of the various keywords in the appropriate IRIS Technical Notes 14 through 25 in the document library (http://iris.lmsal.com/documents.html).

An example run of iris_prep for one of the examples above is as follows:

IDL> t0 = '2013-08-20 15:05:07' & t1 = '2013-08-20T15:22:13'
IDL> nuv_fl = iris_time2files(t0, t1, /nuv, /url, /jsoc2)
IDL> sock_copy, nuv_fl, out_dir='./'    ; puts files in current directory
IDL> f = file_search('./*_nuv.fits', nfits)     ; gets local file listing
IDL> read_iris, f, index, data          ; read those FITS files
IDL> iris_prep, index, data, oindex, odata      ; run iris_prep

now, you can leave this here and the prepped IRIS data is in the (multi-dimensional) odata array and oindex array of structures contains the header information.

If you want to go to the next step (creating a Level 2 IRIS FITS file) you can write the Level 1.5 files into a new sub-directory (here we call it './level1.5/') for each Level 1 FITS file:

IDL> spawn, 'mkdir ./level1.5/'
IDL> for kk =0,nfits-1 do begin &
    read_iris, f[kk], index, data
    iris_prep, index, data, oindex, odata &
    hdr = struct2fitshead(oindex) &
    writefits, './level1.5/'+file[kk], odata, hdr &
    endfor

These Level 1.5 FITS files are ready to be analyzed as is, or can be passed through the code in the following section to create a Level 2 FITS file.

_images/NUV_calibration.jpg

Uncorrected Level 1 NUV spectrograph detector image (left) and the corrected version of the detector image (right) following application of the iris_prep algorithm with default settings (as shown in code above).

_images/FUV_calibration.jpg

Uncorrected Level 1 FUV spectrograph detector image (left) and the corrected version of the detector image (right) following application of the iris_prep algorithm with default settings (as shown in code above).

5.4. Creating Level 2 Data from Level 1.5 Data

IRIS Level 2 data can be created using the iris_level1to2 procedure. The data sent to iris_level1to2 can either be corrected to Level 1 or Level 1.5 (by iris_prep, see above). The header of iris_level1to2 looks like this:

PRO IRIS_level1to2, level1files, outputfolder, xmlparentfolder=xmlparentfolder, $
        l1to2log=l1to2log,_extra=_extra, OBSid=OBSid, debug=debug,$
        maxdeviation=maxdeviation, scaled=scaled

At a very basic level iris_level1to2 creates a small number of Level 2 files from the larger number of Level 1 files where the slit-jaw image and spectrograph data are separated and stored sequentially in a FITS file by wavelength and time.

The IRIS SJI Level 2 files are straightforward in structure. Using the header information to identify the Level 1 frames belonging to each SJI filter, the corresponding Level 2 FITS file is a time-series of the images taken over the duration of the raster or sit-n-stare observation for that filter organized by dimension [Spatial X, Spatial Y, Time].

The raster type of IRIS Level 2 files are organized by dimension [lambda, Spatial Y, Spatial X/time]. When being ordered the list of Level 1.5 (or Level 1 files) are separated using the header information with two counters that appear in the raster file name: rastertype (t***) and rastertype repetition (r*****):

iris_l2_20130829_060935_4000005156_raster_t000_r00000.fits
iris_l2_20130829_060935_4000005156_raster_t000_r00001.fits
iris_l2_20130829_060935_4000005156_raster_t000_r00002.fits

rastertype is defined as an observation which have the following pieces of header information in common:

  • OBSID
  • The Number of steps
  • dX (X is the horizontal part of the Telescope PZT values, not necessarily solar X)
  • Position X
  • Number of Spectral Windows (the line list)
  • The spectral regions MUST be identical regions
  • Exposure Time

Note that the following header values can vary from exposure to exposure and between NUV and FUV spectrographs: * Summing * Exposure Time * Compression Scheme Used (“N” or “K”) * LUT ID * Max and Min Exposure time (if Automatic Exposure Control is On)

Note

If a Level 1 (or 1.5) file is missing, the respective image, or slice in the corresponding Level 2 will be blank.

The code is designed such that the y-range of all windows is made to be exactly the same. More precisely, we find the median of the frame start row (TSRn) values of all of the frames in the sequence, and use the minimum TSR (which is not less than the median minus 30 pixels) as the start row for all windows. The end points are established in exactly the same way using the frame end row (TERn), the maximum TER (which is not more than the median plus 30 pixels).

Note

30 pixels is the default value (used by the mission team), but can be overwritten by the caller of the iris_level1to2 procedure, using the keyword maxdeviation.

An example execution of iris_level1to2 is as follows (taking approximately 10 minutes end to end depending on your network speed):

IDL> t0 = '2013-08-20 15:05:07' & t1 = '2013-08-20T15:22:13'
IDL> fl = iris_time2files(t0, t1, /url)
IDL> spawn, 'mkdir ./level1/'
; puts files in current directory
IDL> sock_copy, fl, out_dir = './level1/'
; gets local file listing
IDL> f = file_search('./level1/*.fits', count=nfits)
IDL> spawn, 'mkdir ./level1.5/'
IDL> for kk =0,nfits-1 do begin &
        read_iris, f[kk], index, data &
        iris_prep, index, data, oindex, odata &
        hdr = struct2fitshead(oindex) &
        file = str_sep(f[kk],' /') &
        file = file[n_elements(file) - 1] &
        writefits, './level1.5/' + file, odata, hdr &
      endfor
IDL> l15files = file_search('./level1.5/*.fits', count=nfits)
IDL> spawn, 'mkdir ./level2/'
IDL> IRIS_level1to2, l15files, './level2/'
IDL> $ls -al
iris_l2_20130820_002000_4182010156.log
iris_l2_20130820_002000_4182010156_SJI_1330_t000.fits
iris_l2_20130820_002000_4182010156_SJI_1400_t000.fits
iris_l2_20130820_002000_4182010156_SJI_2796_t000.fits
iris_l2_20130820_002000_4182010156_SJI_2832_t000.fits
iris_l2_20130820_002000_4182010156_raster_t000_r00000.fits