3. Working with IRIS Level 2 data in Python

This section is aimed to familiarize the reader with the Python jargon and the main module used in this tutorial: extract_irisL2data. This module will allow us to:

  • read the IRIS Level 2 files

  • select those spectral regions of our interest

  • load the data of the spectral windows of our interest

  • visualize, inspect, and interact with them in an easy and comfortable way

  • select those points of our interest or discard the data of a selected spectral window

  • save the selected data and their associated points of interest

  • load, display or share with other colleagues our IRIS data in a simple, robust, and efficient way.

Once the decompressed IRIS Level 2 data are locally hosted, the next step is to read, extract, and inspect them. We have created a module with different methods (or routines) to this aim. This module is called extract_irisL2data, and it is used to load and inspect either raster or SJI files. The extract_irisL2data module has several methods. The most relevant are:

  • info_fits: shows some information about the FITS IRIS Level 2 data file, such as extension numbers, type and size of the data associated to each ;extension, the spectral window label of each extension, i.e. the spectral data contained in that number extension, and the observed spectral range. See Section 3.2.1.

  • only_header: returns the main header (default) or the extension header (through the keyword extension, default is 0) of an IRIS Level 2 data file, as a header class. The fields of the header can be accessed as the keys of a dictionary object, e.g. hdr[‘NAXIS1’]. See Section 3.2.1.

  • only_data: returns the data from an extension of the IRIS Level 2 data file through the keyword extension (default is 1). The extension number and the associated spectral data can be obtained with the method info_fits. See Section 3.2.1.

  • show_lines: shows the spectral windows available in an IRIS Level 2 data file, their dimensions, and their observed spectral range. It returns an array, being each element of this array a string with the information of the spectral window (window_label).

  • raster: a class that builds a Raster of IRIS object or RoI. This object contains the data selected by the user from an IRIS Level 2 raster file, some basic information from the headers, and other values that makes its visualization easier. To this aim, the class raster has a method especially designed to visualize a RoI called quick_look. This method allows to add Points of IRIS or PoI, i.e, points of the user’s interest from the data into the RoI object, which can be easily saved and analyzed.

  • SJI: a class that makes a SJI of IRIS object or SoI. It is similar to a RoI but contains the data from an IRIS SJI Level 2 data file. It also has its own quick_look method, which also allows to add one or more PoI can be added to the SoI

  • load: loads the data for a given spectral window or, in the case of a raster file, for a list of spectral windows. This code use two classes of object, raster and SJI, which returns two objects, RoI and SoI respectively, with 3 main attributes and 1 main method.

    • For the Raster of IRIS object or RoI, the main attributes and method are:
      • filename: contains the name of the parent raster file.

      • windows: contains the requested observed spectral windows. This means, that it is a list with the window_label of the data loaded by the user’s request.

      • raster: a dictionary, which has as many keys as spectral windows have been asked to be load. Thus, each spectral window has its unique descriptive key window_label. Each of these keys are themselves a dictionary object with different keys, such as data, wl, some information from the header and other useful values, and the Point(s) of IRIS or PoI, a list with the selected point of user’s interest.

      • flush: a method, puts the data of all the spectral windows window_label of a RoI stored in numpy.memmap array to the memory of the system. See Section 3.2.

      • tomemmap: a method, puts the data of all the spectral windows window_label of a RoI stored in the memory of the system in a numpy.memmap array, i.e. in a temporary files(s). See Section 3.2.

      • quick_look: a method to inspect interactively the spectral data of the RoI. See Section 3.2.3.

    • For the SJI of IRIS object or SoI, the attributes are:
      • filename: contains the name of the parent SJI file.

      • window: contains the observed spectral window window_label.

      • SJI: a dictionary, which unique key is named with the window_label. This key is itself a dictionary containing several keys, including the data, wl, some information from the header and other useful values, and the PoI list.

      • flush: a method, puts the data of the observed spectral window window_label of a SoI stored in numpy.memmap array to the memory of the system. See Section 3.2.

      • tomemmap: a method, puts the data of the observed spectral window window_label of a SoI stored in the memory of the system in a numpy.memmap array, i.e. in a temporary files(s). See Section 3.2.

      • quick_look, a method to inspect interactively the slit-jaw images contain in the SoI. See Section 3.2.2.

    load method allows to load the data previously saved by the author. See Section 3.3.

  • save: a method that allows to write to a file a RoI, a SoI, or only their associated PoI.

  • show_poi: this method helps us to display a PoI associated to a RoI or to a SoI.

The following diagram describes some of the methods available in extract_irisL2data module:

extract_irisL2data.
                  |_info_fits            Method: prompts information about the extension
                  |                              and the data in a of a raster or SJI IRIS
                  |                              Level 2 data file
                  |
                  |_only_header          Method: returns only the main header (default)
                  |                              or extension header of a raster or SJI
                  |                              IRIS Level 2 data file
                  |
                  |_only_data            Method: returns only data for and extension number
                  |                              (no. 1 by default) of a raster or SJI
                  |                              IRIS Level 2 data file
                  |
                  |_show_lines           Method: returns a list of strings  with the
                  |                              label of the spectral windows stored
                  |                              in an IRIS Level 2 data file |
                  |
                  |_load                 Method: loads a raster or SJI IRIS Level 2 file.
                  |                              It returns either a 'RoI' or a 'SoI'.
                  |                              It can also load a previously saved 'RoI'
                  |                              or a 'SoI'
                  |
                  |_raster.              Class:  builds a 'Raster of IRIS' object or
                  |       |                      'Roi' object.
                  |       |_quick_look   Method: displays  the data of a 'RoI'.
                  |       |                       Allows to add 'Point(s) of IRIS' or
                  |       |                       'PoI' to the 'RoI'
                  |       |_other methods
                  |
                  |_SJI.                 Class:  builds a 'SJI of IRIS' object or
                  |    |                         'SoI' object.
                  |    |_quick_look      Method: displays the data of a 'SoI'.
                  |    |                          Allows to add 'PoI' to the 'SoI'
                  |    |_other methods
                  |
                  |_save                 Method: saves a 'RoI', a 'SoI' or only their
                  |                              associated 'PoI'
                  |
                  |_show_poi             Method: displays a 'PoI'

The RoI, SoI, and PoI objects have attributes of different kind: arrays, tuples, list, or dictionaries, and some methods. extract_irisL2data.save and extract_irisL2data.load methods allows us to save and load easily these objects, despite of having associated methods and a complex structure, for instance a RoI having data from different spectral range, i.e. storing 3D data cubes with different size, and different number of PoI for each spectral window.

3.1. A first look at the IRIS Level 2 data

We are going to work with the data corresponding to the NOAA AR 12480. Thus, we are going to download a data set (of several available) observed by IRIS. With the following commands we will download the data, which consist in a raster file and 5 SJI files corresponding to 5 spectral windows observed by IRIS. Once the files are hosted locally, we will take a look into the header of the files.

The following command lines are the same that the ones in the \(1^{st}\) code block of Section 2. There is no need to execute them again if they have been already run.

>>> from iris_lmsalpy import hcr2fits

# Let's download the data from NOAA AR 12480.
>>> query_text = 'https://www.lmsal.com/hek/hcr?cmd=search-events3&outputformat=json&startTime=2016-01-14T00:00&stopTime=2016-01-15T00:00&minnumRasterSteps=320&hasData=true&minxCen=550&limit=200'
>>> list_urls = hcr2fits.get_fits(query_text)

Requesting the query...
Downloading the file http://www.lmsal.com/solarsoft/irisa/data/level2_compressed/2016/01/14/20160114_230409_3630008076/iris_l2_20160114_230409_3630008076_SJI_1330_t000.fits.gz into /sanhome1/asainz/IRIS_data/ (#1 of 5) ...

...

Decompressing the file iris_l2_20160114_230409_3630008076_SJI_1330_t000.fits.gz into /sanhome1/asainz/IRIS_data/ (#1 of 5) ...

...

Let us recover the header of the raster file and show the description of the observation:

>>> from iris_lmsalpy import extract_irisL2data

>>> raster_filename = 'iris_l2_20160114_230409_3630008076_raster_t000_r00000.fits'
>>> hdr = extract_irisL2data.only_header(raster_filename)
>>> hdr['OBS_DESC']
'Large dense 320-step raster 105.3x120 320s   Deep x 8    Lossless co'

While the SJI files cotain just one spectral window per file, the raster files have several spectral windows per file. We can know what spectral windows are observed in an IRIS Level 2 data file by using the method extract_irisL2data.show_lines.

>>> SJI_filename = 'iris_l2_20160114_230409_3630008076_SJI_2796_t000.fits'
>>> lines = extract_irisL2data.show_lines(SJI_filename)
Extracting information from file iris_l2_20160114_230409_3630008076_SJI_2796_t000.fits...

Available data with size Y x X x Image are stored in a window labeled as:

-------------------------------------------------------------
Index --- Window label --- Y x X x Im --- Spectral range [AA]
-------------------------------------------------------------
  0         SJI_2796       779x1400x80    2794.00 - 2798.00
-------------------------------------------------------------

Observation description:  Large dense 320-step raster 105.3x120 320s   Deep x 8    Lossless co


>>> raster_filename = 'iris_l2_20160114_230409_3630008076_raster_t000_r00000.fits'
>>> lines = extract_irisL2data.show_lines(raster_filename)
Extracting information from file iris_l2_20160114_230409_3630008076_raster_t000_r00000.fits...

Available data with size Y x X x Wavelength are stored in windows labeled as:

--------------------------------------------------------------------
Index --- Window label --- Y x X x WL --- Spectral range [AA] (band)
--------------------------------------------------------------------
  0      C II 1336         779x320x374     1332.66 - 1337.50  (FUV)
  1      Fe XII 1349       779x320x244     1347.64 - 1350.79  (FUV)
  2      O I 1356          779x320x340     1352.19 - 1356.59  (FUV)
  3      Si IV 1394        779x320x411     1390.87 - 1396.08  (FUV)
  4      Si IV 1403        779x320x601     1398.60 - 1406.23  (FUV)
  5      2832              779x320x113     2831.38 - 2834.23  (NUV)
  6      2814              779x320x146     2812.69 - 2816.38  (NUV)
  7      Mg II k 2796      779x320x530     2793.14 - 2806.61  (NUV)
--------------------------------------------------------------------

Observation description:  Very large dense 320-step raster 105.3x175 320s   Deep x 8 Spatial x


# The window labels are stored in the output variable ('lines' in this example)
# as an array of strings.
>>> print(lines)
['C II 1336' 'Fe XII 1349' 'O I 1356' 'Si IV 1394' 'Si IV 1403' '2832'
'2814' 'Mg II k 2796']

Note that the name of the dictionaries returned by extract_irisL2data.load have the same name as the class they belong (raster and SJI). These classes of objects have some methods (or functionalities) that we discuss in the following section.

3.2. Reading and looking at the IRIS Level 2 data

Thus, the object returned by extract_irisL2data.load is either a RoI or a SoI. Because the SJI IRIS Levele 2 data are simpler than the raster files, since they have only one spectral window per file, we start this tutorial working with this data. The philosophy behind the SoI and RoI is the same, although the latter has a more complex structure.

In this tutorial, for the sake of clarity, we have named iris_sji and iris_raster to the SoI and RoI obtained by loading a SJI and a raster IRIS Level 2 data file respectively with extract_irisL2data.load. Of course, the user can name the SoI and RoI as s/he wishes.

We use the following command to read and load the data from a SJI IRIS Level 2 file:

# To read and load the data from IRIS Level 2 file is pretty simple!
>>> iris_sji = extract_irisL2data.load(SJI_filename)
Creating temporary file...  /var/folders/jn/wjxzjh7n2nv58g9t3qjmgdr8000mky/T/tmpjm7s8l4k

The previous command creates a SoI object, wich contains the data and other values in the following structure:

iris_sji.
        |_filename
        |_window = [window_label]
        |_SJI[window_label].
        |                  |_data [ = SJI data cube ]
        |                  |_wl   [ = wavelengths in Angstrom]
        |                  |_clip_ima [ = lower, upper thresholds ]
        |                  |_... more keys
        |
        |_flush()          Method: flushes the data from the
        |                          temporary file to the memory
        |                          of the system
        |
        |_tomemmap()       Method: moves the data from the memory
        |                          of the system to a temporay file
        |
        |_quick_look()     Method: visualizes the data
        |
        |_other methods

Note

extract_irisL2data.load method loads the data from a raster or SJI IRIS Level 2 in a numpy memory-map array, i.e. a numpy.memmap class. That means, the data are written in a temporary file (in the raster case, as many temporary files as spectral windows are requested). Thus, the data are not loaded in the memory of the system, but written in a temporary file. In this way, the memory pressure in the system is much lower, which makes the user experience smoother and more efficient. For instance, in the following example, the memory required to load all the spectral windows of the raster file is about \(2.66 Gbytes\), while the memory used by the system when the data are loaded through a memory-map array is just \(96 Mbytes\). On the other hand, this methods requires to have enough space (\(~ 2.66 Gbytes\) in this example) in a physical storage device to write the temporary files. These temporay files are removed when the RoI or SoI are flushed or deleted.

We use the same command to read the data from a raster IRIS Level 2 data, but it returns a RoI object with the data, wavelengths sampled, and other information stored as:

# To read and load the data from IRIS Level 2 file is pretty simple!
>>> iris_raster = extract_irisL2data.load(raster_filename)
Creating temporary file...  /var/folders/jn/wjxzjh7n2nv58g9t3qjmgdr8000mky/T/tmpvra72uuz

The previous command only loads the data corresponding to the spectral window Mg II k 2796, which is the default option. Through the keyword window_info of `extract_irisL2data.load we can set what spectral window(s) will be loaded. This keyword is a list with the label of the requested spectral window(s). The label of the spectral windows observed by IRIS in a given observation are stored in the keyword TDESCn of the main header of the raster file, where n corresponds to the extension in the file where the data of that spectral window are stored. In addition, it accepts the value window_info = ['all'] to load all the available spectral windows in the raster file. In Section 3.2.3 we explain how to load multiple spectral windows.

The RoI have the same basic attributes of a SoI, but it includes more attributes devoted to the visualization tool.

iris_raster.
           |_filename
           |_windows = [sel_window_label_1, ..., sel_window_label_m], m <=n
           |_all_windows_in_file = [window_label_1, ..., window_label_n]
           |_raster[window_label_1].
           |                       |_data [ = raster data cube ]
           |                       |_wl   [ = wavelengths in Angstrom]
           |                       |_clip_ima [ = lower, upper thresholds ]
           |                       |_lim_yplot [ = lower, upper thresholds ]
           |                       |_... more attributes (or keys)
           |
           |_ ... as many as m-2 selected spectral windows
           |
           |_raster[window_label_m].
           |                       |_data
           |                       |_wl
           |                       |_clip_ima
           |                       |_lim_yplot
           |                       |_... more attributes (or keys)
           |
           |_flush()          Method: flushes the data from the
           |                          temporary file to the memory
           |                          of the system
           |
           |_tomemmap()       Method: moves the data from the memory
           |                          of the system to a temporary file
           |
           |_quick_look()     Method: visualizes the data
           |
           |_other methods

Note that windows in a RoI object is a list that may have one or several spectral window labels, while the SoI is a list with a unique spectral window.

If we take a look at the data in the SoI and the RoI, we can see they are numpy.memmap arrays:

>>> print(type(iris_sji.SJI['SJI_2796'].data))
<class 'numpy.memmap'>

>>> print(type(iris_raster.raster['Mg II k 2796'].data))
<class 'numpy.memmap'>

If, for any reason, the user wants to put the data from the temporary file, i.e. from the numpy.memmap array, to the memory of the system, s/he can use the method flush of the Roi and SoI:

>>> iris_sji.flush()
Removing temporary file...  /var/folders/jn/wjxzjh7n2nv58g9t3qjmgdr8000mky/T/tmpjm7s8l4k
>>> print(type(iris_sji.SJI['SJI_2796'].data))
<class 'numpy.ndarray'>

>>> iris_raster.flush()
Removing temporary file...  /var/folders/jn/wjxzjh7n2nv58g9t3qjmgdr8000mky/T/tmpvra72uuz
>>> print(type(iris_sji.SJI['SJI_2796'].data))
<class 'numpy.ndarray'>

As you can see, the flush method closes and removes the temporary files. This action is automatically done if the variables are deleted (e.g del iris_raster).

If the user wants to recover the data from the raster or SJI IRIS Level 2 files directly into the memory of the system instead that in a temporary file, the user can just simply set the keyword memmap = False:

>>> iris_sji = extract_irisL2data.load(SJI_filename, memmap = False)
>>> print(type(iris_sji.SJI['SJI_2796'].data))
<class 'numpy.ndarray'>

>>> iris_raster = extract_irisL2data.load(raster_filename, memmap = False)
>>> print(type(iris_sji.SJI['SJI_2796'].data))
<class 'numpy.ndarray'>

If the user has already loaded the data into the memory of the system, like in the previous example, and s/he wants to put the date in a numpy.memmap array then s/he can do:

>>> iris_sji.tomemmap()
Creating temporary file...  /var/folders/jn/wjxzjh7n2nv58g9t3qjmgdr8000mky/T/tmpwc7pqs10
>>> print(type(iris_sji.SJI['SJI_2796'].data))
<class 'numpy.memmap'>

>>> iris_raster.tomemmap()
Creating temporary file...  /var/folders/jn/wjxzjh7n2nv58g9t3qjmgdr8000mky/T/tmpyx5pqy3f
>>> print(type(iris_raster.raster['Mg II k 2796'].data))
<class 'numpy.memmap'>

3.2.1. Low-level access to the headers, the data, and other information

The RoI and SoI objects obtained with extract_irisL2data.load allow us to inspect the data from the IRIS Level 2 data files in a comfortable, easy way. These objects have the data for the observed requested data, its wavelengths, and other information as attributes, but not all the information available in the IRIS Level 2 FITS files. Some users may prefer to recover these information and other one not present in the RoI and SoI objects directly from the FITS file. extract_irisL2data has some methods to this aim.

First, we need to understand the structure of the IRIS Level 2 FITS data file. The IRIS Level 2 FITS are multi-extension FITS files. An extension refers to a part of the file containing self-consistent information. This information may be, in the general case, a header or its corresponding data. The first extension is called primary and its extension number is 0.

The extensions in an IRIS Level 2 SJI FITS file has the following numbers:

  • 0: header and data corresponding to the spectral images observed by the SJI.

  • 1: header and auxiliar 31 values from each exposure taken by the SJI in the spectral band of the file. It is an array of float values with dimensions \(no. images \times 31\).

  • 2: header and extra data from each exposure taken by the SJI in the spectral band of the file. It is a record array containing 5 string fields for each exposure. The values of each field can be access as the key in a dictionary or as an attribute. See example in the last code block of this section.

An IRIS Level 2 raster FITS file has the following extensions:

  • 0: a main header with the main information of the observation. This header has information about all the spectral windows contained in the file and other relevant and general information. This extension DOES NOT have spectral data associated.

  • 1 to N: header and data for the N spectral windows cointaned in the file.

  • N+1: header and auxiliar 47 values from each exposure considered in the file. It is an array of float values with dimensions \(no. acquisitions \times 47\).

  • N+2: header and extra information data from each exposure considered in the file. It is a record array containing 9 string fields for each exposure. The values of each field can be access as the key in a dictionary or as an attribute. See example in the last code block of this section.

The method extract_irisL2data.info_fits shows the information of the extensions contained in the IRIS Level 2 file. For the SJI file:

>>> extract_irisL2data.info_fits(SJI_filename)
Filename: iris_l2_20160114_230409_3630008076_SJI_2796_t000.fits
No.    Name      Ver    Type      Cards   Dimensions   Format
0  PRIMARY       1 PrimaryHDU     161   (1400, 779, 80)   int16 (rescales to float32)
1                1 ImageHDU        38   (31, 80)   float64
2                1 TableHDU        33   80R x 5C   [A10, A10, A4, A66, A55]

Observation description:  Large dense 320-step raster 105.3x120 320s   Deep x 8    Lossless co

Extension No. 1 stores data and header of SJI_2796: 2794.00 - 2798.00 AA

To get the main header use :
hdr = extract_irisL2data.only_header(filename)

To get header corresponding to data of SJI_2796 use :
hdr = extract_irisL2data.only_header(filename, extension = 0)

To get the data of SJI_2796 use :
data = extract_irisL2data.only_data(filename, extension = 0, memmap = True)

and for the raster file:

>>> extract_irisL2data.info_fits(raster_filename)
Filename: iris_l2_20160114_230409_3630008076_raster_t000_r00000.fits
No.    Name      Ver    Type      Cards   Dimensions   Format
  0  PRIMARY       1 PrimaryHDU     379   ()
  1                1 ImageHDU        33   (374, 779, 320)   int16 (rescales to float32)
  2                1 ImageHDU        33   (244, 779, 320)   int16 (rescales to float32)
  3                1 ImageHDU        33   (340, 779, 320)   int16 (rescales to float32)
  4                1 ImageHDU        33   (411, 779, 320)   int16 (rescales to float32)
  5                1 ImageHDU        33   (601, 779, 320)   int16 (rescales to float32)
  6                1 ImageHDU        33   (113, 779, 320)   int16 (rescales to float32)
  7                1 ImageHDU        33   (146, 779, 320)   int16 (rescales to float32)
  8                1 ImageHDU        33   (530, 779, 320)   int16 (rescales to float32)
  9                1 ImageHDU        54   (47, 320)   float64
 10                1 TableHDU        53   320R x 7C   [A10, A10, A3, A10, A4, A66, A66]

Observation description:  Large dense 320-step raster 105.3x120 320s   Deep x 8    Lossless co

Extension No. 1 stores data and header of C II 1336:     1332.66 - 1337.50 AA (FUV)
Extension No. 2 stores data and header of Fe XII 1349:   1347.64 - 1350.79 AA (FUV)
Extension No. 3 stores data and header of O I 1356:      1352.19 - 1356.59 AA (FUV)
Extension No. 4 stores data and header of Si IV 1394:    1390.87 - 1396.08 AA (FUV)
Extension No. 5 stores data and header of Si IV 1403:    1398.60 - 1406.23 AA (FUV)
Extension No. 6 stores data and header of 2832:  2831.38 - 2834.23 AA (NUV)
Extension No. 7 stores data and header of 2814:  2812.69 - 2816.38 AA (NUV)
Extension No. 8 stores data and header of Mg II k 2796:  2793.14 - 2806.61 AA (NUV)

To get the main header use :
hdr = extract_irisL2data.only_header(filename)

To get header corresponding to data of C II 1336 use :
hdr = extract_irisL2data.only_header(filename, extension = 1)

To get the data of C II 1336 use :
data = extract_irisL2data.only_data(filename, extension = 1, memmap = True)

If we now want to recover the main header of the raster file, and the header and the data corresponding the spectral window Si IV 1403. Because extension = 0 is the default option, to get the main header of the file we can just type:

>>> hdr = extract_irisL2data.only_header(raster_filename)

# Let's get the individual header corresponding to Si IV 1403
>>> hdr_SiIV_1403 = extract_irisL2data.only_header(raster_filename, extension = 5)

# Let's get the data corresponding to Si IV 1403
>>> data_SiIV_1403 = extract_irisL2data.only_data(raster_filename, extension = 5)
Creating temporal file...  /var/folders/jn/wjxzjh7n2nv58g9t3qjmgdr8000mky/T/tmpan9g9ak8
>>> print(type(data_SiIV_1403))
<class 'numpy.memmap'>

Note that extract_irisL2data.only_data can return the data as a numpy.array, i.e. the data are loaded in the memory of the system by using memmap = False in the extract_irisL2data.only_data method, or as numpy.memmap, i.e. the data are stored in a temporary file in the file system (the default option). In this case, the temporary file is removed from the file system by deleting the associated variable:

>>> del data_SiIV_1403

Because the number of spectral windows in a raster file may vary from an observation to another, a good option to access to the last 2 extensions of the IRIS Level 2 file is to use the negative index in the extension keyword:

# Let's recover the header and the data corresponding to the auxiliar
# information extension
>>> hdr_aux = extract_irisL2data.only_header(raster_filename, extension = -2)
>>> data_aux = extract_irisL2data.only_data(raster_filename, extension = -2)

# Let's recover the data for the extra information extension
>>> data_extra = extract_irisL2data.only_data(raster_filename, extension = -1)

# Let's check the names of the records
>>> data_extra.dtype.names
('FRMID',
 'FUVFDBID',
 'FUVCRSID',
 'NUVFDBID',
 'NUVCRSID',
 'FUVfilename',
 'NUVfilename',
 'FUVtemp',
 'NUVtemp')

We can access to the values of the variables stored in the data corresponding to the extra information extension as an attribute or as a key:

# Let's access to the record 'FUVfilename'
>>> data_extra.FUVfilename # or data_extra['FUVfilename']
array(['/irisa/data/level1/2016/01/14/H2300/iris20160114_23041396_fuv.fits',
     '/irisa/data/level1/2016/01/14/H2300/iris20160114_23042356_fuv.fits',
     ...
     more FUV filenames are displayed
     ...
    '/irisa/data/level1/2016/01/14/H2300/iris20160114_23550181_fuv.fits'],
    dtype='<U66')

3.2.2. Inspection of a SJI IRIS Level 2 data file

As we have already mentioned, the extract_irisL2data.load method identifies whether the file given as input is a raster or a SJI IRIS Level 2 data file or not. Let’s load the SJI data:

>>> SJI_filename = 'iris_l2_20160114_230409_3630008076_SJI_2796_t000.fits'
>>> iris_sji  = extract_irisL2data.load(SJI_filename, verbose = True)

The provided file is a SJI IRIS Level 2 data file containing SJI_2796 data.

Extracting information from file iris_l2_20160114_230409_3630008076_SJI_2796_t000.fits...

Available data with size Y x X x Image are stored in a windows labeled as:

-------------------------------------------------------------
Index --- Window label --- Y x X x Im --- Spectral range [AA]
-------------------------------------------------------------
   0         SJI_2796       779x1400x80    2794.00 - 2798.00
-------------------------------------------------------------

Observation description:  Large dense 320-step raster 105.3x120 320s   Deep x 8    Lossless co


The SJI data are passed to the output variable
(e.g iris_sji) as a dictionary with the following keys:

    - iris_sji.SJI['SJI_2796']

In this case, we have named the output SoI object iris_sji. This object has three attributes: iris_sji.filename, which contains the parent SJI filename; iris_sji.window, which contains the spectral window in the file; and iris_sji.SJI, a dictionary with the key of the spectral window observed, in this example, ‘SJI_2796’. This key is itself a Python dictionary with keys storing some values obtained from the headers (in capital letters), others are calculated for the code (small letters), and the data collected during the observation.

In the examples of this tutorial, the variable named iris_sji is a SoI.

For making the access easier to the keys and the method, they can be called either as a key or as an attribute or method of an object. Let’s see an example:

>>> for i in iris_sji.SJI['SJI_2796'].keys(): print(i)
data
wl
date_in_filename
iris_obs_code
raster_info
date_time
DATE_OBS
DATE_END
TDET
TDESCT
TWAVE
TWMIN
TWMAX
SPCSCL
SPXSCL
SPYSCL
POS_X
POS_Y
SLTPX1IX
SLTPX2IX
date_time_acq
date_time_acq_ok
extent_arcsec_arcsec
extent_px_px
extent_px_arcsec
extent_opt
extent_opt_coords
list_extent
list_extent_coords
clip_ima
_SJI__clip_ima_ini
poi

# Accessing to a value as a key in the parent object dictionary
>>> print(iris_sji.SJI['SJI_2796']['date_in_filename'])
'20160114_230409'

# Accessing to a value as an attribute in the parent object dictionary
>>> print(iris_sji.SJI['SJI_2796'].date_in_filename)
'20160114_230409'

The simplest way to start working with the data stored in a SoI is through its attribute data, which is a numpy.ndarray object, i.e. an array object with all the attribute and methods inherits from the numpy class. Thus, we can operate with that attribute of the SoI object as any other numpy.ndarray object. Let’s see some examples:

>>> print(type(iris_sji.SJI['SJI_2796'].data))
<class 'numpy.ndarray'>

# Let's see the size and dimensions of the data
>>> print(iris_sji.SJI['SJI_2796'].data.shape)
(779, 1400, 80)

# Let's calculate the maximum value (avoiding NaN values)
# of the 2D array corresponding to the 5th
# position in the 3rd dimension. In Python the
# first index of a dimension in a numpy array starts
# at 0, therefore the index for the 5th position is 4.
>>> import numpy as np
>>> np.nanmax(iris_sji.SJI['SJI_2796'].data[:,:,4])
3574.25

Warning

In Python, a 2D array starts by the upper left corner, being the \(1^{st}\) dimension the one usually associated to the vertical axis, and the \(2^{nd}\) dimension to the horizontal axis. In IDL, for instance, a 2D array starts by the lower left corner, being the \(1^{st}\) dimension the one usually associated to the horizontal axis, and the \(2^{nd}\) dimension to the vertical axis. Thus, in the example given, in Python the size of the dimensions of iris_sji.SJI[‘SJI_2796’].data are [Y,X,t] = [779, 1400, 80]

3.2.2.1. Visualizing and selecting SJI IRIS Level 2 data

We have developed a visualization tool to display the SJI data contained in the SoI object. Indeed, the object SoI was built to be easily displayed by this and other functions. The visualization tool is a method of the extract_irisL2data.SJI class, and it accepts a SoI as an input:

# Let's visualize the SJI data
>>> extract_irisL2data.SJI.quick_look(iris_sji)
# or cleaner
>>> vis = extract_irisL2data.SJI.quick_look
>>> vis(iris_sji)

Because iris_sji is a SoI, i.e. an object of the extract_irisL2data.SJI class, we can simply use:

>>> iris_sji.quick_look()
_images/IRIS_slit_viewer.png

Figure 3.1 The output of the iris_sji.SJI.quick_look method applied to the SoI object named iris_sji, which contains data from the IRIS spectral window ‘SJI_2796’.

Figure 3.1 shows the result of the previous command. The bottom panel allows us to move backward and forward the selected position in the \(3^{rd}\) dimension of the data cube by hovering the mouse over this panel.

The SoI object has an attribute that sets the lower and upper limits of the image displayed by the extract_irisL2data.SJI.quick_look method. These limits are automatically calculated by the code when the SoI is built. The user can change these values, which may be useful for a better visualization manually:

>>> print(iris_sji.SJI['SJI_2796'].clip_ima)
[5, 1031.3840103149414]
>>> iris_sji.SJI['SJI_2796'].clip_ima = [30, 500]
>>> iris_sji.quick_look()

or dynamically when the visualization window has been displayed by pressing the keys u/i/o/p (see below).

The extract_irisL2data.SJI.quick_look method, or iris_sji.quick_look in our example, allow us to inspect interactively the data stored in the iris_sji.SJI['SJI_2796'].data attribute. The data dimensions are [Y, X, time_step], being Y the direction along the slit, X the direction perpendicular to the slit or the steps used to sample the solar surface perpendicularly to the slit either temporally (sit-and-stare) or spatially, and time_step, any of the acquisition taken during the observation.

iris_sji.quick_look method uses matplotlib to display the images, we can take advantage of the options provided by that package. Thus, we can use the zoom option to visualize and animate the SJI data (see Figure 3.2). Note that the bottom panel has been also zoomed-in. That allows us to make a mini-loop animation in those steps of our interest, in the example, between the image 30 to 55. We can recover the full access to all the images by pressing h.

_images/IRIS_slit_viewer_zoom.png

Figure 3.2 The output of the iris_sji.SJI['SJI_2796'].quick_look() zoomed in by using the zoom option provided by matplotlib, by clicking in the magnifier icon, the \(5^{th}\) icon in the lower left corner. Note that zooming-in in the bottom panel allows us to make mini-loop animation.

We have developed and integrated new functionalities to the ``matplotlib` figure. These new functions are controled by some shortcut keys that are summarized in the following list and in the Figure 3.3. They allow us to:

  • Space bar: start/stop the animation of displaying the data corresponding to the steps in the \(3^{rd}\) dimension.

  • -/+: controls the speed of the SJI animation. By pressing -/+ the animation is displayed slower/faster.

  • Left/right arrow: allows to move backward/forward step-by-step along the \(3^{rd}\) dimension.

  • v/V: show/hide a dashed vertical line over the slit. by pressing V, the line overplotted is thicker.

  • H: returns to the initial display parameters. This affects to the image displayed (e.g. after being zoomed-in) and to the contrast intensity levels of the image.

  • u/i/o/p: these keys are used to control the contrast intensity of the image displayed in the figure. By pressing u/i the lower limit of the intensity image is decreased/increased respectively. By pressing o/p, the higher limit of the intensity image is decreased/increased respectively. If one of the limits is 0, these keys will not have an effect on that limit. In this case, it may be convenient to change this value in the attribute iris_sji.SJI[window_label].clip_ima, as it has been done above.

  • a: add a Point of IRIS or PoI to the SoI. Thus, the user can store a particular slit acquisition. Note that a PoI contains the data values for a given step, i.e. the 2D data corresponding to the plane [Y,X,selected_step], and some relevant information for the display.

  • e/r/t: these keys allow us to erase, return or go to a saved PoI.

  • ? or /: show help menu.

  • q: quit the visualization tool.

These keys - except e/r/t - and the native matplotlib capabilities can be used during the animation mode. We have to be especially careful when the animation is working and not press continuously the control keys, e.g. the contrast intensity control keys u/i/o/p, and wait any time the keys are pressed. That allows to the animation loop to keep the control on the figure. We can always stop the animation, adjust the configuration of the display, and restart the animation.

_images/shortcut_keys_SJI_quick_look.png

Figure 3.3 Shortcut keys to inspect SJI IRIS Level 2 data with the visualization tool extract_irisL2data.SJI.quick_look.

The performance of the animation mode is limited by the data size and the user’s computer features, but mostly by the capabilities of matplotlib package, which was not designed to visualize animations. A faster solution can be found in A quick Slitjaw Viewer.

3.2.3. Inspection of a raster IRIS Level 2 data file

By default, extract_irisL2data.load loads the spectra corresponding to Mg II k 2796. The keyword window_info allow us to load other spectral windows using a list of window labels. This keyword has to be a list even if we only want to load a single spectral window, e.g.:

>>> iris_raster = extract_irisL2data.load(raster_filename, window_info = ['Si IV 1394'])
>>> print(iris_raster.windows)
['Si IV 1394']

extract_irisL2data.load method has the advantage over the extract_irisL2data.only_data method that the former loads the data for a spectral window (or multiple ones) independently of the number of the extension where the data are stored, as far as the spectral window(s) is (are) actually in the raster file. extract_irisL2data.load makes easier to load the data for one or several spectral windows in different files, even if they are stored in different extension numbers.

Let us load some spectral windows, for instance Si IV 1394, 2814, and Mg II k 2796, from the raster file:

>>> raster_filename = 'iris_l2_20160114_230409_3630008076_raster_t000_r00000.fits'
>>> lines = extract_irisL2data.show_lines(raster_filename)
>>> iris_raster = extract_irisL2data.load(raster_filename,
>>>                                       window_info = lines[[3,6,7]],
>>>                                       verbose = True)
The provided file is a raster IRIS Level 2 data file.

Extracting information from file iris_l2_20160114_230409_3630008076_raster_t000_r00000.fits...

Available data are stored in windows labeled as:

--------------------------------------------------------------------
Index --- Window label --- Y x X x WL --- Spectral range [AA] (band)
--------------------------------------------------------------------
  0      C II 1336         779x320x374     1332.66 - 1337.50  (FUV)
  1      Fe XII 1349       779x320x244     1347.64 - 1350.79  (FUV)
  2      O I 1356          779x320x340     1352.19 - 1356.59  (FUV)
  3      Si IV 1394        779x320x411     1390.87 - 1396.08  (FUV)
  4      Si IV 1403        779x320x601     1398.60 - 1406.23  (FUV)
  5      2832              779x320x113     2831.38 - 2834.23  (NUV)
  6      2814              779x320x146     2812.69 - 2816.38  (NUV)
  7      Mg II k 2796      779x320x530     2793.14 - 2806.61  (NUV)
--------------------------------------------------------------------

Observation description:  Large dense 320-step raster 105.3x120 320s   Deep x 8    Lossless co

Creating temporary file...  /var/folders/jn/wjxzjh7n2nv58g9t3qjmgdr8000mky/T/tmpz43rtuc8
Creating temporary file...  /var/folders/jn/wjxzjh7n2nv58g9t3qjmgdr8000mky/T/tmpvn8vwi7v
Creating temporary file...  /var/folders/jn/wjxzjh7n2nv58g9t3qjmgdr8000mky/T/tmpwpuby2h_


The selected data are passed to the output variable
(e.g iris_raster) as a dictionary with the following keys:

 - iris_raster.raster['Si IV 1394']
 - iris_raster.raster['2814']
 - iris_raster.raster['Mg II k 2796']

The spectral windows stored in an output object can be found in the windows attribute, or with the method keys in the dictionary object attribute of the output object. In our example, the RoI object is called iris_raster. Let us take a look at its attributes:

>>> print(type(iris_raster))
<class 'extract_irisL2data.raster'>

# Let's verify the spectral windows loaded
>>> print(iris_raster.windows)
['Si IV 1394' '2814' 'Mg II k 2796']

# The attribute 'raster' of the  'Roi' is a dictionary with keys named
# with the labels of the spectral windows loaded
>>> print(iris_raster.raster.keys())
dict_keys(['Si IV 1394', '2814', 'Mg II k 2796'])

#Let's see the keys in the iris_raster.raster['Mg II k 2796'] dictionary
>>> print(iris_raster.raster['Mg II k 2796'].keys())
dict_keys(['data', 'wl', 'date_in_filename', 'iris_obs_code', 'raster_info',
'DATE_OBS', 'DATE_END', 'TDET', 'TDESCT', 'TWAVE', 'TWMIN', 'TWMAX', 'SPCSCL',
'SPXSCL', 'SPYSCL', 'EXPTIME', 'STEPT_AV', 'POS_X', 'POS_Y',
'date_time_acq', 'date_time_acq_ok', 'number_ext', 'binxy', 'binwl',
'extent_arcsec_arcsec', 'extent_px_px', 'extent_px_arcsec', 'extent_time_px',
'extent_time_arcsec', 'extent_opt', 'extent_opt_coords', 'list_extent',
'list_extent_coords', 'extent_display', 'extent_display_coords',
'clip_ima', 'lim_yplot', 'delay', 'poi', '_raster__count_poi',
'_raster__move_count_poi', '_raster__count_coords', '_raster__z_pos',
'_raster__z_pos_ori', '_raster__z_pos_ext', '_raster__dim_data',
'_raster__xlim1', '_raster__ylim1', '_raster__xlim2', '_raster__ylim2',
'_raster__xlim3', '_raster__ylim3', 'arr_y_ax1', 'arr_x_ax1', 'arr_x_ax3'])

The capital-case keys are directly taken from the original headers. The small-case keys are derived from the headers or created for the code to be used during the visualization of the RoI. Likely the more important attributes or keys are data and wl. Note that they can be accessed as an attribute or a key:

>>> print(iris_raster.raster['Mg II k 2796'].wl.shape)
(530,)
>>> print(iris_raster.raster['Mg II k 2796']['wl'].shape)
(530,)

The dimensions of the data attribute in a Roi are [Y,X,spectral_sampling], where the spatial dimensions Y and X are the same as in the SJI file, and the spectral_sampling corresponds to the sampling in the spectral positions (given in Å), and contained in the variable iris_raster.raster[window_label].wl.

# Let's see the size and dimensions of the data stored in each key of
# the output dictionary.
>>> for i in iris_raster.raster.keys():
>>>     print("Size and dimensions of data stored in the "
>>>           "iris_raster.raster[\'{}\']: {}".format(i, iris_raster.raster[i].data.shape))
Size and dimensions of data stored in the iris_raster.raster['Si IV 1394']: (779, 320, 411)
Size and dimensions of data stored in the iris_raster.raster['2814']: (779, 320, 146)
Size and dimensions of data stored in the iris_raster.raster['Mg II k 2796']: (779, 320, 530)

3.2.3.1. Visualizing and selecting raster IRIS Level 2 data

We can visualize the data stored in a key corresponding to a spectral window using the method extract_irisL2data.raster.quick_look, or directly from this method in the RoI object. Thus, to visualize the data in the Mg II k 2796 spectral window contained in the RoI called iris_raster in our example, we use:

# Let's visualize the data in the spectral window ``Mg II k 2796``
>>> iris_raster.quick_look()

Figure 3.4 shows the output of the iris_raster.quick_look() method. Three panels are shown: i) in the upper left panel, the spatial distribution of intensity for a given spectral position with the title XY map at a wavelength given in Å and in the data cube position (\(3^{rd}\) dimension)`; ii) in the upper right panel, the full spectra along the slit for a given sampling step (either spatial or temporal) with the title Slit spectra at the position in the XY map and in the data cube (\(2^{nd}\) dimension); the bottom panel, titled Spectra, shows the spectral distribution at the [X,Y] position of the XY map, which also corresponds to the intensity along the X direction (wavelength in pixels) at the Y position (along the slit) of the Slit spectra panel.

_images/quick_look_raster_OK.png

Figure 3.4 The output of the quick_look method of the RoI called iris_raster, i.e. of iris_raster.quick_look().

The three panels are dynamically connected, which means that the movement of the mouse in one panel has an impact on the others and their titles. This may make the inspection of the data a bit slow, depending of the size of the data and the hardware features of the user’s machine.

Again, as we are using matplotlib package, we can use its options, e.g. to save the image displayed we can type s to save the current figure, or use the zoom option to look in detail an interesting region.

We have added an new feature related with the zoom options. Now, when a panel is zoomed-in, the user can automatically matches the axes shared with other panels. Thus, if we zoom in the Slit spectra, we are changed the region showed both in the axis corresponding to the Slit (axis Y in the XY map panel) and in the axis corresponding to the wavelengths (axis X in the Spectra). The user can automatically fits sharing axes by pressing v.

One of the most useful new capability developed for the visualization tool is to be able to easily show several spectral windows stored in a RoI. The following commands load and store 4 spectral regions in a RoI, which are visualized with iris_raster.quick_look.

# Let's check again the available spectral windows that we stored
# in the 'lines' variable
>>> for j, i in enumerate(lines): print(j, i)
0 C II 1336
1 Fe XII 1349
2 O I 1356
3 Si IV 1394
4 Si IV 1403
5 2832
6 2814
7 Mg II k 2796

# Let's load 4 spectral windows of out interest
>>> iris_raster = extract_irisL2data.load(raster_filename, window_info = lines[[0,3,5,7]])
# Let's visualize the 4 spectral windows
>>> extract_irisL2data.raster.quick_look(iris_raster)

The way to go backward/forward from a spectral window to another is by pressing n/m. Note that the panels for the different spectral windows are independently considered. Thus, for instance, we can zoom-in in the XY map panel for the window showing the data of Mg II k 2796 and not in the others, or change the contrast intensity in just the Si IV 1394 panels. Of course, we can be interested in to inspect in detail a particular region in all the selected spectral windows. After, zoomed in that region of interest in the XY map of a particular spectral window, if we press N/M we will move backward/forward through the spectral windows showing always the same region in the XY map panel. The animation in Figure 3.5 shows how this looks when the we inspect the same zoomed-in area in all the selected spectral windows by pressing N/M.

_images/example_ql_zoom_300_00.png

Figure 3.5 Inspecting simultaneously several spectral windows with the iris_raster.quick_look() method. iris_raster is a RoI that, in addition to the quick_look methods, contains data of ‘C II 1336’, ‘Si IV 1394’, ‘2832’, and ‘Mg II k 2796’ spectra. Note that the Y axis of the SlitSpectra has been automatically fit to the slit portion zoomed-in in the XY map by pressing v when the mouse was on this last panel.

Because the raster IRIS data may be very different in their spatial or temporal dimensions, e.g. a raster scan over an active region or a sit-and-stare observation during 1h, the visualization code allows to choose between several combinations of spatial/temporal scales by pressing y. By default, extract_irisL2data.raster.quick_look uses that ratio of the scales in the X and Y axes closest to 1.

The capabilities in the extract_irisL2data.raster.quick_look or the iris_raster.quick_look method, allows us to:

  • Space bar: start/stop the animation of displaying the data. The animation activated on the XY map runs either in direction perpendicular to the slit, or over time for the sit-and-stare data. The animation activated over the SlitSpectra runs along the slit. The animation activated over the Spectra runs along the wavelengths. The animation also runs on zoomed areas, and can be zoomed-in while the animation is running.

  • -/+: control the speed of the SJI animation. By pressing -/+ the animation is displayed slower/faster.

  • Left/right/up/down arrows: move manually the focus point on the X-Y directions in the XY map. That can be done even when the panels are blocked, which is useful for a detail inspection. By pressing simultaneously alt and an arrow key the incremental step is 5 pixels in that direction.

  • z/x: move backward/forward in wavelength. By pressing simultaneously shift and these keys the incremental step is 5 pixels in wavelength.

  • n/m: move backward/forward between spectral windows. We can inspect the same region in the XY map by pressing N/M. See Figure 3.5.

  • v: fit the sharing axes when a panel has been zoomed-in.

  • h/H: by pressing h over a panel we can return it to its initial dimensions after being zoomed-in. To change all the panels of a spectral windows press H.

  • y: chose between different scales for the XY map.

  • Y: toggles the title in the SlitSpectra panel between the acquisition number and the date and time of that acquisition.

  • u/i/o/p: these keys are used to control the contrast intensity of the image displayed in the figure. By pressing u/i the lower limit of the intensity image is decreased/increased respectively. By pressing o/p, the higher limit of the intensity image is decreased/increased respectively. We can also change the upper/lower limits of the Y axis of the plot shows in the Spectral with these keys.

  • a: add a Point of IRIS or PoI to the SoI. Thus, the user can store a particular slit acquisition. Note that a PoI contains the data values for a given position [X, Y, wavelength] in the spectral window currently displayed. By pressing A, a PoI per spectral window and the current wavelength in each spectral window is saved, i.e. the point information for the [X,Y] is saved simultaneously in for all the spectral windows. In particular, a PoI stores the images (i.e. data) displayed in the XY map, the SlitSpectra, and the Spectra panels. It also contains all the information relevant for the display. plane [Y,X,selected_step], and some relevant information for the display.

  • e/r/t: these keys allow us to erase, return or go to a saved PoI.

  • d: delete all data corresponding to a spectral window from the RoI.

  • ? or /: show help menu.

  • q: quit the visualization tool.

Figure 3.6 visually summarized all the shortcut keys associated to the extract_irisL2data.raster.quick_look

_images/shortcut_keys_raster_quick_look.png

Figure 3.6 The shortcut keys for extract_irisL2data.raster.quick_look and their meaning.

3.2.4. Customizing and initializing the visualization tool

Although the quick_look method allows to change many visualization parameters dynamically when the RoI or the SoI are being inspected, some basic ones can be directly changed from the attributes of these objects.

We can use the attributes from the RoI and SoI objects to make figures and plots manually and customize the aspect of the visualization.

Tip

When show = True in iris_sji.build_figure or iris_raster.build_figure methods, the user can change the size of the figure by resizing the displayed matplotlib window. The new size will be automatically stored in the attribute figsize of window_label of the SoI or the RoI. This also happens when the visualization is done through the quick_look method. Thus, the user can play with the size of the IRIS SJI or raster viewer to get a figure more appealing in terms of aspect and white spaces between the panels.

If the user tries to adjust the spaces between panels with the Adjust Tool of matplotlib by clicking in the \(6^{th}\) icon shown in the Figure 3.4 , the IRIS viewer window losses the control of the keys. To recover the control of the keys the user can click in the Save icon (the \(7^{th}\) icon shown in the Figure 3.4), and when the Save dialog window is open then press on the Cancel button.

The main attributes of the SoI, in our example of iris_sji.SJI['SJI_2796'] are:

  • xlimN: the upper and lower limits of the X axis in the panel N, with N = 1 corresponding to the SlitJawImage panel, and N = 2 for the SlideBar panel.

  • ylimN: the upper and lower limits of the Y axis in the panel N, with N = 1 corresponding to the SlitJawImage panel, and N = 2 for the SlideBar panel.

  • clip_ima: the upper and lower limits of the values displayed in the SlitJawImage panel.

  • cmap: the color map used to display the image in the SlitJawImage panel.

  • slit_acqnum: the number of the slit-jaw image acquisition to be displayed.

  • show_slit: if True, it plots a dashed line over the slit in the displayed slit-jaw image. Default is False.

  • set_IRIS: if True, it sets the default values of a SoI, which include thresholds for the displayed images and plot, and the color map for the spectral window stored in the SoI.

Thus, we can customize or initialize the IRIS SJI viewer:

>>> iris_sji.SJI['SJI_2796'].xlim1 = [200, 700]
>>> iris_sji.SJI['SJI_2796'].xlim2 = [0, 30]
>>> iris_sji.SJI['SJI_2796'].clip_ima = [50,500]
>>> iris_sji.SJI['SJI_2796'].figsize = [5,6.5]
>>> iris_sji.SJI['SJI_2796'].slit_acqnum = 15
>>> iris_sji.SJI['SJI_2796'].show_slit = True
>>> iris_sji.quick_look()

or to make a figure and save it in a file with the keyword filename of the iris_sji.build_figure method (see Figure 3.7):

>>> iris_sji.build_figure(show = True, filename = 'SJI_Mg_II_2796_seahorse_plage.pdf')
_images/example_build_figure_SJI.png

Figure 3.7 Output of the customized iris_sji object.

We can also customize the iris_raster and create a PDF file of the resulting figure:

>>> iris_raster.raster['Mg II k 2796'].clip_ima = [50,500]
>>> iris_raster.raster['Mg II k 2796'].lim_yplot  = [0,900]
>>> iris_raster.raster['Mg II k 2796'].xlim1 = [50,100]
>>> iris_raster.raster['Mg II k 2796'].ylim1 = [40,110]
>>> iris_raster.raster['Mg II k 2796'].x_pos_ext = 66
>>> iris_raster.raster['Mg II k 2796'].y_pos_ext = 81
>>> iris_raster.raster['Mg II k 2796'].z_pos_ext = 2803.80
>>> iris_raster.raster['Mg II k 2796'].figsize =  [8.5, 7.21]
>>> iris_raster.raster['Mg II k 2796'].set_IRIS = False
>>> iris_raster.build_figure(window_label='Mg II k 2796', show = True,
>>>                          filename='example_build_figure.pdf')

or to initialize the quick_look:

>>> iris_raster.quick_look()

Thus, the visualization aspect is tighter, as we can see in Figure 3.8:

_images/example_build_figure_raster.png

Figure 3.8 Output of the customized iris_raster object.

Note that the method iris_raster.build_figure requires to have a valid loaded spectral window passed through the keyword window_label (‘Mg II k 2796’ in the example).

The main customizable attributes of iris_raster.raster['window_label'] are:

  • xlimN: the upper and lower limits of the X axis in the panel N, with N = 1 corresponding to the XY map panel, N = 2 for the Slit Spectra panel, and N = 3 for the Spectra panel.

  • ylimN: the upper and lower limits of the Y axis in the panel N, with N = 1 corresponding to the XY map panel, N = 2 for the Slit Spectra panel, and N = 3 for the Spectra panel.

  • x_pos_ext: the position in the X axis in the XY map panel. It always given in the coordinates shown in that panel.

  • y_pos_ext: the position in the Y axis in the XY map panel. It always given in the coordinates shown in that panel.

  • z_pos_ext: the position in the Z axis of the Spectra panel. This value is given in Å, since this panel is shown by default in these units.

  • clip_ima: the upper and lower limits of the values displayed in the XY map and Slit Spectra panels.

  • cmap: the color map used to display the images in the XY map and Slit Spectra panels.

  • extent_display: the values of extent keyword, i.e the values of the X and Y axes in the XY map panel. The default available list of values accepted by this attribute is stored in output_variable.[window_label].list_extent.

  • extent_display_coords: the coordinates corresponding to the extent_display. The default available list of values accepted by this attribute is stored in output_variable.[window_label].list_extent_coords.

  • set_IRIS: if True, it sets the default values of a RoI, which include spectral positions of interest, thresholds for the images and plot displayed, and color map for each spectral window stored in the RoI.

Of course, we can use the RoI attributes to reproduce the Spectra of Figure 3.8 using the standard plot utilities of matplotlib. Thus, we can do:

plt.figure(figsize=(12,3))
xdata = iris_raster.raster['Mg II k 2796'].wl
ydata = iris_raster.raster['Mg II k 2796'].data[486, 188,:]
ylim = iris_raster.raster['Mg II k 2796'].lim_yplot
plt.plot(xdata, ydata)
plt.ylim(*ylim)
plt.xlabel('Wavelength [AA]')
plt.ylabel('Intensity [DNs]')
plt.tight_layout()
plt.show()

In Section 3.5 we show how to make figures of the panels showed by the quick_look method from a PoI of a SoI or a RoI.

3.3. Saving your data and selected points

Note

For the right functioning of this section and the following ones, the user must have added PoI to the spectral windows of the RoI and SoI by pressing a/A. The name of the files given through the keyword filename in this section is at user’s discretion, but it has to be the same during the execution of the code. Note that the extension ‘.jbl.gz’ is automatically added to the filename if this does not have this extension.

The module extract_irisL2data has a method or function devoted to save either a RoI, or a SoI or only the PoI in a simple, easy way. This method is called save and uses saveall (see Saving your search (and other varibles)) to write these objects into a file:

# Saving the 'iris_raster' object to a file
>>> extract_irisL2data.save(iris_raster)
Saving variable(s) ...
raster2save
... in roi_iris_l2_20160114_230409_3630008076_raster_t000_r00000.jbl.gz

# Saving the 'iris_sji' object to a file
>>> extract_irisL2data.save(iris_sji)
Saving ...
sji2save
... in soi_iris_l2_20160114_230409_3630008076_SJI_2796_t000.jbl.gz

By default, extract_irisL2data.save sets automatically the filename by preceeding the original IRIS Level 2 data file with the tags roi_ or soi_, and changes the .fits extension to the .jbl.gz extension. If the destination file already exists, the code asks to verify whether the file should be actually overwritten or not. To avoid this safety question the user can set the keyword force equals True:

# Saving the 'iris_sji' object to an existing file
>>> extract_irisL2data.save(iris_sji)
# Answer 'n'
File soi_iris_l2_20160114_230409_3630008076_SJI_2796_t000.jbl.gz exists. Do you want to overwrite it?  Y/[n]
Nothing has been done.

>>> extract_irisL2data.save(iris_sji, force = True)
Saving ...
sji2save
... in soi_iris_l2_20160114_230409_3630008076_SJI_2796_t000.jbl.gz

If we want to save our data in another directory than the current one or/and with a different name, we can use the keyword output_dir:

# Saving the 'iris_raster' object to a file called 'roi_noaa_12480_IRIS.myext' in '/scratch/asainz/'
>>> extract_irisL2data.save(iris_raster, filename = 'roi_noaa_12480_IRIS.myext', out_dir='/scratch/asainz/')
Saving variable(s) ...
raster2save
... in /scratch/asainz/roi_noaa_12480_IRIS.myext.jbl.gz

Note that the codes automatically adds the extension .jbl.gz to the filename if this does not have that extension.

We may be interested in saving only the saved PoI in a RoI or a SoI and discard the rest of the data, since they can be read again from the original IRIS Level 2 data. To this aim, we can use the keyword only_poi. That will change the original data from the attribute data of RoI or the SoI by an array of 1x1x1 dimension with value 0. Thus, the RoI and the SoI conserve their structure but save space in disk. Let’s see an example

# Saving the 'iris_raster' object to a file in '/scratch/asainz/'
>>> extract_irisL2data.save(iris_raster, only_poi = True)
Saving variable(s) ...
raster2save
... in poi_roi_iris_l2_20160114_230409_3630008076_raster_t000_r00000.jbl.gz

# Saving only the PoI of 'iris_raster' object to a file named 'mypoints_noaa12480_IRIS.myext'
>>> my_poi_filename = 'mypoints_12480_IRIS.myext'
>>> extract_irisL2data.save(iris_raster, filename = my_poi_filename, only_poi = True)
Saving variable(s) ...
raster2save
... in mypoints_12480_IRIS.myext.jbl.gz

3.4. Loading your data and selected points

To load a RoI, a SoI, or a only-PoI object we use the same method that we used for the IRIS Level 2 data. The code verifies that the input file is a gzip compressed file, and for the tags roi_, soi_, or poi_. Therefore, we strongly suggest to the user to name the files including these flags. If for any reason the user do not want to use these tags in the filename, these can be passed with the keywords roi_file and soi_file. Let us load the files saved during the previous section:

>>> myIRISdata = extract_irisL2data.load('roi_iris_l2_20160114_230409_3630008076_raster_t000_r00000.jbl.gz')
>>> print(type(myIRISdata))
<class 'extract_irisL2data.raster'>
>>> print(myIRISdata.windows)
['C II 1336', 'Si IV 1394', '2832', 'Mg II k 2796']
# Let's check the size of the data of the 'C II 1336' spectral window
>>> print(myIRISdata.raster['C II 1336'].data.shape)
(779, 320, 374)

# Loading an only-PoI RoI object named manually by the user
>>> my_poi_filename = 'mypoints_12480_IRIS.myext.jbl.gz' # Note the extension .jbl.gz
>>> myIRISdata = extract_irisL2data.load(my_poi_filename, roi_file = True)
# Let's check the size of the data of the 'C II 1336' spectral window
>>> print(myIRISdata.raster['C II 1336'].data.shape)
(1, 1, 1)
# Let's check how many PoI have the 'C II 1336' spectral window
>>> print(len(myIRISdata.raster['C II 1336'].poi))

3.5. Making figures from your selected data and points

Once the data are restored, we can easily work again with then using their corresponding quick_look. For the only-PoI data, this method will do nothing, since it has no data (actually it is 1-point array). In this case, we can use the method extract_irisL2data.show_poi. This method can be use with RoI and SoI having PoI or with only-PoI data.

# Showing the 3-panel figure corresponding to the 2nd PoI of the spectral window 'Si IV 1394'
# of the 'RoI' called myIRISdata restored from the file 'mypoints_noaa12480_IRIS.myext.jbl.gz'
>>> extract_irisL2data.show_poi(myIRISdata.raster['Si IV 1394'].poi[1])

extract_irisL2data.show_poi accepts the following keywords:

  • show_XY_map: boolean, it shows the XY_map panel only. Default is False

  • show_SlitSpectra: boolean, it shows the SlitSpectra only. Default is False

  • show_Spectra: boolean, it shows the Spectra panel only. Default is False

  • show_SJI_slit: boolean, it shows the line overplotted over the slit position. Default is True

  • colorbar1: boolean, it shows/hides the color bar in XY map or in the SJI figure. Default is True (shows)

  • colorbar2: boolean, it shows/hides the color bar in SlitSpectra. Default is True (shows)

# Showing the XY-map panel corresponding to the 1st PoI of the spectral window '2832'
# of the 'RoI' called myIRISdata restored from the file 'mypoints_noaa12480_IRIS.myext.jbl.gz'
>>> extract_irisL2data.show_poi(myIRISdata.raster['2832'].poi[0], show_XY_map=True)

# Show the 'PoI' #2 of the spectral window 'SJI_2796' of the 'SoI' called 'iris_sji'
# without the line overplotted on the slit position
>>> extract_irisL2data.show_poi(iris_sji.SJI['SJI_2796'].poi[1], show_SJI_slit=False)

3.6. Straight to the point : 12-line code for working with IRIS Level 2 data in 5 min

If you are not interested in to learn the concepts behind the code, and you just want to start working in Python, you just need to copy the following lines:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
>>> from iris_lmsalpy import hcr2fits, extract_irisL2data
>>> query_text = 'https://www.lmsal.com/hek/hcr?cmd=search-events3&outputformat='\
>>>              'json&startTime=2016-01-14T00:00&stopTime=2016-01-15T00:00&'\
>>>              'minnumRasterSteps=320&hasData=true&minxCen=550&limit=200'
>>> list_urls = hcr2fits.get_fits(query_text)
>>> raster_filename = 'iris_l2_20160114_230409_3630008076_raster_t000_r00000.fits'
>>> SJI_filename = 'iris_l2_20160114_230409_3630008076_SJI_2796_t000.fits'
>>> lines = extract_irisL2data.show_lines(raster_filename)
>>> iris_raster = extract_irisL2data.load(raster_filename, window_info = lines[[3,6,7]])
>>> iris_raster.quick_look()
>>> extract_irisL2data.save(iris_raster)
>>> iris_sji = extract_irisL2data.load(SJI_filename)
>>> iris_sji.quick_look()
>>> extract_irisL2data.save(iris_sji)
The only lines you should change are:
  • Line 2-4: include your own query_text. See Section 2 to learn how to look for and download IRIS Level 2 data

  • Lines 6 and 7: set your own raster and SJI filenames.

  • Line 9: select the spectral windows of your interest in lines

Your comments and suggestions about hcr2fits, extract_irisL2data, and saveall modules can be sent to Alberto Sainz Dalda (asainz.solarphysics@gmail.com).