# 2. Working with IRIS Level 2 data in Python#

This section is aimed to familiarize the reader with the Python jargon and the main module used in this tutorial: extract_irisL2data. This module will allow us to:

• read IRIS Level 2 files

• select spectral regions of interest

• load the data of these spectral windows

• visualize, inspect, and interact with them in an easy and comfortable way

• select points of our interest or discard the data of a selected spectral window

• save the selected data and their associated points of interest

• load, display or share with other colleagues IRIS data in a simple, robust, and efficient way.

Once the decompressed IRIS Level 2 data are hosted locally, the next step is to read, extract, and inspect them. We have created a module with different methods (or routines) to this aim. This module is called extract_irisL2data, and it is used to load and inspect either raster or SJI files. The extract_irisL2data module has several methods. The most relevant are:

• info_fits: shows specific information about the FITS IRIS Level 2 data file. Such as extension numbers, type and size of the data associated to each extension, the spectral window label of each extension, i.e., the spectral data contained in that number extension, and the observed spectral range. See Section 2.3.

• only_header: returns the main header or the extension header (through the keyword extension) of an IRIS Level 2 data file, as a “header” object. The fields of the header can be accessed as the keys of a dictionary object, e.g., hdr['NAXIS1']. See Section 2.3.

• only_data: returns the data from an extension of the IRIS Level 2 data file through the keyword extension (default is 1). The extension number and the associated spectral data can be obtained with the method info_fits. See Section 2.3.

• show_lines: shows the spectral windows available in an IRIS Level 2 data file, their dimensions, and their observed spectral range. It returns an array, being each element of this array a string with the information of the spectral window (window_label).

• raster: a class that builds a “Raster of IRIS” object or RoI. This object contains the data selected by the user from an IRIS Level 2 raster file, some basic information from the headers, and other values that makes its visualization easier. To this aim, the class raster has a method especially designed to visualize a RoI called quick_look. This method allows to add “Points of IRIS” or “PoI”, i.e., points of the user’s interest from the data into the RoI object, which can be easily saved and analyzed.

• SJI: a class that makes a “SJI of IRIS” object or SoI. It is similar to a RoI but contains the data from an IRIS SJI Level 2 data file. It also has its own quick_look method, which also allows to add one or more “PoI” can be added to the SoI.

• load: loads the data for a given spectral window or, in the case of a raster file, for a list of spectral windows.

• save: a method that allows to write to a file a RoI, a SoI, or only their associated PoI.

• show_poi: this method helps us to display a PoI associated to a RoI or to a SoI.

This code uses two classes of object, raster and SJI, which returns two objects,

RoI and SoI respectively, with 3 attributes and 1 method.

For the “Raster of IRIS” object or RoI, the attributes and method are:
• filename: contains the name of the parent raster file.

• windows: contains the requested observed spectral windows. This means that it is a list with the window_label of the data loaded by the user’s request.

• raster: a dictionary, which has as many keys as spectral windows have been asked to be load. Thus, each spectral window has its unique descriptive key window_label. Each of these keys are themselves a dictionary object with different keys, such as data, wl with some information from the header and other useful values like the PoI.

• flush: a method, puts the data of all the spectral windows window_label of a RoI stored in numpy.memmap array into the memory of the system. See Section 2.2.

• tomemmap: a method, puts the data of all the spectral windows window_label of a RoI stored in the memory of the system in a numpy.memmap array. See Section 2.2.

• quick_look: a method, to inspect interactively the spectral data of the RoI. See Section 2.5.

For the “SJI of IRIS” object or SoI, the attributes are:
• filename: contains the name of the parent SJI file.

• window: contains the observed spectral window window_label.

• SJI: a dictionary, which unique key is named with the window_label. This key is itself a dictionary containing several keys, including the data, wl, some information from the header and other useful values, and the PoI list.

• flush: a method, puts the data of the observed spectral window window_label of a SoI stored in numpy.memmap array to the memory of the system. See Section 2.2.

• tomemmap: a method, puts the data of the observed spectral window window_label of a SoI stored in the memory of the system in a numpy.memmap array. See Section 2.2.

• quick_look: a method to inspect interactively the slit-jaw images contained in the SoI. See Section 2.4.

• load: a method allows to load the data previously saved by the author. See Section 2.6.

The following diagram describes some of the methods available in extract_irisL2data module:

extract_irisL2data.
|_info_fits            Method: prompts information about the extension
|                              and the data in a of a raster or SJI IRIS
|                              Level 2 data file
|
|                              or extension header of a raster or SJI
|                              IRIS Level 2 data file
|
|_only_data            Method: returns only data for and extension number
|                              (no. 1 by default) of a raster or SJI
|                              IRIS Level 2 data file
|
|_show_lines           Method: returns a list of strings  with the
|                              label of the spectral windows stored
|                              in an IRIS Level 2 data file |
|
|                              It returns either a 'RoI' or a 'SoI'.
|                              It can also load a previously saved 'RoI'
|                              or a 'SoI'
|
|_raster.              Class:  builds a 'Raster of IRIS' object or
|       |                      'Roi' object.
|       |_quick_look   Method: displays  the data of a 'RoI'.
|       |                       Allows to add 'Point(s) of IRIS' or
|       |                       'PoI' to the 'RoI'
|       |_other methods
|
|_SJI.                 Class:  builds a 'SJI of IRIS' object or
|    |                         'SoI' object.
|    |_quick_look      Method: displays the data of a 'SoI'.
|    |                          Allows to add 'PoI' to the 'SoI'
|    |_other methods
|
|_save                 Method: saves a 'RoI', a 'SoI' or only their
|                              associated 'PoI'
|
|_show_poi             Method: displays a 'PoI'


The RoI, SoI, and PoI objects have attributes of different kind: arrays, tuples, list, or dictionaries, and some methods.

Note

From now on we will be referring to extract_irisL2data as ei to shorten the text. In the examples we will also be doing the same, i.e.,

from iris_lmsalpy import extract_irisL2data as ei


ei.save and ei.load methods allows us to save and load these objects, despite having associated methods and a complex structure. For instance a RoI having data from different spectral range, i.e., storing 3D data cubes with different sizes, and different number of PoI for each spectral window.

Your comments and suggestions about the modules outlined here can be sent to Alberto Sainz Dalda (asainz.solarphysics@gmail.com).

## 2.1. A quick look at IRIS Level 2 data#

We are going to work with the data corresponding to the NOAA AR 12480. Thus, we are going to download a subset of data observed by IRIS. With the following commands we will download the data, which consist of one raster file and five SJI files corresponding to 5 spectral windows observed by IRIS. Once the files are hosted locally, we will take a look into the header of the files.

The following command lines are the same that the ones in the $$1^{st}$$ code block of Section 1. There is no need to execute them again if they have been already run.

>>> from iris_lmsalpy import hcr2fits
>>> query_text = 'https://www.lmsal.com/hek/hcr?cmd=search-events3&outputformat=json&startTime=2016-01-14T00:00&stopTime=2016-01-15T00:00&minnumRasterSteps=320&hasData=true&minxCen=550&limit=200'
>>> list_urls = hcr2fits.get_fits(query_text)
Requesting the query...
...
Decompressing the file iris_l2_20160114_230409_3630008076_SJI_1330_t000.fits.gz into /home/user/IRIS_data/ (#1 of 5) ...
...


Let us recover the header of the raster file and show the description of the observation:

>>> from iris_lmsalpy import extract_irisL2data as ei

>>> raster_filename = 'iris_l2_20160114_230409_3630008076_raster_t000_r00000.fits'
>>> hdr['OBS_DESC']
'Large dense 320-step raster 105.3x120 320s   Deep x 8    Lossless co'


While the SJI files contain just one spectral window per file, the raster files have several spectral windows per file. We can know what spectral windows are observed in an IRIS Level 2 data file by using the method ei.show_lines.

>>> SJI_filename = 'iris_l2_20160114_230409_3630008076_SJI_2796_t000.fits'
>>> lines = ei.show_lines(SJI_filename)
Extracting information from file iris_l2_20160114_230409_3630008076_SJI_2796_t000.fits...

Available data with size Y x X x Image are stored in a window labeled as:

-------------------------------------------------------------
Index --* Window label --* Y x X x Im --* Spectral range [AA]
-------------------------------------------------------------
0         SJI_2796       779x1400x80    2794.00 * 2798.00
-------------------------------------------------------------

Observation description:  Large dense 320-step raster 105.3x120 320s   Deep x 8    Lossless co

>>> raster_filename = 'iris_l2_20160114_230409_3630008076_raster_t000_r00000.fits'
>>> lines = ei.show_lines(raster_filename)
Extracting information from file iris_l2_20160114_230409_3630008076_raster_t000_r00000.fits...

Available data with size Y x X x Wavelength are stored in windows labeled as:

--------------------------------------------------------------------
Index --* Window label --* Y x X x WL --* Spectral range [AA] (band)
--------------------------------------------------------------------
0      C II 1336         779x320x374     1332.66 * 1337.50  (FUV)
1      Fe XII 1349       779x320x244     1347.64 * 1350.79  (FUV)
2      O I 1356          779x320x340     1352.19 * 1356.59  (FUV)
3      Si IV 1394        779x320x411     1390.87 * 1396.08  (FUV)
4      Si IV 1403        779x320x601     1398.60 * 1406.23  (FUV)
5      2832              779x320x113     2831.38 * 2834.23  (NUV)
6      2814              779x320x146     2812.69 * 2816.38  (NUV)
7      Mg II k 2796      779x320x530     2793.14 * 2806.61  (NUV)
--------------------------------------------------------------------

Observation description:  Very large dense 320-step raster 105.3x175 320s   Deep x 8 Spatial x

>>> # The window labels are stored in the output variable ('lines' in this example)
>>> # as an array of strings.
>>> print(lines)
['C II 1336' 'Fe XII 1349' 'O I 1356' 'Si IV 1394' 'Si IV 1403' '2832'
'2814' 'Mg II k 2796']


Note that the name of the dictionaries returned by ei.load have the same name as the class they belong to (raster and SJI). These classes of objects have some methods (or functionalities) that we discuss in the following section.

## 2.2. Reading the IRIS Level 2 data#

Thus, the object returned by ei.load is either a RoI or a SoI. As the SJI IRIS Level 2 data are simpler than the raster files, since they have only one spectral window per file, we will start here. The philosophy behind the SoI and RoI is the same, although the latter has a more complex structure.

In this tutorial, for the sake of clarity, we have named iris_sji and iris_raster to the SoI and RoI obtained by loading a SJI and a raster IRIS Level 2 data file respectively with ei.load. Of course, the user can name the SoI and RoI as they wish. We use the following command to read and load the data from a SJI IRIS Level 2 file:

>> # To read and load the data from IRIS Level 2 file is pretty simple!
Creating temporary file...  /var/folders/jn/wjxzjh7n2nv58g9t3qjmgdr8000mky/T/tmpjm7s8l4k


The previous command creates a SoI object, which contains the data and other values in the following structure:

iris_sji.
|_filename
|_window = [window_label]
|_SJI[window_label].
|                  |_data [ = SJI data cube ]
|                  |_wl   [ = wavelengths in Angstrom]
|                  |_clip_ima [ = lower, upper thresholds ]
|                  |_... more keys
|
|_flush()          Method: flushes the data from the
|                          temporary file to the memory
|                          of the system
|
|_tomemmap()       Method: moves the data from the memory
|                          of the system to a temporary file
|
|_quick_look()     Method: visualizes the data
|
|_other methods


Note

ei.load method loads the data from a raster or SJI IRIS Level 2 in a numpy memory-map array, i.e., a numpy.memmap class. That means the data are written to a temporary file (in the raster case, as many temporary files as spectral windows are requested). Thus, the data are not loaded in the memory of the system, but written in a temporary file. In this way, the memory pressure in the system is much lower, which makes the user experience smoother and more efficient. For instance, in the following example, the memory required to load all the spectral windows of the raster file is $$~2.66 Gbytes$$, while the memory used by the system when the data are loaded through a memory-map array is just $$96 Mbytes$$. On the other hand, this methods requires to have enough space ($$~2.66 Gbytes$$ in this example) in a physical storage device to write the temporary files. These temporary files are removed when the RoI or SoI are “flushed” or deleted.

We use the same command to read the data from a raster IRIS Level 2 data, but it returns a RoI object with the data, wavelengths sampled, and other information stored as:

>>> # To read and load the data from IRIS Level 2 file is pretty simple!
Creating temporary file...  /var/folders/jn/wjxzjh7n2nv58g9t3qjmgdr8000mky/T/tmpvra72uuz


The previous command only loads the data corresponding to the spectral window “Mg II k 2796”, which is the default option. Using the keyword window_info of ei.load we can set what spectral window(s) will be loaded. This keyword is a list with the label of the requested spectral window(s). The label of the spectral windows observed by IRIS in a given observation are stored in the keyword TDESCn of the main header of the raster file, where n corresponds to the extension in the file where the data of that spectral window are stored. In addition, it accepts the value window_info=['all'] to load all the available spectral windows in the raster file. In Section 2.5 we explain how to load multiple spectral windows.

The RoI have the same basic attributes of a SoI, but it includes more attributes devoted to the visualization tool.

iris_raster.
|_filename
|_windows = [sel_window_label_1, ..., sel_window_label_m], m <=n
|_all_windows_in_file = [window_label_1, ..., window_label_n]
|_raster[window_label_1].
|                       |_data [ = raster data cube ]
|                       |_wl   [ = wavelengths in Angstrom]
|                       |_clip_ima [ = lower, upper thresholds ]
|                       |_lim_yplot [ = lower, upper thresholds ]
|                       |_... more attributes (or keys)
|
|_ ... as many as m-2 selected spectral windows
|
|_raster[window_label_m].
|                       |_data
|                       |_wl
|                       |_clip_ima
|                       |_lim_yplot
|                       |_... more attributes (or keys)
|
|_flush()          Method: flushes the data from the
|                          temporary file to the memory
|                          of the system
|
|_tomemmap()       Method: moves the data from the memory
|                          of the system to a temporary file
|
|_quick_look()     Method: visualizes the data
|
|_other methods


Note that windows in a RoI object is a list that may have one or several spectral window labels, while the SoI is a list with a unique spectral window.

If we take a look at the data in the SoI and the RoI, we can see they are numpy.memmap arrays:

>>> print(type(iris_sji.SJI['SJI_2796'].data))
<class 'numpy.memmap'>
>>> print(type(iris_raster.raster['Mg II k 2796'].data))
<class 'numpy.memmap'>


If, for any reason, the user wants to put the data from the temporary file, i.e., from the numpy.memmap array, to the memory of the system, they can use the method flush of the Roi and SoI:

>>> iris_sji.flush()
Removing temporary file...  /var/folders/jn/wjxzjh7n2nv58g9t3qjmgdr8000mky/T/tmpjm7s8l4k
>>> print(type(iris_sji.SJI['SJI_2796'].data))
<class 'numpy.ndarray'>
>>> iris_raster.flush()
Removing temporary file...  /var/folders/jn/wjxzjh7n2nv58g9t3qjmgdr8000mky/T/tmpvra72uuz
>>> print(type(iris_sji.SJI['SJI_2796'].data))
<class 'numpy.ndarray'>


As you can see, the flush method closes and removes the temporary files. This action is automatically done if the variables are deleted, e.g., del iris_raster.

If the user wants to recover the data from the raster or SJI IRIS Level 2 files directly into the memory of the system instead that in a temporary file, the user can just simply set the keyword memmap = False:

>>> iris_sji = ei.load(SJI_filename, memmap = False)
>>> print(type(iris_sji.SJI['SJI_2796'].data))
<class 'numpy.ndarray'>
>>> iris_raster = ei.load(raster_filename, memmap = False)
>>> print(type(iris_sji.SJI['SJI_2796'].data))
<class 'numpy.ndarray'>


If the user has already loaded the data into the memory of the system, like in the previous example, and they want to put the data into a numpy.memmap array:

>>> iris_sji.tomemmap()
Creating temporary file...  /var/folders/jn/wjxzjh7n2nv58g9t3qjmgdr8000mky/T/tmpwc7pqs10
>>> print(type(iris_sji.SJI['SJI_2796'].data))
<class 'numpy.memmap'>

>>> iris_raster.tomemmap()
Creating temporary file...  /var/folders/jn/wjxzjh7n2nv58g9t3qjmgdr8000mky/T/tmpyx5pqy3f
>>> print(type(iris_raster.raster['Mg II k 2796'].data))
<class 'numpy.memmap'>


The RoI and SoI objects obtained with ei.load allow us to inspect the data from the IRIS Level 2 data files in a comfortable, easy way. These objects have the data for the observed requested data, its wavelengths, and other information as attributes, but not all the information available in the IRIS Level 2 FITS files. Some users may prefer to recover these information and other one not present in the RoI and SoI objects directly from the FITS file. ei has some methods to this aim.

First, we need to understand the structure of the IRIS Level 2 FITS data file. The IRIS Level 2 FITS are multi-extension FITS files. An “extension” refers to a part of the file containing self-consistent information. This information will be in the general case, a header and its corresponding data. The first extension is named “primary” and its “extension number” is 0.

The extensions in an IRIS Level 2 SJI FITS file has the following numbers:

• 0: header and data corresponding to the spectral images observed by the SJI.

• 1: header and auxiliary 31 values from each exposure taken by the SJI in the spectral band of the file. It is an array of float values with dimensions $$no. images \times 31$$.

• 2: header and extra data from each exposure taken by the SJI in the spectral band of the file. It is a record array containing 5 string fields for each exposure. The values of each field can be access as the key in a dictionary or as an attribute. See example in the last code block of this section.

An IRIS Level 2 raster FITS file has the following extensions:

• 0: header with the main information of the observation. This header has information about all the spectral windows contained in the file and other relevant information. This extension DOES NOT have spectral data associated.

• 1 to N: header and data for the N spectral windows contained in the file.

• N+1: header and auxiliary 47 values from each exposure considered in the file. It is an array of float values with dimensions $$no. acquisitions \times 47$$.

• N+2: header and extra information data from each exposure considered in the file. It is a record array containing 9 string fields for each exposure. The values of each field can be access as the key in a dictionary or as an attribute. See example in the last code block of this section.

The method ei.info_fits shows the information of the extensions contained in the IRIS Level 2 file. For the SJI file:

>>> ei.info_fits(SJI_filename)
Filename: iris_l2_20160114_230409_3630008076_SJI_2796_t000.fits
No.    Name      Ver    Type      Cards   Dimensions   Format
0  PRIMARY       1 PrimaryHDU     161   (1400, 779, 80)   int16 (rescales to float32)
1                1 ImageHDU        38   (31, 80)   float64
2                1 TableHDU        33   80R x 5C   [A10, A10, A4, A66, A55]

Observation description:  Large dense 320-step raster 105.3x120 320s   Deep x 8    Lossless co

Extension No. 1 stores data and header of SJI_2796: 2794.00 * 2798.00 AA

To get the main header use :

To get header corresponding to data of SJI_2796 use :
hdr = ei.only_header(filename, extension = 0)

To get the data of SJI_2796 use :
data = ei.only_data(filename, extension = 0, memmap = True)


and for the raster file:

>>> ei.info_fits(raster_filename)
Filename: iris_l2_20160114_230409_3630008076_raster_t000_r00000.fits
No.    Name      Ver    Type      Cards   Dimensions   Format
0  PRIMARY       1 PrimaryHDU     379   ()
1                1 ImageHDU        33   (374, 779, 320)   int16 (rescales to float32)
2                1 ImageHDU        33   (244, 779, 320)   int16 (rescales to float32)
3                1 ImageHDU        33   (340, 779, 320)   int16 (rescales to float32)
4                1 ImageHDU        33   (411, 779, 320)   int16 (rescales to float32)
5                1 ImageHDU        33   (601, 779, 320)   int16 (rescales to float32)
6                1 ImageHDU        33   (113, 779, 320)   int16 (rescales to float32)
7                1 ImageHDU        33   (146, 779, 320)   int16 (rescales to float32)
8                1 ImageHDU        33   (530, 779, 320)   int16 (rescales to float32)
9                1 ImageHDU        54   (47, 320)   float64
10                1 TableHDU        53   320R x 7C   [A10, A10, A3, A10, A4, A66, A66]

Observation description:  Large dense 320-step raster 105.3x120 320s   Deep x 8    Lossless co

Extension No. 1 stores data and header of C II 1336:     1332.66 * 1337.50 AA (FUV)
Extension No. 2 stores data and header of Fe XII 1349:   1347.64 * 1350.79 AA (FUV)
Extension No. 3 stores data and header of O I 1356:      1352.19 * 1356.59 AA (FUV)
Extension No. 4 stores data and header of Si IV 1394:    1390.87 * 1396.08 AA (FUV)
Extension No. 5 stores data and header of Si IV 1403:    1398.60 * 1406.23 AA (FUV)
Extension No. 6 stores data and header of 2832:  2831.38 * 2834.23 AA (NUV)
Extension No. 7 stores data and header of 2814:  2812.69 * 2816.38 AA (NUV)
Extension No. 8 stores data and header of Mg II k 2796:  2793.14 * 2806.61 AA (NUV)

To get the main header use :

To get header corresponding to data of C II 1336 use :
hdr = ei.only_header(filename, extension = 1)

To get the data of C II 1336 use :
data = ei.only_data(filename, extension = 1, memmap = True)


If we now want to recover the main header of the raster file, and the header and the data corresponding the spectral window “Si IV 1403”. As extension = 0 is the default option, to get the main header of the file we can just type:

>>> hdr = ei.only_header(raster_filename)
# Let's get the individual header corresponding to Si IV 1403
>>> hdr_SiIV_1403 = ei.only_header(raster_filename, extension = 5)

# Let's get the data corresponding to Si IV 1403
>>> data_SiIV_1403 = ei.only_data(raster_filename, extension = 5)
Creating temporal file...  /var/folders/jn/wjxzjh7n2nv58g9t3qjmgdr8000mky/T/tmpan9g9ak8
>>> print(type(data_SiIV_1403))
<class 'numpy.memmap'>


Note that ei.only_data can return the data as a numpy.array, i.e. the data are loaded into the memory of the system by using memmap = False in the ei.only_data method, or as numpy.memmap, i.e., the data are stored in a temporary file in the file system (the default option). In this case, the temporary file is removed from the file system by deleting the associated variable:

>>> del data_SiIV_1403


Because the number of spectral windows in a raster file may vary from an observation to another, a good option to access to the last 2 extensions of the IRIS Level 2 file is to use the negative index in the extension keyword:

# Let's recover the header and the data corresponding to the auxiliary
# information extension
>>> hdr_aux = ei.only_header(raster_filename, extension = -2)
>>> data_aux = ei.only_data(raster_filename, extension = -2)

# Let's recover the data for the extra information extension
>>> data_extra = ei.only_data(raster_filename, extension = -1)

# Let's check the names of the records
>>> data_extra.dtype.names
('FRMID',
'FUVFDBID',
'FUVCRSID',
'NUVFDBID',
'NUVCRSID',
'FUVfilename',
'NUVfilename',
'FUVtemp',
'NUVtemp')


We can access to the values of the variables stored in the data corresponding to the extra information extension as an attribute or as a key:

# Let's access to the record 'FUVfilename'
>>> data_extra.FUVfilename # or data_extra['FUVfilename']
array(['/irisa/data/level1/2016/01/14/H2300/iris20160114_23041396_fuv.fits',
'/irisa/data/level1/2016/01/14/H2300/iris20160114_23042356_fuv.fits',
...
more FUV filenames are displayed
...
'/irisa/data/level1/2016/01/14/H2300/iris20160114_23550181_fuv.fits'],
dtype='<U66')


## 2.4. Inspection of a SJI IRIS Level 2 data file#

As we have already mentioned, the ei.load method identifies whether the file given as input is a raster or a SJI IRIS Level 2 data file or not. Let us load the SJI data:

>>> SJI_filename = 'iris_l2_20160114_230409_3630008076_SJI_2796_t000.fits'
>>> iris_sji  = ei.load(SJI_filename, verbose = True)

The provided file is a SJI IRIS Level 2 data file containing SJI_2796 data.

Extracting information from file iris_l2_20160114_230409_3630008076_SJI_2796_t000.fits...

Available data with size Y x X x Image are stored in a windows labeled as:

-------------------------------------------------------------
Index --* Window label --* Y x X x Im --* Spectral range [AA]
-------------------------------------------------------------
0         SJI_2796       779x1400x80    2794.00 * 2798.00
-------------------------------------------------------------

Observation description:  Large dense 320-step raster 105.3x120 320s   Deep x 8    Lossless co

The SJI data are passed to the output variable
(e.g iris_sji) as a dictionary with the following keys:

* iris_sji.SJI['SJI_2796']


In this case, we have named the output SoI object iris_sji. This object has three attributes:

• iris_sji.filename, which contains the parent SJI filename

• iris_sji.window, which contains the spectral window in the file

• iris_sji.SJI, a dictionary with the key of the spectral window observed.

In this example, “SJI_2796”, this key is itself a Python dictionary with keys storing certain values obtained from the headers (in capital letters), others are calculated for the code (small letters), and the data collected during the observation.

For making the access easier to the keys and the method, they can be called either as a key or as an attribute or method of an object. Let’s see an example:

>>> iris_sji.SJI['SJI_2796'].keys():
dict_keys[
"data"
"wl"
"date_in_filename"
"iris_obs_code"
"raster_info"
"date_time"
"DATE_OBS"
"DATE_END"
"TDET"
"TDESCT"
"TWAVE"
"TWMIN"
"TWMAX"
"SPCSCL"
"SPXSCL"
"SPYSCL"
"POS_X"
"POS_Y"
"SLTPX1IX"
"SLTPX2IX"
"date_time_acq"
"date_time_acq_ok"
"extent_arcsec_arcsec"
"extent_px_px"
"extent_px_arcsec"
"extent_opt"
"extent_opt_coords"
"list_extent"
"list_extent_coords"
"clip_ima"
"_SJI__clip_ima_ini"
"poi"
])

# Accessing to a value as a key in the parent object dictionary
>>> print(iris_sji.SJI['SJI_2796']['date_in_filename'])
'20160114_230409'

# Accessing to a value as an attribute in the parent object dictionary
>>> print(iris_sji.SJI['SJI_2796'].date_in_filename)
'20160114_230409'


The simplest way to start working with the data stored in a SoI is through its attribute data, which is a numpy.ndarray object, i.e., an array object, with all the normal numpy attribute and methods. Thus, we can operate with that attribute of the SoI object as any other numpy.ndarray object. Let’s see some examples:

>>> print(type(iris_sji.SJI['SJI_2796'].data))
<class 'numpy.ndarray'>

# Let's see the size and dimensions of the data
>>> print(iris_sji.SJI['SJI_2796'].data.shape)
(779, 1400, 80)

# Let's calculate the maximum value (avoiding NaN values)
# of the 2D array corresponding to the 5th
# position in the 3rd dimension. In Python the
# first index of a dimension in a numpy array starts
# at 0, therefore the index for the 5th position is 4.
>>> import numpy as np
>>> np.nanmax(iris_sji.SJI['SJI_2796'].data[:,:,4])
3574.25


Warning

In Python, a 2D array starts by the upper left corner, being the $$1^{st}$$ dimension the one usually associated to the vertical axis, and the $$2^{nd}$$ dimension to the horizontal axis. In IDL, for instance, a 2D array starts by the lower left corner, being the $$1^{st}$$ dimension the one usually associated to the horizontal axis, and the $$2^{nd}$$ dimension to the vertical axis. Thus, in the example given, in Python the size of the dimensions of iris_sji.SJI['SJI_2796'].data are “[Y,X,t] = [779, 1400, 80]”

## 2.5. Inspection of a raster IRIS Level 2 data file#

By default, ei.load loads the spectra corresponding to “Mg II k 2796”. The keyword window_info allow us to load other spectral windows using a list of window labels. This keyword has to be a list even if we only want to load a single spectral window, e.g.,

>>> iris_raster = ei.load(raster_filename, window_info = ['Si IV 1394'])
>>> print(iris_raster.windows)
['Si IV 1394']


ei.load make it easier to load the data for one or several spectral windows in different files, even if they are stored in different extension numbers.

Let us load some spectral windows, for instance “Si IV 1394”, “2814”, and “Mg II k 2796”, from the raster file:

>>> raster_filename = 'iris_l2_20160114_230409_3630008076_raster_t000_r00000.fits'
>>> lines = ei.show_lines(raster_filename)
>>>                                       window_info = lines[[3,6,7]],
>>>                                       verbose = True)
The provided file is a raster IRIS Level 2 data file.

Extracting information from file iris_l2_20160114_230409_3630008076_raster_t000_r00000.fits...

Available data are stored in windows labeled as:

--------------------------------------------------------------------
Index --- Window label --- Y x X x WL --- Spectral range [AA] (band)
--------------------------------------------------------------------
0      C II 1336         779x320x374     1332.66 - 1337.50  (FUV)
1      Fe XII 1349       779x320x244     1347.64 - 1350.79  (FUV)
2      O I 1356          779x320x340     1352.19 - 1356.59  (FUV)
3      Si IV 1394        779x320x411     1390.87 - 1396.08  (FUV)
4      Si IV 1403        779x320x601     1398.60 - 1406.23  (FUV)
5      2832              779x320x113     2831.38 - 2834.23  (NUV)
6      2814              779x320x146     2812.69 - 2816.38  (NUV)
7      Mg II k 2796      779x320x530     2793.14 - 2806.61  (NUV)
--------------------------------------------------------------------

Observation description:  Large dense 320-step raster 105.3x120 320s   Deep x 8    Lossless co

Creating temporary file...  /var/folders/jn/wjxzjh7n2nv58g9t3qjmgdr8000mky/T/tmpz43rtuc8
Creating temporary file...  /var/folders/jn/wjxzjh7n2nv58g9t3qjmgdr8000mky/T/tmpvn8vwi7v
Creating temporary file...  /var/folders/jn/wjxzjh7n2nv58g9t3qjmgdr8000mky/T/tmpwpuby2h_


The selected data are passed to the output variable (e.g iris_raster) as a dictionary with the following keys:

• iris_raster.raster['Si IV 1394']

• iris_raster.raster['2814']

• iris_raster.raster['Mg II k 2796']

The spectral windows stored in an output object can be found in the windows attribute. In our example, the RoI object is called iris_raster, let us take a look at its attributes:

>>> print(type(iris_raster))
<class 'ei.raster'>

# Let's verify the spectral windows loaded
>>> print(iris_raster.windows)
['Si IV 1394' '2814' 'Mg II k 2796']

# The attribute 'raster' of the  'Roi' is a dictionary with keys named
# with the labels of the spectral windows loaded
>>> print(iris_raster.raster.keys())
dict_keys(['Si IV 1394', '2814', 'Mg II k 2796'])

#Let's see the keys in the iris_raster.raster['Mg II k 2796'] dictionary
>>> print(iris_raster.raster['Mg II k 2796'].keys())
dict_keys(['data', 'wl', 'date_in_filename', 'iris_obs_code', 'raster_info',
'DATE_OBS', 'DATE_END', 'TDET', 'TDESCT', 'TWAVE', 'TWMIN', 'TWMAX', 'SPCSCL',
'SPXSCL', 'SPYSCL', 'EXPTIME', 'STEPT_AV', 'POS_X', 'POS_Y',
'date_time_acq', 'date_time_acq_ok', 'number_ext', 'binxy', 'binwl',
'extent_arcsec_arcsec', 'extent_px_px', 'extent_px_arcsec', 'extent_time_px',
'extent_time_arcsec', 'extent_opt', 'extent_opt_coords', 'list_extent',
'list_extent_coords', 'extent_display', 'extent_display_coords',
'clip_ima', 'lim_yplot', 'delay', 'poi', '_raster__count_poi',
'_raster__move_count_poi', '_raster__count_coords', '_raster__z_pos',
'_raster__z_pos_ori', '_raster__z_pos_ext', '_raster__dim_data',
'_raster__xlim1', '_raster__ylim1', '_raster__xlim2', '_raster__ylim2',
'_raster__xlim3', '_raster__ylim3', 'arr_y_ax1', 'arr_x_ax1', 'arr_x_ax3'])


The capital-case keys are directly taken from the original headers. The small-case keys are derived from the headers or created for the code to be used during the visualization of the RoI. The more important attributes or keys are data and wl. Note that they can be accessed as an attribute or as a key:

>>> print(iris_raster.raster['Mg II k 2796'].wl.shape)
(530,)
>>> print(iris_raster.raster['Mg II k 2796']['wl'].shape)
(530,)


The dimensions of the data attribute in a Roi are “[Y,X,spectral_sampling]”, where the spatial dimensions Y and X are the same as in the SJI file, and the “spectral_sampling” corresponds to the sampling in the spectral positions (given in Angstroms), and contained in the variable iris_raster.raster[window_label].wl.

# Let's see the size and dimensions of the data stored in each key of
# the output dictionary.
>>> for i in iris_raster.raster.keys():
>>>     print("Size and dimensions of data stored in the "
>>>           "iris_raster.raster[\'{}\']: {}".format(i, iris_raster.raster[i].data.shape))
Size and dimensions of data stored in the iris_raster.raster['Si IV 1394']: (779, 320, 411)
Size and dimensions of data stored in the iris_raster.raster['2814']: (779, 320, 146)
Size and dimensions of data stored in the iris_raster.raster['Mg II k 2796']: (779, 320, 530)


## 2.6. Saving your data and selected points#

Note

For the right functioning of this section and the following ones, the user must have added PoI to the spectral windows of the RoI and SoI by pressing a/A. The name of the files given through the keyword filename in this section is at user’s discretion, but it has to be the same during the execution of the code. Note that the extension ‘.jbl.gz’ is automatically added to the filename if this does not have this extension.

The module ei has a method or function devoted to save either a RoI, or a SoI or only the PoI in a simple, easy way. This method is called save and uses saveall (see Section 1.1) to write these objects into a file:

# Saving the 'iris_raster' object to a file
>>> ei.save(iris_raster)
Saving variable(s) ...
raster2save
... in roi_iris_l2_20160114_230409_3630008076_raster_t000_r00000.jbl.gz

# Saving the 'iris_sji' object to a file
>>> ei.save(iris_sji)
Saving ...
sji2save
... in soi_iris_l2_20160114_230409_3630008076_SJI_2796_t000.jbl.gz


By default, ei.save sets the filename by preceding the original IRIS Level 2 data file with the tags “roi_” or “soi_”, and changes the “.fits” extension to the “.jbl.gz” extension. If the destination file already exists, the code asks to verify whether the file should be actually overwritten or not. To avoid this safety question the user can set the keyword force equals True:

# Saving the 'iris_sji' object to an existing file
>>> ei.save(iris_sji)
"File soi_iris_l2_20160114_230409_3630008076_SJI_2796_t000.jbl.gz exists. Do you want to overwrite it?  Y/[n]"

>>> ei.save(iris_sji, force = True)
Saving ...
sji2save
... in soi_iris_l2_20160114_230409_3630008076_SJI_2796_t000.jbl.gz


If we want to save our data in another directory than the current one or/and with a different name, we can use the keyword output_dir:

# Saving the 'iris_raster' object to a file called 'roi_noaa_12480_IRIS.myext' in '/scratch/asainz/'
>>> ei.save(iris_raster, filename = 'roi_noaa_12480_IRIS.myext', out_dir='/scratch/asainz/')
Saving variable(s) ...
raster2save
... in /scratch/asainz/roi_noaa_12480_IRIS.myext.jbl.gz


Note that the codes automatically adds the extension .jbl.gz to the filename if this does not have that extension.

We may be interested in saving only the saved PoI in a RoI or a SoI and discard the rest of the data, since they can be read again from the original IRIS Level 2 file. To this aim, we can use the keyword only_poi. That will change the original data from the attribute data of RoI or the SoI by an array of 1x1x1 dimension with value 0. Thus, the RoI and the SoI conserve their structure but save space in disk. Let’s see an example:

# Saving the 'iris_raster' object to a file in '/scratch/asainz/'
>>> ei.save(iris_raster, only_poi = True)
Saving variable(s) ...
raster2save
... in poi_roi_iris_l2_20160114_230409_3630008076_raster_t000_r00000.jbl.gz

# Saving only the PoI of 'iris_raster' object to a file named 'mypoints_noaa12480_IRIS.myext'
>>> my_poi_filename = 'mypoints_12480_IRIS.myext'
>>> ei.save(iris_raster, filename = my_poi_filename, only_poi = True)
Saving variable(s) ...
raster2save
... in mypoints_12480_IRIS.myext.jbl.gz


To load a RoI, a SoI, or a only-PoI object we use the same method that we used for the IRIS Level 2 data. The code verifies that the input file is a gzip compressed file, and for the tags “roi_”, “soi_”, or “poi_”. Therefore, we strongly suggest to the user to name the files including these flags. If for any reason the user do not want to use these tags in the filename, these can be passed with the keywords roi_file and soi_file. Let us load the files saved during the previous section:

>>> myIRISdata = ei.load('roi_iris_l2_20160114_230409_3630008076_raster_t000_r00000.jbl.gz')
>>> print(type(myIRISdata))
<class 'extract_irisL2data.raster'>
>>> print(myIRISdata.windows)
['C II 1336', 'Si IV 1394', '2832', 'Mg II k 2796']
# Let's check the size of the data of the 'C II 1336' spectral window
>>> print(myIRISdata.raster['C II 1336'].data.shape)
(779, 320, 374)

>>> my_poi_filename = 'mypoints_12480_IRIS.myext.jbl.gz' # Note the extension .jbl.gz
>>> myIRISdata = ei.load(my_poi_filename, roi_file = True)
# Let's check the size of the data of the 'C II 1336' spectral window
>>> print(myIRISdata.raster['C II 1336'].data.shape)
(1, 1, 1)
# Let's check how many PoI have the 'C II 1336' spectral window
>>> print(len(myIRISdata.raster['C II 1336'].poi))


### 2.6.2. Making figures from your selected data and points#

Once the data are restored, we can easily work again with then using their corresponding quick_look. For the only-PoI data, this method will do nothing, since it has no data (actually it is 1-point array). In this case, we can use the method ei.show_poi. This method can be use with RoI and SoI having PoI or with only-PoI data.

# Showing the 3-panel figure corresponding to the 2nd PoI of the spectral window 'Si IV 1394'
# of the 'RoI' called myIRISdata restored from the file 'mypoints_noaa12480_IRIS.myext.jbl.gz'
>>> ei.show_poi(myIRISdata.raster['Si IV 1394'].poi[1])


ei.show_poi accepts the following keywords:

# Showing the XY-map panel corresponding to the 1st PoI of the spectral window '2832'
# of the 'RoI' called myIRISdata restored from the file 'mypoints_noaa12480_IRIS.myext.jbl.gz'
>>> ei.show_poi(myIRISdata.raster['2832'].poi[0], show_XY_map=True)

# Show the 'PoI' #2 of the spectral window 'SJI_2796' of the 'SoI' called 'iris_sji'
# without the line overplotted on the slit position
>>> ei.show_poi(iris_sji.SJI['SJI_2796'].poi[1], show_SJI_slit=False)