Primeros pasos in IDL

3. Input/Output

3.1. Save/Restore

The most simple way to save and restore variables, system variables and IDL routines  is by using  SAVE and RESTORE procedures. Variables and routines cannot be saved in the same file.  If a set of variables are saved, they can be fully restored after reseting a session. The only dissadvantage is that  variables and routines are only saved in IDL format. The syntaxis is as follows:

SAVE, var1,var2,...,varn,filename='nameofthefile'
RESTORE,nameofthefile

There are a number of useful keywords:

VARIABLES
Set this keyword to save all variables in the current program unit. This option is the default.

ROUTINES
Set this keyword to save user defined procedures and functions in a machine independent, binary form.

COMPRESS
Set this keyword to compress your file (it does not work on IDL v.5.2). Be careful of applying this keyword twice.
 

EXAMPLES:

IDL> a=findgen(5)
IDL> b=a+10
IDL> SAVE,a,b,/VARIABLES,filename='ab.sav'
;now RESET the session to destroy a and b arrays.
IDL> RESTORE,'ab.sav'
IDL> print,a,b

> pro abab
> a=findgen(5)
> b=a+10
> end
> IDL> SAVE,/ROUTIINES,filename='abab.sav'
> ;now RESET session
IDL> restore,'abab.sav' ;note that only the procedure was saved but not the variables a and b
 

3.2. Reading/Writing

To work with text and binary files three steps are required: opening the file, reading/writing and closing.
Three reading options are allowed:

OPENR, lun,'namefile'         Open a file for reading.
OPENW, lun,'namefile'        Open a file for writing.
OPENU, lun,'namefile'         Open a file for updating, i.e., reading/writing

lun  is a logical unit number, to which the file is attached. You can make use of the 1-99 numbers for this purpose. If you prefer that IDL manage the lun you should make use of Get_Lun,according to the following syntaxis:

OPENR, lun,'namefile',/Get_Lun

The numbers reserverd for Get_Lun are 100-128.

Once the file is opened and attached to a lun, you can read or write your binary or ascii data. For reading/writing binary data:

READU,lun,var1,var2,...,varn
WRITEU,lun,var1,var2,...,varn

For reading/writing ascii data:

READF,lun,var1,var2,...,varn
PRINTF,lun,var1,var2,...,varn

Files are closed in the following way:

CLOSE,lun    where lun is the number that you have specified

Free_Lun,lun    if IDL managed the lun

EXAMPLES:

IDL> a=findgen(5)
IDL> OPENW,1,'file_a'
IDL> PRINTF,1,a
IDL> CLOSE,1

IDL> b=fltarr(5) ;create the variable to be read
IDL> OPENR,lun,'file_a',/Get_Lun
IDL> READF,lun,b
IDL> Free_Lun,lun
IDL> print,b[0:2]

IDL> mybinaryfile=FILEPATH('head.dat',SUBDIRECTORY=['examples','data'])
IDL> OPENR,a,mybinaryfile,/Get_Lun
IDL> mybinaryfile=bytarr(80,100,57)
IDL> READU,a,mybinaryfile
IDL> Free_Lun,a

IDL> galaxy=filepath('galaxy.dat',SUBDIRECTORY=['examples','data'])
IDL> openr,lun,galaxy,/get_lun
IDL> g=bytarr(256,256)
IDL> readu,lun,g
IDL> free_lun,lun
IDL> window,xsize=256,ysize=256
IDL> tvscl,g

3.2.1. Explicit file formatting

Users can read and write with an explicit file format, using READF and PRINTF commands and specifying the FORMAT keyword. The syntax of FORMAT is similar to the format specifications in Fortran codes. The most common format specifiers are:

I    specify integers
F   for floating point data
D   for double precision data
E   for scientifc notation
A   for strings
nX skip n character spaces

The syntaxis of FORMAT is as follows:

myformat='(specifier1,specifier2,...,specifiern)'

When reading or writing you just need to include after the variables or arrays:

FORMAT=myformat

EXAMPLES:

IDL> a=findgen(12)
IDL> b='This is a:'
IDL> myformat='(A10,2X,6(F5.2,1X))'
IDL> print,b,a[0:5],FORMAT=myformat
IDL> mynewformat='(A2,1X,6(I2,1X))'
IDL> print,b,a[0:5],FORMAT=mynewformat

3.2.2. Reading from the prompt

For reading from the prompt, you should define first the variable type and then use PROMPT in READ.

EXAMPLE:

IDL>  a='' ;if you wish to read a string from the prompt
IDL>  READ,a,PROMPT='a?'
IDL>  PRINT,a

3.2.3. Reading from a string

READS allows you to read free format or explicitly formatted input from a string variable. This command is particularly useful when you need to access to numerical information from a file header.

EXAMPLE:

IDL> thisisheader='10 12 2001 This is the date of my file'
IDL> day=0
IDL> month=0
IDL> year=0
IDL> todaystring=''
IDL> READS,thisisheader,day,month,year,todaystring
IDL> print,day,month,year

3.2.4. Alternative reading facilities

In astronomy there is a very popular way for reading free-format ASCII files by making use of the READCOL facility provided by the free-software IDL Astronomy Library  ( http://idlastro.gsfc.nasa.gov/homepage.html). This library is currently installed in /usr/pkg/rsi/idl_local/lib/astro. Lines of data not meeting the specified format (e.g. comments) are ignored.  Columns may be separated by commas, tabs or spaces. Its syntaxis is as follows:

READCOL, name, v1, [ v2, v3, v4, v5, ...  v25 , COMMENT=,DELIMITER= ,FORMAT = , /DEBUG ,  /SILENT , SKIPLINE = , NUMLINE = ]

name = Name of ASCII data file.

v1, v2, v3, v4, v5, ...  v25 = IDL vectors to contain columns of data. Up to 25 columns may be read.

FORMAT = scalar string containing a letter specifying an IDL type for each column of data to be read.  Allowed letters are A - string data, B - byte, D - double precision, F- floating point, I - integer, L - longword, Z - longword hexadecimal, and X - skip a column.
Columns without a specified format are assumed to be floating point.  Examples of valid values of FMT are
;
;       'A,B,I'        ;First column to read as a character string, then
;                       1 column of byte data, 1 column integer data
;       'L,L,L,L'       ;Four columns will be read as longword arrays.
;       ' '             ;All columns are floating point
;
If a FORMAT keyword string is not supplied, then all columns are assumed to be floating point.

/SILENT = Normally, READCOL will display each line that it skips over. If SILENT is set and non-zero then these messages will be suppressed.

/DEBUG = If this keyword is non-zero, then additional information is printed as READCOL attempts to read and interpret the file.

COMMENT = single character specifying comment signal.  Any line beginning with this character will be skipped.   Default is no comment lines.

DELIMITER = single character specifying delimiter used to separate columns.   Default is either a comma, tab, or a blank.

SKIPLINE = Scalar specifying number of lines to skip at the top of file before reading.   Default is to start at the first line.

NUMLINE = Scalar specifying number of lines in the file to read.  Default is to read the entire file.

EXAMPLES:

IDL> myfile=FILEPATH('ascii.txt',SUBDIRECTORY=['examples','data'])
IDL> READCOL, myfile, v1, v2, v3, v4, v5,  /SILENT
IDL> print,v1,v3

IDL> READCOL, myfile, v1, v2, v3, v4, v5, v6, v7,FORMAT='F,F,I,I,I,A,A'
IDL> print,v1,v3,v6
 

An alternative way for reading files is to make use of READ_ASCII or READ_BINARY to read an ascii or binary file. You need to call  the widget ASCII_TEMPLATE or BINARY_TEMPLATE for allowing you to choose your data among columns and rows. The syntaxis is as follows:

mytemplate=ASCII_TEMPLATE(nameoffile)
mydata=READ_ASCII(nameoffile,TEMPLATE=mytemplate)

mytemplate=BINARY_TEMPLATE(nameoffile)
mydata=READ_BINARY(nameoffile,TEMPLATE=mytemplate)

Once you have read the data, the result (mydata) is an IDL structure variable containing a number of fields (field1,field2,...,fieldn) as shown by the widget. These fields can be renamed as you wish.To see these fields you can type:

Help,mydata,/STRUCTURE

To pull the vectors out of the structure:

vector1=mydata.field1
vector2=mydata.field2
....
vectorn=mydata.fieldn

EXAMPLE:

IDL> myfile=FILEPATH('ascii.txt',SUBDIRECTORY=['examples','data'])
IDL> mytemplate=ASCII_TEMPLATE(myfile)
IDL> mydata=READ_ASCII(myfile,TEMPLATE=mytemplate)
IDL> vector2=mydata.field2
IDL> print,vector2

Note: This method is mostly used for reading column format data files. A direct way to read these files can be done in a loop. First you have to define a string array to read the header, and another array(s) of the appropiate type to read the rest of the file. Obviously, you have to edit the file in advance to check its content.

3.2.5. Working with Fortran files

Suppose you would like to play around within IDL with an array that will be generated by a Fortran code. When writing your Fortran code you should specify the option FORM='UNFORMATTED' (in the Fortran code) when opening a file for writing your array, and include the otpion F77_UNFORMATTED when opening this file in IDL.

Within the Fortran code:

       open(1,file='My_Fortran_file',status='new',FORM='UNFORMATTED')
c     now you have to write in the following way:
       write(1) My_array
       close(1)

Within IDL:

You should use the keyword f77_unformatted to open your Fortran code generated file:

openr,lun,'My_Fortran_file',/f77_unformatted,/Get_Lun

EXAMPLE:

; Let us first to write the following Fortran code:

     program myarray
     integer i,j,k,n
     dimension n(2,4,4)
     character*50 myfile
     write(*,*)'name of your file?'
     read(*,*)myfile
     do i=1,2
         do j=1,4
             do k=1,4
                  n(i,j,k)=i*j*k
             enddo
         enddo
     enddo
     open(1,file=myfile,status='new',FORM='UNFORMATTED')
     WRITE(1)n
     close(1)
     do i=1,2
         write(*,*)'These are the values of n for i =',i
         do j=1,4
               write(*,*)'  The values of n when j =',j,' are ',(n(i,j,k),k=1,4)
          enddo
     enddo
     end

;From within IDL:

> pro testfortran
> spawn,'f77 -o myarray myarray.f'  ;Compile the Fortran code
> spawn,' echo myfortranarray | myarray'   ;Provide the name of the ourput file and execute the Fortran code
> openr,lun,'myfortranarray',/f77_unformatted,/Get_Lun
> n=lonarr(2,4,4)
> readu,lun,n
> Free_Lun,lun
> myformat='(A24,x,I2,x,A3,x,4(I2,x))'
> myjword1='The values of n when j ='
> myjword2='are'
> ;print,n(1:1,3:3,*),format=myformat
> i=0
> while i le 1 do begin
>   print,'These are the values of n for i =',i+1
>   for j=0,3 do print,myjword1,j+1,myjword2, $
>                                n(i:i,j:j,*),format=myformat
>    i=i+1
> endwhile
> end ;please compile and run

3.3. Associated Variables

Associated variables map the organizational structure of a file, which is treated as an array of these repeating units. Unlike a normal variable the associated variable does not keep the data set in memory but, when referenced, it only loads the requested data set unit, and therefore the size of the data is not limited by memory. These variables do not require any read or write command. The syntaxis is as follows:

Result = ASSOC( Unit, Array_Structure [, Offset])

where Offset refers to the offset in the file to the start of the data in the file.

EXAMPLE:

IDL> myfile=filepath('abnorm.dat',sub=['examples','data'])
IDL> openr,lun,myfile,/get_lun
IDL> a = ASSOC(lun, bytarr(64,64))
IDL> b=ASSOC(lun, bytarr(64,64),4096)  ;we skip the header of each image
IDL> tvscl,a(4) ;display this data set unit
IDL> c=a(0) ;we build up a new variable on the basis of the associated variable
 

3.4. Reading and writing with popular formats

IDL supports most of the standard formats. Specific routines to query read and write files are available. Among the most popular ones we write:

QUERY_BMP,  READ_BMP,  WRITE_BMP
QUERY_JPEG,  READ_JPEG,  WRITE_JPEG
QUERY_PNG,  READ_PNG,  WRITE_PNG
QUERY_PPM,  READ_PPM,  WRITE_PPM
QUERY_SRF,  READ_SRF,  WRITE_SRF
QUERY_TIFF,  READ_TIFF,  WRITE_TIFF
QUERY_WAV,READ_WAV,WRITE_WAV
QUERY_MPEG,  READ_MPEG,  WRITE_MPEG (for video)

QUERY_ routines allow users to obtain information about files without having to read them into memory. QUERY_ routines return a status, which determines if the file is appropriate to use the corresponding READ_ routine.

IDL also provides general routines for this purpose. QUERY_IMAGE function determines whether a file is recognized as a supported image file. READ_IMAGE can read most types of image files supported by IDL, and WRITE_IMAGE can write most types of image files supported by IDL. The corresponding  graphical interface routines for reading and writing are DIALOG_READ_IMAGE and DIALOG_WRITE_IMAGE.

Note: GIF format is no longer supported in IDL v5.4. You can however work with them in IDL v5.2. If you wish to work with a GIF file within IDL v5.4 then you should convert it to another file with an IDL supported format, e.g. PNG. You can use, e.g., imagemagick orxv software.

EXAMPLES:

IDL> myfile=filepath('rose.jpg',sub=['examples','data'])
IDL> queryfile=QUERY_JPEG(myfile,myfileinfo)
IDL> help,myfileinfo,/structure  ;you get the information on myfile
IDL> print,myfileinfo.dimensions ;it will let you know as well the dimensions of myfile

IDL> READ_JPEG,myfile,dd ;load this file into the array dd
IDL> window,xsize=256,ysize=256
IDL> tvscl,dd[0,*,*]
IDL> ddnew=congrid(dd,3,255,255)
IDL> tvscl,ddnew[1,*,*]
 

3.4.1. FITS format reading/writing routines

IDL does not provide any routine for reading/writing FITS files, which are largely used by the astronomical community. The IDL Astronomy Library  (http://idlastro.gsfc.nasa.gov/homepage.html) provides however the required procedures for this purpose: WRITEFITS and READFITS. The syntaxis is:

 WRITEFITS, filename, data [, header, NaNvalue = , /APPEND]

filename = String containing the name of the file to be written.

data = Image array to be written to FITS file.

header = String array containing the header for the FITS file. If variable HEADER is not given, the program will generate a minimal FITS header.

NaNvalue - Value in the data array to be set to the IEEE NaN condition.   This is the FITS representation of undefined values

APPEND - If this keyword is set then the supplied header and data array are assumed to be an extension and are appended onto the end of an existing FITS file.

The restrictions are:

       (1) It recommended that BSCALE and BZERO not be used (or set equal to 1. and 0) with REAL*4 or REAL*8 data.

       (2) WRITEFITS will remove any group parameters from the FITS header

 result = READFITS( filename,[ Header, /NOSCALE, EXTEN_NO = , /SILENT , NaNVALUE = , STARTROW = , NUMROW = ] )

 filename = Name of the FITS file  (including extension) to be read.   If the filename has a *.gz extension, it will be treated as a gzip compressed file.  If it has a .Z extension, it will be treated as a Unix compressed file.

 result = FITS data array constructed from designated record.

 Header = String array containing the header from the FITS file. Optional

 NOSCALE - If present and non-zero, then the ouput data will not be scaled using the optional BSCALE and BZERO keywords in the  FITS header.  Default is to scale.

 SILENT - Normally, READFITS will display the size the array at the terminal.  The SILENT keyword will suppress this

 NaNVALUE - It specifies the value to translate any IEEE "not a number" values in the FITS data array.    In addition, if the data is stored as integer (BITPIX = 16 or 32), and BSCALE is present, then NaNValue gives the values to pixels assigned with the BLANK keyword.

 EXTEN_NO - scalar integer specify the FITS extension to read.

 POINT_LUN  -  Position (in bytes) in the FITS file at which to start reading.   Useful if READFITS is called by another procedure which needs to directly read a FITS extension.  Should  always be a multiple of 2880.

 STARTROW - This keyword only applies when reading a FITS extension. It specifies the row (scalar integer) of the extension table at which to begin reading. Useful when one does not want to read the entire table.

 NUMROW -  This keyword only applies when reading a FITS extension. If specifies the number of rows (scalar integer) of the extension table to read.   Useful when one does not want to read the entire table.
 

 EXAMPLES:

;Write a randomn 50 x 50 array as a FITS file creating a minimal header.
IDL> im = randomn(seed, 50, 50)        ;Create array
IDL> WRITEFITS, 'test.fits', im             ;Write to a FITS file "test"

;Read a FITS file test.fits into an IDL image array, IM and FITS, header array, H. No scaling with BSCALE and BZERO.
IDL> im2 = READFITS('test.fits', h, /NOSCALE)

;Let us work with a real image
IDL> $cp /net/caballa/scratch/vazdekis/IDL/cursosIDL/n3379.fits .
IDL> n3379= READFITS('n3379.fits', h, /NOSCALE)
IDL> plot,n3379
 
 
 

EXCERSISES:

1.- Write a procedure in which you read a 256 elements binary file called 'dirty_sine.dat' of the IDLdistribution. Plot the array from the left to the right and from the right to the left using solid lines. Using dotted lines overplot the mean, maximum and minimum values of this file. Using dashed lines plot this file ordering it from the smallest to the largest value and viceversa.

2.- Stellar populations in galaxies are known to show non-solar metallicities as well as non-solar abundance trends for elements such as Mg or Ca. Very recent modeling allows us to predict the line-strengths of a given number of absorption spectral lines for such stellar populations. The latex-style table 'Ca4227.models' shows a set of such models, which predict the strength of the Ca4227 line for different non solar Ca abundance trends for stellar populations of different ages (in Gigayears) and metallicities ([Fe/H]).
Write a procedure to read this table.
Convert the first column (Ages) to real numbers.
Create a single array with the same number of columns as the original latex table.
Using different line types plot the age versus the Ca4227 line-strength for each of the three abundance trends for models of solar metallicity ([Fe/H]=0.0).
Finally, using the same format as in the original file, write the solar metallicity models to a new file.