Please note that all the SIEpedia's articles address specific issues or questions raised by IAC users, so they do not attempt to be rigorous or exhaustive, and may or may not be useful or applicable in different or more general contexts.

Software Installation at the IAC

Here we describe how astronomical software is installed at the IAC and distributed to all software servers. You will see that the process is a little more convoluted than an installation in a single-user machine or laptop.

We use sextractor as a practical example.

Download and compilation

All software packages are download and compiled in a specific machine, avellano, where /usr/pkg and /usr/local are mounted locally (i.e. they are physical directories in the machine's hard disk), not on a remote server.

• Download the package into a specific directory, and untar it
  cd /export/invsoft1/sextractor
wget -cNS ftp://ftp.iap.fr/pub/from_users/bertin/sextractor/sextractor-2.5.0.tar.gz
tar -zxvf sextractor-2.5.0.tar.gz
• Configure and compile. Here the important thing to notice is the --prefix flag, which specifies the directory into which the software is installed (see note below).
  cd sextractor-2.5.0
./configure --prefix=/usr/pkg/sextractor/sextractor-2.5.0
make
make install
  This latter command will copy all relevant files from the compile directory to the installation directory (the one specified in the --prefix flag).

Softlinks in /usr/local

For an executable to be launched by just typing its name, it must be inside one of the directories included in the PATH variable. On the same line, for a dynamic library (shared object) to be found, it must be inside one of the directories included in the LD_LIBRARY_PATH variable.

So, in order to be able to launch sextractor by just typing the command "sex" (the name of the executable), in principle we must add the directory /usr/pkg/sextractor/sextractor-2.5.0/bin to the PATH variable. However, it's clear that by doing so for all executables in dozens of different packages and directories would lead to a humongous, unmanageable PATH variable, perhaps even exceeding the maximum length allowed by the OS.

A much more practical solution is to collect all the executables in one directory, in our case /usr/local/bin, not by physically copying or moving them, but rather by creating a symlink (aka softlink), that is a "pointer" to the actual file (google symlink or softlink for details). The same recipe applies for man files, libraries, include files, etc.

• Create softlinks:
  cd /usr/local/bin
ln -sf /usr/pkg/sextractor/sextractor-2.5.0/bin/sex     (the -f flag allows overwriting an existing link)
  cd /usr/local/man/man1
ln -sf /usr/pkg/sextractor/sextractor-2.5.0/man/man1/sex.1
cd ../manx
ln -sf /usr/pkg/sextractor/sextractor-2.5.0/man/manx/sex.x
  

For other packages, as for instance cfitsio, we would create the appropriate softlinks in /usr/local/lib and /usr/local/include. Other packages may also require links in /usr/local/share, /usr/local/info, etc.

Test

With the steps above, installation of sextractor is completed in the installation machine. Typically we do a few tests. Some packages come with a set of scripts to test them; for sextractor, we simply run it on some test images and check that it works as expected.

For other packages, we might find such problems as that they need some new environment variable to be defined, or discover incompatibilities with other libraries or applications, or realize that the help widget does not open, etc. These problems are then analyzed and (hopefully) resolved. Other, generally more subtle or obscure, problems are detected by end users and may be more complicated to fix (or are genuine software bugs).

Distribution to the test server

The software package is then copied over to the test server (rubens), which "feeds" a couple of PCs where the package is tested again.

• Copy installation by means of a dedicated script:
  ${HOME}/mirrors/avellano/avellano_2_rubens_pkg sextractor/sextractor-2.5.0
  This script copies the software to the appropriate directory in the test server; in this case, the directory /usr/pkg/sextractor/sextractor-2.5.0/ is copied over to /net/rubens/export/soft/linux/pkg/sextractor/sextractor-2.5.0/
• Create softlinks in test server:
  cd /net/rubens/export/soft/linux/local/bin
ln -sf /usr/pkg/sextractor/sextractor-2.5.0/bin/sex     (the -f flag allows overwriting an existing link)
  cd /net/rubens/export/soft/linux/local/man/man1
ln -sf /usr/pkg/sextractor/sextractor-2.5.0/man/man1/sex.1
cd ../manx
ln -sf /usr/pkg/sextractor/sextractor-2.5.0/man/manx/sex.x
• Execute some further tests from one of the machines which mount the software from rubens, and make sure that the software work correctly.

Distribution to production servers

Using a set of dedicated scripts, the software is distributed to all the production servers in the IAC Headquarters, the Astronomy Department at the ULL, the CCA and the Observatories. This ensures that exactly the same packages and same versions are available in each of the IAC sites.

Whenever a single package is distributed, also the entire /usr/local/ tree is distributed too. This may cause some problems when a number of new packages are distributed, as new symlinks are created for executables and libraries which haven't be copied over to the server yet. Thus, these new symlinks will be broken, and the yet-to-be-distributed updated packages won't work. However, this disruption is only momentary and ceases as soon as the affected packages are copied over to the server.

32 and 64 bits

• In the last months, many new 64-bit PCs have been installed. As they are fully compatible with 32-bit software, so far they mount the same software are their 32-bit cousins (except for a few packages such as IDL, which automatically detects the "bitness" of the CPU and launches the corresponding binaries).
• In the future, and depending on their availability, specific 64-bit binaries will be installed, or code will be compiled in 64-bit.