SIENews

New "burros"

Two new high-performance machines have been recently installed, named "dejar" and "denso". Each of them has 32 physical cores (64 logical ones with hyperthreading), 384 GB of RAM, and about 20 TB of disk space. These new machines will progressively replace the old burros ("dardo", "diosa", etc.) that are very out-dated and out of warranty, and will be dedicated to parallel programs tests, massive data reduction, intensive computing jobs, etc. These new machines have been installed with the most recent version of Fedora, 26, which is 5 releases newer than the Fedora version installed on the desktop PCs (21). Due to this gap, it's possible that your programs compiled or tested under Fedora 21 do not work straightaway under Fedora 26, and may require some (hopefully minor) modifications or recompiling. This must be taken into account especially if you use HTCondor (you may wish to restrict your job to Fedora-21 machines). On the other hand, should you wish to upgrade your desktop PC to Fedora 26 (a process which usually takes a few hours), please let us know.
Usage policy will be the same as for the older "burros", that is, there will be no restrictions "a priori" and every user will be able to use those machines, just following a few basic rules:

• Before running any job on them, please check the load (use commands like uptime or htop): if it is higher than 32, wait a bit till it goes down before launching your application. Also check that the load does not exceed 32 after your program starts.
• If you are testing your parallel codes, check how many cores are being used and don't take up all the 32 cores. This machine should be used only when developing or testing your parallel programs: if you need to run a parallel aplication for hours or days on a large number of cores, there are better alternatives, such as TeideHPC or LaPalma.
• These machines have a huge disk space (about 20TB). Don't abuse it! There are no backups of your data there, so don't use it like a storage system, and move your data to other locations once your executions are done.

New powerful Supercomputing resources are going to be added to the IAC soon: in due time we'll give a talk to inform you all about them.

IAC80 Virtual Observatory archive updated

As you may know, the CAMELOT/IAC80 public image archive can be accessed through the Virtual Observatory (VO) protocols. Since 2016, we have been adding regularly new images as soon as the one-year proprietary period ends: recently, we have uploaded the data from May 2015 to September 2016. The IAC80-VO archive now contains more than 160 000 images, all of them reduced with the standard CAMELOT pipeline and appropriately astrometrized. The easiest way to retrieve such images is using Aladin's All-VO tool, selecting "IAC" among the available sources.
The IAC80-VO archive is maintained by the Support Astronomers Group (GAS) and the SIE. The GAS gives us the reduced images and we adapt the metadata and the database to comply with the VO standards using specific software tools. If you have images, spectra and catalogues and wish to make them world-wide available through the Virtual Observatory, do not hesitate to contact us to help you set them up.

New IDL licenses

Starting with version 8.6, IDL has changed its licensing system, which from now on is based on an "Activation Code" rather than on a license file. This simplifies significantly the process of installing an IDL license, and allows to migrate the license from an old PC to a new one in a very easy way (it can be done by the user herself, and no longer requires the heinous "lmhostid" string). We recommend that all users with a valid license on their laptops upgrade IDL to v8.6.1 (the latest release) and migrate to this new license system. Please get in touch with us for instructions: we just need to know the "Installation number:" and "Licensed for use by:" values shown in the "welcome message" printed on the terminal when starting IDL.

CNV and CVA

With the aim of providing better and wider support to our researchers, in 2006 we created the SIEpedia, where one can find detailed information about many software packages installed at IAC, with several Tutorials, Manuals, HOWTOs, usage tips, etc. For some time now we have seen that users from other institutions too are actively visiting our SIEpedia. In past editions of SIEnews we mentioned that our HOWTOs about HTCondor are usually the most visited. This time we wish to point out the sudden peak of visits to the HOWTOs pages for the CVN (Curriculum Vitae Normalizado), which are referenced in several presentations and talks about it. This peak coincides with the deadline of the last call for projects for the "Plan Nacional I+D+i", around June/July 2017. We'd like to remind you that we offer support on editing and updating your CVN or CVA, and help import your publications from ADS directly to CVN (yes, you don't need to add by hand all your new publications in the CVN editor, you can import them from ADS!). For further information, please visit our CVN SIEpedia page

Python profiling and display using RunSnakeRun

Has ever happened to you that your python application runs at a snail's pace and takes ages to complete? Sometimes it is possible to increase the speed of your code by doing some modifications, but it is not easy to find where you should focus your programming efforts on (maybe there are few lines which are called thousands of times, so a small optimization there will considerably speed up the whole execution, or maybe you are using slow external functions that can be replaced by faster ones, ...). To help you in this topic, there are some tools called "profilers" which gather statistics about the execution of your code, and show its most time-consuming parts, where the bottlenecks are, etc. Once those areas are identified, you can optimize them and reduce significantly the execution time. Among the several profilers and viewers for Python, we like cProfile and the viewer RunSnakeRun, both already installed on our desktop Linux PCs. They will display a very intuitive graph showing how you application behaves by means of a "square map" which provides a good overview of your executions. To use them, simply run these commands:

• Get profiling data: python -m cProfile -o profiling.dat your_code.py
• Display data: /usr/pkg/python/Python-2.7.9/bin/runsnake profiling.dat

And speaking about python, let us recommend another very useful tool that you might not know yet: Python Glue. This is a powerful library to explore relationships within and among related datasets, making complex graphs in a very simple and quick way (it also allows you to select points directly from the graphs in the same way TOPCAT does).

Mount remotely from Linux a directory in a Mac (and v.v.) with sshfs

There exist several tools to mount directories on a remote system, the most known of which are NFS (the one used to cross-mount scratch disks in our Linux network), and Samba, which also works on Windows and macOS PCs. A practical alternative for Unix-based systems, which is implemented in user-land (that is, no administrative/root access is required), is sshfs. As the name suggests, it mounts a remote file system via ssh. With this, for instance, you can run Linux tools on data stored in your Mac, or vv. run Mac applications on data in a Linux box. Here goes a couple of pretty dumb examples (with a fictional username and machine).

• Use Acroread installed in your Mac to open a file in the Linux home directory
  mkdir -p ~/homelinux ; sshfs filemon@mortadelo:/home/filemon/ ~/homelinux
  cd ~/homelinux/ ; open file.pdf
• Use the Intel compiler (only available in Linux) to compile and execute a file in your Mac (instead of transferring files between the two platforms)
  mkdir -p ~/homemac ; sshfs filemon@IPoftheMac:/Users/filemon/ ~/homemac
  cd ~/homemac/ ; icc -o simul simul.c ; ./simul
• To unmount: fusermount -u ~/homelinux (same for other mountpoints)

We cannot say whether sshsf is faster or more robust (or not) than nfs or samba, YMMV.

SIE de Investigación y Enseñanza :: N. 59 - September/October 2017 - Contact: