Presentation Supercomputing Zulip @ IAC SIEpedia IT News

Index of all issues November 2021 June 2021 March 2021 December 2020 Summer 2018 March / April 2018 January / February 2018 November / December 2017 September / October 2017 May / June 2017 January / February 2017 November / December 2016 July / August 2016
For earlier issues, please browse the Index

SIENews

The newsletter of the SIE de Investigación y Enseñanza N. 61 - January/February 2018

New touch-based information panel in main corridor

Chances are you have already noticed the new Research Area's touch monitor on the IAC's main corridor. It displays the pictures of all Research and Graduate Studies Division staff members, as well as the latest research news, submitted preprints, and upcoming talks. In addition, you can find astronomical pictures and some presentations for outreach purposes. The software behind the panel, which runs on a modest Raspberry Pi microcomputer, was fully developed by us using touch-based web technologies. It gathers information from the IAC's SAP system, the IAC Preprints and Publications archives, and from the IACTalks database. Plans for the future includes adding data for all the IAC personnel, and the possibility of uploading PDF presentations which can then be used for guided visits. We'd like to remark that the panel is still in beta phase, and that we are open to comments and suggestions.

SIE's training program

In the last few weeks we have given two workshops and a seminar (the last jointly with SIC). The first 4-day workshop, titled "Python for Astronomy", and given by our own Jorge Pérez Prieto, was addressed to support astronomers, telescope technicians and IACTEC staff. It was followed a few weeks later by another worskhop with the same title, this time open to all IAC researchers, with about 30 participants. The survey we made after the workshops shows high satisfaction ratings, with about half of the participants expressing interest in advanced topics.

A few days ago the seminar "Supercomputing at IAC" was given by Antonio Dorta (SIE) and Ubay Dorta (SIC), in which all Supercomputing resources available to IAC's researchers were presented: features, main usage, policy of use, etc., with special focus on the new machines: the new "Burros" added last September, the Severo Ochoa Supercomputer last November and the LaPalma3 Supercomputer a few weeks ago.
Our supercomputing facilities are of interest not only to our researchers, but also to anyone working at IAC (engineers, technical staff, etc.), who might take advantage of the them. The webcast is available at IACtalks, and the slides are also available. General information about Supercomputing at IAC is available at the SIE Supercomputing webpage

We are now preparing a new edition of our "Dale alas a tus programas" (give wings to your programs) workshop series, scheduled for late April. Besides teaching how to use HTCondor, as in past workshops, it will also tackle such new topics as submitting jobs with SLURM (the cluster management and job scheduling installed in the Severo Ochoa server, LaPalma3 and TeideHPC), and basic code parallelization techniques.
If you are interested in any other supercomputing topics, or in some programming languages or astronomical software packages, and would like them to be included in future seminars or workshops, please do let us know.

Supercomputing Statistics for 2017

HTCondor has experienced a noticeable increase in the number of CPU hours in 2017, when it consumed 1,766,302 hours, 16% more than in the previous year.
As for TeideHPC, the interruption of executions for more than one month, due to high load and the additional restrictions that were imposed, clearly affected our usage, so that the total time of 3,138,633 hours was about 5% lower than in 2016, consuming only 69% of the total time available on TeideHPC (we usually range between 70% and 80%).
For the LaPalma Supercomputer we only have statistics data for the first semester, where a total of 363,337 hours were used: as you know, it was shut down and disassembled in the second semester to make room for the new version of the Supercomputer, LaPalma3, which has now been running for a few weeks.
Finally, we welcome a new machine, the Severo Ochoa Supercomputer, which has executed 21,137 hours since it was switched on in late November 2017.

Adding up all these numbers, the total amount of CPU hours used by IAC researchers in 2017 was 5,289,409 hours.

New/updated software on burros (Fedora 26)

All of our high performance PCs (aka "burros") run on Fedora 26. We have installed new tools as well as updated existing software packages:
  • PGI C++ and Fortran compilers: updated to version 18.1, which fixes some incompatibilities between older versions and Fedora 26.
  • Intel compilers: updated to version 2018-update1 (this release is incompatible with older Fedora releases).
  • ESO pipelines: several of them have been installed in denso: efosc, fors, giraf, kmos, muse, sinfo, uves, vimos, visir, xshoo. Should you need a pipeline not included in this list, please let us know.
  • ALFA: Automated Line Fitting Algorithm, especially useful for big datacubes.
  • ndcube: "SunPy-affiliated package designed for handling n-dimensional datacubes described by a WCS (World Coordinate System) translation" - only on Python3.
  • IDL v8.7: See the IDL 8.7 Release Notes to see new routines and features, fixed issues, etc. If using IDL remotely on another computer, type idl87p or idl85p if the "regular" IDL session hangs (which is due to issues with the graphics drivers on remote connections - the commands above are workarounds).

Local installation of Python packages

With the ever increasing number of IAC Python users, from time to time we receive requests to install or update this or that Python module. We'll be happy to do it for you: we'll install it in all Linux desktops so that everybody can take advantage of the new/updated module. However, if you are in a hurry, you can easily do it yourself, using pip with the --user flag, which does not require being root or using sudo, for instance:
pip install --upgrade --user matplotlib     # (Python2)
pip3 install --upgrade --user matplotlib     # (Python3)

This is the easiest and safest way. The module will be installed (on Linux)into: ~/.local/lib/pythonX.X/site-packages/ (where X.X is the Python version — this directory is automatically included in the Python path), while any accompanying executables will go into ~/.local/bin/.
On a laptop, you can use the OS's package manager to install the packages you need (for instance, on Fedora: dnf install python3-scipy python3-spyder ), or use the local installation method described above for those packages not available in the OS repositories.
We strongly recommend against running pip as root or with sudo, as this on one side creates potential security problems, on the other it may interfere with the system-installed python packages. See for instance the explanation given in https://askubuntu.com/questions/802544/is-sudo-pip-install-still-a-broken-practice or the note in the Astropy installation page: "Do not install Astropy or other third-party packages using sudo unless you are fully aware of the risks."
SIE de Investigación y Enseñanza :: N. 61 - January/February 2018 - Contact: