Presentation Supercomputing Zulip @ IAC SIEpedia IT News

Index of all issues October 2024 August 2024 May 2024 March 2024 October 2023 July 2023 May 2023 March 2023 January 2023 October 2022 May 2022 February 2022
For earlier issues, please browse the Index
For the former SIENews, please go to the SIENews index

ITNews

The newsletter of the SIE de Investigación y Enseñanza and the Servicios Informáticos N. 6 - May 2023

Status of upgrades to Ubuntu 22.04

The upgrade of burros and desktops to Ubuntu 22.04 is underway; the currently installed version, Ubuntu 18.04, has reached the end of the standard, five-year maintenance window for Long-Term Support (LTS) releases, at the end of May this year. So far, about 20 machines have been upgraded, with no major issues.

Some notable changes, with respect to Ubuntu 18.04, are the migration to Python 3.10, the installation of the latest releases of the NVIDIA and Intel compilers, and the upgrade of several other software packages (TOPCAT, astrometry.net, gnuastro, julia, HEASoft, etc.). For more details, see our dedicated page at https://vesta.ll.iac.es/SIE/wiki/pmwiki.php?n=SIEInvEns.Ubuntu2204. Further updates and changes will be announced through the #computing/astrosoft@iac channel in Zulip @ IAC

Slurm Workload Manager for a more efficient use of the "burros"

Slurm Workload Manager (formerly known as Simple Linux Utility for Resource Management (SLURM), or simply Slurm, is a free and open-source job scheduler for Linux and Unix-like kernels, used by many of the world's supercomputers and computer clusters. Many of you are probably already familiar with it if you have used deimos/diva, LaPalma, TeideHPC, MareNostrum, PizDaint, etc.

Along with the upgrades to Ubuntu 22.04, we are installing Slurm in the public "burros" (and also in the project "burros" that request it). This was considered necessary because when a workstation is shared amongst several users it is easy to oversubscribe it (e.g. to run more processes than available cores), which makes all applications to perform far worse and it is usually detrimental to all users.

We are still learning how to best use Slurm in this scenario, and thus Slurm is installed so far only in a few "burros", but we expect to extend it gradually to most other burros, since it allows a more efficient and balanced use of the Linux servers. While using Slurm is quite easy, we are aware that it involves some changes for you. To help you with the transition, we have prepared the following resources:

  • Instructions on the use of Slurm (in general and specific to the IAC burros): https://vesta.ll.iac.es/SIE/hpc/burros/burros.html
  • A specific "stream" (aka "channel") in IAC-Zulip to discuss all things related to Slurm@IAC (see #computing/slurm at https://iac-es.zulipchat.com)
  • A talk (date pending) about Slurm (in general but also specific to the Slurm systems at IAC: deimos/diva, LaPalma, burros), where we'll discuss basic usage, and also common mistakes, tips, etc.

Deletion of > 1 yr old files in public burros

And speaking of burros, we would like to remind that scratch disks on burros are meant to store those data (input, output, associated programs, etc.) needed to run the desired computational jobs, not as temporary (or permanent) generic storage areas. Data no longer necessary should be removed. Note that there is no backup whatsoever of data on scratch disks.

To manage the scratch disks in a more efficient way, and make sure that sufficient disk space is available to those who need it, a periodic script that deletes all those files older than 1 year has been running - since November 2019 - twice a month on public burros (deimos, diva, dejar and denso).

You can check yourself what files are older than 1 year. Cd to your directory, and run the following command:
find . -type f -ctime +365 -exec ls -alh {} \;
(ctime stands for "change time", which includes changes in the content and in the metadata)
while the total size (in GB) occupied by those files can be obtained with:
find . -type f -ctime +365 -printf '%s\n' | awk '{total=total+$1}END{printf "%12.3f \n", total/1073741824}'

If you have files approaching the 1 year time limit, and want to keep them, please make a copy in some other place. Also check if there are files you are interested in, but belong to some collaborators of yours who have left the IAC (usually the owner has an _ext or -ext suffix, or has a numeric ID if their account was closed).

New IDL 8.9 release

IDL version 8.9 has been recently released. To check the new features and updated brought about by this release, please check What's New in IDL 8.9. If you have a Mac or Linux laptop with IDL installed, and are using a local (node-locked) license, you may wish to upgrade IDL. The License Administrator should be able to detect the existing license, and migrate it automatically. You can download the appropriate installer from the IAC cloud (ask us for the link). Take into account though that IDL 8.9 cannot be installed on MacOS releases older than 11.3 (Big Sur).

IDL 8.9 will also be installed in the Ubuntu 22.04 burros and desktops, though this might be a bit more complicated since IDL 8.9 uses a completely new licensing system which must be installed in the server, configured, started, while avoiding any possible conflicts with the existing, Flexera-based system (which IDL did away with because it was "plagued by massive difficulties from unresolvable failures and limitations".). We'll also need to reconfigure the machines to use the new licencing system.

As for IDL running natively on new M1/M2 Macs, a recent post in the IDL forum states that "IDL 8.9 does not yet natively support Apple Silicon chips but it will in the near-future. No ETA on that ARM64 version of IDL yet, though. Still hoping for a 2023 release."

IACNet DNS Firewall Protection Service

The IAC has implemented this protection feature using a service provided by RedIRIS, called DNS Firewall. Through this service, all queries that pass through our DNS servers will pass through the enabled filters, which will guarantee safer access and avoid possible threats.

Most legitimate and non-legitimate communication begins with a DNS query that obtains the IP address of an online resource or service. The primary purpose of a DNS firewall is to monitor DNS traffic to provide protection for:

  • Block user access to malicious websites.
  • Prevent phishing attacks.
  • Block communications from within our network that use the DNS as a means of communication with the outside, such as malware or botnets.
  • Detect infected machines in our infrastructure.

When a client sends a DNS request to your DNS Firewall server, it performs a series of tasks to find out how dangerous the request is and what actions to take with it, including allowing, blocking, discarding, reporting...

This service is also available to the rest of the IACNet user institutions that wish to do so, they simply have to redirect their DNS queries to the IAC DNS servers.

(Original text from https://www.iac.es/en/outreach/news/iacnet-dns-firewall-protection-service)

SIE de Investigación y Enseñanza :: N. 6 - May 2023 - Contact: