In the second half of 2024 HTCondor has seen a decreased usage with respect to the first six months of the same year, with a total number of 646500 CPU hours (compared to 970500 for 2024, 1st half). This value corresponds to an average of 144 CPU cores devoted to HTCondor jobs.
About 10.0 million CPU hours have been used in the LaPalma supercomputer in the second semester of 2024, a ~ 34% decrease compared to 15.2 million CPU hours for the first semester. The total for 2024 is 25.2 million, a small decline with respect to the 27.3 million CPUh for year 2023.
As for the Burros, we have collected usage statistics for those that work with the Slurm queue management system. Data show that 85 users have used up about 2.7 million CPU hours, with a great heterogeneity of jobs, most with duration of one or two days, and some requiring a large amount of memory, up to 3 TB.
In October 2024 20 additional Overleaf "Standard" licenses were added to our pool, bringing the total number to 80. However, Overleaf licenses go like hotcakes, and we have already run out of them, while the waiting list is steadily growing. In the same month of October, we achieved an all-time record of 506 Active Projects (number of unique projects edited at least once) and 5076 Editing Sessions (an Editing Session starts when a user opens a project and lasts until there's 30 minutes of inactivity; activity here includes moving the cursor, typing and clicking). You can check the Overleaf usage statistics
While we hope nobody will suffer from the type of catastrophic situation depicted
in this famous XKCD comic,
it's a fact of life that sometimes different python packages cannot coexist peacefully
in the same Python environment. At the IAC we (SIE) are offering two main Python
environments: for Python 3.10 and for Python 3.12. The former is basically "frozen",
meaning no updates are currently done, while the latter is maintained, and new
packages can be added according to demand. However, there are Python packages
that require their own specific habitat, being incompatible with either environment,
so we provide a few specialized environments too. Check them with command:
module avail.
"Personal" Python virtual environments are easy to set up and use, and doing so does not require any administrative privileges. Among the many tutorials that teach you how to create and use them, we can (unabashedly) recommend the Slides and the Tutorial from a Smack Talk given a couple of years ago here at the IAC.
As most of you have already heard, because of heavy budget restrictions this year we could not renew the maintenance of IDL and Mathematica licenses. However, do not panic: in the short term little or nothing changes, in the sense that existing floating and node-locked licenses will still work as usual.
The downsides (in the medium/long run) are:
As the IDL usage statistics graph shows, use of IDL has decreased a lot in the last few years, and we expect that this trend carries on, as more and more users migrate towards free, open-source solutions (e.g. Python). As for Mathematica, it's actively used only by a handful of researchers.
Installation of the ESO Instruments pipelines has traditionally been a little pain in the neck, in particular on Ubuntu where they had to be compiled from scratch, and only worked with some old Java versions. On Mac, the situation was not as dire, as they could be installed via MacPorts.
Fortunately, the installation procedure on Linux has been improved and
simplified a lot, and they now work using Apptainer containers. Installation
takes just a few minutes and, what's even better, we (SIE) are providing
ready-to-run containers for those pipelines most widely used at the IAC
(additional pipelines can be added with minimal efforts). If interested, do the following:
ls -al /opt/SIE/ESOpipelines to see the available pipelines:
module load apptainer
apptainer exec /opt/SIE/ESOpipelines/molecfit_4.4.2.sif esoreflex -l see the available workflows
apptainer exec /opt/SIE/ESOpipelines/molecfit_4.4.2.sif esoreflex molecfit run the molecfit workflow
(just replace molecfit_4.4.2.sif by the name of the .sif
container you want to use).
For further details see the ESO Pipeline Containers webpage. Especially relevant are sections 3 to 5 about the tutorials and the binding of local directories.
As you can see in our Burros webpage, there are now two new public burros available: doner and durum. Despite not having impressive hardware specs, each one has two RTX2080Ti GPU cards, which might be useful to some of you and which brings the number of publicly available GPU cards in the IAC burros from two to six.
Both of them are configured as "public" burros, with HTCondor and Slurm. Both of them should work in a similar way to the other public burros (and similar to deimos for GPU usage). Check the Burros documentation if you are not sure how to use them.
[In case you are curious, these two new burros have been recycled from two of the servers running the (now retired) VideoWall in the Burbuja room (the VideoWall never took off and its usage was minimal, so we considered that they would offer a better service as "burros") together with the four GPUs of the "drogon" project burro, which had severe stability issues for some time and it is also being retired].
