Isaac, on behalf of the SIE, will be giving a talk at the "Día de Nuestra Ciencia", titled: "SIE for Science: an update".
Abstract: The SIE (Servicios Informáticos Específicos) of the Research Area aims to provide IAC researchers with all the specific software tools they need to carry out their activity, along with all the information, guidance and support required for their most efficient and productive use. During the last two years, the SIE has increased in size with the incorporation of three permanent research software specialists, allowing to improve its scope and productivity. In this talk, we'll briefly summarize what the SIE does for the scientific community at the IAC, what services it provides, and some success stories. In addition, the new CAT web application, a key project for the IAC and the SIE, will be briefly showcased. Regarding supercomputing, details about previous and future hardware updates will be given, with an special emphasis on the usability and efficiency recommendations for the available resources at the IAC.
We all look forward to seeing you at the DNC!
We are pleased to announce that the installation of CernIE, the software distribution service utilizing the CernVM File System, has been successfully completed, and the system is functioning optimally.
As previously highlighted, the SIE supports an extensive list of software packages. The transition from the traditional method of copying software packages to local disks has been a strategic move, addressing the challenges of maintaining synchronization across the IAC, particularly for laptops.
The CernIE approach has successfully resolved these issues, offering several advantages, including reduced disk space consumption for software in the "/opt" location and seamless synchronization of software versions across all IAC systems, including laptops. While there was an initial concern regarding potential hardware failure and the need for clients to connect to a software server, this risk has been mitigated through the implementation of a redundant server configuration.
After a month-long trial period involving more than ten machines, we are delighted to report that CernIE has performed flawlessly, with no issues encountered. This successful deployment reinforces our confidence in the system's robustness and reliability. For further details and comprehensive documentation, please refer to our CernIE documentation page.
If you'd like to have CernIE implemented on your laptop (Ubuntu 22.04 only), and thus have exactly the same astronomical software packages as installed on the "burros", please let us know.
Every now and then, some user asks for the installation of a specific software package in their laptops. In most cases, those packages may be useful to other users too, so, whenever possible, we install them IAC-wide, so that they are available on any Ubuntu 22.04 desktop or server, as well as on any (Linux) laptop that has been "CernIE-d". The most recent installations are:
(All descriptions taken from Wikipedia)
Note though that the free version of both Obsidian and PyCharm requires that you use them for non-commercial purposes. To clarify the usage rules in an academic/research environment, Obsidian's FAQ say the following "If your research involves producing a commercial product – for example if your research contributes to revenue-generating commercial activities – you need a commercial license for your usage. If your research doesn't contribute to a commercial product, or if your research has not yet led to commercial activities, you don't need a license." A similar statement applies to PyCharm.
If you like to be on the state-of-the-art regarding computing, the
SIE has you covered! We have made available Python 3.12.3 to all
the burros and laptops having CernIE. To use it, simply call
module load python/3.12
. We have installed a typical set of modules
for astronomy and data processing (numpy, scipy, matplotlib...) but
if you find something important missing, let us know.
If you are still using Python 3.10, do not worry, as
module load python
will still default to that version,
and its end-of-life is scheduled for 2026.
We have made a series of improvements in our statistics webpage, where you can find a lot of information that we gather about the use of software and hardware managed by the SIE, from the different licenses to the efficiency of the jobs running in the burros.
Regarding HTcondor, we now include real-time summary plots. In the upper panel, the total amount of available CPU cores (more than 1.5k!) is divided by those used by the owner (or slurm) in yellow and those available to HTcondor (i.e., idle cores) in red. The current amount of CPUs used by jobs running in HTcondor is shown in blue. In summary, if there is no red, most of the available computing time is being used by users of HTcondor. In the lower panel, each color represent the CPU usage for different users. If you want to know what your fellow researchers use HTcondor for, you can read the usage reports.
For burros that have Slurm for managing the queue, we have now detailed
statistics of their current usage and the efficiencies of the
last run
jobs here. Also, we provide a
tool
for generating reports of the usage of the machines per month/year.
This also takes into account the efficiency of the jobs being run (an example figure is shown below), and
can be a useful tool for, e.g., users wanting to improve the performance of
their jobs; or PIs who want to justify the funds used for buying a machine.
Lastly, we have improved the module usage statistics to make it easier to navigate through the historical data.