Presentation Supercomputing Zulip @ IAC SIEpedia IT News

Index of all issues November 2021 June 2021 March 2021 December 2020 Summer 2018 March / April 2018 January / February 2018 November / December 2017 September / October 2017 May / June 2017 January / February 2017 November / December 2016 July / August 2016
For earlier issues, please browse the Index

SIENews

The newsletter of the SIE de Investigación y Enseñanza N. 53 - September/October 2015
SIENews is 10 years old!

With this issue n.53, SIENews celebrates its 10th birthday. The first issue was sent out back on September 1st 2005, and since then we have been publishing, on average, a new issue every two months or so. If you have joined the IAC recently, and enjoy our SIENews, please note that we maintain a full, easy-to-browse archive of all past issues.

HTCondor updated to v8.4.0, with many important improvements

Last Tuesday (Sept 15) we updated HTCondor to version 8.4.0 released a week ago. Among the many new features, those related to the Submit Description Files are probably the most noticeable and interesting changes for HTCondor users in the last few years.

The new job submission capabilities are much more powerful and flexible than before:

  • now it is possible to specify directly which files you want to process, even if they have arbitrary names (for instance, now you don't need to rename or relocate your input files, you can easily specify them using globbing like *.dat, input*, etc.: HTCondor will find which files match your description and automatically generate jobs for each one);
  • new automatic variables and function macros have been added;
  • working with operating variables and macros is now much easier thanks to an intuitive syntax;
  • you can specify simple or complex lists of arguments to be expanded, slices, include external files and even execute external scripts!;
  • you can use conditional statements inside the Submit File (if, else...);
  • and test the behaviour of your submit file without actually submitting the jobs (dry-run option).

If you were using scripts (perl, bash, python, etc.) to dynamically create condor files and then submit them, please use from now on the new syntax, so as to reduce the complexity of your submission files, increase the efficiency of the job execution, and get more reusable submit files (get in touch with us if you need any help with this).

More details and examples specially adapted for this new HTCondor version will be soon posted in our SIEpedia, but you can also read about them in the HTCondor official documentation.

Finally, we would like to remind that HTCondor jobs are executed on other researchers' machines, so there are a few basic rules that we all should follow, such as using HTCondor's transfer files system rather than using network shared locations (/net/nasX, /net/scratch, etc.) in order to avoid network congestion, submitting all jobs in the same cluster whenever possible, etc.

eGroupWare migrated to a new server

Not without some considerable efforts, we have finally managed to migrate most of the eGroupWare domains from our old server, where it was at version 1.4.005, to a new server (http://casanova/egroupware/), where the latest version, 14.3, is installed. The most difficult task has been migrating the old "File Manager" application, which has been replaced by a new, totally backward-incompatible one. We were able however, with the help of some MySQL black magic, to achieve the desired result: all the documents uploaded in the older server should be present and visible in the new one, though some details may have been lost or changed, such as associated comments, original permissions, creation/modification date, etc. Another important change is the users' authentication, which is now done with Active Directory (the same system used to authenticate users on Linux and Windows). If you are a manager or a user of an active eGroupWare domain, we invite you to try out the new version, and check that nothing has been lost in the migration. Please let us know of any problems you may have, including being unable to log in. We thank Hector Rodriguez Rodriguez for helping us understand the many eGroupWare intricacies, and for doing a thorough check of the post-migration "Instrumentación" domain.

A quick note: if you are looking for documents that were in the "MyDMS" folder in the old version, go to "File Manager" and change the Path to "/" to obtain a complete list.

Supercomputing usage statistics for the 1st semester of 2015 (HTCondor, TeideHPC, LaPalma)

Statistics data on Supercomputing usage in the last semester have been published on SIE webpages. During the first 6 months in 2015, IAC's researchers have used 1.036.722,3 CPU hours on our Linux Desktops using HTCondor. This corresponds to more than 47 years of uninterruptedly computation in just 6 months (for a breakdown of HTCondor usage, see our Hall of Fame post).

During this same period, LaPalma executed a total of 1.374.966 hours, with IAC users accounting for 248.609 hours, and RES (Red Española de Supercomputación) users for the rest. This number is rather low since at this stage all our users have successfully migrated their jobs to the TeideHPC Supercomputer, a much more powerful machine (in fact, it is the second most powerful of Spain, with a similar architecture as MareNostrumIII). Our researchers have executed on TeideHPC a total of 2.139.079 hours, almost 100 years of computation in 6 months, using virtually 100% of the allocated time. Summing up, the total Supercomputing CPU time for the first semester of 2015 was 3.424.410 hours, equivalent to 156 years.

We would like to kindly remind you that, if you use any of the above Supercomputing resources in your research, a short acknowledgement text must be included in your publications.

We encourage all those new users that have recently joined the IAC, as well all others who think about using our Supercomputing facilities to get their computational results in much less time, to visit the Supercomputing section in the SIE website. We are giving active Supercomputing support to all IAC users, so do not hesitate to contact us if you need it. We have also organized several workshops on HTCondor, MPI/OpenMP, etc. (see https://research.iac.es/sieinvens/SINFIN/Main/sie_courses.php), so please contact us if you are willing to attend one of them, as we are keen on organizing new seminars or workshops on Supercomputing topics if there are enough interested people.

MATE as alternative to KDE/Plasma

Unfortunately the recent upgrade of our Linux desktop machines to Fedora 21 has been suffering from some problems, in particular affecting the KDE desktop environment, which showed a tendency to freeze or crash every so often. Not even the upgrade to Plasma5 seems to have solved these problems, which may have a variety of causes hard to identify and fix. Here at the SIE we have been trying an alternative Desktop Environment, called MATE, which is a continuation of Gnome 2. So far it has proven to be much more stable and less demanding on computer resources than KDE. MATE has a sort of "vintage" look and feel, but it is very easy to use and customize. If you wish to leave KDE and try MATE out, please let us or the CAU know as MATE must be installed on your Linux desktop first.

Removal of personal websites of those users who have left the IAC

The last years have seen a considerable increase of people with a personal website in our external server, website that not only provide important information on the scientific and technical work carried out at the IAC, but also show a more human view of our staff (personal background, general interests, etc.)
The problem arises of what to do with the personal websites of those users who have left the IAC. Keeping all of them is an additional burden for us (in terms of usage of disk space, backups, server upgrades, installation of patches, etc.), and may also entail possible security issues. For this reason we would like to implement a new policy whereby we allow the user who has just left the IAC adequate time (say two months) to migrate their website to their new workplace (or to some hosting facility), and then we'll remove the website at the IAC, leaving a redirect so that the http://research.iac.es/galeria/username/ URL will still work, but will just point to the new location.

SIE de Investigación y Enseñanza :: N. 53 - September/October 2015 - Contact: