SIE Supercomputing documentation

Welcome!

Here you can find in a single place all the available computational resources for IAC researchers and how to use them.

For questions, comments or suggestions regarding this documentation please reach us at sinfin@iac.es. For technical support contact us sending an email to cau@iac.es.

All the work carried out by researchers makes use of computational resources. In many cases, a single laptop or small desktop computer may be just enough to be productive. However, it is common to find that more computational power is required. For example, if your scripts start to take hours rather than a few minutes... that is a clear symptom that your computer may limit your productivity.

The IAC provides different ways to offload computations from the researchers' computers to specialised servers. In addition, you can modify your code to make it run faster using parallelisation. For example, analysing more than one image or object at a time.

We have prepared a simple workflow that you can follow to determine which systems (in green) and parallelisation options (red) are more suitable for your case. We can help you to:

  • choose the best option depending on your problem,

  • compile your applications and transfer your data to these environments,

  • prepare your submit files, etc.

digraph test { fontname="Helvetica,Arial,sans-serif" node [fontname="Helvetica,Arial,sans-serif"] edge [fontname="Helvetica,Arial,sans-serif"] node [shape=box, fontname="Arial"]; { node [shape=circle, fontname="Arial"]; "Par Yes"; "Par No"; "Div Yes"; "Div No"; } { node [style="filled,dashed", fillcolor=lightcoral]; "cupy"; "numba"; "MPI"; "openMP"; } { node [style="filled,dashed", fillcolor=lightgreen]; "HTCondor"; "Burros"; "Large burros"; "Burros w/GPUs"; "LaPalma"; "TeideHPC"; } "My code" [shape=circle, label="My code", width=0.25] "Already parallel?" [shape=diamond] "Par Yes" [group=paratype, label="Yes"] "Par No" [group=paratype, label="No"] "Type of workload" [shape=diamond, group=worktype] "Large arrays" [group=worktype] "Independent inputs" [group=worktype] // "Tightly coupled" [group=worktype] "Divisible?" [shape=diamond] "Div Yes" [label="Yes"] "Div No" [label="No"] "GPUs" [group=paratype] "Shared memory" [group=paratype] "Distributed memory" [group=paratype] "LaPalma" [href="../lapalma/lapalma.html", target="_top", style="filled,dashed"] "Burros" [href="../burros/index.html", target="_top", style="filled,dashed"] "Large burros" [href="../burros/index.html", target="_top"] "Burros w/GPUs" [href="../burros/index.html#gpus", target="_top"] "TeideHPC" [href="../teidehpc/index.html", target="_top"] "HTCondor" [href="../htcondor/index.html#htcondor", target="_top"] "MPI" [href="https://en.wikipedia.org/wiki/Message_Passing_Interface", target="_top"] "openMP" [href="https://en.wikipedia.org/wiki/OpenMP", target="_top"] "cupy" [href="https://cupy.dev/", target="_top"] "numba" [href="https://numba.pydata.org/", target="_top"] "My code" -> "Already parallel?" "Already parallel?" -> "Par Yes" "Already parallel?" -> "Par No" "Par No" -> "Type of workload" "Type of workload" -> "Large arrays" "Type of workload" -> "Independent inputs" // "Type of workload" -> "Tightly coupled" "Independent inputs" -> "HTCondor" // "Tightly coupled" -> "Good luck!" "Large arrays" -> "Divisible?" "Divisible?" -> "Div Yes" "Divisible?" -> "Div No" "Div Yes" -> "HTCondor" "HTCondor" -> "Burros" "Div Yes" -> "But very large" "But very large" -> "MPI" "MPI" -> "LaPalma" "Div No" -> "Python" "Python" -> "numba" "Python" -> "cupy" "Div No" -> "C, FORTRAN" "C, FORTRAN" -> "openMP" "openMP" -> "Burros" "openMP" -> "Large burros" "numba" -> "Burros" "numba" -> "Large burros" "cupy" -> "Burros w/GPUs" "Par Yes" -> "Shared memory" "Par Yes" -> "Distributed memory" "Par Yes" -> "GPUs" "GPUs" -> "Burros w/GPUs" "Shared memory" -> "Small jobs" "Shared memory" -> "Large jobs" "Small jobs" -> "Burros" "Large jobs" -> "Large burros" "Distributed memory" -> "LaPalma" "LaPalma" -> "TeideHPC" }

Tip

The elements with dashed lines are clickable and will lead you to the corresponding section of the documetation or to external resources!

Note

The above workflow may be incomplete, specially regarding parallelisation options for Python. Feel free to explore or ask about other options.

Others