Computing resources

We provide two HPC clusters, three data storage systems, and a tape based archive/backup system to support the OIST research community in your scientific computing activities, and to store your research data (see our page on Research storage at OIST). HPC clusters and the data storage systems are hosted inside the campus at the Onna village site, whereas the archive and backup system is located in a facility in Nago city.

HPC Clusters

We operate two high performance computing (HPC) clusters: Deigo and Saion. which are located in the main campus data center.

Deigo is the main research computing cluster and consists of 66584 compute cores (processor units) with a total of 337 TByte (tera-bytes) of memory.

Saion is a cluster for accelerator-based computing, and currently houses 29 GPU compute nodes equipped with NVIDIA P100 and V100 GPUs; 4 nodes with NVIDIA A100 GPUs; and a small number of CPU nodes for data postprocessing. It is especially well suited for image processing and deep-learning related computing.

Deigo

<img src=”/assets/images/computing-resources/deigo.png” width=”600” height=”434” alt=”A diagram of Deigo, depicting the different kinds of nodes and storage it has”./>

Deigo has two types of nodes, with CPUs from AMD and from Intel. The main AMD nodes have dual AMD Epyc 7702 CPUs with 128 cores and 512GB memory. The secondary Intel nodes have dual Xeon Gold 6230 CPUs with 40 cores and 512GB memory. The system also houses a number of nodes with fewer cores and more memory, as well as nodes for data copying and other tasks.

The working storage (called “Flash”) is a 500TB SSD Lustre system from DDN. It is a high-performance read-write temporary storage system that you can access very fast from the compute nodes. Each research unit has a non-expandable 10TB allocation.

“Deigo” is a common Okinawan flower, and is the official prefectural flower of Okinawa.

Saion

Saion has a mix of hardware types. The main GPU nodes have dual Intel CPUs, with either four NVIDIA Tesla P100 or four Tesla V100 GPUs. Each host node has 36 cores and 512G memory, and the GPUs have 16G memory each. We have another 4 nodes each with 128 core AMD CPUs, 2 TB memory; and 8 NVIDIA A100 GPUs with 80GB on-device memory each.

In addition there are three CPU nodes with dual Intel Xeon Gold 6148 CPUs with a total of 40 cores and 512GB memory for postprocessing and testing.

The compute storage is a 200TB SSD (solid state) storage system, with a non-expandable 10TB per unit allocation.

Saion was a Okinawan scholar, regent and reformer during the late 1600’s to the mid 1700’s, and an important historical figure.

Software

We have a large collection of both free and licensed software on the clusters. You are welcome to build and install your own software, but we recommended that you use the software we provide on the cluster if you can. You can read about how to use the installed software here. Very briefly:

module avail gives you a list of the available software, and module spider searches the installed software for a specific keyword:

$ module avail
$ module spider julia

You load a piece of software with module load:

$ module load julia

The page on using the module system will give you much more detail on using software on the clusters.

File systems

We provide research data storage that you can access both from your workstation or laptop, and from the HPC clusters. The data storage can be used with the HPC clusters through the following file systems.

Parallel File Systems (High performance):

The high-performance file systems are used during computations. They are very fast, but have a limited size and are not backed up.

NFS File Systems:

This storage is intended for long-term storage of data and results; or for your personal configuration files, source code and so on.

Use the /flash or /work partition for all your computing work. You have access to your unit directory on the parallel high-performance file servers. These system is much faster than your home directory, and can handle very large amounts of data.

Use /bucket for long-term storage of research data. It is high-capacity and is backed-up, with an off-site backup in Nago city. Bucket is read-only on the computing nodes, so read data from bucket, do your computing on /flash or /work, then move the results back to bucket at the end. If you need to move very large amounts of data, you can use the special datacp partition on Deigo.

Use your /home only for documents, configuration files, source code and such things. The file system is quite slow and storage space is limited so do not use it for any computation. It is also not backed up, so please use bucket to store anything important.