3. Storage system¶

In this system, you can use the Lustre file system for the high-speed and the large-scale storage area as well as home directory. And local scratch area on each compute node is also available.

3.1. Home directory¶

HOME directory is 25GiB availalbe per user.

You can check usage status with t4-user-info disk home command.

$ t4-user-info disk home
  uid name         b_size(GB) b_quota(GB)    i_files    i_quota
---------------------------------------------------------------
 2011 TESTUSER              7          25     101446    2000000

Of the 25 GB quota limit, 7 GB is used, regarding the inode limit, we can check the situation that we are using approximately 100,000 out of the 2 million quota limit.

Please note that new writing cannot be performed if the quota limit is exceeded. If you remove some files and free space is available again, you can create new files.

Even if you delete files, sometimes the quota limit status has remains. In this case, it takes a day at most until refreshing the status to recover.

3.2. Group disk¶

Group disk is used to share files among specific group members. The high-speed or the large-scale storage area are available depend on your purpose.

To purchase group disk, please refer TSUBAME Portal User''s Guide.
The characteristics of each storage area are as follows.

Storage area	PATH	Group disk capacity setting	Available sizes	Number of files that can be saved(inode)
High-speed storage area (SSD)	/gs/fs	100GB unit	MAX 3TB	2,000,000 / 1TB
Large-scale (Big) storage area (HDD)	/gs/bs	1TB unit	MAX 100TB	2,000,000 / 1TB

3.2.1. High-speed storage area¶

High-speed storage area is mounted at /gs/fs.
The high-speed storage area consists of the Lustre file system using SSD and you can be used by purchasing it as a group disk.
Compared to the Large-scale (Big) storage area, the total disk space is smaller, but file access is faster, so it is recommended for “execution environments where file access speed is important” and “frequently accessed data files”.

3.2.2. Large-scale (Big) storage area¶

Large-scale (Big) storage area is mounted at /gs/bs.
The large-scale (big) storage area consists of the Lustre file system using HDD and you can be used by purchasing it as a group disk.
Although file access is slower than in high-speed storage areas, the available storage size is large and the cost is low, so it is recommended for “data for which file access speed is not required” and “storage of files such as backup data.

3.2.3. Usage status of group disk¶

You can check usage status of specific group disk with t4-user-info disk group command.

$ t4-user-info disk group -g TESTGROUP
                                              /gs/bs                                 /gs/fs                     
  gid group_name                 size(TB) quota(TB)   file(M)  quota(M)  size(TB) quota(TB)   file(M)  quota(M) 
----------------------------------------------------------------------------------------------------------------
 xxxx TESTGROUP                    59.78       100      7.50       200      0.00         0      0.00         0

In this case, TESTGROUP purchased /gs/bs and used 60TB out of 100TB quota limit, its inode used 7.5M out ob 200M.

3.2.4. Confirmation of group disk usage¶

Group disks are purchased on a monthly basis.
If the amount of group disk purchased in the previous month is different from the current month (decrease), the actual usage may be greater than the amount of group disk purchased, depending on the usage situation.
Please use one of the following procedures to check whether the amount of group disk usage exceeds the purchased amount.

[How to check]
1. when checking on TSUBAME4.0 : Usage status of group disk
2. When checking on TSUBAME Portal : Confirmation of group disk usage

Info

Only group administrators and authorized group members can check on TSUBAME portal.

3.2.4.1. What to do when the group disk usage exceeds the reserved size¶

If the amount of group disk usage exceeds the purchased amount, the following two levels of restrictions will apply
1. prohibition of writing files that exceed the purchased amount (at the beginning of the month)
2. prohibition of access to anything on the group disk (specific timing)

In the case of condition 1, please delete files to keep the purchased amount within the limit, or change the purchased size of the group disk by referring to Increase or decrease group disk capacity to exceed the current usage amount.
In case of condition 2., file deletion is not possible. Please change the purchased size of the group disk with reference to Increase or decrease group disk capacity to exceed the current usage.

3.3. CIFS access from inside campus¶

In TSUBAME4.0, you can access group disk from the Windows / Mac terminal in the university using the CIFS protocol.

\\gshs.t4.gsic.titech.ac.jp

You need TSUBAME4.0 account. The password is the one you set in the portal.
Note that "@TSUBAME" is needed subsequent your account when you use Windows.


User	(TSUBAME4.0_ACCOUNT)@TSUBAME
Password	(TSUBAME4.0_PASSWORD)

/gs/bs, /gs/fs are displayed as t4_bs and t4_fs respectively. Access to your group disk beneath them.

TSUBAME4 t4cifs

Info

Home directory is not accessible with CIFS. If the connection fails, please also see Can not establish CIFS connection to the group disk, Unable to open TSUBAME group disk on Windows.

3.4. Local scratch area¶

Local scratch area is located in SSD on each compute node and available for workspace or temporarily area to read/write data rapidly. See Local scratch area for detail.