Applications¶

Info

The command line examples on this page use the following notation
[login]$ : login node
[rNnN]$ : compute node
[login/rNnN]$ : login node or compute node
[yourPC]$ : environment from which the login node is connected

I want to process a large number of tasks consecutively (I want to automatically submit a large number of jobs)¶

Due to the overhead associated with initialization at the start of each job, TSUBAME’s job scheduler and billing system are not suitable for operations involving the submission of a large number of small jobs.
Furthermore, there have been several instances where a large number of jobs were submitted due to configuration errors in automated processes or job scripts, resulting in an overload of the billing system or the consumption of a significant amount of points.
(On TSUBAME4, jobs with execution times of less than 5 minutes are billed as 5 minute jobs; therefore, submitting a large number of error jobs that complete in tens of seconds will result in unnecessary charges. See Charging rules of TSUBAME4.0 supercomputer, Institute of Science Tokyo.

If you wish to process a large number of small tasks, we recommend processing them sequentially within a single large job. TSUBAME provides sample scripts for sequentially processing multiple tasks within a job; please make use of them as appropriate.

Introduction page (Japanese only) : 簡易型ジョブ内ワークキュー
Sample script : dirworkqueue.sh

Info

This script is part of an experimental service and is not covered by support.

Additionally, if you are automating job submission using your own environment, please take the following precautions to prevent infinite loops or massive simultaneous submissions:

Consider whether you can consolidate multiple jobs into a single job
Before production deployment, conduct tests with the number of iterations limited to a few times
Check periodically during automated processing to ensure no unexpected behavior occurs

X application (GUI) doesn't work¶

In this page, X applicatoin indicates the application that is installed in TSUBAME4.0 and can work on the X environment, that is GUI application.
Please check the troubleshooting below.

X server application is installed and active on the client PC¶

Windows
There are a lot of X server applications for Windows.
Please confirm that one of them is installed and on active.
For information on SSH clients available for Windows, please see here.

Mac
Please confirm XQuartz is installed and configured.
https://support.apple.com/en-us/100724

Linux
Please confirm both of the X11 server application and its libraries are installed.

The X transfer option in the terminal is enabled.¶

A Terminal on Windows (Except for Cygwin)
The setting method differs depending on your terminal and X server application.
Please check the manual of each application.

Linux/Mac/Windows(Cygwin) Please confirm the ssh command contains the option -y and -c (these are the options for X transfer)

[yourPC]$ ssh <account_name>@login.t4.gsic.titech.ac.jp -i <key> -YC

Example: in case of tsubame_user as account_name and ~/.ssh/t4-key as key, then

[yourPC]$ ssh tsubame_user@login.t4.gsic.titech.ac.jp -i ~/.ssh/t4-key -YC

Please refer to the output of the following command for ssh option.

[yourPC]$ man ssh

Error reproduces in another terminal/Xserver¶

There are various free terminal softwares/X server applications for Windows.
Please check the same error occurs another terminal/X server.
It may be due to compatibility between terminal and X server.
It may be compatible with commercial application.
If it does not reproduce in other applications, there is a possibility that it is an application specific problem.
In that case we can not respond even if you contact us, please understand.

In addition, depending on the X application, command options may be required.
Please check the manual of X application you want to use.

Some GL applications that do not work with normal X forwarding/VNC connection may work with VirtualGL, so please give it a try if needed.

For the detail of VirtualGL, please refer to User's Guide.

Operation check¶

If it is in an interactive node, the standard terminal emulator of X Window System is started with the following command. Please confirm whether to start.

[login]$ xterm

If xterm works but the X application you want to use does not work, please try "3. Error reproduces in another terminal/Xserver"

Example of failure

xterm: Xt error: Can't open display: 
xterm: DISPLAY is not set

Please check 1 and 2 if the error occurs.

Application use¶

Do not execute programs that occupy the CPU at login nodes. Please use compute nodes for Full-scale use including visualization.

Please refer to the FAQ below for information on using the GUI application at the compute node. Reference: FAQ "How to transfer X with qrsh"

When using node_f, X transfer can be performed with the ssh -Y command.

Please inform us of the following when you inquire

Operating System you use (Example: Windows11,Debian13,macOS 26)
( We only accept Operating System within the officially supported range. )
Terminal environment that the error occurs (Cygwin, PuTTY/VcXsrv, Rlogin/Xming)
Version
For Windows, the both versions of the terminal and X server application.
see the manuals for applications for checking versions.
Please inform the version of SSH in case of using Linux/Mac with the command below.
```
[yourPC]$ ssh -V
```
Please send us the contents you tried so far, or if you get an error, please describe the error.

I would like to use an application not provided by TSUBAME 4.0¶

Installing applications not provided by TSUBAME 4.0¶

Please check if it applies to the following items.
If applicable, you can install it freely at your own risk.
Please check the installation manual and the license agreement of the application.

Works with OS installed in TSUBAME4.0. Software requiring Windows or Mac OS won't work.
Not requiring administrator privilege (root) to install it.
Possible to install it to your own home directory or group disk. (It is not allowed to install it to any specified nodes' local disk.)
With a valid license.
Not requiring the change to the settings for the kernel, libraries or the system itself.
If only under these conditions, you can install it and use it on your own responsibility.
No need for CII support.

0.Notes
As described above, CII will not help anything about the applications brought by users, as we do not know anything about it.

In case of problems, users themselves must distinguish whether it comes from the application itself or the general issue of TSUBAME, and ask application vendors for application-specific problems.

The versions of libraries and drivers may be changed at the time of the regular maintenance of TSUBAME etc. In that case, you might need to reconfigure the application you had used. Please be aware of the risk of losing compatibility in the future.

1.Installation directory
You can install in the following two places.
Please choose suited according to your operation.
If you need to share within the TSUBAME group such as members of the laboratory, please use the high speed storage area.
Even if you change the permissions by chmod or some commands in the home directory, you can not share that.

Home directory
Group disk

Reference: TSUBAME 4.0 User's Guide "Strage system"

2.Installation method
Please install application according to the manual or README or community forum of the application to be installed.
Depending on the application, it is necessary to compile the library or module or something from the source file by yourself.
Below are some notes and a typical installation.

Application management software such as zypper can not be used, you have to compile from the source file basically.
If CUDA is used, it must be compiled on the compute node. (since the login node does not have a GPU).
The TSUBAME4.0 OS is not a Debian distribution, so the apt command is not available.
dnf commands cannot be used because they require root privileges. If you want to use them, please use Use Containers.

Example 1) executing configure script, generating makefile, then make, make test and make install:

[login/rNnN]$ ./configure --prefix=$HOME/install  
[login/rNnN]$ make && make test  
[login/rNnN]$ make install

Example 2) creating a directory for buid, cmake and make install:

[login/rNnN]$ mkdir build && cd build  
[login/rNnN]$ cmake .. -DCMAKE_INSTALL_PREFIX=$HOME/install  
[login/rNnN]$ make install

Example 3) example 3)installin with install script:

[login/rNnN]$ ./install.sh

There is a problem with the operation of the distribution application¶

Since most of the troubles arising with regard to the distribution application are caused by the environment, we do not support them individually. Please solve yourself as you accepted at the time of application.

Even if you contact us, we can not respond.

The range of support by T4 Helpdesk about the program error such as segmentation fault¶

General
Please check the following related FAQ first
About common errors in Linux
"Disk quta exceeded" error is output
Error handling for each commercial application

1.For commercial applications
Supported without ABAQUS/ABAQUS CAE. Please inform the following information through inquiry.

Application name
　Eg)Abaqus/Explicit
Error message
　Eg)buffer overflow detected
JOB_ID
　Eg)181938
Host name where the error occurred
　Eg)r6n5
The situation in detail
　Eg)The error occured when logged into r6n5 interactively with qrsh and executed the following command. Details are as follows:

[rNnN]$ module purge
[rNnN]$ module load abaqus intel
[rNnN]$ abq2017 interactive job=TEST input=Job1 cpus=6 scratch=$T4TMPDIR mp_mode=mpi
 #Error#
　Run package
*** buffer overflow detected ***: 
/pathto/package terminated
======= Backtrace: =========
/lib64/libc.so.6(+0x721af)[0x2aaab0c001af]
...
(The rest is ommited)

ABAQUS is an academic license, so there is no technical support.
It is necessary to register on the SIMULIA documentation site and resolve it yourself.
For information on the documentation site, please contact us from "Contact Us".

2.For the application compiled yourself

Not supported. Please resolve it yourself. See "I would like to use an application not provided by TSUBAME 4.0".

Error information is output when compiling with the traceback option.

How to use VNC from MobaXterm¶

If you have a GUI application on TSUBAME and X forwarding fails to draw or the performance is insufficient, TurboVNC may improve the situation.
Since MobaXterm has a built-in VNC client function, it is relatively easy to use.
Please refer to User's Guide. for how to start a VNC server on a compute node and how to connect it from MobaXterm.

The basic configuration of Module file¶

The basic configuration of Module file is described below.

Module file listed as [Application name]/[Version].
If you do not specify a version, it loads the preset default version

If multiple versions exist, the default version is loaded.

$ module avail cuda
---------------------
cuda/12.0.0  cuda/12.1.0  cuda/12.3.2
$ module list
Currently Loaded Modulefiles:
 1) cuda/12.3.2

For applications with dependencies such as MPI, they can be used by loading them in advance.

$ module load openmpi/5.0.2-intel
Loading openmpi/5.0.2-intel
  Loading requirement: intel/2024.0.2 cuda/12.3.2

I want to change the directory where cache files, user files, etc. used by the application are stored¶

If you want to change the directory where cache files, user files, etc. used by the application are stored, please consider the following procedure.

Info

If the used capacity of the group disk exceeds the set capacity, access will be disabled.
Please be careful not to exceed the set capacity of the group disk, as this is done on a monthly basis.
In addition, the procedures for changing the settings and any problems that may occur as a result of changing the settings are not covered by this inquiry.

using the settings provided by the application
Some applications allow the storage location of cache files, user files, etc. to be changed.
The method of changing these settings differs depending on the application, such as environment variables, etc. Please refer to the man or other documentation.
replace the relevant directory with a symbolic link.
If the setting does not exist on the application side, there is a way to move to the group disk using a symbolic link.
Note that this procedure does not guarantee normal execution by all applications.
Here is an example of moving the ${HOME}/.cache directory to /gs/bs/tga-xxxxxxx/tsubametarou.
```
[login]$ cd ${HOME}
[login]$ mv -i .cache  /gs/bs/tga-xxxxx/tsubametarou     # Move the current .cache directory to the group disk  
[login]$ ln -nfs /gs/bs/tga-xxxxx/tsubametarou/.cache    # Symbolic link to ${HOME}/.cache  
```