6. Commercial Applications

Under the license agreement, users who can use commercial applications is limited.
Users other than "1. Student/Staff ID" who belong to Science Tokyo can only use the following commercial applications.

• Gaussian/Gauss View
• AMBER (Only users affiliated with academic institutions)
• Intel oneAPI Compiler
• NVIDIA HPC SDK Compiler
• Linaro forge(ex:Arm Forge)

Info

Each application fee is required for the use of some commercial applications. For more details, please refer to Fare Overview Commercial Applications (Partially charged in TSUBAME4.0).

Commercial applications are listed below:

Software name	Description
ANSYS	Analysis Software
ANSYS/Fluent	Analysis Software
ANSYS/LS-DYNA	Analysis Software
ABAQUS	Analysis Software
ABACUS CAE	Analysis Software
Gaussian	Quantum chemistry calculation program
GaussView	Quantum chemistry calculation program Pre-Post tool
AMBER	Molecular dynamics calculation program
Materials Studio	Chemical Simulation Software
Discovery Studio	Chemical Simulation Software
Mathematica	Mathematical Processing Software
COMSOL	Analysis Software
Schrodinger	Chemical Simulation Software
MATLAB	Numerical calculation software
VASP	Quantum-mechanical molecular dynamics program
Linaro forge(ex:Arm Forge)	Debugger
Intel oneAPI Compiler	Compiler, Development tool
NVIDIA HPC SDK Compiler	Compiler, Development tool

Info

For "module" command, please refer to Switch User Environment.

6.1. ANSYS

The procedure for using GUI is shown below.

$ module load ansys
$ launcher

The procedure for using CLI is shown below.

$ module load ansys
$ mapdl

Following command is also available instead of mapdl.

（As of ANSYS24.1. It depends on the version.）

$ ansys241

Enter exit, then the program ends.

Specify input file, you can run it interactively.

Example-1
$ mapdl [options] < inputfile > outputfile

Example-2
$ mapdl [options] -i inputfile -o outputfile

To use the batch queue system, create a shell script and run it in the CLI as follows

$ qsub sample.sh

Script example：MPI parallel processing

#!/bin/bash
#$ -cwd
#$ -V
#$ -l node_f=2
#$ -l h_rt=0:10:0

module load ansys

mapdl -b -dis -np 56 < inputfile > outputfile

Script example：With GPU

#!/bin/bash
#$ -cwd
#$ -V
#$ -l node_f=1
#$ -l h_rt=0:10:0

module load ansys

mapdl -b -dis -np 28 -acc nvidia -na 4 < inputfile > outputfile

The license usage of ANSYS can be checked as below:

$ lmutil lmstat -S ansyslmd -c *****@kvm5.ini.t4.gsic.titech.ac.jp:*****@kvm6.ini.t4.gsic.titech.ac.jp:*****@ldap2.ini.t4.gsic.titech.ac.jp

Info

The port number for the license is visible after you purchase the application with Application activation (TSUBAME4).
/apps/t4/rhel9/isv/ansys_inc/t4-license
The port number is changed in mid April every year.

6.2. ANSYS/Fluent

ANSYS/Fluent is a thermal fluid analysis application. The usage procedure is shown below.

GUI startup procedure is shown below.

$ module load ansys
$ fluent

CLI startup procedure is shown below.

$ module load ansys
$ fluent -g

Entering exit ends the program.

To run interactively using the journal file, execute the command as follows

journal file is fluentbench.jou, 3D

$fluent 3d -g -i fluentbench.jou

To use the batch queue system, create a shell script and run it in the CLI as follows

$ qsub sample.sh

Sample scripts for SMP parallel, MPI parallel, and GPU are shown below.

Sample script: SMP

#!/bin/bash
#$ -cwd
#$ -l node_f=1
#$ -l h_rt=:10:

module load ansys

JOURNAL=rad_a_1.jou
OUTPUT=rad_a_1.out
VERSION=3d

fluent -mpi=intel -g ${VERSION} -scheduler_pe=${PE_HOSTFILE} -i ${JOURNAL} > ${OUTPUT} 2>&1

Sample script: MPI, CPU

#!/bin/bash
#$ -cwd
#$ -l node_f=1
#$ -l h_rt=:10:

module load ansys

JOURNAL=rad_a_1.jou
OUTPUT=rad_a_1.out
VERSION=3d
NCPUS=192

fluent -mpi=intel -g ${VERSION} -t${NCPUS} -scheduler_pe=${PE_HOSTFILE} -i ${JOURNAL} > ${OUTPUT} 2>&1

Sample script： MPI, GPU

#!/bin/bash
#$ -cwd
#$ -l node_f=1
#$ -l h_rt=:10:

module load ansys

JOURNAL=rad_a_1.jou
OUTPUT=rad_a_1.out
VERSION=3d
GPGPU=4
NCPUS=192

fluent -mpi=intel -g ${VERSION} -t${NCPUS} -gpgpu=${GPGPU} -scheduler_pe=${PE_HOSTFILE} -i ${JOURNAL} > ${OUTPUT} 2>&1

Info

Please use #$ -l {resource type}=1 (e.g. #$ -l node_f=1) since Fluent does not allow cross-resource configuration.
( -ncheck option is obsolete in v24. )

The license usage of ANSYS/Fluent can be checked as below:

$ lmutil lmstat -S ansyslmd -c *****@kvm5.ini.t4.gsic.titech.ac.jp:*****@kvm6.ini.t4.gsic.titech.ac.jp:*****@ldap2.ini.t4.gsic.titech.ac.jp

Info

The port number for the license is visible after you purchase the application with Application activation (TSUBAME4).
/apps/t4/rhel9/isv/ansys_inc/t4-license
The port number is changed in mid April every year.

6.3. ANSYS/LS-DYNA

6.3.1. ANSYS/LS-DYNA overview

LS-DYNA is a program that analyzes the large deformation behavior of structures with an explicit time history, and is a highly reliable program with a world-class installation record in these fields, demonstrating its power in crash/impact analysis, drop analysis, plastic forming analysis, and penetration/crack/fracture analysis.

6.3.2. Launching ANSYS/LS-DYNA

ANSYS/LS-DYNA is used in batch jobs. An example of a batch script is shown below.

Please read and execute the script according to the version you wish to use.

Script example：MPP single-precision version

    #!/bin/bash
    #$ -cwd
    #$ -V
    #$ -l node_h=1
    #$ -l h_rt=5:00:0


    module load ansys intel-mpi

    export dynadir=/apps/t4/rhel9/isv/ansys_inc/v241/ansys/bin/linx64
    export exe=$dynadir/lsdyna_sp_mpp.e
    export dbo=$dynadir/lsl2a_sp.e

    export NCPUS=4
    export INPUT=$base_dir/sample/airbag_deploy.k

    mpiexec -np ${NCPUS} ${exe} i=${INPUT}

    ${dbo} binout*

Script example ：MPP double-precision edition

    #!/bin/bash
    #$ -cwd
    #$ -V
    #$ -l node_h=1
    #$ -l h_rt=5:00:0

    module load ansys intel-mpi

    export dynadir=/apps/t4/rhel9/isv/ansys_inc/v241/ansys/bin/linx64
    export exe=$dynadir/lsdyna_dp_mpp.e
    export dbo=$dynadir/lsl2a_dp.e

    export NCPUS=4
    export INPUT=$base_dir/sample/airbag_deploy.k

    mpiexec -np ${NCPUS} ${exe} i=${INPUT}

    ${dbo} binout*

Scripts should be modified to suit the user's environment. In the example script above, the input file is specified as INPUT=inputfile in the shell script.

6.4. ABAQUS

The procedure for interactive use is shown below.

$ module load abaqus
$ abaqus job=inputfile [options]

To use the batch queue system, create a shell script and run it in the CLI as follows

$ qsub sample.sh

Scrip example：MPI Parallel processing

#!/bin/bash
#$ -cwd
#$ -V
#$ -l node_o=1
#$ -l h_rt=0:10:0


module load abaqus

# ABAQUS settings.
INPUT=s2a
ABAQUS_VER=2024
ABAQUS_CMD=abq${ABAQUS_VER}
SCRATCH=${TMPDIR}
NCPUS=4

${ABAQUS_CMD} interactive \
job=${INPUT} \
cpus=${NCPUS} \
scratch=${SCRATCH} \
mp_mode=mpi > ${INPUT}.`date '+%Y%m%d%H%M%S'`log 2>&1

6.5. ABAQUS CAE

The procedure for using ABAQUS CAE is shown below.

$ module load abaqus
$ abaqus cae

Click File > Exit on the menu bar to exit.

6.6. Gaussian

The interactive usage procedure is shown below.

$ module load gaussian/revision
$ g16 inputfile

Specify the revision you are using for revision; for Gaussian16 Rev.C02, it is as follows

$ module load gaussian/16C2_cpu
$ g16 inputfile

To use the batch queue system, create a shell script and run it in the CLI as follows

$ qsub sample.sh

Script example：Intra-node parallel processing

Sample scripts for calculating structural optimization and vibrational analysis (IR+Raman intensity) of Glycine.

The calculation can be performed by placing the following files, glycine.sh and glycine.gjf, in the same directory and executing the following command. After calculation, glycine.log and glycine.chk are generated.

The results of the analysis are explained in GaussView.

$ qsub glycine.sh

glycine.sh

#!/bin/bash
#$ -cwd
#$ -l node_f=1
#$ -l h_rt=0:10:0
#$ -V


module load gaussian

g16 glycine.gjf

glycine.gjf

chk=glycine.chk
cpu=0-191    <--Not necessary if the module that automatically sets the environment variables GAUSS_CDEF/GAUSS_GDEF is loaded.
mem=700GB

P opt=(calcfc,tight,rfo) freq=(raman)

glycine Test Job

 2
N                0   -2.15739574   -1.69517043   -0.01896033 H
H                0   -1.15783574   -1.72483643   -0.01896033 H
C                0   -2.84434974   -0.41935843   -0.01896033 H
C                0   -1.83982674    0.72406557   -0.01896033 H
H                0   -3.46918274   -0.34255543   -0.90878333 H
H                0   -3.46918274   -0.34255543    0.87086267 H
O                0   -0.63259574    0.49377357   -0.01896033 H
O                0   -2.22368674    1.89158057   -0.01896033 H
H                0   -2.68286796   -2.54598119   -0.01896033 H

1 2 1.0 3 1.0 9 1.0
2
3 4 1.0 5 1.0 6 1.0
4 7 1.5 8 1.5
5
6
7
8
9

6.7. GaussView

GaussView is an application that visualizes Gaussian results.

The procedure for using GaussView is shown below.

$ module load gaussian gaussview
$ gview.exe

Click File > Exit from the menu bar to exit.

Example: glycine.log

The following explanation uses as an example the analysis of a result file obtained by executing the script shown as a sample in the Gaussian section.

$ module load gaussian gaussview
$ gview.exe glycine.log

You can check the analysis results from Result.

You can see a summary of the calculation in Result>Summary, the charge information in Result>ChageDistribution..., and the results of the vibration analysis in Vibration....

In the sample, vibration analysis is performed, so you can see the vibration from StartAnimation in the Vibration dialog.

6.8. AMBER

AMBER was originally developed for molecular dynamics calculations of proteins and nucleic acids, but recently parameters for sugars have also been developed, making it an increasingly useful tool for chemical and biological research. If you wish to use AMBER in your own research, you should carefully study the manual and related papers to understand the limitations of the models and theories employed by AMBER, as well as the scope of application. Currently, AMBER does not allow unlimited copying of its source code, but it is possible to use it internally at Science Tokyo, so you can incorporate further developed methods based on it.

Please read the following as appropriate for the version you wish to use.

The following is the procedure for interactive sequential processing.

$ module load amber
$ sander [-O|A] -i mdin -o mdout -p prmtop -c inpcrd -r restrt

The following is the procedure for using parallel processing (sander.MPI) in an interactive manner.

$ module load amber
$ mpirun -np -[parallel number] -x LD_LIBRARY_PATH sander.MPI [-O|A] -i mdin -o mdout -p prmtop -c inpcrd -r restrt

The following is the procedure for using GPU sequential processing (pmemd.cuda) in interactive mode.

$ module load amber
$ pmemd.cuda [-O] -i mdin -o mdout -p prmtop -c inpcrd -r restrt

The following is the procedure for using GPU parallel processing (pmemd.cuda.MPI) in interactive.

$ module load amber
$ mpirun -np -[parallel number] -x LD_LIBRARY_PATH pmemd.cuda.MPI [-O] -i mdin -o mdout -p prmtop -c inpcrd -r restrt

The following is the procedure for using the system in the case of a batch queue system.

$ qsub parallel.sh

Script example：CPU Parallel processing

#!/bin/bash
#$ -cwd
#$ -l node_f=2
#$ -l h_rt=0:10:00
#$ -V
export NSLOTS=56

in=./mdin
out=./mdout_para
inpcrd=./inpcrd
top=./top

cat <<eof > $in
 Relaxtion of trip cage using
&cntrl                                                                       
  imin=1,maxcyc=5000,irest=0, ntx=1,
  nstlim=10, dt=0.001,
  ntc=1, ntf=1, ioutfm=1
  ntt=9, tautp=0.5,
  tempi=298.0, temp0=298.0,
  ntpr=1, ntwx=20,
  ntb=0, igb=8,
  nkija=3, gamma_ln=0.01,
  cut=999.0,rgbmax=999.0,
  idistr=0
 /
eof


module load amber

mpirun -np $NSLOTS -x LD_LIBRARY_PATH \
sander.MPI -O -i $in -c $inpcrd -p $top -o $out < /dev/null

/bin/rm -f $in restrt

Script example：GPU Parallel processing

#!/bin/bash
#$ -cwd
#$ -l node_f=2
#$ -l h_rt=0:10:0
#$ -V

export NSLOTS=56

in=./mdin
out=./mdout
inpcrd=./inpcrd
top=./top

cat <<eof > $in
FIX (active) full dynamics ( constraint dynamics: constant volume)
&cntrl
   ntx = 7,       irest = 1,
   ntpr = 100,     ntwx = 0,     ntwr = 0,
   ntf = 2,       ntc = 2,       tol = 0.000001,
   cut = 8.0,
   nstlim = 500,  dt = 0.00150,
   nscm = 250,
   ntt = 0,
   lastist = 4000000,
   lastrst = 6000000,
 /
eof


module load amber

mpirun -np $NSLOTS -x LD_LIBRARY_PATH \
pmemd.cuda.MPI -O -i $in -c $inpcrd -p $top -o $out < /dev/null

/bin/rm -f $in restrt

6.9. Materials Studio

6.9.1. License connection setting

BIOVIA > Licensing > License Administrator x.x.xx from the Windows [Start menu] with system administrator privileges.

Click [Connections] -[Set] , and open "Set License Server" dialog.

Select Redundant Server and type each host name and a port number.

Host name	kvm5.ini.t4.gsic.titech.ac.jp
Host name	kvm6.ini.t4.gsic.titech.ac.jp
Host name	ldap2.ini.t4.gsic.titech.ac.jp

Info

The port number for the license is seen after you purchase the application with Application distribution for Labs. They are described in the install manual.
The port number is changed in mid April every year.

If the Server Status displays "Connected", the setting is complete.

In order to use Materials Studio, a connection must be established to the license server on two or more hosts.

6.9.2. License Usage Status

6.9.2.1. How to check on Windows

BIOVIA > Licensing > License Administrator x.x.xx > Utilities (FLEXlm LMTOOLs) from the Windows [Start menu] .

Open [Service/License File] tab and slect [Configulation using License File] .

Make sure that MSI_LICENSE_FILE is displayed.

Open [Server Status] tab, click [Perform Status Enqurity] and you can see usage status of the license.

If you want to display only specific licenses, enter the license name that you want to display in [Individual Feature] and execute [Perform Status Enqurity].

6.9.2.2. How to check on login nodes

oExecute the following command to view the usage status.

$ lmutil lmstat -S msi -c *****@kvm5.ini.t4.gsic.titech.ac.jp,*****@kvm6.ini.t4.gsic.titech.ac.jp,*****@ldap2.ini.t4.gsic.titech.ac.jp

Info

The port number for the license is visible after you purchase the application with Application activation (TSUBAME4).
/apps/t4/rhel9/isv/materialsstudio/t4-license
The port number is changed in mid April every year.

6.9.3. Launching Materials Studio

On Windows that Materials Studio installed, go to Start menu and click BIOVIA > Materials Studio 2024 to launch it.

6.10. Discovery Studio

6.10.1. How to set up a license connection

From the Start menu, run BIOVIA > Licensing > License Administrator X.X.X as administrator.

Open [Connections] and click [Set] to open the Set License Server dialog.

Check Redundant servers, enter the host name and port number as shown below, and click OK.

Host name	kvm5.ini.t4.gsic.titech.ac.jp
Host name	kvm6.ini.t4.gsic.titech.ac.jp
Host name	ldap2.ini.t4.gsic.titech.ac.jp

Info

The port number for the license is seen after you purchase the application with Application distribution for Labs. They are described in the install manual.
The port number is changed in mid April every year.

If the Server Status shows "Connected", the setup is complete.

In order to use Discovery Studio, a connection must be established to at least two license server hosts.

6.10.2. How to check license usage

6.10.2.1. How to check on Windows

Run BIOVIA > Licensing > License Administrator X.X.X > Utilities (FLEXlm LMTOOLs) from Windows start menu.

Open [Service/License File] tab, select [Configulation using License File].

Make sure MSI_LICENSE_FILE is displayed.

Open [Server Status] tab, click [Perform Status Enqurity] then the staus of license usage is shown.

If you want to check only specific license, input the license name into [Individual Feature], then run [Perform Status Enqurity].

6.10.3. Launching Discovery Studio

On Windows that Discovery Studio is installed, click BIOVIA > Discovery Studio 2024 64-bit Client from Start menu.

6.11. Mathematica

Info

Starting with Mathematica version 14.1, the application startup command names have changed.
Please refer to the instructions according to the version you are using.
To check the version that is enabled by default and the list of available versions, execute the following command.
$ module avail mathematica
The underlined versions in the output version list are enabled by default.

6.11.1. When using Mathematica 14.0

The following is the procedure for starting with the CLI.

$ module load mathematica/14.0
$ math
Mathematica 14.0.0 Kernel for Linux x86 (64-bit)
Copyright 1988-2023 Wolfram Research, Inc.

In[1]:=

Entering Quit ends the program.

You can use it with GUI as follows.

$ module load mathematica/14.0
$ Mathematica

6.11.2. When using Mathematica 14.1 or later

The following is the procedure for starting with the CLI.

$ module load mathematica/14.1
$ math
Wolfram 14.1.0 Kernel for Linux x86 (64-bit)
Copyright 1988-2024 Wolfram Research, Inc.

In[1]:=

Entering Quit ends the program.

You can use it with GUI as follows.

$ module load Mathematica/14.1
$ WolframNB

6.12. COMSOL

COMSOLの利用手順を以下に示します。

You can launch COMSOL as follows.

$ module load comsol
$ comsol

Click File > Exit from menu bar, then the program ends.

You can check the status of COMSOL licenses as follows.

$ lmutil lmstat -S LMCOMSOL -c *****@kvm5.ini.t4.gsic.titech.ac.jp:*****@kvm6.ini.t4.gsic.titech.ac.jp:*****@ldap2.ini.t4.gsic.titech.ac.jp

Info

The port number for the license is visible after you purchase the application with Application activation (TSUBAME4).
/apps/t4/rhel9/isv/comsol/t4-license
The port number is changed in mid April every year.

6.13. Schrodinger

The procedure for using Schrodinger is shown below.

Ligprep with CLI：

$ module load schrodinger
SMILES format input file, MAE format output file
$ ligprep -ismiinputfile -omaeoutputfile

If you want to use it with GUI, launch Maestro.

$ module load schrodinger
$ maestro

Click File > Exit on menu bar will ends the program.

You can check license usage of Schrodinger as follows.

$ lmutil lmstat -S SCHROD -c *****@kvm5.ini.t4.gsic.titech.ac.jp

Info

The port number for the license is visible after you purchase the application with Application activation (TSUBAME4).
/apps/t4/rhel9/isv/schrodinger/t4-license
The port number is changed in mid April every year.

6.14. MATLAB

MATLAB is an application that enables numerical calculations such as matrix calculations and data visualization.

Examples of how to use MATLAB are shown below.

$ module load matlab
$ matlab

CLI

$ module load matlab
$ matlab -nodisplay

Enter exit when you stop using the application.

You can check MATLAB license usage status as follows.

$ lmutil lmstat -S MLM -c *****@kvm5.ini.t4.gsic.titech.ac.jp:****@kvm6.ini.t4.gsic.titech.ac.jp:*****@ldap2.ini.t4.gsic.titech.ac.jp

Info

The port number for the license is visible after you purchase the application with Application activation (TSUBAME4).
/apps/t4/rhel9/isv/matlab/t4-license
The port number is changed in mid April every year.

6.15. VASP

Info

When using VASP, users must have their own license.

Info

For VASP inquiries, the response by TSUBAME4.0 contacts is limited due to licensing terms.
Please refer to TSUBAME4.0 FAQ or refer to and post on VASP Forum.

There are two ways to use VASP with TSUBAME 4.0.

• Use pre-compiled binaries
  We've acquired VASP licenses for supercomputers and provide compiled binaries to VASP license holders.
  If you have a valid VASP6 license and want to use compiled binaries on TSUBAME4.0, please register through this application form.
  After the procedure is completed, set permissions to access binaries on TSUBAME. (It may take some time as confirmation with the license issuer is required.)
  VASP compiled binary usage is shown below.

$ module load VASP
$ vasp_std (or vasp_ncl etc...)

• Build the source code yourself
  If you build the source code yourself, no application is required. Please prepare the source code yourself.
  Also, you do not need to specify the module command. Specify environment variables according to your own environment.
  As reference information, the procedure for building VASP6.4.2 on TSUBAME4.0 is available.
  • I would like to know the procedure to build a VASP with TSUBAME4.0.

6.16. Linaro forge(ex:Arm Forge)

Run Linaro forge as follows.

$ module load forge
$ forge

Select File > Exit will end the program.