4. Mini Compose

A minimal Flask-based API that launches, monitors, and controls multi-container applications defined in a docker-compose.yml. It can run the services with Docker, Singularity, or Apptainer (from a single Python process) and exposes a small HTTP interface for starting, stopping, and reloading the stack. In an Apptainer container environment, PortShift enables the deployment of multiple container groups on the same host.

4.1. Key Features

• Compose-aware launcher – Parses docker-compose.yml, resolves depends_on, builds images when necessary, and starts containers in a correct order.
• Multi-runtime support – Switch between docker | singularity | apptainer (default = apptainer) at runtime via the /set_apptainer endpoint.
• Health-check loop – Executes healthcheck.test commands from compose and sets container status to HEALTHY / UNHEALTHY.
• Environment injection – POST /container_settings/ with JSON to pass arbitrary environment variables (including bind-mount settings) to every launched container.
• Graceful shutdown – /stop/<id>/ stops all running containers, /stopServer terminates the Flask server itself.
• Minimal UI – Root path (/) serves a tiny HTML page with Start, Stop, Reload, and Shutdown buttons—handy for manual testing.
• PortShift (Optional) - When launching servers that use the same TCP port within Apptainer, port collisions occur, making simultaneous operation impossible. By integrating with the PortShift framework (Please refer to the latter part of this page), however, port numbers can be automatically offset to avoid conflicts, allowing multiple services to run concurrently. These offsets are assigned per container group, and both server and client processes are automatically given their corresponding port offsets.

4.2. Prerequisites

Install Python deps:

pip install flask pyyaml pytimeparse asyncio

4.3. Project Layout

your-app/
 ├── app.py                # mini-compose script
 ├── docker-compose.yml    # services definition – **required**
 └── working_dir/          # Apptainer(Singularity) image cache / build context

app.py can be copied from $MINICOMPOSE_LIB directory, which will be available after invoking module load mini-compose

4.4. Overview of Usage

Mini-Compose involves three main stages:

1. Preparing the container images in a private environment where Docker / Docker-Compose are available
2. Transferring the container images to TSUBAME 4.0
3. Running the images on the TSUBAME 4.0 cluster

4.4.1. Create the Apptainer container images in a private Docker environment

1. Prepare a docker-compose.yml file in your private environment.
2. Perform thorough testing with Docker-Compose.
3. Switch Mini-Compose to Docker mode and run additional tests locally. Please switch runtime = 'docker' (Please disable runtime = 'singularity' and runtime = 'apptainer') in app.py
4. Enable image export and run Mini-Compose once more in Docker mode. Please switch is_save = True in app.py. Container images for each service will be generated under the working_dir directory.

4.4.2. Move the images to TSUBAME 4.0

1. Compress the your-app directory you created in the private environment (e.g., with tar).
2. Transfer the archive to TSUBAME 4.0.
3. Decompress it on TSUBAME 4.0.

4.4.3. Execute on TSUBAME 4.0

Every subsequent step is performed on TSUBAME 4.0 cluster.

1. Prepare a script that assigns IP addresses and apply any minor image fixes if necessary. Please switch runtime = 'apptainer' in app.py.
2. Create a job script that launches Mini-Compose on TSUBAME 4.0.

4.5. Running the Server

Choose an IP address of the host that containers should use to reach each other. The script appends this IP with every service name into a generated container_hosts file and bind-mounts it at /etc/hosts inside each container.

export HOST_IP=192.168.1.100   # ← adjust

Start the API:

python app.py "$HOST_IP"

Flask listens on 0.0.0.0:5000 by default.

4.6. HTTP Endpoints

Method & Path	Description
GET /	Serves a simple HTML control panel
GET /reload/	Re-reads docker-compose.yml and rebuilds the dependency graph
GET /set_apptainer/	Switch backend to Apptainer (same effect for Singularity/Docker if you edit the code)
POST /container_settings/	JSON body of key-value pairs → exported as --env to every container
GET /start/<id>/	Launches all services once (the <id> value is ignored except for the response text)
GET /stop/<id>/	Sends docker …
GET /status/<id>/	Dummy endpoint that just tells whether <id> is in the jobs dict
GET /stopServer	Terminates the Flask process (SIGINT)

4.7. PortShift : System Overview

PortShift lets you run many copies of the same network-centric application on a single machine without editing their source code or hard-coding unique port numbers. It consists of two lightweight, cooperating components:

Component	Role	Runs as	Key Responsibilities
Port Offset Server	Central authority that hands out port-offset windows (100 ports each) on demand.	Stand-alone daemon on one TCP port (default 25000).	• Tracks { APP_GRP → offset } in memory. • Guarantees that every group receives a disjoint window. • Re-issues the same offset to repeat callers from the same group.
LD_PRELOAD Port-Offset Library	Transparent shim that adds the assigned offset to every bind()/connect() made by the process.	Injected into target process via LD_PRELOAD.	• Queries the Offset Server at startup using environment variable APP_GRP. • Offsets IPv4 & IPv6 ports on both server and client sides. • Respects EXCEPT_PORTS to leave privileged or shared ports untouched. • (Optional) Rewrites file paths in open() so that each group also gets its own log/config directory.

4.7.1. Why Port Offsetting?

Traditional approaches (Docker networks, separate VMs, hand-edited configs) carry non-trivial overhead or maintenance cost. Port offsetting offers a zero-footprint alternative that is ideal when you need to:

• Spin up dozens of integration-test stacks on CI runners.
• Allow different teams to share a jump-box while developing micro-services that talk to fixed “well-known” ports.
• Replay production traffic against multiple binary versions on the same host.

Because the offset is injected below the application layer, any language, framework, or binary—even closed-source—works unmodified.

4.7.2. High-Level Flow

1. Startup – The shim contacts the Offset Server, sending its APP_GRP.
2. Assignment – The server returns the group’s unique offset (e.g., 100).
3. Runtime – Every subsequent bind() or connect() call has the offset added transparently.
4. Re-use – Additional processes with the same APP_GRP skip allocation and immediately receive the cached value.

4.7.3. Key Properties

• Stateless clients – If the server restarts, clients simply re-query and continue.
• 100-port windows – Large enough for typical service stacks (HTTP, gRPC, DB, cache…).
• Collision-free – Offsets are multiples of 100, avoiding accidental overlap.
• Opt-out list – Critical ports (22/80/443…) can be excluded per process.
• IPv4 & IPv6 – Works uniformly across address families.

4.8. Port Offset Server

A lightweight TCP service that assigns per-application port-offsets at runtime.
Client processes (typically instrumented via an LD_PRELOAD shim) connect to this server, submit their APP_GRP name, and receive a numeric offset that is later added to every bind()/connect() port.
This enables multiple instances of identical software stacks to coexist on the same node without hard-coding unique port numbers.

4.8.1. How It Works

client (APP_GRP = "groupA") ─┐
client (APP_GRP = "groupB") ─┤──► Port-Offset-Server (port 25000)
client (APP_GRP = "groupA") ─┘          │
                                        ├── maintains {groupName → offset}
                                        └── replies with offset (100, 200 …)

- First request for a new group → the server allocates the next free 100-port window (100, 200, 300 …). - Subsequent requests for the same group receive the same offset. - Up to 100 groups can be stored in memory (tune MAX_GROUPS, g_nextInc).

4.8.2. Running

module load mini-compose
port_offset_server             # listens on 0.0.0.0:25000

4.8.3. Configuration Constants

Macro / Variable	Purpose	Default
SERVER_PORT	Listen port for the offset service	25000
MAX_GROUPS	Max group entries in memory	100
g_nextOffset	Starting offset for the first group	100
g_nextInc	Size of each port window	100

4.9. LD_PRELOAD Port-Offset Library

libpreload_port_offset.so transparently adds a per-application port offset to all bind() and connect() calls (IPv4 & IPv6) and can optionally rewrite file paths on open().

Pair it with the Port Offset Server to launch many copies of the same service stack on one host—each isolated to its own window of 100 ports—without modifying source code.

4.9.1. Features

Hooked Call	Behaviour
bind()	Adds offset to the requested port unless it is listed in EXCEPT_PORTS.
connect()	Likewise applies the offset when dialing remote services (skipped while talking to the offset server itself).

4.9.2. Quick Start

4.9.2.1. Start the offset server (one per host)

module load mini-compose
port_offset_server &

4.9.2.2. Launch your application with LD_PRELOAD

module load mini-compose
export APP_GRP=myteam          # assigns offset 100
export EXCEPT_PORTS=22,80,443  # leave SSH/HTTP unchanged
export LD_PRELOAD=$MINICOMPOSE_LIB/libpreload_port_offset.so
./my_server_binary

Every bind(…, 8080) made by my_server_binary will be translated to bind(…, 8180); a teammate starting the same binary with APP_GRP=yourteam would receive offset 200, and so on.

4.9.3. Environment Variables

Variable	Default	Description
APP_GRP	(unset)	Logical group name. Required—otherwise offset 0 is used.
OFFSET_SERVER_ADDR	127.0.0.1	Hostname/IP of the Port Offset Server.
EXCEPT_PORTS	(none)	Comma-separated list of ports that must not receive an offset.
EXTENSIONS	(none)	Comma-separated file extensions (txt,log) that trigger path rewriting in open().