Submit Grid Job¶

This page is under construction (obviously)

1. Before you can submit Grid jobs you need to prepare your setup.

Finish first the steps described here: https://p55cc-redmine.utu.fi/projects/user-s-page/wiki/Prepare_For_Grid_Usage

2. Other preparation

You use the Grid resources via the ARC (Advanced Resource Connector) middleware, developed by the Nordugrid community.

There are basically two ways to submit Grid job using ARC:

1. From ANY other computer but Pleione or Titan Login frontend
2. From Pleione or Titan Login frontend. (i.e. pleione.utu.fi or titan.utu.fi)

2.1 Using the Grid from other computers¶

Using the Grid from other computers than Pleione or Titan you need to install ARC client software and configure it. Such tasks
are documented here: https://research.csc.fi/fgci-arc-middleware#1.3.2

2.2 Using the Grid from Pleione or Titan frontend¶

The rest of this guide is just short version of this guide: https://research.csc.fi/fgci-using-arc-middleware

So you might want to read the comprehensive guide before running the example.

3. Example of running a simple Grid job

To run your binary as a Grid jog you need the following files:

- compiled program (the binary)
- job description file
- batch file
- possibly input file(s)

The c-source of an example program gtest.c :

#include <stdio.h>

int main(void) {

char *line = NULL;
size_t size;

  printf("Hello UTU.\n");

  while (getline(&line, &size, stdin) != -1)
    printf("%s", line);

  return(0);

}

Compile the source:

gcc -Wall -o gtest gtest.c

The job description file gtest.xrsl :

&(executable=gtest.sh)
(jobname=g_test)
(runtimeenvironment>="ENV/FGCI")
(join="yes")
(stdout=std.out)
(cpuTime="1 hours")
(count="12")
(memory="1000")
(inputfiles=
   ("gtest" "" )
   ("gtest.txt" "" )
)
(outputfiles=
   ("gtest.tgz" "" )
)

The batch file gtest.sh :

#!/bin/sh
echo "Running gtest" 
module load OpenMPI
chmod u+x gtest
mpirun ./gtest < gtest.txt > gtest.out
tar czf gtest.tgz gtest.out
echo "Done" 
exit 0

Optional input file gtest.txt :

Hello FGCI.

Get proxy to run the job:

$ arcproxy

Enter pass phrase for private key:

Your identity: /DC=org/DC=terena/DC=tcs/C=FI/O=Turun yliopisto/CN=Timo Eronen tke@utu.fi
Proxy generation succeeded
Your proxy is valid until: 2016-12-08 19:46:02

Run the job:

$ arcsub gtest.xrsl

Job submitted with jobid: gsiftp://io-grid.fgci.csc.fi:2811/jobs/3dxMDmPc7Ypn9NOVEmGrhjGmABFKDmABFKDmXGKKDmABFKDmWJfHjm

Query status of the job:

$ arcstat gsiftp://io-grid.fgci.csc.fi:2811/jobs/3dxMDmPc7Ypn9NOVEmGrhjGmABFKDmABFKDmXGKKDmABFKDmWJfHjm

Job: gsiftp://io-grid.fgci.csc.fi:2811/jobs/3dxMDmPc7Ypn9NOVEmGrhjGmABFKDmABFKDmXGKKDmABFKDmWJfHjm
 Name: g_test
 State: Finishing

Status of 1 jobs was queried, 1 jobs returned information

Job was not yet finished but after awhile:

$ arcstat gsiftp://io-grid.fgci.csc.fi:2811/jobs/3dxMDmPc7Ypn9NOVEmGrhjGmABFKDmABFKDmXGKKDmABFKDmWJfHjm
Job: gsiftp://io-grid.fgci.csc.fi:2811/jobs/3dxMDmPc7Ypn9NOVEmGrhjGmABFKDmABFKDmXGKKDmABFKDmWJfHjm
 Name: g_test
 State: Finished
 Exit Code: 0

Status of 1 jobs was queried, 1 jobs returned information

Get the results:

$ arcget gsiftp://io-grid.fgci.csc.fi:2811/jobs/3dxMDmPc7Ypn9NOVEmGrhjGmABFKDmABFKDmXGKKDmABFKDmWJfHjm

Results stored at: 3dxMDmPc7Ypn9NOVEmGrhjGmABFKDmABFKDmXGKKDmABFKDmWJfHjm
Jobs processed: 1, successfully retrieved: 1, successfully cleaned: 1

And then:

$ ls -l 3dxMDmPc7Ypn9NOVEmGrhjGmABFKDmABFKDmXGKKDmABFKDmWJfHjm/
total 8
-rw------- 1 tke admin 148 Dec  8 09:14 gtest.tgz
-rw------- 1 tke admin  19 Dec  8 09:14 std.out

cd 3dxMDmPc7Ypn9NOVEmGrhjGmABFKDmABFKDmXGKKDmABFKDmWJfHjm/

$ tar xf gtest.tgz

$ cat std.out
Running gtest
Done

$ cat gtest.out

Hello UTU.

Hello FGCI.