EasyGrid - Job Submission System.

James Werner

Running root applications with Grid - easyapp

If you want to run a compiled root application, follow this steps:

login in your afs account in linux8(outside firewall):
```
   ssh -l you linux8.hep.man.ac.uk
   mkdir grid 
   cd grid 
   . /afs/hep.man.ac.uk/g/bfactory/etc/hepix/bashrc         
```
Warning: dot space /afs... You want to source definitions from BaBar bashrc file in your shell.
Edit your main code and compile it. The first line of your code should be:
void Source_root(TString path,TString dataset,TString initset,TString numgrp,TString par1, ... ,TString parN){
where par1, ... , parN are parameters for your code. If you do not have parameters, use only the 4 first parameters (mandatory):
void Source_root(TString path,TString dataset,TString initset,TString numgrp){
If you are using other language, you have to read the parameters in the code following the same order. If you just want to run your code, use the same function definition but do not use the parameters in your code and type any mumbers in easyapp.
The first time you submit:
easyapp Root_release Dataset_filename Dataset_name app_binary inic fim group par1 par2 ... parN
where Dataset_filename is the name of the files inside the dataset directory, Dataset_name is the dataset name, app_binary is the name of your executable, inic is the first data file number (for example 1), fim is the last data file (for example 3) and you want group files together (for example 3 will join files 1, 2 and 3 in the job 1, 4,5, and 5 in the job 4, etc). par1 par2 ... parN are optional parameters for your code.
```
For example:

easyapp Root-04.01-02 data1 run1OnPeak main 1 20 20
easyapp Root-04.01-02 data1 run1OnPeak main 1 20 20 140 230 MC

The first case does not have any parameters.
```
After submit, to recover results and listings:
easygrid Dataset
if you have no parameters, or
easygrid Dataset-par1-par2-...-parN
if you have parameters in your code.
Warning: you will receive only results from jobs done. Results from jobs in scheduled or running state will not be available. You have to try "easygrid dataset" again until all jobs have returned results!!!
You can know if there are jobs pending typing "ls *.tok". If there are no files means all tokens have been deleted, everything was recovered, and there is nothing pending.
Results will be available in the directories Dataset-par1-par2-...-parN-seq if you submit with parameters or Datasetname-seq if you do not. seq is the sequence number of each's job first dataset.

A simple session submiting to grid:


[jamwer@bf39 grid_troot]$ easyapp Root-04.01-02 data1 run1OnPeak main 1 20 20

######################################################################
#       e a s y a p p  - Job Submission system for applications      #
#                                                                    #
#  If you need any support: James Cunha Werner  jamwer@hep.man.ac.uk #
#                           Room 7-11  Phone 0161 275 4150           #
#                           www.geocities.com/jamwer2002             #
#  Documentation:   http://www.hep.man.ac.uk/u/jamwer/               #
#                   University of Manchester                         #
######################################################################
Welcome, jamwer !


Job Submission User Interface version  lcg2.1.69
Searching previous handlers.


        W A R N I N G ! ! ! Handlers not found.

Type yes if you want to run everything again - some previous results may be lost!:
yes

Handlers not found. Submiting to GRID . Wait end of process...
Your identity: /C=UK/O=eScience/OU=Manchester/L=HEP/CN=james werner
Enter GRID pass phrase for this identity:
Creating proxy .................................. Done
Your proxy is valid until: Sun Jan 29 14:50:06 2006

Sub Tau11-1 Thu Jan 19 14:50:07 GMT 2006

Selected Virtual Organisation name (from --config-vo option): babar
Connecting to host bf31.hep.man.ac.uk, port 7772
Logging to host bf31.hep.man.ac.uk, port 9002


*********************************************************************************************
                               JOB SUBMIT OUTCOME
 The job has been successfully submitted to the Network Server.
 Use edg-job-status command to check job current status. Your job identifier (edg_jobId) is:

 - https://bf31.hep.man.ac.uk:9000/2QL_6CpqPACTvrSIGFNETw


*********************************************************************************************


               -  *  -  *  -  *  -  *  -  *  -  *  -  *  -  *  -


Sub Tau11-6 Thu Jan 19 14:50:28 GMT 2006

Selected Virtual Organisation name (from --config-vo option): babar
Connecting to host bf31.hep.man.ac.uk, port 7772
Logging to host bf31.hep.man.ac.uk, port 9002


*********************************************************************************************
                               JOB SUBMIT OUTCOME
 The job has been successfully submitted to the Network Server.
 Use edg-job-status command to check job current status. Your job identifier (edg_jobId) is:

 - https://bf31.hep.man.ac.uk:9000/SE7f-bzUyOYwa8V_pxJazg


*********************************************************************************************


               -  *  -  *  -  *  -  *  -  *  -  *  -  *  -  *  -

[jamwer@bf39 grid_troot]$ easygrid Tau11

######################################################################
#       e a s y g r i d - Job Submission system for Babar analysis   #
#                                                                    #
#  If you need any support: James Cunha Werner  jamwer@hep.man.ac.uk #
#                           Room 7-11  Phone 0161 275 4150           #
#                           www.geocities.com/jamwer2002             #
#  Documentation:   http://www.hep.man.ac.uk/u/jamwer/               #
#                   University of Manchester                         #
######################################################################
Welcome, jamwer !


Job Submission User Interface version  lcg2.1.69
Searching pre selected skimdata.
Searching previous handlers.
Checking if jobs finished.

### SubFile Tau11-1.tok  Handle -> https://bf31.hep.man.ac.uk:9000/2QL_6CpqPACTvrSIGFNETw
    Current Status: Scheduled

### SubFile Tau11-6.tok  Handle -> https://bf31.hep.man.ac.uk:9000/SE7f-bzUyOYwa8V_pxJazg
    Current Status: Scheduled
0 jobs aborted ! Try again using easyresub DATASET FILE_NUMBER

[jamwer@bf39 grid_troot]$ easygrid Tau11

######################################################################
#       e a s y g r i d - Job Submission system for Babar analysis   #
#                                                                    #
#  If you need any support: James Cunha Werner  jamwer@hep.man.ac.uk #
#                           Room 7-11  Phone 0161 275 4150           #
#                           www.geocities.com/jamwer2002             #
#  Documentation:   http://www.hep.man.ac.uk/u/jamwer/               #
#                   University of Manchester                         #
######################################################################
Welcome, jamwer !


Job Submission User Interface version  lcg2.1.69
Searching pre selected skimdata.
Searching previous handlers.
Checking if jobs finished.

### SubFile Tau11-1.tok  Handle -> https://bf31.hep.man.ac.uk:9000/2QL_6CpqPACTvrSIGFNETw
    Current Status: Done

Retrieving files from host: bf31.hep.man.ac.uk ( for https://bf31.hep.man.ac.uk:9000/2QL_6CpqPACTvrSIGFNETw )

*********************************************************************************
                        JOB GET OUTPUT OUTCOME

 Output sandbox files for the job:
 - https://bf31.hep.man.ac.uk:9000/2QL_6CpqPACTvrSIGFNETw
 have been successfully retrieved and stored in the directory:
 /home/jamwer/grid_troot/jamwer_2QL_6CpqPACTvrSIGFNETw

*********************************************************************************

    Exit code: 0

### SubFile Tau11-6.tok  Handle -> https://bf31.hep.man.ac.uk:9000/SE7f-bzUyOYwa8V_pxJazg
    Current Status: Done

Retrieving files from host: bf31.hep.man.ac.uk ( for https://bf31.hep.man.ac.uk:9000/SE7f-bzUyOYwa8V_pxJazg )

*********************************************************************************
                        JOB GET OUTPUT OUTCOME

 Output sandbox files for the job:
 - https://bf31.hep.man.ac.uk:9000/SE7f-bzUyOYwa8V_pxJazg
 have been successfully retrieved and stored in the directory:
 /home/jamwer/grid_troot/jamwer_SE7f-bzUyOYwa8V_pxJazg

*********************************************************************************

    Exit code: 0
0 jobs aborted ! Try again using easyresub DATASET FILE_NUMBER
All jobs done. Available results recovered in your folder.

WARNING: Next time easygrid will submit everything AGAIN!!!

Top

Last modified:

Feedback to: jamwer@hep.man.ac.uk