Workflow 534, Stage 1

Priority	50
Processors	1
Wall seconds	80000
RSS bytes	2097152000 (2000 MiB)
Max distance for inputs	0.0
Enabled input RSEs	CERN_PDUNE_EOS, DUNE_CA_SFU, DUNE_CERN_EOS, DUNE_ES_PIC, DUNE_FR_CCIN2P3_DISK, DUNE_IN_TIFR, DUNE_IT_INFN_CNAF, DUNE_UK_GLASGOW, DUNE_UK_LANCASTER_CEPH, DUNE_UK_MANCHESTER_CEPH, DUNE_US_BNL_SDCC, DUNE_US_FNAL_DISK_STAGE, FNAL_DCACHE, FNAL_DCACHE_STAGING, FNAL_DCACHE_TEST, MONTECARLO, NIKHEF, PRAGUE, QMUL, RAL-PP, RAL_ECHO, SURFSARA, T3_US_NERSC
Enabled output RSEs	CERN_PDUNE_EOS, DUNE_CA_SFU, DUNE_CERN_EOS, DUNE_ES_PIC, DUNE_FR_CCIN2P3_DISK, DUNE_IN_TIFR, DUNE_IT_INFN_CNAF, DUNE_UK_GLASGOW, DUNE_UK_LANCASTER_CEPH, DUNE_UK_MANCHESTER_CEPH, DUNE_US_BNL_SDCC, DUNE_US_FNAL_DISK_STAGE, FNAL_DCACHE, FNAL_DCACHE_STAGING, FNAL_DCACHE_TEST, NIKHEF, PRAGUE, QMUL, RAL-PP, RAL_ECHO, SURFSARA, T3_US_NERSC
Enabled sites	BR_CBPF, CA_SFU, CA_Victoria, CERN, CH_UNIBE-LHEP, CZ_FZU, ES_CIEMAT, ES_PIC, FR_CCIN2P3, IN_TIFR, IT_CNAF, NL_NIKHEF, NL_SURFsara, RU_JINR, UK_Bristol, UK_Brunel, UK_Durham, UK_Edinburgh, UK_Imperial, UK_Lancaster, UK_Liverpool, UK_Oxford, UK_QMUL, UK_RAL-PPD, UK_RAL-Tier1, UK_Sheffield, US_BNL, US_Caltech, US_Colorado, US_FNAL-FermiGrid, US_FNAL-T1, US_Michigan, US_MIT, US_Nebraska, US_NotreDame, US_PuertoRico, US_SU-ITS, US_Swan, US_UChicago, US_UConn-HPC, US_UCSD, US_Wisconsin, US_WSU
Scope	testpro
Events for this stage

Output patterns

	Destination	Pattern	Lifetime	For next stage	RSE expression
1	Rucio testpro:test-01-w534s1p1	sha1sum-*.txt	604800	False	DUNE_US_FNAL_DISK_STAGE

File states

Total files	Finding	Unallocated	Allocated	Outputting	Processed	Not found	Failed
456	0	0	0	0	439	0	17

Job states

Total	Submitted	Started	Processing	Outputting	Finished	Notused	Aborted	Stalled	Jobscript error	Outputting failed	None processed
1374	0	0	0	0	502	0	263	25	0	584	0

RSEs used

Name	Inputs	Outputs
DUNE_US_FNAL_DISK_STAGE	1300	458
DUNE_CA_SFU	0	26
DUNE_US_BNL_SDCC	0	18

Stats of processed input files as CSV or JSON, and of uploaded output files as CSV or JSON (up to 10000 files included)

File reset events, by site

Site	Allocated	Outputting
US_FNAL-FermiGrid	219	492
US_FNAL-T1	42	91

Jobscript

#!/bin/bash
#
# Example jobscript that makes sha1sum checksums of all the files
# referred to by the MQL expression give on the justin command line.
#
# Submit with something like this:
#
# ./justin simple-workflow \
#  --mql "rucio-dataset protodune-sp:np04_raw_run_number_5769" \
#  --scope test --output-pattern 'sha1sum-*.txt:test-01' \
#  --jobscript simple.jobscript 
#
# Then monitor with dashboard or ./justin show-jobs --workflow-id ID
# where ID is the value printed by the first command
#

# Get an unprocessed file from this stage
did_pfn_rse=`$JUSTIN_PATH/justin-get-file`
did=`echo $did_pfn_rse | cut -f1 -d' '`
pfn=`echo $did_pfn_rse | cut -f2 -d' '`
rse=`echo $did_pfn_rse | cut -f3 -d' '`

# These files are found by the wrapper job using the --output-pattern
# form given on the justin command line (see above).
xrdcp --nopbar --debug 1 $pfn tmp
sha1sum tmp >sha1sum-$(date -u +%Y%m%dT%H%M%SZ).txt

# We say we processed whatever we were given
echo "$did" > justin-processed-dids.txt

# Log what we did
echo "Processed $did at $pfn for Workflow $JUSTIN_WORKFLOW_ID, Stage $JUSTIN_STAGE_ID"

exit 0