Jobsub ID 168204.0@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID	168204.0@justin-prod-sched02.dune.hep.ac.uk
Workflow Testing	Yes
Workflow ID	500
Stage ID	1
User name	amcnab@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	GPU	No
	RSS bytes	1073741824 (1024 MiB)
	Wall seconds limit	3600 (1 hours)
Submitted time	2025-04-01 09:40:10
Site	US_UConn-HPC
Entry	GLUEX_US_UConn-HPC_osgce
Last heartbeat	2025-04-02 01:38:42
From worker node	Hostname	cn416
	cpuinfo	AMD EPYC 7452 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	1073741824 (1024 MiB)
	Wall seconds limit	172800 (48 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Allocator name	justin-allocator-int.dune.hep.ac.uk
Started	2025-04-02 01:37:34
Input files
Jobscript	Exit code	0
	Real time	0m (43s)
	CPU time	0m (10s = 23%)
	Max RSS bytes	39129088 (37 MiB)
Outputting started	2025-04-02 01:38:17
Output files
Finished	2025-04-02 01:38:42
Saved logs	justin-logs:168204.0-justin-prod-sched02.dune.hep.ac.uk.logs.tgz
List job events Wrapper job log
Jobscript log (last 10,000 characters)

OthTpIR9o --timeout 1200' returns 99


---------------------------------------------------------------------
US_UConn-HPC QMUL davs root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/testpro/bb/7f/awt-download-2023-03-07-01.txt
'xrdcp --force --nopbar --verbose root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/testpro/bb/7f/awt-download-2023-03-07-01.txt downloaded.txt' returns 0
{
    "created_timestamp": null,
    "creator": "dunepro",
    "fid": "CKxzLi5vTqedWkVA",
    "metadata": {},
    "name": "awt-1743557856-ricNsJi3G8",
    "namespace": "testpro",
    "retired": false,
    "retired_by": null,
    "retired_timestamp": null,
    "size": 0,
    "updated_by": null,
    "updated_timestamp": null
}
metacat file declare returns 0
GFAL_CONFIG_DIR:    GFAL_PLUGIN_DIR: 
justin-rucio-upload attempt 1
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): dune-rucio.fnal.gov:443
DEBUG:urllib3.connectionpool:https://dune-rucio.fnal.gov:443 "GET /auth/x509_proxy HTTP/1.1" 401 132
DEBUG:charset_normalizer:Encoding detection: ascii is most likely the one.
DEBUG:charset_normalizer:Encoding detection: ascii is most likely the one.
DEBUG:charset_normalizer:Encoding detection: ascii is most likely the one.
--- Upload try 1/1
--- Rucio upload 1/1 fails: Cannot authenticate.
Details: Cannot authenticate to account justinreadonly with given credentials
--- Exit with 99
'justin-rucio-upload --rse QMUL --protocol davs --scope testpro --dataset awt-uploads-202513 awt-1743557856-ricNsJi3G8 --timeout 1200' returns 99


---------------------------------------------------------------------
US_UConn-HPC RAL-PP davs root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/testpro/bb/7f/awt-download-2023-03-07-01.txt
'xrdcp --force --nopbar --verbose root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/testpro/bb/7f/awt-download-2023-03-07-01.txt downloaded.txt' returns 0
{
    "created_timestamp": null,
    "creator": "dunepro",
    "fid": "sR49QguNQ5SgaDQv",
    "metadata": {},
    "name": "awt-1743557856-NP4gE9tTDG",
    "namespace": "testpro",
    "retired": false,
    "retired_by": null,
    "retired_timestamp": null,
    "size": 0,
    "updated_by": null,
    "updated_timestamp": null
}
metacat file declare returns 0
GFAL_CONFIG_DIR:    GFAL_PLUGIN_DIR: 
justin-rucio-upload attempt 1
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): dune-rucio.fnal.gov:443
DEBUG:urllib3.connectionpool:https://dune-rucio.fnal.gov:443 "GET /auth/x509_proxy HTTP/1.1" 401 132
DEBUG:charset_normalizer:Encoding detection: ascii is most likely the one.
DEBUG:charset_normalizer:Encoding detection: ascii is most likely the one.
DEBUG:charset_normalizer:Encoding detection: ascii is most likely the one.
--- Upload try 1/1
--- Rucio upload 1/1 fails: Cannot authenticate.
Details: Cannot authenticate to account justinreadonly with given credentials
--- Exit with 99
'justin-rucio-upload --rse RAL-PP --protocol davs --scope testpro --dataset awt-uploads-202513 awt-1743557856-NP4gE9tTDG --timeout 1200' returns 99


---------------------------------------------------------------------
US_UConn-HPC RAL_ECHO davs root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/testpro/bb/7f/awt-download-2023-03-07-01.txt
'xrdcp --force --nopbar --verbose root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/testpro/bb/7f/awt-download-2023-03-07-01.txt downloaded.txt' returns 0
{
    "created_timestamp": null,
    "creator": "dunepro",
    "fid": "PSqJPS1ORvi2Wyc3",
    "metadata": {},
    "name": "awt-1743557856-cdw69YIX8a",
    "namespace": "testpro",
    "retired": false,
    "retired_by": null,
    "retired_timestamp": null,
    "size": 0,
    "updated_by": null,
    "updated_timestamp": null
}
metacat file declare returns 0
GFAL_CONFIG_DIR:    GFAL_PLUGIN_DIR: 
justin-rucio-upload attempt 1
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): dune-rucio.fnal.gov:443
DEBUG:urllib3.connectionpool:https://dune-rucio.fnal.gov:443 "GET /auth/x509_proxy HTTP/1.1" 401 132
DEBUG:charset_normalizer:Encoding detection: ascii is most likely the one.
DEBUG:charset_normalizer:Encoding detection: ascii is most likely the one.
DEBUG:charset_normalizer:Encoding detection: ascii is most likely the one.
--- Upload try 1/1
--- Rucio upload 1/1 fails: Cannot authenticate.
Details: Cannot authenticate to account justinreadonly with given credentials
--- Exit with 99
'justin-rucio-upload --rse RAL_ECHO --protocol davs --scope testpro --dataset awt-uploads-202513 awt-1743557856-cdw69YIX8a --timeout 1200' returns 99


---------------------------------------------------------------------
US_UConn-HPC SURFSARA davs root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/testpro/bb/7f/awt-download-2023-03-07-01.txt
'xrdcp --force --nopbar --verbose root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/testpro/bb/7f/awt-download-2023-03-07-01.txt downloaded.txt' returns 0
{
    "created_timestamp": null,
    "creator": "dunepro",
    "fid": "NWSh4eaxREef4SbG",
    "metadata": {},
    "name": "awt-1743557856-i8rDoPnwK6",
    "namespace": "testpro",
    "retired": false,
    "retired_by": null,
    "retired_timestamp": null,
    "size": 0,
    "updated_by": null,
    "updated_timestamp": null
}
metacat file declare returns 0
GFAL_CONFIG_DIR:    GFAL_PLUGIN_DIR: 
justin-rucio-upload attempt 1
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): dune-rucio.fnal.gov:443
DEBUG:urllib3.connectionpool:https://dune-rucio.fnal.gov:443 "GET /auth/x509_proxy HTTP/1.1" 401 132
DEBUG:charset_normalizer:Encoding detection: ascii is most likely the one.
DEBUG:charset_normalizer:Encoding detection: ascii is most likely the one.
DEBUG:charset_normalizer:Encoding detection: ascii is most likely the one.
--- Upload try 1/1
--- Rucio upload 1/1 fails: Cannot authenticate.
Details: Cannot authenticate to account justinreadonly with given credentials
--- Exit with 99
'justin-rucio-upload --rse SURFSARA --protocol davs --scope testpro --dataset awt-uploads-202513 awt-1743557856-i8rDoPnwK6 --timeout 1200' returns 99


subject   : /C=UK/O=eScience/OU=Manchester/L=HEP/CN=justin-jobs-production.dune.hep.ac.uk/CN=17710010/CN=174355785421
issuer    : /C=UK/O=eScience/OU=Manchester/L=HEP/CN=justin-jobs-production.dune.hep.ac.uk/CN=17710010
identity  : /C=UK/O=eScience/OU=Manchester/L=HEP/CN=justin-jobs-production.dune.hep.ac.uk/CN=17710010
type      : RFC compliant proxy
strength  : 2048 bits
path      : /home/awt-proxy.pem
timeleft  : 167:59:20
key usage : Digital Signature, Key Encipherment, Key Agreement
=== VO dune extension information ===
VO        : dune
subject   : /C=UK/O=eScience/OU=Manchester/L=HEP/CN=justin-jobs-production.dune.hep.ac.uk
issuer    : /DC=org/DC=incommon/C=US/ST=Illinois/O=Fermi Research Alliance/CN=voms1.fnal.gov
attribute : /dune/Role=Production/Capability=NULL
attribute : /dune/Role=NULL/Capability=NULL
timeleft  : 145:49:48
uri       : voms1.fnal.gov:15042

===== Results =====

Download/upload commands:
xrdcp --force --nopbar --verbose $read_pfn downloaded.txt
echo '{"namespace":"testpro","name":"FILENAME","size":0}' >tmp.json
metacat file declare --json -f tmp.json "dune:all"
justin-rucio-upload --rse $rse_name --protocol $write_protocol --scope testpro --dataset awt-uploads-202513 --timeout 1200 FILENAME
Use the wrapper job link on the page for the job on the justIN Dashboard to find the full log file, with errors from these commands

Each line: $JUSTIN_SITE_NAME $rse_name $download_retval $upload_retval $read_pfn $write_protocol
==awt== US_UConn-HPC DUNE_CA_SFU 0 99 root://lcg-dunese1.sfu.computecanada.ca:1094//dune/RSE/testpro/bb/7f/awt-download-2023-03-07-01.txt davs
==awt== US_UConn-HPC DUNE_CERN_EOS 0 99 root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/testpro/bb/7f/awt-download-2023-03-07-01.txt davs
==awt== US_UConn-HPC DUNE_ES_PIC 0 99 root://xrootd.pic.es:1094/pnfs/pic.es/data/dune/RSE/testpro/bb/7f/awt-download-2023-03-07-01.txt davs
==awt== US_UConn-HPC DUNE_FR_CCIN2P3_DISK 0 99 root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/testpro/bb/7f/awt-download-2023-03-07-01.txt davs
==awt== US_UConn-HPC DUNE_IT_INFN_CNAF 0 99 root://xrootd-archive.cr.cnaf.infn.it:1096//dune/testpro/bb/7f/awt-download-2023-03-07-01.txt davs
==awt== US_UConn-HPC DUNE_UK_GLASGOW 0 99 root://cephc02.gla.scotgrid.ac.uk:1094//cephfs/dune/RSE/testpro/bb/7f/awt-download-2023-03-07-01.txt davs
==awt== US_UConn-HPC DUNE_UK_LANCASTER_CEPH 0 99 root://xgate.hec.lancs.ac.uk:1094//cephfs/grid/dune/testpro/bb/7f/awt-download-2023-03-07-01.txt davs
==awt== US_UConn-HPC DUNE_UK_MANCHESTER_CEPH 0 99 root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/testpro/bb/7f/awt-download-2023-03-07-01.txt davs
==awt== US_UConn-HPC DUNE_US_BNL_SDCC 0 99 root://dcdndoor.sdcc.bnl.gov:1094//pnfs/sdcc.bnl.gov/data/dune/RSE/testpro/bb/7f/awt-download-2023-03-07-01.txt davs
==awt== US_UConn-HPC DUNE_US_FNAL_DISK_STAGE 0 99 root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/testpro/bb/7f/awt-download-2023-03-07-01.txt davs
==awt== US_UConn-HPC NIKHEF 0 99 root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/testpro/bb/7f/awt-download-2023-03-07-01.txt davs
==awt== US_UConn-HPC PRAGUE 0 99 root://golias100.farm.particle.cz:1094/dpm/farm.particle.cz/home/dune/RSE/testpro/bb/7f/awt-download-2023-03-07-01.txt davs
==awt== US_UConn-HPC QMUL 0 99 root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/testpro/bb/7f/awt-download-2023-03-07-01.txt davs
==awt== US_UConn-HPC RAL-PP 0 99 root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/testpro/bb/7f/awt-download-2023-03-07-01.txt davs
==awt== US_UConn-HPC RAL_ECHO 0 99 root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/testpro/bb/7f/awt-download-2023-03-07-01.txt davs
==awt== US_UConn-HPC SURFSARA 0 99 root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/testpro/bb/7f/awt-download-2023-03-07-01.txt davs