Jobsub ID 360199.0@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID	360199.0@justin-prod-sched01.dune.hep.ac.uk
Workflow Testing	Yes
Workflow ID	500
Stage ID	1
User name	amcnab@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	GPU	No
	RSS bytes	1073741824 (1024 MiB)
	Wall seconds limit	3600 (1 hours)
Submitted time	2025-04-02 09:42:40
Site	CA_Victoria
Entry	DUNE_CA_Victoria_dune-condor_whole
Last heartbeat	2025-04-02 23:30:15
From worker node	Hostname	dune--arbutus--3456932580--220759867907155
	cpuinfo	Intel Xeon E3-12xx v2 (Ivy Bridge, IBRS)
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	1073741824 (1024 MiB)
	Wall seconds limit	171000 (47 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Allocator name	justin-allocator-int.dune.hep.ac.uk
Started	2025-04-02 22:59:28
Input files
Jobscript	Exit code	0
	Real time	24m (1476s)
	CPU time	0m (27s = 1%)
	Max RSS bytes	31981568 (30 MiB)
Outputting started	2025-04-02 23:24:28
Output files
Finished	2025-04-02 23:30:15
Saved logs	justin-logs:360199.0-justin-prod-sched01.dune.hep.ac.uk.logs.tgz
List job events Wrapper job log
Jobscript log (last 10,000 characters)

t-1743635936-smaYcZaFd1 --timeout 1200' returns 99


---------------------------------------------------------------------
CA_Victoria QMUL davs root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/testpro/bb/7f/awt-download-2023-03-07-01.txt
'xrdcp --force --nopbar --verbose root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/testpro/bb/7f/awt-download-2023-03-07-01.txt downloaded.txt' returns 0
{
    "created_timestamp": null,
    "creator": "dunepro",
    "fid": "ntA6Y7ciQ0CuVE8V",
    "metadata": {},
    "name": "awt-1743635936-qphN5oHG4V",
    "namespace": "testpro",
    "retired": false,
    "retired_by": null,
    "retired_timestamp": null,
    "size": 0,
    "updated_by": null,
    "updated_timestamp": null
}
metacat file declare returns 0
GFAL_CONFIG_DIR:    GFAL_PLUGIN_DIR: 
justin-rucio-upload attempt 1
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): dune-rucio.fnal.gov:443
DEBUG:urllib3.connectionpool:https://dune-rucio.fnal.gov:443 "GET /auth/x509_proxy HTTP/1.1" 401 132
DEBUG:charset_normalizer:Encoding detection: ascii is most likely the one.
DEBUG:charset_normalizer:Encoding detection: ascii is most likely the one.
DEBUG:charset_normalizer:Encoding detection: ascii is most likely the one.
--- Upload try 1/1
--- Rucio upload 1/1 fails: Cannot authenticate.
Details: Cannot authenticate to account justinreadonly with given credentials
--- Exit with 99
'justin-rucio-upload --rse QMUL --protocol davs --scope testpro --dataset awt-uploads-202513 awt-1743635936-qphN5oHG4V --timeout 1200' returns 99


---------------------------------------------------------------------
CA_Victoria RAL-PP davs root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/testpro/bb/7f/awt-download-2023-03-07-01.txt
'xrdcp --force --nopbar --verbose root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/testpro/bb/7f/awt-download-2023-03-07-01.txt downloaded.txt' returns 0
{
    "created_timestamp": null,
    "creator": "dunepro",
    "fid": "uJNUIX7PQpGvvqeD",
    "metadata": {},
    "name": "awt-1743635936-IWN7Q0kE0l",
    "namespace": "testpro",
    "retired": false,
    "retired_by": null,
    "retired_timestamp": null,
    "size": 0,
    "updated_by": null,
    "updated_timestamp": null
}
metacat file declare returns 0
GFAL_CONFIG_DIR:    GFAL_PLUGIN_DIR: 
justin-rucio-upload attempt 1
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): dune-rucio.fnal.gov:443
DEBUG:urllib3.connectionpool:https://dune-rucio.fnal.gov:443 "GET /auth/x509_proxy HTTP/1.1" 401 132
DEBUG:charset_normalizer:Encoding detection: ascii is most likely the one.
DEBUG:charset_normalizer:Encoding detection: ascii is most likely the one.
DEBUG:charset_normalizer:Encoding detection: ascii is most likely the one.
--- Upload try 1/1
--- Rucio upload 1/1 fails: Cannot authenticate.
Details: Cannot authenticate to account justinreadonly with given credentials
--- Exit with 99
'justin-rucio-upload --rse RAL-PP --protocol davs --scope testpro --dataset awt-uploads-202513 awt-1743635936-IWN7Q0kE0l --timeout 1200' returns 99


---------------------------------------------------------------------
CA_Victoria RAL_ECHO davs root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/testpro/bb/7f/awt-download-2023-03-07-01.txt
'xrdcp --force --nopbar --verbose root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/testpro/bb/7f/awt-download-2023-03-07-01.txt downloaded.txt' returns 0
{
    "created_timestamp": null,
    "creator": "dunepro",
    "fid": "N78mf3xKSniZpDCg",
    "metadata": {},
    "name": "awt-1743635936-FKP2dBCPn4",
    "namespace": "testpro",
    "retired": false,
    "retired_by": null,
    "retired_timestamp": null,
    "size": 0,
    "updated_by": null,
    "updated_timestamp": null
}
metacat file declare returns 0
GFAL_CONFIG_DIR:    GFAL_PLUGIN_DIR: 
justin-rucio-upload attempt 1
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): dune-rucio.fnal.gov:443
DEBUG:urllib3.connectionpool:https://dune-rucio.fnal.gov:443 "GET /auth/x509_proxy HTTP/1.1" 401 132
DEBUG:charset_normalizer:Encoding detection: ascii is most likely the one.
DEBUG:charset_normalizer:Encoding detection: ascii is most likely the one.
DEBUG:charset_normalizer:Encoding detection: ascii is most likely the one.
--- Upload try 1/1
--- Rucio upload 1/1 fails: Cannot authenticate.
Details: Cannot authenticate to account justinreadonly with given credentials
--- Exit with 99
'justin-rucio-upload --rse RAL_ECHO --protocol davs --scope testpro --dataset awt-uploads-202513 awt-1743635936-FKP2dBCPn4 --timeout 1200' returns 99


---------------------------------------------------------------------
CA_Victoria SURFSARA davs root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/testpro/bb/7f/awt-download-2023-03-07-01.txt
'xrdcp --force --nopbar --verbose root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/testpro/bb/7f/awt-download-2023-03-07-01.txt downloaded.txt' returns 0
{
    "created_timestamp": null,
    "creator": "dunepro",
    "fid": "IP74gBAcQZaGtr3B",
    "metadata": {},
    "name": "awt-1743635936-56QwHg3aDI",
    "namespace": "testpro",
    "retired": false,
    "retired_by": null,
    "retired_timestamp": null,
    "size": 0,
    "updated_by": null,
    "updated_timestamp": null
}
metacat file declare returns 0
GFAL_CONFIG_DIR:    GFAL_PLUGIN_DIR: 
justin-rucio-upload attempt 1
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): dune-rucio.fnal.gov:443
DEBUG:urllib3.connectionpool:https://dune-rucio.fnal.gov:443 "GET /auth/x509_proxy HTTP/1.1" 401 132
DEBUG:charset_normalizer:Encoding detection: ascii is most likely the one.
DEBUG:charset_normalizer:Encoding detection: ascii is most likely the one.
DEBUG:charset_normalizer:Encoding detection: ascii is most likely the one.
--- Upload try 1/1
--- Rucio upload 1/1 fails: Cannot authenticate.
Details: Cannot authenticate to account justinreadonly with given credentials
--- Exit with 99
'justin-rucio-upload --rse SURFSARA --protocol davs --scope testpro --dataset awt-uploads-202513 awt-1743635936-56QwHg3aDI --timeout 1200' returns 99


subject   : /C=UK/O=eScience/OU=Manchester/L=HEP/CN=justin-jobs-production.dune.hep.ac.uk/CN=1987735510/CN=174363476883
issuer    : /C=UK/O=eScience/OU=Manchester/L=HEP/CN=justin-jobs-production.dune.hep.ac.uk/CN=1987735510
identity  : /C=UK/O=eScience/OU=Manchester/L=HEP/CN=justin-jobs-production.dune.hep.ac.uk/CN=1987735510
type      : RFC compliant proxy
strength  : 2048 bits
path      : /home/awt-proxy.pem
timeleft  : 167:35:27
key usage : Digital Signature, Key Encipherment, Key Agreement
=== VO dune extension information ===
VO        : dune
subject   : /C=UK/O=eScience/OU=Manchester/L=HEP/CN=justin-jobs-production.dune.hep.ac.uk
issuer    : /DC=org/DC=incommon/C=US/ST=Illinois/O=Fermi Research Alliance/CN=voms1.fnal.gov
attribute : /dune/Role=Production/Capability=NULL
attribute : /dune/Role=NULL/Capability=NULL
timeleft  : 161:13:03
uri       : voms1.fnal.gov:15042

===== Results =====

Download/upload commands:
xrdcp --force --nopbar --verbose $read_pfn downloaded.txt
echo '{"namespace":"testpro","name":"FILENAME","size":0}' >tmp.json
metacat file declare --json -f tmp.json "dune:all"
justin-rucio-upload --rse $rse_name --protocol $write_protocol --scope testpro --dataset awt-uploads-202513 --timeout 1200 FILENAME
Use the wrapper job link on the page for the job on the justIN Dashboard to find the full log file, with errors from these commands

Each line: $JUSTIN_SITE_NAME $rse_name $download_retval $upload_retval $read_pfn $write_protocol
==awt== CA_Victoria DUNE_CA_SFU 0 99 root://lcg-dunese1.sfu.computecanada.ca:1094//dune/RSE/testpro/bb/7f/awt-download-2023-03-07-01.txt davs
==awt== CA_Victoria DUNE_CERN_EOS 0 99 root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/testpro/bb/7f/awt-download-2023-03-07-01.txt davs
==awt== CA_Victoria DUNE_ES_PIC 0 99 root://xrootd.pic.es:1094/pnfs/pic.es/data/dune/RSE/testpro/bb/7f/awt-download-2023-03-07-01.txt davs
==awt== CA_Victoria DUNE_FR_CCIN2P3_DISK 0 99 root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/testpro/bb/7f/awt-download-2023-03-07-01.txt davs
==awt== CA_Victoria DUNE_IT_INFN_CNAF 0 99 root://xrootd-archive.cr.cnaf.infn.it:1096//dune/testpro/bb/7f/awt-download-2023-03-07-01.txt davs
==awt== CA_Victoria DUNE_UK_GLASGOW 0 99 root://cephc02.gla.scotgrid.ac.uk:1094//cephfs/dune/RSE/testpro/bb/7f/awt-download-2023-03-07-01.txt davs
==awt== CA_Victoria DUNE_UK_LANCASTER_CEPH 0 99 root://xgate.hec.lancs.ac.uk:1094//cephfs/grid/dune/testpro/bb/7f/awt-download-2023-03-07-01.txt davs
==awt== CA_Victoria DUNE_UK_MANCHESTER_CEPH 0 99 root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/testpro/bb/7f/awt-download-2023-03-07-01.txt davs
==awt== CA_Victoria DUNE_US_BNL_SDCC 0 99 root://dcdndoor.sdcc.bnl.gov:1094//pnfs/sdcc.bnl.gov/data/dune/RSE/testpro/bb/7f/awt-download-2023-03-07-01.txt davs
==awt== CA_Victoria DUNE_US_FNAL_DISK_STAGE 0 99 root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/testpro/bb/7f/awt-download-2023-03-07-01.txt davs
==awt== CA_Victoria NIKHEF 0 99 root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/testpro/bb/7f/awt-download-2023-03-07-01.txt davs
==awt== CA_Victoria PRAGUE 0 99 root://golias100.farm.particle.cz:1094/dpm/farm.particle.cz/home/dune/RSE/testpro/bb/7f/awt-download-2023-03-07-01.txt davs
==awt== CA_Victoria QMUL 0 99 root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/testpro/bb/7f/awt-download-2023-03-07-01.txt davs
==awt== CA_Victoria RAL-PP 0 99 root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/testpro/bb/7f/awt-download-2023-03-07-01.txt davs
==awt== CA_Victoria RAL_ECHO 0 99 root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/testpro/bb/7f/awt-download-2023-03-07-01.txt davs
==awt== CA_Victoria SURFSARA 0 99 root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/testpro/bb/7f/awt-download-2023-03-07-01.txt davs