Queuedata

The following list describes the queuedata fields currently used by the Pilot.

Data member	Meaning
acopytools	Dictionary mapping transfer activity names (e.g. "pr" for read, "pw" for write) to lists of allowed copy tool names. Used by the data layer to select the correct copy tool per activity. Populated directly from CRIC queue data; falls back to the keys of copytools wrapped in a "default" activity if not set.
acopytools_schemas	Dictionary mapping transfer activities and copy tool names to lists of allowed protocol schemas (e.g. root, srm, webdav). Format per activity: either a list applied to all copy tools, or a dict keyed by copy tool name. Example: {"pr": {"gfalcopy": ["webdav"]}, "default": ["root"]}. Queried via resolve_allowed_schemas().
allow_lan	Boolean. If True, LAN-based transfers are permitted during stage-in regardless of the copy method used. Default: True.
allow_wan	Boolean. If True, WAN-based transfers are permitted during stage-in regardless of the copy method used. Default: False.
altstageout	Resolved boolean controlling alternative stage-out behaviour. Derived at load time from the "allow_altstageout" key inside the params dict: True forces alt stage-out, False disables it, None means no preference. Accessed via jobdata to decide whether to attempt failover storage.
appdir	Optional path to the directory in which experiment software releases are installed on the worker node. Raises a NoSoftwareDir exception at payload setup time if the path does not exist.
aprotocols	Dictionary of allowed transfer protocol schemas indexed by activity. Superseded by acopytools_schemas for most use cases. Not currently referenced in the pilot code and may be deprecated.
astorages	Dictionary mapping transfer activity names to associated DDM storage endpoint (RSE) names. Used by the data layer (api/data.py) to resolve the source or destination RSE for a given file transfer activity.
catchall	Free-form string carrying preliminary or site-specific parameters not yet promoted to dedicated fields. The pilot parses it for singularity_options / apptainer_options to populate container_options if that field is not set explicitly in CRIC.
container_options	String of command-line options passed directly to Singularity/Apptainer when launching the payload container (e.g. "-B /cvmfs,${workdir} --contain"). If absent from CRIC, extracted from the catchall field. The pilot automatically appends ",${workdir}" if --contain is present but the workdir bind mount is missing.
container_type	Parsed dictionary specifying which container runtime is used by which actor. Raw CRIC format is semicolon-separated "container_name:user_name" pairs (e.g. "apptainer:pilot;docker:wrapper"). Stored internally as {"pilot": "apptainer", "wrapper": "docker"}. Used to decide whether the pilot or the wrapper is responsible for container setup.
copytools	Dictionary of available copy tool configurations keyed by copy tool name (e.g. {"rucio": {...}, "mv": {...}}). Used directly when acopytools is not set, wrapping all keys under a "default" activity. Also consulted to check whether the mv copy tool is present for non-deterministic DDM handling.
corecount	Integer. Number of CPU cores available on the worker node for multi-core payload execution (e.g. AthenaMP). Defaults to 1 if not set or zero. Used when calculating the effective memory kill threshold (maxrss scaled by corecount).
direct_access_lan	Boolean. If True, the pilot prefers remote I/O (direct access, no local copy) over LAN for input files. If False, files are always copied to the scratch area (copy-to-scratch). Default: False.
direct_access_wan	Boolean. If True, the pilot prefers remote I/O over WAN for input files. If False, files are always copied to the scratch area. Default: False.
environ	String of semicolon-separated "KEY=VALUE" export statements prepended to the payload command before execution (e.g. "export ATLAS_POOLCOND_PATH=/cvmfs/..."). Populated from the CRIC "environ" field and applied in atlas/common.py via prepend_env_vars().
es_stageout_gap	Integer (seconds). Minimum time gap between consecutive Event Service stage-out operations. Mapped from CRIC field "zip_time_gap". If not set in CRIC, defaults to 600 s for opportunistic resources (pledgedcpu == -1) or 7200 s for normal resources.
is_cvmfs	Boolean. True if CVMFS is installed and accessible on the worker node. Passed to container setup functions to decide whether CVMFS bind mounts should be included in the Apptainer invocation. Default: True. If False, the pilot returns exit code 64 to the wrapper (CVMFSISNOTALIVE).
maxinputsize	Integer (MB). Maximum total size of all input files permitted for stage-in. A value of -1 disables the check entirely. Validated in api/data.py before stage-in begins; raises SIZETOOLARGE if the limit is exceeded.
maxrss	Integer (MB). Maximum allowed RSS memory for the payload process, as configured in CRIC. Used by the memory monitor (atlas/memory.py) together with pilot_rss_grace and corecount to compute the kill threshold. Set to 0 to disable memory monitoring.
maxtime	Integer (seconds). Maximum allowed wall-clock lifetime for the pilot on this resource. Used by job_monitor to abort the job when remaining time falls below the 10-minute threshold. A value of 0 is ignored; the pilot falls back to its default or command-line -l value.
maxwdir	Integer (MB). Maximum allowed size of the job work directory. Used to check available local disk space before stage-in and to enforce directory size limits during monitoring. Raises USERDIRTOOLARGE or NOLOCALSPACE if exceeded. Typically 14336 + 2000 MB at ATLAS sites.
memkillgrace	Integer (percentage). Grace margin above the RSS limit before the pilot sends a kill signal. A value of 100 means the kill threshold equals maxrss exactly; values above 100 give the payload extra headroom. Default: 100. Converted to a float multiplier via get_memkillgrace() in atlas/memory.py.
name	Internal name of the PanDA queue. Mapped from the CRIC field "nickname". Used in log messages and as the canonical queue identifier for server communication.
params	Dictionary of miscellaneous queue-level parameters not covered by dedicated fields. Used by the job monitor to extract per-job max walltime and start time. Also consumed by clean() to derive pilot_walltime_grace, pilot_rss_grace, and pilot_maxwdir_grace. Recognised keys: "allow_altstageout", "pilot_walltime_grace", "pilot_rss_grace", "pilot_maxwdir_grace".
pilot_maxwdir_grace	Float multiplier applied to maxwdir when enforcing work directory size checks. Derived from params["pilot_maxwdir_grace"] as 1.0 + value/100; default raw value is 20, giving a multiplier of 1.2. Set by set_pilot_maxwdir_grace() during clean().
pilot_rss_grace	Float multiplier applied to maxrss when computing the RSS kill threshold. Derived from params["pilot_rss_grace"] as 1.0 + value/100; default raw value is 100, giving a multiplier of 2.0. Set by set_pilot_rss_grace() during clean().
pilot_walltime_grace	Float multiplier applied to the effective maximum walltime before triggering a walltime abort. Derived from params["pilot_walltime_grace"] as 1.0 + value/100; default raw value is 0, giving a multiplier of 1.0 (no grace). Set by set_pilot_walltime_grace() during clean().
platform	Platform and architecture string, equivalent to cmtconfig in Pilot 1 (e.g. "x86_64-centos7-gcc8-opt"). Mapped from the CRIC field "cmtconfig". Used to select the correct software release build for the payload.
pledgedcpu	Integer. Pledged number of CPU cores for this queue as declared in CRIC. A value of -1 marks an opportunistic resource, which reduces the default es_stageout_gap to 600 s instead of 7200 s.
resource	Name of the PanDA resource associated with this queue. Mapped from the CRIC field "panda_resource". Used in logging and server communication.
resource_type	String describing the type of compute resource (e.g. "grid", "hpc", "k8s"). Used to apply resource-specific logic; for example, NOJOBSINPANDA is set on HPC resources after repeated failed job retrieval attempts.
site	ATLAS site name associated with this queue. Mapped from the CRIC field "atlas_site", falling back to "gstat" if not set.
state	AGIS/CRIC PanDA Queue state string (e.g. "ACTIVE", "INACTIVE", "BROKEROFF"). Checked at pilot startup: if the state is not "ACTIVE" the pilot raises a PilotException and aborts immediately.
status	PanDA Queue status string (e.g. "online", "offline"). Consulted in control/job.py when determining the job retrieval label to pass to the server.
timefloor	Integer (seconds). Minimum remaining pilot lifetime required before requesting another job. Stored in CRIC in minutes and converted to seconds by clean__timefloor(). A value of 0 disables the check. Used in proceed_with_getjob() to avoid accepting a job too close to pilot expiry.
type	String describing the queue type (e.g. "unified", "analysis", "production"). Used to determine proxy handling and stage-out behaviour: on unified queues an analysis job may fall back to the production proxy if a dedicated analysis proxy is absent.
use_pcache	Boolean. If True, passes --usepcache=True to the stage-in middleware command, enabling the PanDA cache (pcache) for input file staging. Also forwarded to the container stage-in script. Default: False.