microsoft-psrp securestring jinja2.exceptions.TemplateAssertionError

[noderat]

Apache Airflow Provider(s)

microsoft-psrp

Versions of Apache Airflow Providers

apache-airflow-providers-microsoft-psrp==2.8.0

Apache Airflow version

2.10.2

Operating System

Debian GNU/Linux 12 (bookworm)

Deployment

Official Apache Airflow Helm Chart

Deployment details

Using official helm chart and the following slightly extended Dockerfile:

FROM apache/airflow:2.10.2
RUN pip install --no-cache-dir "apache-airflow[microsoft-psrp]==${AIRFLOW_VERSION}" \
    --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-${AIRFLOW_VERSION}/constraints-3.9.txt"

What happened

jinja2.exceptions.TemplateAssertionError: No filter named 'securestring'

It appears the securestring template filter is not working as intended.

Summarized Error:

ERROR - Exception rendering Jinja template for task 'Write-Output', field 'arguments'. Template: ["{{ 'foo' | securestring }}"]
  ( ... lengthy stack trace redacted ... )
jinja2.exceptions.TemplateAssertionError: No filter named 'securestring'.

Full execution log including the redacted stack trace:

[2024-11-13, 16:11:36 MST] {taskinstance.py:2865} INFO - Starting attempt 1 of 1
[2024-11-13, 16:11:36 MST] {taskinstance.py:2888} INFO - Executing <Task(PsrpOperator): Write-Output> on 2024-11-13 23:11:35.867453+00:00
[2024-11-13, 16:11:36 MST] {warnings.py:112} WARNING - /home/airflow/.local/lib/python3.12/site-packages/airflow/task/task_runner/standard_task_runner.py:70: DeprecationWarning: This process (pid=14633) is multi-threaded, use of fork() may lead to deadlocks in the child.
  pid = os.fork()
[2024-11-13, 16:11:36 MST] {standard_task_runner.py:72} INFO - Started process 14644 to run task
[2024-11-13, 16:11:36 MST] {standard_task_runner.py:104} INFO - Running: ['airflow', 'tasks', 'run', 'psrpoperator_securestring_error_reproduction', 'Write-Output', 'manual__2024-11-13T23:11:35.867453+00:00', '--job-id', '802', '--raw', '--subdir', 'DAGS_FOLDER/psrp_operator_securestring_error_repoduction.py', '--cfg-path', '/tmp/tmpwrdenr76']
[2024-11-13, 16:11:36 MST] {standard_task_runner.py:105} INFO - Job 802: Subtask Write-Output
[2024-11-13, 16:11:37 MST] {task_command.py:467} INFO - Running <TaskInstance: psrpoperator_securestring_error_reproduction.Write-Output manual__2024-11-13T23:11:35.867453+00:00 [running]> on host experimental-temp-airflow-worker-0.experimental-temp-airflow-worker.experimental-temp-airflow.svc.cluster.local
[2024-11-13, 16:11:37 MST] {abstractoperator.py:778} ERROR - Exception rendering Jinja template for task 'Write-Output', field 'arguments'. Template: ["{{ 'foo' | securestring }}"]
Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/models/abstractoperator.py", line 770, in _do_render_template_fields
    rendered_content = self.render_template(
                       ^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/template/templater.py", line 183, in render_template
    return [self.render_template(element, context, jinja_env, oids) for element in value]
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/template/templater.py", line 170, in render_template
    template = jinja_env.from_string(value)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/jinja2/environment.py", line 1108, in from_string
    return cls.from_code(self, self.compile(source), gs, None)
                               ^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/jinja2/environment.py", line 768, in compile
    self.handle_exception(source=source_hint)
  File "/home/airflow/.local/lib/python3.12/site-packages/jinja2/environment.py", line 939, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<unknown>", line 1, in template
jinja2.exceptions.TemplateAssertionError: No filter named 'securestring'.
[2024-11-13, 16:11:37 MST] {taskinstance.py:3310} ERROR - Task failed with exception
Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/models/taskinstance.py", line 273, in _run_raw_task
    TaskInstance._execute_task_with_callbacks(
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/models/taskinstance.py", line 3114, in _execute_task_with_callbacks
    task_orig = self.render_templates(context=context, jinja_env=jinja_env)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/models/taskinstance.py", line 3533, in render_templates
    original_task.render_template_fields(context, jinja_env)
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/models/baseoperator.py", line 1419, in render_template_fields
    self._do_render_template_fields(self, self.template_fields, context, jinja_env, set())
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/models/abstractoperator.py", line 770, in _do_render_template_fields
    rendered_content = self.render_template(
                       ^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/template/templater.py", line 183, in render_template
    return [self.render_template(element, context, jinja_env, oids) for element in value]
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/airflow/template/templater.py", line 170, in render_template
    template = jinja_env.from_string(value)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/jinja2/environment.py", line 1108, in from_string
    return cls.from_code(self, self.compile(source), gs, None)
                               ^^^^^^^^^^^^^^^^^^^^
  File "/home/airflow/.local/lib/python3.12/site-packages/jinja2/environment.py", line 768, in compile
    self.handle_exception(source=source_hint)
  File "/home/airflow/.local/lib/python3.12/site-packages/jinja2/environment.py", line 939, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<unknown>", line 1, in template
jinja2.exceptions.TemplateAssertionError: No filter named 'securestring'.
[2024-11-13, 16:11:37 MST] {taskinstance.py:1225} INFO - Marking task as FAILED. dag_id=psrpoperator_securestring_error_reproduction, task_id=Write-Output, run_id=manual__2024-11-13T23:11:35.867453+00:00, execution_date=20241113T231135, start_date=20241113T231136, end_date=20241113T231137

What you think should happen instead

The minimal example provided should return through XCom the ToString representation of a powershell SecureString which would be "System.Security.SecureString"

How to reproduce

The following DAG can be used to reproduce the issue and does not require a remote connection as the task fails before attempting any connection:

import datetime
from airflow import DAG
from airflow.providers.microsoft.psrp.operators.psrp import PsrpOperator

with DAG(
    dag_id="psrpoperator_securestring_error_reproduction",
    start_date=None,
    schedule_interval=None,
    render_template_as_native_obj=True,
):
    PsrpOperator(
        psrp_conn_id = "SomeConnection",
        cmdlet = "Write-Output", 
        arguments = [ "{{ 'foo' | securestring }}" ],
    )

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.