(Towards #3060) intrinsic return types #3119

LonelyCat124 · 2025-09-02T09:57:02Z

I'm putting this up early so if anyone has time to have a quick look at the implementation before I go too far down the rabbit hole with how I'm implementing the return type and tests.

I have added two more members to the IAttr namedtuple, return_type and reference_accesses, which can either be a Callable or specific value (depending on whats required).

To get the return_type of an IntrinsicCall, my plan is to do something like:

def return_type(self):
    if isinstance(self.intrinsic.return_type, Callable):
        try:
            return self.intrinsic.return_type(self)
         except:
             # The idea here is to handle all of the "bad input cases", e.g. if we have an Unresolved or UnsupportedType in the input
             # I'd rather have a except here than handle it in every single return_type callable.
             return UnresolvedType()
    # If its not a callable we just return the value.
    return self.intrinsic.return_type

The return type implementations are started - there are 3 helper functions at the moment (for cases I expect to be used a lot), e.g. _get_first_argument_type, wheras other's have their own lambda (for example see AINT).

I'm unsure how much to avoid code duplication here, for example AINT and ANINT have the same lambda for their return_type, so I'm not sure whether its worth moving this out (and whether the result should be a lambda or function) every time I have any 2 intrinsic calls with the same return type? Feedback on this specific question would be appreciated as early as possible (probably one for @arporter to answer perhaps).

To test the return types, my plan was to have standalone test for every "helper" function (or even helper lambda later).
I was then planning to create a parametrize test for all other intrinsics who have their own specific lambda. My one concern is this parametrize would become very large - again feedback/thoughts on this approach would be helpful. You can see an initial versoin of how this parametrize might look at intrinsic_call_test::650.

NB. This is dependent on #3110 and I think I will rebase onto that branch for now so I can have passing tests.

LonelyCat124 · 2025-09-02T10:10:39Z

One other coding style question - are we happy with statements like:

    return ScalarType(ScalarType.Intrinsic.REAL,
                      (node.arguments[node.argument_names.index("kind")]
                       if "kind" not in node.argument_names else
                       node.arguments[0].datatype.precision))

Or would you prefer to pull out the if statement? (This was required when this was a lambda, but its turning into a function as its getting a lot of reuse so I'm happy to rewrite it if it is preferred).

LonelyCat124 · 2025-09-02T10:25:19Z

Also one thing to note is PSyclone appears to support more IntrinsicCall than are created by Fparser - e.g. BESSEL_ functions get created as ArrayReferences - I'm not sure if this will be resolved by #3041 .

Also a question for @sergisiso - can you always refer to arguments with their names? I see for example CSHIFT is defined as
RESULT = CSHIFT(ARRAY, SHIFT [, DIM]), but could you do:
RESULT = CSHIFT(shift=3, ARRAY=array)?

LonelyCat124 · 2025-09-02T14:45:00Z

I would also say - there are some cases of reusing precision througout this code. I'm not sure if this is a good idea with the new "precision can be DataNodes" - if not then the review might need to request me to fix that by copying if they're a DataNode. This probably in some cases means some significant rewrites, but I'll wait for the review (I think that datatype.copy might also have this issue?).

LonelyCat124 · 2025-09-02T15:43:40Z

I applied the black formatter to these files as well - I couldn't work out how to make formatting happy myself for a couple of the lambdas so I had to make black do it for me.

arporter · 2025-09-03T08:52:49Z

src/psyclone/psyir/nodes/intrinsic_call.py

+            False,
+            ArgDesc(1, 1, DataNode),
+            {},
+            None,


Strictly speaking, as this is a call to a routine, the type is lambda: NoType(). I was going to say we need to make sure this is consistent with Call.datatype but that method isn't implemented yet :-)

arporter · 2025-09-03T08:55:45Z

src/psyclone/psyir/nodes/intrinsic_call.py

-            'ANINT', True, True, False,
-            ArgDesc(1, 1, DataNode), {"kind": DataNode})
+            "ANINT",
+            True,


Not yours but to make this code easier to understand, I think it would be good to use keyword arguments when we construct the IAttr objects, e.g.:

ANINT = IAttr( name = "ANINT", is_pure = True, ...

I can work on this after I reach the end of return types yeah.

I have previously felt it is hard to work out what is being referred to so would be a good change, especially as their sizes grow.

arporter · 2025-09-03T09:05:17Z

src/psyclone/psyir/nodes/intrinsic_call.py

+    return ArrayType(dtype, new_shape)
+
+
+def _get_first_argument_logical_kind_with_optional_dim(node) -> DataType:


Is it possible to replace the various _get_first_argument_{int,real,logical...}_kind_with... with a version that just takes the intrinsic-type as an argument?

I think I'd need to see an example of how you mean - I'm not sure I have many of these functions that only vary by the Intrinsic type - I think most have some other variation (either have an optional dim or kind or work on scalars vs arrays etc.). Some that take the same named arguments don't behave the same either - some have kind of first argument if no kind whilst others have default kind if no kind.

I did consider a decorator to try to make a single one that was cleaner, but I didn't like the design of that due to the level of disparity between the functions - especially now I've reached some of the more complex return types that really needed to be a function and not a lambda to be readable.

arporter · 2025-09-03T09:15:04Z

One other coding style question - are we happy with statements like:
    return ScalarType(ScalarType.Intrinsic.REAL,
                      (node.arguments[node.argument_names.index("kind")]
                       if "kind" not in node.argument_names else
                       node.arguments[0].datatype.precision)) 
Or would you prefer to pull out the if statement? (This was required when this was a lambda, but its turning into a function as its getting a lot of reuse so I'm happy to rewrite it if it is preferred).

I think I'd prefer a separate if for this - it's quite hard to parse :-)

arporter · 2025-09-03T09:16:22Z

To get the return_type of an IntrinsicCall, my plan is to do something like:

def return_type(self):
    if isinstance(self.intrinsic.return_type, Callable):
        try:
            return self.intrinsic.return_type(self)
         except:
             # The idea here is to handle all of the "bad input cases", e.g. if we have an Unresolved or UnsupportedType in the input
             # I'd rather have a except here than handle it in every single return_type callable.
             return UnresolvedType()
    # If its not a callable we just return the value.
    return self.intrinsic.return_type

I'm a bit confused by the check on whether it is Callable. Could we avoid this by always having a lambda or am I missing something?

arporter · 2025-09-03T09:17:24Z

Thanks Aidan, I think it's looking mostly as I'd expect although, as commented above, I was anticipating always having a Callable - whether a lambda or a separate routine if it's complicated enough.

EDIT: scrub that - I was getting confused between the definition of an IntrinsicCall and an Intrinsic. I think what you're suggesting is fine actually.

LonelyCat124 · 2025-09-03T09:30:55Z

Thanks Aidan, I think it's looking mostly as I'd expect although, as commented above, I was anticipating always having a Callable - whether a lambda or a separate routine if it's complicated enough.

EDIT: scrub that - I was getting confused between the definition of an IntrinsicCall and an Intrinsic. I think what you're suggesting is fine actually.

I could always have a lambda - it just felt overkill for cases where the return type is just an INTEGER_TYPE - I'll finish implementing things as they are now and at review time if the datatype of the IntrinsicCall routine is a bit of a mess we can re-evaluate.

LonelyCat124 · 2025-09-03T14:49:57Z

I will say I semi-lost the will to carry on for specifically THIS_IMAGE - there are no GNU docs for it and I'm not sure our current version is correct anyway so I just gave up and made it UnresolvedType().

I'll clean up the remaining test suite issues before I have "finished" return_type and probably it would be good to have a closer look before I move on to implementing reference_accesses.

codecov · 2025-09-03T15:19:05Z

Codecov Report

❌ Patch coverage is 99.63235% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 99.89%. Comparing base (769cf00) to head (d108c89).
⚠️ Report is 84 commits behind head on master.

Files with missing lines	Patch %	Lines
src/psyclone/lfric.py	75.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3119      +/-   ##
==========================================
- Coverage   99.90%   99.89%   -0.01%     
==========================================
  Files         374      374              
  Lines       52383    52624     +241     
==========================================
+ Hits        52332    52569     +237     
- Misses         51       55       +4

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

LonelyCat124 · 2025-09-03T17:19:54Z

I fixed the remaining failing test and added the keywords for IAttr.
@arporter @sergisiso if either of you have time for a quick look through how I've implemented the return type functiosn and have any (specific) feedback it would be appreciated, as I'll then apply that when I look at reference_accesses as well.
I think we don't need to have a detailed overall review yet (including if the review wants to check I've not made mistakes), as that can wait until later.

LonelyCat124 · 2025-09-04T08:53:04Z

Fixed up the remaining coverage I can do with fortran_reader now - the remaining coverage is uncoveted as PSyclone can only made CodeBlock[StructureAccess] from the inputs.

LonelyCat124 · 2025-09-08T09:10:01Z

@arporter @sergisiso ready for a look now - Andy suggested doing reference accesses as its own PR separately.

There is coverage missing, but PSyclone/fparser doesn't support the inputs that could result in those - I want to leave the correct results in the code for when we do, but I'll leave it to the reviewer to decide.

Edit: Note that this PR incorporates the kind stuff from #3110 - so ignore anything that looks like it comes from changes to kinds.

Edit2: The other thing I'm unsure about for both this PR and the following PR is if we have optional arguments declared on an IntrinsicCall do they HAVE to be named in Fortran? I.e. is only integer(x, kind=wp) legal or can you just do integer(x, wp)?

LonelyCat124 · 2025-09-09T12:02:07Z

Also one note - I think TEAM_IMAGE is a typo/made up intrinsic we have? I think it should be TEAM_NUMBER (https://gcc.gnu.org/onlinedocs/gfortran/TEAM_005fNUMBER.html). If so, then let me know and I'll fix its name at least - I'll leave it to the reviewer as to whether we want return_type (and later reference_Accesses) for it.

LonelyCat124 · 2025-09-11T08:22:43Z

One note - this probably needs a todo w.r.t #2302 - I am rewriting the reference_accesses code to handle that, but this does not handle unexpected naming of arguments (that would cause IntrinsicCall.create to fail).

arporter

I realised that it's a bit early to review this as it's branched off the PR that makes precision a DataNode. Therefore, I've only done a limited look, mainly focused on intrinsic_call.py (once I realised about the branching).
I like the way it's going and thanks for adding the keywords to the arguments to the many IAttr constructors. Mainly it's the usual request for comments plus it would be really helpful to write down the rules that are implemented by the various help methods - if you could do that for all of them (in their docstrings) then that would be great. I think there's also some scope to reduce duplication.

arporter · 2025-09-10T15:23:05Z

examples/psyir/create_structure_types.py


 # Shorthand for a scalar type with REAL_KIND precision
-SCALAR_TYPE = ScalarType(ScalarType.Intrinsic.REAL, REAL_KIND)
+SCALAR_TYPE = ScalarType(ScalarType.Intrinsic.REAL, Reference(REAL_KIND))


I see what you mean about precisions and types now. Maybe SCALAR_TYPE needs to become a routine that returns a new object? Similarly for all the other shorthands we have here.

arporter · 2025-09-10T15:25:37Z

src/psyclone/core/access_sequence.py

+        :raises InternalError: if the variable does not have READ acccess.
+        '''
+        if self._access_type != AccessType.READ:
+            raise InternalError("Trying to change variable to 'TYPE_INFO' "


I can see you've just adapted this from the existing one but I think the text could be better. Perhaps "...change variable access from 'READ' to 'TYPE_INFO' but access type is '{self._access_type}'"

arporter · 2025-09-10T15:29:24Z

src/psyclone/core/signature.py

        # pylint: disable=import-outside-toplevel
        from psyclone.psyir.backend.debug_writer import DebugWriter
        from psyclone.psyir.nodes import Literal, Node, Reference
+        from psyclone.psyir.symbols import DataSymbol, INTEGER_TYPE


Presumably you hit a circular dependence? Did you try making the imports more specific (e.g. from psyclone.psyir.symbols.datasymbol import DataSymbol)?

Yeah - these were the circular dependencies I mentioned w.r.t. what Joerg was saying in Teams that I couldn't resolve any other way.

arporter · 2025-09-10T15:30:27Z

src/psyclone/domain/common/transformations/kernel_module_inline_trans.py

                # Compare the routine to be inlined with the one that
                # is already present.
                new_rts = self._prepare_code_to_inline([kernel_schedule])
+                print(new_rts[0] == routine, len(new_rts))


arporter · 2025-09-10T15:37:45Z

src/psyclone/tests/domain/common/transformations/kernel_module_inline_trans_test.py

    end module my_mod
    ''')

+    print("----------------------------------------------")


Please rm these prints.

arporter · 2025-09-11T11:02:14Z

src/psyclone/psyir/nodes/intrinsic_call.py

+    argtype2 = node.arguments[1].datatype
+    shape1 = argtype1.shape
+    shape2 = argtype2.shape
+    stype1 = ScalarType(argtype1.intrinsic, argtype1.precision)


mea culpa on lack of comments here. I presume I do all this so we can re-use the existing machinery for deciding on the type of the result of a BinaryOperation (e.g. real*integer => real). If you agree, please add a comment.

arporter · 2025-09-11T11:02:54Z

src/psyclone/psyir/nodes/intrinsic_call.py

+    arg2 = Reference(DataSymbol("b", stype2))
+    binop = BinaryOperation.create(BinaryOperation.Operator.MUL,
+                                   arg1, arg2)
+    # TODO - make this a public method?


Probably we do want to make it (BinaryOperation._get_result_scalar_type) a public method.

arporter · 2025-09-11T11:04:10Z

src/psyclone/psyir/nodes/intrinsic_call.py

+    #  a31 a32        a31*b1 + a32*b2
+    #  3 x 2 * 2 x 1 = 3 x 1
+    #  rank 2  rank 1  rank 1
+    if len(shape1) == 1:


mea culpa again but comments please, e.g. I think this first case is "vector-matrix" and the second is "matrix-vector".

arporter · 2025-09-11T12:31:29Z

src/psyclone/psyir/nodes/intrinsic_call.py

+def _maxval_return_type(node) -> DataType:
+    """ Helper function for the MAXVAL (and similar) intrinsic return
+    types.
+


Please document the rules.

arporter · 2025-09-11T12:34:16Z

src/psyclone/psyir/nodes/intrinsic_call.py

+            required_args=ArgDesc(1, 1, DataNode),
+            optional_args={"kind": DataNode},
+            return_type=lambda node: (
+                ScalarType(


I think you may have a utility that does this now?

…one into 2302_intrinsic_argument_names

This reverts commit 18ab68e.

…input

…one into 2302_intrinsic_argument_names

LonelyCat124 · 2025-11-06T15:46:27Z

I've rebased this now to branch off #3150, I'm not sure whether I'll do the same for the reference_accesses or just restart that at the point where its being worked on, it will depend how hard it is to rebase that since this has already been done. I can potentially restart looking at this with the argument names update now which means a more sensible way of implementing these is possible.

LonelyCat124 force-pushed the 3060_intrinsic_types branch from 5911f97 to 9f0799f Compare September 2, 2025 15:43

arporter reviewed Sep 3, 2025

View reviewed changes

This was referenced Sep 4, 2025

Function Calls vs ArrayAccessors vs Type Constructors #2823

Open

(Closes #3087) Handle expressions in the kind attribute of a declaration #3110

Merged

LonelyCat124 marked this pull request as ready for review September 8, 2025 09:02

LonelyCat124 added the ready for review label Sep 8, 2025

LonelyCat124 requested a review from sergisiso September 8, 2025 09:02

LonelyCat124 mentioned this pull request Sep 8, 2025

(Closes #3060) reference accesses for intrinsicCall #3125

Draft

LonelyCat124 changed the title ~~(Closes #3060) intrinsic return types and reference accesses~~ (Towards #3060) intrinsic return types Sep 8, 2025

arporter added under review and removed ready for review labels Sep 10, 2025

arporter requested changes Sep 11, 2025

View reviewed changes

arporter removed the under review label Sep 11, 2025

LonelyCat124 added 27 commits October 7, 2025 14:06

Fixed error in SIGN

36426fb

linting fixW

a8c13b6

Fix failing tests

67a161b

Merge branch 'master' into 2302_intrinsic_argument_names

755956e

Added argument_by_name to call

d91c9cc

Merge branch '2302_intrinsic_argument_names' of github.com:stfc/PSycl…

4c42db1

…one into 2302_intrinsic_argument_names

Revert "Fix for NEMO SIGN overriding"

38b267b

This reverts commit 18ab68e.

Adds switches to change the output behaviour for intrinsics from the …

a1c480d

…input

Added doc updates

ab0225e

Merged master

8621281

First set of fixes towards review

c04f77a

linting issues

30af84f

Merge branch 'master' into 2302_intrinsic_argument_names

f40f5eb

Changes for review

9fff0bd

Merge branch '2302_intrinsic_argument_names' of github.com:stfc/PSycl…

fbf5002

…one into 2302_intrinsic_argument_names

Add documentation about overriding intrinsics

77cf8a6

Merged master

75c2e84

[skip-ci] Initial version of intrinsic call extension

91ca311

[skip-ci] Initial version of intrinsic call extension

dd21eb5

[skip-ci] Added more intrisic return types and applied black formatter

f5aeddf

[skip-ci] linting

b9d24f1

Finished intrinsic return types...

7118834

Finish failing tests

a2270bc

Fixed remaining possible coverage

6404214

Missing coverage line

b4326b1

Added the ability to change READ access to TYPE_INFO

7d0f504

Linting

4285bdb

LonelyCat124 force-pushed the 3060_intrinsic_types branch from d108c89 to 4285bdb Compare November 6, 2025 15:45

fix code mistakes

b7e0f1c

		return ArrayType(dtype, new_shape)


		def _get_first_argument_logical_kind_with_optional_dim(node) -> DataType:

(Towards #3060) intrinsic return types #3119

Are you sure you want to change the base?

(Towards #3060) intrinsic return types #3119

Uh oh!

Conversation

LonelyCat124 commented Sep 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

LonelyCat124 commented Sep 2, 2025

Uh oh!

LonelyCat124 commented Sep 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

LonelyCat124 commented Sep 2, 2025

Uh oh!

LonelyCat124 commented Sep 2, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

arporter commented Sep 3, 2025

Uh oh!

arporter commented Sep 3, 2025

Uh oh!

arporter commented Sep 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

LonelyCat124 commented Sep 3, 2025

Uh oh!

LonelyCat124 commented Sep 3, 2025

Uh oh!

codecov bot commented Sep 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

LonelyCat124 commented Sep 3, 2025

Uh oh!

LonelyCat124 commented Sep 4, 2025

Uh oh!

LonelyCat124 commented Sep 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

LonelyCat124 commented Sep 9, 2025

Uh oh!

LonelyCat124 commented Sep 11, 2025

Uh oh!

arporter left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LonelyCat124 commented Sep 2, 2025 •

edited

Loading

LonelyCat124 commented Sep 2, 2025 •

edited

Loading

arporter commented Sep 3, 2025 •

edited

Loading

codecov bot commented Sep 3, 2025 •

edited

Loading

LonelyCat124 commented Sep 8, 2025 •

edited

Loading