NXxas refactoring

[woutdenolf]

WIP: everything will be moved to contributed_definitions once this is up for review.

Contributing: PR to the main branch of https://github.com/XraySpectroscopy/nexus_definitions (merge commits will be removed regularly).

Rendering: https://hdf5.gitlab-pages.esrf.fr/nexus/nxxas/classes/applications/NXxas_new.html

Example files: https://pynxxas.readthedocs.io/

NXDL questions:

- how to re-use an enum
- when enum field X has a specific value, make field Y required OR how to define enum properties
- possibly link to
- how to make a field required in a base class

[lukaspie]

Hi @woutdenolf, I came across this PR by chance and noticed that you are working on a base class for describing an electron level. In the context of photoemission spectroscopy, we in FAIRmat have also been working on something very similar: https://github.com/FAIRmat-NFDI/nexus_definitions/blob/fairmat/contributed_definitions/NXelectron_level.nxdl.xml. In fact, some of the base class we developed was originally adapated from another XAS-related PR: #1293. Would you be open to harmonizing the concept that we were developing with your approach here? I think the two approaches are already very similar right now.

[woutdenolf]

@lukaspie Yes since we are doing the same thing we should try to converge. We are currently discussing this very topic: XraySpectroscopy#2. Feel free to get involved.

The XAS working group will work in https://github.com/XraySpectroscopy/nexus_definitions. It would be helpful to add comments in that repo. This PR will be a draft and is subject to lots of changes until we are finished.

[lukaspie]

@woutdenolf great, thanks for the pointers. I will add some comments there.

[woutdenolf]

another XAS-related PR: #1293

This was the first attempt before we created the XAS working group. It will most likely be closed.

[whs92]

@woutdenolf after asking a question to the nexus mailing list and reading:

#1011
#1293

I arrived here :)

I have had a look at the proposal you made to the NXxas_new

I am particularly interested in the topic raised in #1011 of adding the ability to work with multiple detectors. I see that you've added the entry intensity that is still tied to a single mode (TFY, PFY, TEY etc). How might I add other signals acquired at the same time? How might I add more than one PFY signal?

Do you plan to include the suggestions from @padraic-shafer in https://github.com/padraic-shafer/definitions/blob/1011-multixas/contributed_definitions/NXmultixas.nxdl.xml

When is the next planned meeting of the working group? Can I come along?

[woutdenolf]

Currently we only support a single XAS spectrum. I'm very much open to support multi-detector and/or multi-scan. You are very welcome to join the XAS working group. I'll contact you directly.

[benajamin]

If you want to allow the "electron energy loss" mode, then you should also allow an "electron" probe type.

[benajamin]

What if I want to do a scan that includes multiple absorption edges (e.g. Carbon K-edge and K L-edge)? Can I specify multiple elements?

[benajamin]

Does this include Auger electron yield? Can we include a way to specify the observed electron energy range? Or should that be a detail for NXdetector?

[benajamin]

The i0 should have a corresponding NXmonitor group. There are also many ways of measuring an i0 that is useful to document in the file. For example, is the i0 measured in parallel with the sample absorption (or its proxy), or is it measured in sequence (e.g. before or after the sample), or is it sequential, but interleaved?

There is also question of how "clean" the i0 data is, as in how directly/accurately does the i0 measurement follow the ideal incident flux signal? This issue is particularly significant for C K-edge measurements - one can read my article on the topic to further understand this issue.

[ounsy]

#1352 (comment)
@whs92
@woutdenolf
I also got a request for that from one of our beamlines at SOLEIL

[mretegan]

The two options for the updated application definition described above and in Telco 20260211 are implemented below.

1. Inheritance - NXxas is a parent class. Each mode is a separate application definition that extends NXxas and adds mode-specific fields.

NXobject
 `-- NXxas (parent)
      |-- NXxas_trans
      |-- NXxas_tfy
      |-- NXxas_pfy
      |-- NXxas_herfd
      |-- NXxas_tey
      |-- NXxas_pey
      `-- ...

Rendered documentation: https://nexus-definitions.readthedocs.io/en/xas-using-inheritance/classes/applications/NXxas.html

2. Composition - a single NXxas application definition. Each measurement mode is a base class that extends NXobject (e.g., NXherfd) included as a group. The mode group carries all mode-specific metadata and optional instrument details.

NXobject
  |-- NXtrans  
  |-- NXtfy  
  |-- NXpfy  
  |-- NXherfd 
  |-- NXtey 
  |-- NXpey  
  `-- ...

NXxas (single application definition, extends NXobject)
  `-- uses any of the above modes

Rendered documentation: https://nexus-definitions.readthedocs.io/en/xas-using-modes/classes/applications/NXxas.html

[newville]

@mretegan strongly opposed to having separate NXpey, Nxtey - etc.

NXxas is sufficient. We are communicating normalized mu(E), and indicating how it was collected.

The subclasses could be fine, but they have proven to be a distraction.

[maurov]

@mretegan strongly opposed to having separate NXpey, Nxtey - etc.

NXxas is sufficient. We are communicating normalized mu(E), and indicating how it was collected.

The subclasses could be fine, but they have proven to be a distraction.

@newville I personally prefer the separate NX classes and I think there are good arguments for it. Over all, the fact that can be reused in other application definitions, like NXXmcd, independent of NXXas. I think the best would be to have a meeting for discussing this.

[newville]

@maurov in general, it is OK to have subclasses. Are these meant to be complete? People measure XAS-like data in lots of different ways that develop and change.

The NXxas definition should communicate normalized mu(E). Yes, different modes may need different kinds of corrections or have different interpretations (surface-sensitivity, energy resolution, magnetic moment).

Still, most of these can be communicated as normalized mu(E), and so be useful in comparison to some other normalized mu(E) data. Just calling them XAS and listing a named mode would be fine, at least for the overwhelming majority of data. With 4 or 10 "XAS modes", now you have a problem that people will forever be unsure whether NXxeol can be compared to NXpey.

Adoption of NXxas was always going to be difficult. NeXuS has very real problems of coming from a tiny, self-selected group that claims the expertise to be a standards body. The current NXxas definition is utterly useless and demonstrates that core weakness of NeXuS. A revised definition for NXxas has to overcome all this baggage and demonstrate value in the result and in the process.

The longer it takes, the more it appears to come from a small group of people, the more it appears to come from one facility, the harder it is to convince people. Months were spent on how to spell the names of emission lines, and adding the complexity of subclasses, and yet there is currently no usable definition. I think that "perfect" has defeated "good".

[rayosborn]

@newville, I can't comment on the substance of your criticisms because I'm not an XAS expert and have not been involved in the NXxas discussions, but I have to push back on your characterization of the NeXus standard body. The current NIAC has members from twenty facilities, all of whom have been approved by their management to represent them. They are dedicated to advancing the standard, even if they cannot claim to have expertise in every single neutron and x-ray scattering technique. However, they are committed to supporting those who do. If the NXxas effort has stalled for some reason, then I encourage you to attend the next NIAC teleconference explaining what you believe the problems are. I think you will find the committee very receptive to constructive criticism.

Creating consensus around standards such as NeXus is hard, but the payoffs of having metadata-rich data formats are worth it at a time when we are being mandated to provide shareable data repositories. Please encourage your colleagues to get involved. Things won't improve without people committing their own time and effort.