Skip to content

Conversation

@yarikoptic
Copy link
Collaborator

@yarikoptic yarikoptic commented Feb 14, 2024

Replaces #1352 submitted from a fork outside of bids-specification.

Add specification for microelectrode electrohpysiology datasets based on the BEP032 proposal. old google doc

Note

We meet regularly and everyone is welcome

Next meeting: insert date on URL to join

Communication channel: https://framalistes.org/sympa/info/neuroscience-data-structure


DONEs

TODOs

Issues this PR would likely to address

Issues to see being addressed while working on this BEP (likely to move above) or not (moved below):

Other issues which relate but not in scope here and provided for reference/backreference

Spreadsheet with correspondence to ProbeInterface: https://docs.google.com/spreadsheets/d/1O0bZzD-n4MjR68r1GlcH3d2JLXBLAU1PfsDceD3IPeo/edit?usp=sharing

@TheChymera TheChymera closed this Mar 7, 2024
@TheChymera TheChymera deleted the bep032 branch March 7, 2024 22:13
@TheChymera TheChymera restored the bep032 branch March 7, 2024 22:15
@TheChymera TheChymera reopened this Mar 11, 2024
@TheChymera
Copy link
Collaborator

TheChymera commented Mar 12, 2024

@TheChymera
Copy link
Collaborator

@Remi-Gau Remi-Gau changed the title [ENH] Add BEP032 (microelectrode electrophysiology) specification [ENH] microelectrode electrophysiology specification (BEP032) Apr 18, 2024
@yarikoptic
Copy link
Collaborator Author

bids:: prefix means the file is inside the current dataset, and probes/neuronexus2.json is the root-relative path to the file.

@effigies so you suggest to establish yet another folder with structured data but which will not be formalized by BIDS in terms of schema/structure under it? or should we strive to come up with some consistent with BIDS overall and/or like e.g. BEP044 approaches to formalize under stimuli/ - https://bids.neuroimaging.io/extensions/beps/bep_044.html and consider making it probes/probe-<label>_probeinterface.json so we

  • have it explicitely stating to be in probeinterface format
  • have probe-<label> to be able to say probe_id (probe-<label>) eventually (and <label> here I think matches the "name")

@effigies
Copy link
Collaborator

@ree-gupta Apologies for the delay. I had unsubscribed so that I would see mentions but not other activity on this PR, so I did not see your response. Please @ me in the future, as I will be doing so again after today.

I had just a small question about how should the probes directory at root be correctly recognized by the schema/validator? Would you suggest adding it to src.schema.objects.files.yaml? like how stimuli directory is specified?

Yes, that or phenotype are the most direct parallels.

@yarikoptic

so you suggest to establish yet another folder with structured data but which will not be formalized by BIDS in terms of schema/structure under it?

The probes are a separate standard with their own contents and file-naming conventions. The intended progression appears to be:

  1. Stick new probe definitions into /probes and refer to them with bids::probes/*.
  2. Upload probes as-is to the global probe namespace.
  3. Replace the bids::probes/ prefix with a full URL prefix.

Given that they already have a BIDS-incompatible naming convention (there are hyphens in the names), it does not seem particularly appealing to me to put an effort to come up with a mapping. Any tool that wants to find the probe definitions is going to need to resolve a URI in any case, so querying using BIDS entities doesn't seem like it adds much utility.

We can probably check that the file name follows probe interface conventions as a schema check, if not a normal filename rule.

That said, if making the directory more BIDSy feels like a good use of people's time, I'm not here to stop you.

Copy link
Collaborator Author

@yarikoptic yarikoptic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some recommendations


The Microelectrode Electrophysiology modality encompasses recordings made with micrometer-scale electrodes,
distinguishing it from related BIDS modalities (EEG, MEG, iEEG) that use larger electrodes.
This modality is primarily used in animal research.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO here -- add an image describing relation between probes electrodes and channels.

smth like

image

based on what is in the spikeinterfaces docs

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know. The issue is that electrodes exist in physical space and channels don't really. They are often the result of some sort of wiring configuration, but at the end of the day a channel is a data stream, not a physical object, and the attributes of that table describe information about the data and data transformations, not the physical device on which it was collected.

**Intracellular electrophysiology example:**

```tsv
name type units sampling_frequency recording_mode gain ground status
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in addition to recording_mode we need to describe how clamping is done following e.g. types described in OpenMINDS at https://github.com/openMetadataInitiative/openMINDS_instances/tree/main/instances/latest/terminologies/patchClampVariation which we can name patch_variation and describe permitted values. (might need extra value for jaxtracellulular patching... loose patch recording according to google search by @apdavison)

I would also vote to define all the values allowed explicitly so could be validated.

### Recommended Channel Type Values

For the `type` column we recommend to use the following terms (adapted from [iEEG](intracranial-electroencephalography.md#channels-description-_channelstsv))

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: make a macro from the schema! Tireless @ree-gupta promised to do that in a separate PR and adopt here to avoid hardcoding

**Extracellular electrophysiology example:**

```tsv
probe_name type AP ML DV AP_angle ML_angle rotation_angle hemisphere manufacturer device_serial_number electrode_count width height depth dimension_unit coordinate_reference_point associated_brain_region associated_brain_region_id reference_atlas material
Copy link
Collaborator Author

@yarikoptic yarikoptic Oct 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note: at our meeting Oct 15th 2025 we briefly touched on a possible topic of introducing probe-<label> entity! It would reflect that actual recordings directly from hardware do ATM store each probe separately and it could be that there are different manufacturers/types of probes. That would then mandate pretty much to have probe_id (probe-<label>) as the leading column in the *probes.tsv file (which would exist then to summarize across probes, potentially even may be benefitting from inheritance principle for common metadata?).

Here probe_name came from analogy to name of electrodes.tsv inherited from EEG. But the question remains whether we might want to standardize this right away. WDYT @effigies ? Should we may be right away introduce probe entity?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I don't think I have enough to go on from this comment and the immediately surrounding text. I would probably need to read the BEP in some detail to form an opinion.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I understand. The primary key of the probes table should not map 1-1 with the json files. The probe jsons describe probe models. It is quite common for labs to use multiple of the exact same model in a session. In that case, multiple probes would map to a single probe model, which might be specified using a json. This is why we need a separate column to specify the model and can't use the probe_id.

Now if you wanted to have a separate tsv for probe models that would make sense from a relational perspective but I don't think it would be practical because the data we want to express about probe models does not fit well into a tabular structure and we would rather point to an external reference anyway.

@yarikoptic
Copy link
Collaborator Author

@effigies

so you suggest to establish yet another folder with structured data but which will not be formalized by BIDS in terms of schema/structure under it?

The probes are a separate standard with their own contents and file-naming conventions.

I do not think there is any particular filename convention outside of their internal library. Here we are to provide files in an alien to that library location, so then tools could load using their "standard" https://probeinterface.readthedocs.io/en/main/api.html#probeinterface.io.read_probeinterface function (correct @bendichter ?) . Then the question of filename sanitization is also not a problem, we could e.g. replace all - with now allowed +.

### Default probe-relative coordinate systems

When no [`space-<label>`](../appendices/entities.md#space) entity is specified in the filename,
electrode positions in `*_electrodes.tsv` are **probe-relative coordinates**:
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

chatting with @CodyCBakerPhD who raises a good point (I think) that if we are to describe probe geometries , either standard, or custom under /probes, then why do we want this yet another way to describe coordinates for the electrodes on the probes, and in the _probes.tsv table in particular? they would not be different for the same probe manufacturer/model, so why to repeat across the files? if very custom, a file under /probes could still be used to describe it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this is a bit redundant.

In the case where you are referencing a probe model in the probe interface library, the advantage would be that you have electrode positions in the BIDS dataset without needing to consult an external source. This is nice, because the electrode positions are very important for data analysis (much more important than the rest of the information in the probe specification files).

In the case where you are defining a custom probe, this is more redundant, since you would be able to extract this information directly from the probeinterface specification file. In this case, it's nice to have for uniformity with the library models, though I do appreciate this is creating redundancy that is undesirable.

@bendichter
Copy link
Contributor

@effigies

so you suggest to establish yet another folder with structured data but which will not be formalized by BIDS in terms of schema/structure under it?

The probes are a separate standard with their own contents and file-naming conventions.

I do not think there is any particular filename convention outside of their internal library. Here we are to provide files in an alien to that library location, so then tools could load using their "standard" https://probeinterface.readthedocs.io/en/main/api.html#probeinterface.io.read_probeinterface function (correct @bendichter ?) . Then the question of filename sanitization is also not a problem, we could e.g. replace all - with now allowed +.

I don't feel strongly about filenames for the probe json files. The probe interface library is clearly adhering to a convention of [manufacturer]/[model]/[model].json but the standard really lies in the json, not in the filenames. Using the probe interface standard does not require any particular file naming convention. I'd be fine with trying to follow the probe interface library or with trying to do something BIDSy. I just don't want this to be a rabbit hole.

probes/probe-<label>_probeinterface.json where label is the model would be a perfectly reasonable choice IMO. We would just need to make sure that the model specifier strings from custom probe models don't have slashes or any other forbidden chars in them.

In the case of long data recordings that exceed a file size of 2Gb,
`.fif` files are conventionally split into multiple parts.
Each of these files has an internal pointer to the next file.
Files with a lot of data may be split into multiple parts.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Files with a lot of data may be split into multiple parts.
Large files of time series data may be split into multiple parts.

Comment on lines +345 to +352
e001 probe01 L 0 0 0 1.2 0 15 iridium-oxide MOp
e002 probe01 L 0 0 25 1.1 0 15 iridium-oxide MOp
e003 probe01 L 0 0 50 1.3 0 15 iridium-oxide MOp
e004 probe01 L 0 0 75 1.4 0 15 iridium-oxide MOp
e005 probe02 R 0 0 0 2.1 n/a 12 tungsten CA1
e006 probe02 R 0 0 15 2.3 n/a 12 tungsten CA1
e007 probe02 R 0 0 30 1.9 n/a 12 tungsten CA1
e008 probe02 R 0 0 45 2.0 n/a 12 tungsten CA1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
e001 probe01 L 0 0 0 1.2 0 15 iridium-oxide MOp
e002 probe01 L 0 0 25 1.1 0 15 iridium-oxide MOp
e003 probe01 L 0 0 50 1.3 0 15 iridium-oxide MOp
e004 probe01 L 0 0 75 1.4 0 15 iridium-oxide MOp
e005 probe02 R 0 0 0 2.1 n/a 12 tungsten CA1
e006 probe02 R 0 0 15 2.3 n/a 12 tungsten CA1
e007 probe02 R 0 0 30 1.9 n/a 12 tungsten CA1
e008 probe02 R 0 0 45 2.0 n/a 12 tungsten CA1
e001 probe01 L 0 0 n/a 1.2 0 15 iridium-oxide MOp
e002 probe01 L 0 25 n/a 1.1 0 15 iridium-oxide MOp
e003 probe01 L 0 50 n/a 1.3 0 15 iridium-oxide MOp
e004 probe01 L 0 75 n/a 1.4 0 15 iridium-oxide MOp
e005 probe02 R 0 0 n/a 2.1 n/a 12 tungsten CA1
e006 probe02 R 0 15 n/a 2.3 n/a 12 tungsten CA1
e007 probe02 R 0 30 n/a 1.9 n/a 12 tungsten CA1
e008 probe02 R 0 45 n/a 2.0 n/a 12 tungsten CA1

Copy link
Contributor

@bendichter bendichter Oct 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change this to have the values be expressed in y instead of z and to have z be n/a when unused because:

  1. This is a more common convention, e.g. in ProbeInterface
  2. This is more consistent with the iEEG modality:
Image

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would also be fine with moving position to x and having both y and z be n/a, but I think what I have proposed is the more common convention

Comment on lines +359 to +361
patch01 pipette01 L 0 0 0 5.2 K-gluconate 1.5 2.5 borosilicate-glass VISp2/3
patch02 pipette02 R 0 0 0 4.8 K-gluconate 1.5 2.5 borosilicate-glass VISp2/3
sharp01 pipette03 L 0 0 0 80 3M KCl 0.5 1.0 borosilicate-glass PL5
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel these should be n/a when position data is irrelevant

Comment on lines +268 to +272
ch001 ref01 LFP uV 1000 HighpassFilter LowpassFilter 500 good n/a
ch002 ref01 LFP uV 1000 HighpassFilter LowpassFilter 500 good n/a
ch003 ref01 HP uV 30000 HighpassFilter n/a 500 good n/a
ch004 ref01 HP uV 30000 HighpassFilter n/a 500 bad high_noise
ch005 ref02 LFP uV 1000 HighpassFilter LowpassFilter 500 good n/a
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
ch001 ref01 LFP uV 1000 HighpassFilter LowpassFilter 500 good n/a
ch002 ref01 LFP uV 1000 HighpassFilter LowpassFilter 500 good n/a
ch003 ref01 HP uV 30000 HighpassFilter n/a 500 good n/a
ch004 ref01 HP uV 30000 HighpassFilter n/a 500 bad high_noise
ch005 ref02 LFP uV 1000 HighpassFilter LowpassFilter 500 good n/a
ch001 e001 LFP uV 1000 HighpassFilter LowpassFilter 500 good n/a
ch002 e002 LFP uV 1000 HighpassFilter LowpassFilter 500 good n/a
ch003 e001 HP uV 30000 HighpassFilter n/a 500 good n/a
ch004 e002 HP uV 30000 HighpassFilter n/a 500 bad high_noise
ch005 e003 LFP uV 1000 HighpassFilter LowpassFilter 500 good n/a

These channels clearly correspond to recordings from electrodes, so reference should point to names of electrodes

"Units": "year",
"Maximum": 89,
}
AP_angle:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to add anatomical_reference_point for when you want to use something other than begma

Comment on lines +590 to +603
If electrode positions are known in multiple coordinate systems (for example, probe-relative, StereoTaxic,
and AllenCCFv3), these spaces can be distinguished by the optional [`space-<label>`](../appendices/entities.md#space)
field, see the [`*_electrodes.tsv`-section](#electrodes-description-_electrodestsv)
for more information.
Note that the [`space-<label>`](../appendices/entities.md#space) fields must correspond
between `*_electrodes.tsv` and `*_coordsystem.json` if they refer to the same
data.

For examples:
- `*_space-StereoTaxic` (electrodes are localized in stereotactic coordinate system with bregma origin)
<!-- TODO: Add 'StereoTaxic', 'AllenCCFv3', 'PaxinosWatson', etc coordinate systems to appendix coordinate-systems.md under "Microelectrode Electrophysiology Specific Coordinate Systems" with appropriate definitions for each standard reference frame used in animal electrophysiology -->
- `*_space-individual` (electrodes are localized in subject-specific anatomical coordinate system)
- `*_space-AllenCCFv3` (electrodes are mapped to Allen Common Coordinate Framework v3)
- `*_space-PaxinosWatson` (electrodes are mapped to Paxinos-Watson rat brain atlas coordinates)
Copy link
Contributor

@bendichter bendichter Nov 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
If electrode positions are known in multiple coordinate systems (for example, probe-relative, StereoTaxic,
and AllenCCFv3), these spaces can be distinguished by the optional [`space-<label>`](../appendices/entities.md#space)
field, see the [`*_electrodes.tsv`-section](#electrodes-description-_electrodestsv)
for more information.
Note that the [`space-<label>`](../appendices/entities.md#space) fields must correspond
between `*_electrodes.tsv` and `*_coordsystem.json` if they refer to the same
data.
For examples:
- `*_space-StereoTaxic` (electrodes are localized in stereotactic coordinate system with bregma origin)
<!-- TODO: Add 'StereoTaxic', 'AllenCCFv3', 'PaxinosWatson', etc coordinate systems to appendix coordinate-systems.md under "Microelectrode Electrophysiology Specific Coordinate Systems" with appropriate definitions for each standard reference frame used in animal electrophysiology -->
- `*_space-individual` (electrodes are localized in subject-specific anatomical coordinate system)
- `*_space-AllenCCFv3` (electrodes are mapped to Allen Common Coordinate Framework v3)
- `*_space-PaxinosWatson` (electrodes are mapped to Paxinos-Watson rat brain atlas coordinates)
If electrode positions are known in multiple coordinate systems, these spaces can be distinguished by the optional [`space-<label>`](../appendices/entities.md#space)
field, see the [`*_electrodes.tsv`-section](#electrodes-description-_electrodestsv)
for more information.
Note that the [`space-<label>`](../appendices/entities.md#space) fields must correspond
between `*_electrodes.tsv` and `*_coordsystem.json` if they refer to the same
data.
For examples:
<!-- TODO: Add 'AllenCCFv3', 'PaxinosWatson', etc coordinate systems to appendix coordinate-systems.md under "Microelectrode Electrophysiology Specific Coordinate Systems" with appropriate definitions for each standard reference frame used in animal electrophysiology -->
- `*_space-individual` (electrodes are localized in subject-specific anatomical coordinate system)
- `*_space-AllenCCFv3` (electrodes are mapped to Allen Common Coordinate Framework v3)
- `*_space-PaxinosWatson` (electrodes are mapped to Paxinos-Watson rat brain atlas coordinates)

stereotaxic coordinates are expressed in the probes.csv, and probe-relative are in the electrodes.csv and (optionally) (in the probeinterface_library OR in the custom probeinterface file). The different coordinate systems here would be useful for individual vs. common spaces, or if a brain was mapped to different common spaces

Comment on lines +619 to +620
"sub-01_space-StereoTaxic_electrodes.tsv": "",
"sub-01_space-StereoTaxic_coordsystem.json": "",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"sub-01_space-StereoTaxic_electrodes.tsv": "",
"sub-01_space-StereoTaxic_coordsystem.json": "",
"sub-01_space-AllenCCFv3_electrodes.tsv": "",
"sub-01_space-AllenCCFv3_coordsystem.json": "",

@bendichter
Copy link
Contributor

bendichter commented Nov 5, 2025

New appendix for coordinate systems: #2247

ch003 ref01 HP uV 30000 HighpassFilter n/a 500 good n/a
ch004 ref01 HP uV 30000 HighpassFilter n/a 500 bad high_noise
ch005 ref02 LFP uV 1000 HighpassFilter LowpassFilter 500 good n/a
ch006 n/a SYNC V 30000 n/a n/a 1 good n/a
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we might see to add description for this value with the .json file to state where the SYNC comes from etc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

BEP ephys Proposed BEP see https://bids.neuroimaging.io/collaboration/governance.html#proposed-bep

Projects

None yet

Development

Successfully merging this pull request may close these issues.