Add pyiron_tables: HDF5-based job table nodes with no pyiron_base - #207
Open
prabhath-c wants to merge 2 commits into
Open
Add pyiron_tables: HDF5-based job table nodes with no pyiron_base#207prabhath-c wants to merge 2 commits into
prabhath-c wants to merge 2 commits into
Conversation
…endency
Reads job data directly from project directories and job HDF5 files
(index_project, BuildTable) instead of going through pyiron_base's SQL
database or job objects. Each table property is one small get_<property>
function plus a matching Add<Property> node that daisy-chains a
{property: function} dict into BuildTable, documented in
databases/pyiron_tables.md for extending with new properties. Includes
pyiron_table_potential_scan.py as a worked reference workflow
(bulk_modulus/potential/lattice_parameter per potential, from Murnaghan jobs).
prabhath-c
changed the title
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #207 +/- ##
==========================================
+ Coverage 31.25% 40.66% +9.41%
==========================================
Files 90 94 +4
Lines 10184 11786 +1602
==========================================
+ Hits 3183 4793 +1610
+ Misses 7001 6993 -8 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces a new, dependency-free implementation of pyiron's job-table data mining functionality in the
pyiron_nodescodebase, along with comprehensive documentation and a reference workflow. The new approach replaces the original pyiron_base/pyiron_atomistics dependency with lightweight, file-based operations, and establishes a clear, extensible pattern for adding new table properties as nodes. The most important changes are:Core functionality: File-based job-table and property extraction
databases/pyiron_tables.py: Implements a full replacement for pyiron_base's job-table and PyironTable data mining, using only file-system and HDF5 access (no SQL/database or job objects). This includes functions for project indexing, job filtering, property extraction, and table construction, all operating directly on HDF5 files.Node interface and extensibility
databases/pyiron_tables.py: Defines a clear, consistent pattern for adding new property-extracting nodes (AddBulkModulus,AddPotential,AddLatticeParameter), each wrapping a single property extraction function. These nodes can be daisy-chained to build up the set of columns for the job table, and are designed for easy extension.databases/pyiron_tables.md: Provides detailed developer documentation describing the design, calling conventions, and extension checklist for adding new table property nodes, ensuring maintainability and ease of onboarding.Reference workflow
Workflows/pyiron_table_potential_scan.py: Adds a reference workflow that reproduces the "Data mining using pyiron tables" notebook using the new node-based, dependency-free approach. This serves as both an example and a test for the new system.These changes together establish a robust, extensible, and lightweight framework for job-table data mining in pyiron_nodes, decoupled from the legacy pyiron_base infrastructure.