Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 9 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,14 +42,20 @@ For a folder containing the data archive and the `optimade.yaml` file (such as i
- `optimake convert .` to convert the entry into the JSONL format (see below).
- `optimake serve .` to start the OPTIMADE API (this also converts the entry, if needed);

For more detailed information see also `optimake --help`.
For more detailed information, see `optimake --help`.

### Tutorial

The sections below provide a high-level overview of the functionality, but a step-by-step notebook tutorial is available in `examples/00_tutorial/tutorial.ipynb`, demonstrating the full workflow from raw data to a running OPTIMADE API and example queries.

To run the tutorial locally, install `optimade-maker[tutorial]` and open the notebook with Jupyter.

### Annotating with `optimade.yaml`

To annotate your structural data for `optimade-maker`, the data archive needs to be accompanied by an `optimade.yaml` config file. The following is a simple example for a zip archive (`structures.zip`) of cif files together with an optional property file (`properties.csv`):
To annotate your structural data for `optimade-maker`, the data archive needs to be accompanied by an `optimade.yaml` config file. The following is a simple example for a ZIP archive (`structures.zip`) of CIF files together with an optional property file (`properties.csv`):

```yaml
config_version: 0.1.1
config_version: 0.2.0
database_description: Simple DB

entries:
Expand Down
1 change: 1 addition & 0 deletions examples/00_tutorial/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
optimade.jsonl
1 change: 1 addition & 0 deletions examples/00_tutorial/.testing/first_entry.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"id": "Ba", "type": "structures", "links": null, "meta": null, "attributes": {"immutable_id": "cifs/Ba.cif", "last_modified": "2026-04-20T17:03:04.262764", "elements": ["Ba"], "nelements": 1, "elements_ratios": [1.0], "chemical_formula_descriptive": "Ba", "chemical_formula_reduced": "Ba", "chemical_formula_hill": null, "chemical_formula_anonymous": "A", "dimension_types": [1, 1, 1], "nperiodic_dimensions": 3, "lattice_vectors": [[4.22762646696287, 0.0, 0.0], [-1.4092088223209558, 3.9858444574842284, 0.0], [-1.4092088223209558, -1.9929222287421124, 3.4518425557147467]], "space_group_symmetry_operations_xyz": null, "space_group_symbol_hall": null, "space_group_symbol_hermann_mauguin": null, "space_group_symbol_hermann_mauguin_extended": null, "space_group_it_number": null, "cartesian_site_positions": [[0.0, 0.0, 0.0]], "nsites": 1, "species": [{"name": "Ba", "chemical_symbols": ["Ba"], "concentration": [1.0], "mass": null, "original_name": null, "attached": null, "nattached": null}], "species_at_sites": ["Ba"], "assemblies": null, "structure_features": [], "_optimake_density": 3920.0}, "relationships": null}
33 changes: 33 additions & 0 deletions examples/00_tutorial/cifs/Ba.cif
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@

##########################################################################
# Crystallographic Information Format file
# Produced by PyCifRW module
#
# This is a CIF file. CIF has been adopted by the International
# Union of Crystallography as the standard for data archiving and
# transmission.
#
# For information on this file format, follow the CIF links at
# http://www.iucr.org
##########################################################################

data_0

loop_
_atom_site_label
_atom_site_fract_x
_atom_site_fract_y
_atom_site_fract_z
_atom_site_type_symbol
Ba1 0.0 0.0 0.0 Ba
_cell_angle_alpha 109.47122063449069
_cell_angle_beta 109.47122063449069
_cell_angle_gamma 109.47122063449069
_cell_length_a 4.22762646696287
_cell_length_b 4.22762646696287
_cell_length_c 4.22762646696287
loop_
_symmetry_equiv_pos_as_xyz
'x, y, z'
_symmetry_int_tables_number 1
_symmetry_space_group_name_H-M 'P 1'
37 changes: 37 additions & 0 deletions examples/00_tutorial/cifs/BaTiO3.cif
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@

##########################################################################
# Crystallographic Information Format file
# Produced by PyCifRW module
#
# This is a CIF file. CIF has been adopted by the International
# Union of Crystallography as the standard for data archiving and
# transmission.
#
# For information on this file format, follow the CIF links at
# http://www.iucr.org
##########################################################################

data_0

loop_
_atom_site_label
_atom_site_fract_x
_atom_site_fract_y
_atom_site_fract_z
_atom_site_type_symbol
Ba1 0.0 0.0 0.0 Ba
Ti1 0.5 0.5 0.5 Ti
O1 0.0 0.5 0.5 O
O2 0.5 0.0 0.5 O
O3 0.5 0.5 0.0 O
_cell_angle_alpha 90.0
_cell_angle_beta 90.0
_cell_angle_gamma 90.0
_cell_length_a 3.9745754582274
_cell_length_b 3.9745754582274
_cell_length_c 3.9745754582274
loop_
_symmetry_equiv_pos_as_xyz
'x, y, z'
_symmetry_int_tables_number 1
_symmetry_space_group_name_H-M 'P 1'
46 changes: 46 additions & 0 deletions examples/00_tutorial/cifs/SrC6.cif
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@

##########################################################################
# Crystallographic Information Format file
# Produced by PyCifRW module
#
# This is a CIF file. CIF has been adopted by the International
# Union of Crystallography as the standard for data archiving and
# transmission.
#
# For information on this file format, follow the CIF links at
# http://www.iucr.org
##########################################################################

data_0

loop_
_atom_site_label
_atom_site_fract_x
_atom_site_fract_y
_atom_site_fract_z
_atom_site_type_symbol
Sr1 0.33333333333333 0.6666666666666755 0.24999999999999484 Sr
Sr2 0.6666666666666466 0.3333333333333244 0.7499999999999949 Sr
C1 0.6665864979541027 0.6665864979514344 0.5 C
C2 0.6665864979541027 0.6665864979514344 0.0 C
C3 0.6665864979568098 6.996148258805112e-21 0.5 C
C4 0.6665864979568098 0.0 0.0 C
C5 0.999999999997293 0.6665864979514344 0.5 C
C6 0.999999999997293 0.6665864979514344 0.0 C
C7 0.9999999999893362 0.3334135020218611 0.5 C
C8 0.9999999999893362 0.3334135020218611 6.856887615856403e-37 C
C9 0.33341350203251724 0.3334135020218611 0.5 C
C10 0.33341350203251724 0.3334135020218611 0.0 C
C11 0.33341350204319026 1.2871350630101513e-20 0.5 C
C12 0.33341350204319026 0.0 3.598182623059814e-39 C
_cell_angle_alpha 90.0
_cell_angle_beta 90.0
_cell_angle_gamma 120.00000000006627
_cell_length_a 4.3196827147571
_cell_length_b 4.319682714748347
_cell_length_c 9.7058202670626
loop_
_symmetry_equiv_pos_as_xyz
'x, y, z'
_symmetry_int_tables_number 1
_symmetry_space_group_name_H-M 'P 1'
17 changes: 17 additions & 0 deletions examples/00_tutorial/optimade.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
config_version: 0.2.0
database_description: Simple DB

entries:
- entry_type: structures
entry_paths:
- path: cifs
matches: ["*.cif"]

property_paths:
- path: properties.csv
property_definitions:
- name: density
title: Density
description: Density of the material
unit: kg/m^3
type: float
4 changes: 4 additions & 0 deletions examples/00_tutorial/properties.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
id,density
SrC6,3381
BaTiO3,6167
Ba,3920
189 changes: 189 additions & 0 deletions examples/00_tutorial/tutorial.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,189 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "a355627a",
"metadata": {},
"source": [
"## `optimade-maker` tutorial\n",
"\n",
"This tutorial guides users through the main functionality of the toolkit. The goal is to start an OPTIMADE API server from raw crystal structure data, and demonstrate how to use the API.\n",
"\n",
"### 1. Source data\n",
"\n",
"The tutorial source data consists of \n",
"\n",
"- `./cifs/` - folder containing CIFs (Crystallographic Information Files) downloaded from the MC3D (Materials Cloud 3D Structures Database);\n",
"- `./properties.csv` -- file containing the densities for each structure (in kg/m^3).\n",
"\n",
"### 2. Installing the toolkit\n",
"\n",
"In order to start the OPTIMADE API from this data, one should install the toolkit with:\n",
"\n",
"```bash\n",
"> pip install optimade-maker[tutorial]\n",
"```\n",
"\n",
"(note that the `[tutorial]` extra dependency just installs support for this jupyter notebook).\n",
"\n",
"### 3. Preparing the `optimade.yaml`\n",
"\n",
"To make the source data understandable by `optimade-maker`, one needs to prepare an `optimade.yaml` configuration file. The Pydantic schema of this configuration file is available in the `config.py` file of the library. For this tutorial, the file has already been prepared, but we'll explain it by blocks below.\n",
"\n",
"```yaml\n",
"config_version: 0.2.0\n",
"database_description: Simple DB\n",
"```\n",
"\n",
"The first two lines contain the version of the config file, and a string to describe the dataset. The config version, in most cases, should typically match the latest version defined by the schema (either checked from `config.py` or the main README).\n",
"\n",
"```yaml\n",
"entries:\n",
" - entry_type: structures\n",
" entry_paths:\n",
" - path: cifs\n",
" matches: [\"*.cif\"]\n",
"```\n",
"\n",
"The `entries:` block contains the description of OPTIMADE entries (e.g. structures, references, ...). In this tutorial, we are serving only crystal structures, which is described by the `entry_type: structures` line, and the `entry_paths:` describe where to look for the structures. Here, we only define a single path, the `cifs` directory, and look for files that match the `*.cif` pattern.\n",
"\n",
"```yaml\n",
" property_paths:\n",
" - path: properties.csv\n",
" property_definitions:\n",
" - name: density\n",
" title: Density\n",
" description: Density of the material\n",
" unit: kg/m^3\n",
" type: float\n",
"```\n",
"\n",
"This block (nested inside `entry_type: structures`) shows which file contains the custom properties, and contains its metadata.\n",
"\n",
"### 4. Starting the OPTIMADE API\n",
"\n",
"We are now ready to start the OPTIMADE API from this dataset. To do this, run the `optimake` CLI in a separate terminal in the tutorial folder:\n",
"\n",
"```bash\n",
"> optimake serve .\n",
"2026-04-20 17:03:03 INFO optimade-maker: optimade.jsonl doesn't exist. Converting archive.\n",
"Parsing structures files: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 206.09it/s]\n",
"Constructing OPTIMADE structures entries: 3it [00:00, 1847.44it/s]\n",
"Parsing property files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 365.33it/s]\n",
"2026-04-20 17:03:04 INFO optimade-maker: Preparing to start the API...\n",
"2026-04-20 17:03:04 INFO optimade-maker: Using the MongoMock backend.\n",
"INFO: [optimade] Using: Mock MongoDB (mongomock) @ localhost:27017\n",
"2026-04-20 17:03:04 INFO optimade-maker: Populating the database...\n",
"2026-04-20 17:03:04 INFO optimade-maker: Inserted 3 rows from the JSONL file\n",
"2026-04-20 17:03:04 INFO optimade-maker: Starting the API\n",
"INFO: [optimade] Loaded settings from /home/kristjan/.optimade.json\n",
"INFO: Started server process [1752662]\n",
"INFO: Waiting for application startup.\n",
"INFO: Application startup complete.\n",
"INFO: Uvicorn running on http://127.0.0.1:5000 (Press CTRL+C to quit)\n",
"```\n",
"\n",
"Note, for other functionality and options of the CLI, see `optimake --help`.\n",
"\n",
"### 5. Querying the API through Python\n",
"\n",
"While the API is running in the background, you can query it according to the OPTIMADE specification (https://www.optimade.org/specification). The following cells show a minimal example on how to do this with Python (converting the structures to ASE (Atomic Simulation Environment) format):"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "c6764a32",
"metadata": {},
"outputs": [],
"source": [
"import requests\n",
"from optimade.adapters import Structure\n",
"\n",
"BASE_URL = \"http://127.0.0.1:5000\"\n",
"\n",
"\n",
"def get_structures(filter_):\n",
" response = requests.get(f\"{BASE_URL}/structures?filter={filter_}\")\n",
" response.raise_for_status()\n",
" return [Structure(entry).as_ase for entry in response.json()[\"data\"]]"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "af1a9742",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Atoms(symbols='BaTiO3', pbc=True, cell=[3.9745754582274, 3.9745754582274, 3.9745754582274])\n",
"Atoms(symbols='Sr2C12', pbc=True, cell=[[4.3196827147571, 0.0, 0.0], [-2.1598413573785002, 3.740954967258099, 0.0], [0.0, 0.0, 9.7058202670626]])\n"
]
}
],
"source": [
"# OPTIMADE query: Get all structures with more than 1 element\n",
"for structure in get_structures(\"nelements > 1\"):\n",
" print(structure)"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "885faa54",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Atoms(symbols='Ba', pbc=True, cell=[[4.22762646696287, 0.0, 0.0], [-1.4092088223209558, 3.9858444574842284, 0.0], [-1.4092088223209558, -1.9929222287421124, 3.4518425557147467]])\n",
"Atoms(symbols='BaTiO3', pbc=True, cell=[3.9745754582274, 3.9745754582274, 3.9745754582274])\n"
]
}
],
"source": [
"# OPTIMADE query: filter based on the custom density property\n",
"# Note that the default prefix for custom properties is \"_optimake_\" for optimade-maker\n",
"for structure in get_structures(\"_optimake_density > 3800\"):\n",
" print(structure)"
]
},
{
"cell_type": "markdown",
"id": "d3a960cb",
"metadata": {},
"source": [
"### 6. Using the API with an external client\n",
"\n",
"The local API can also be queried with existing clients, such as the Materials Cloud OPTIMADE Client (https://optimadeclient.materialscloud.io), which allows to connect to a custom API URL.\n",
"\n",
"To try it out, while the API is running in the background, open: https://optimadeclient.materialscloud.io/?base_url=http://127.0.0.1:5000"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "optimade-maker",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.12"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ version_scheme = "post-release"
pymatgen = ["pymatgen >= 2023.9"]
aiida = ["aiida-core ~= 2.6"]
ingest = ["optimade-maker[pymatgen,aiida]"]
tutorial = ["jupyterlab"]
tests = ["pytest~=8.3", "pytest-cov~=6.0", "numpy >= 1.22, < 3"]
dev = ["optimade-maker[tests]", "ruff", "pre-commit", "mypy"]

Expand Down
Loading
Loading