Pydantic v2 migration #1415

ml-evs · 2025-11-11T01:22:10Z

Supersedes #1398 and #1285 which were stuck in merge/rebase hell

codecov · 2025-11-11T02:03:02Z

Codecov Report

❌ Patch coverage is 87.62712% with 73 lines in your changes missing coverage. Please review.
✅ Project coverage is 79.39%. Comparing base (4ed958f) to head (5cd6855).

Files with missing lines	Patch %	Lines
pydatalab/src/pydatalab/models/utils.py	79.01%	17 Missing ⚠️
pydatalab/src/pydatalab/routes/v0_1/items.py	73.17%	11 Missing ⚠️
pydatalab/src/pydatalab/models/traits/__init__.py	88.50%	10 Missing ⚠️
pydatalab/src/pydatalab/models/cells.py	87.03%	7 Missing ⚠️
pydatalab/src/pydatalab/models/people.py	81.25%	6 Missing ⚠️
...datalab/src/pydatalab/models/traits/collectable.py	83.33%	6 Missing ⚠️
pydatalab/src/pydatalab/config.py	92.50%	3 Missing ⚠️
pydatalab/src/pydatalab/models/entries.py	86.66%	2 Missing ⚠️
...datalab/src/pydatalab/models/starting_materials.py	92.59%	2 Missing ⚠️
pydatalab/src/pydatalab/routes/v0_1/info.py	92.59%	2 Missing ⚠️
... and 7 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1415      +/-   ##
==========================================
- Coverage   80.04%   79.39%   -0.65%     
==========================================
  Files          70       71       +1     
  Lines        4836     4946     +110     
==========================================
+ Hits         3871     3927      +56     
- Misses        965     1019      +54

Files with missing lines	Coverage Δ
pydatalab/src/pydatalab/apps/chat/blocks.py	`75.19% <100.00%> (ø)`
pydatalab/src/pydatalab/apps/xrd/blocks.py	`90.27% <100.00%> (ø)`
pydatalab/src/pydatalab/backups.py	`62.96% <100.00%> (ø)`
pydatalab/src/pydatalab/blocks/base.py	`94.11% <100.00%> (ø)`
pydatalab/src/pydatalab/login.py	`84.12% <100.00%> (ø)`
pydatalab/src/pydatalab/main.py	`64.86% <100.00%> (ø)`
pydatalab/src/pydatalab/models/__init__.py	`100.00% <100.00%> (ø)`
pydatalab/src/pydatalab/models/blocks.py	`100.00% <100.00%> (ø)`
pydatalab/src/pydatalab/models/collections.py	`94.11% <100.00%> (-0.62%)`	⬇️
pydatalab/src/pydatalab/models/equipment.py	`100.00% <100.00%> (ø)`
... and 22 more

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

ml-evs · 2025-11-11T02:08:06Z

pydatalab/tests/server/test_samples.py

+    #! "relationships" are not in the response because of "ItemModel = ITEM_MODELS[doc["type"]]", need to find a way to fix this.
+    # assert (
+    #     len([d for d in response.json["item_data"]["relationships"] if d["type"] == "collections"])
+    #     == 1
+    # )


@BenjaminCharmes

ml-evs · 2025-11-11T02:09:24Z

pydatalab/src/pydatalab/mongo.py

+    global ITEMS_FTS_FIELDS
+
+    if not ITEMS_FTS_FIELDS:
+        ITEMS_FTS_FIELDS = get_items_fts_fields()
+
+    if not ITEMS_FTS_FIELDS:
+        raise ValueError("Cannot create text indices: no fields available for full-text search")


This is super error prone and hard to test -- better to just call the function everywhere

cypress · 2025-11-11T02:23:50Z

datalab Run #4085

Run Properties: Passed #4085 • 026dc170f9 ℹ️: Merge 5cd68559d2d921d590bb8eca41563850bac67cc5 into 4ed958ffbe1c3e89ee6d7fc23f1a...

Project	`datalab`
Branch Review	`ml-evs/bump-pydantic-from-scratch`
Run status	`Passed #4085`
Run duration	`07m 48s`
Commit	`026dc170f9 ℹ️: Merge 5cd68559d2d921d590bb8eca41563850bac67cc5 into 4ed958ffbe1c3e89ee6d7fc23f1a...`
Committer	`Matthew Evans`
View all properties for this run ↗︎

Test results
Failures	`0`
Flaky	`0`
Pending	`0`
Skipped	`0`
Passing	`354`
View all changes introduced in this branch ↗︎

Remove accidental repo

Rework ITEMS_FTS_FIELDS lookup

Chatblock patch; set default values in block

- Reorder imports for better isolation - Additional test cases for schema functionality

BenjaminCharmes

LGTM, everything working great ! 👍

be-smith · 2025-11-14T16:50:15Z

I have played around doing everything I can think of and seems to be working well for me so far!

ml-evs force-pushed the ml-evs/bump-pydantic-from-scratch branch from b2abba0 to 312bcb6 Compare November 11, 2025 01:25

ml-evs changed the base branch from ml-evs/bump-pydantic-redo to main November 11, 2025 01:25

ml-evs force-pushed the ml-evs/bump-pydantic-from-scratch branch 3 times, most recently from 2895a36 to 45fca0a Compare November 11, 2025 02:00

ml-evs commented Nov 11, 2025

View reviewed changes

ml-evs force-pushed the ml-evs/bump-pydantic-from-scratch branch from 45fca0a to 9818765 Compare November 11, 2025 02:10

ml-evs force-pushed the ml-evs/bump-pydantic-from-scratch branch from 9818765 to 41e8ea1 Compare November 11, 2025 09:47

ml-evs marked this pull request as ready for review November 11, 2025 09:48

ml-evs requested review from BenjaminCharmes and jdbocarsly as code owners November 11, 2025 09:48

BenjaminCharmes and others added 13 commits November 12, 2025 22:54

'admin' changes -- update dependency versions across various files

6961422

Refactor server config for pydantic v2

9ea6f1d

Refactor traits into own submodule

7560046

Remove accidental repo

Refactor model imports to avoid circular deps

91c49c6

Rework all models into pydantic v2 amenable form

25c3274

Load FTS fields for MongoDB from model schemas directly

4bc47ea

Rework ITEMS_FTS_FIELDS lookup

Misc pydantic v2 bumps around code

97ed5c4

Update various routes for pydantic v2

a807f59

Update various blocks for pydantic v2

8524773

Chatblock patch; set default values in block

Various test updates needed for pydantic v2:

60ecd4d

- Reorder imports for better isolation - Additional test cases for schema functionality

FIXME: leftover missing test case that cannot be ported

8e3ad85

Patch constituent table for new default values

28d35ea

Defend against name being unset in database

28e43c3

ml-evs force-pushed the ml-evs/bump-pydantic-from-scratch branch from 41e8ea1 to 28e43c3 Compare November 12, 2025 22:54

ml-evs requested a review from be-smith November 12, 2025 22:54

Fix auth routes with careful handling of immutable ID

5cd6855

BenjaminCharmes approved these changes Nov 14, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Pydantic v2 migration #1415

Pydantic v2 migration #1415

Uh oh!

ml-evs commented Nov 11, 2025 •

edited

Loading

Uh oh!

codecov bot commented Nov 11, 2025 •

edited

Loading

Uh oh!

ml-evs Nov 11, 2025

Uh oh!

ml-evs Nov 11, 2025

Uh oh!

ml-evs Nov 11, 2025

Uh oh!

cypress bot commented Nov 11, 2025 •

edited

Loading

Uh oh!

BenjaminCharmes left a comment

Uh oh!

be-smith commented Nov 14, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Pydantic v2 migration #1415

Are you sure you want to change the base?

Pydantic v2 migration #1415

Uh oh!

Conversation

ml-evs commented Nov 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Nov 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

ml-evs Nov 11, 2025

Choose a reason for hiding this comment

Uh oh!

ml-evs Nov 11, 2025

Choose a reason for hiding this comment

Uh oh!

ml-evs Nov 11, 2025

Choose a reason for hiding this comment

Uh oh!

cypress bot commented Nov 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

datalab Run #4085

Uh oh!

BenjaminCharmes left a comment

Choose a reason for hiding this comment

Uh oh!

be-smith commented Nov 14, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ml-evs commented Nov 11, 2025 •

edited

Loading

codecov bot commented Nov 11, 2025 •

edited

Loading

cypress bot commented Nov 11, 2025 •

edited

Loading