Build: better support for empty default branch #12462

stsewd · 2025-09-04T21:49:17Z

The basic idea is to have the value of latest in sync with the value from the default branch, which we already do. But, in case the default branch is left empty, we were falling back to master, which is just a guess of the default, so instead of guessing the value, we just don't update it if we don't know its value. When cloning the repo, we have access to the default branch, so that's when we update latest for those projects.

Note this is only relevant for projects that aren't linked to a remote repository, since for those we know the default branch. So, when we try to build the latest version of a project that doesn't have a default branch, we:

Omit the ref from the fetch command.
Use the default branch from the repository for the checkout (we could skip this step, but since we have a new feature that keeps the files from a previous clone, we need to do a checkout to make sure we are building from the given branch/commit).
Update the identifier from latest to the default branch we detected from the cloned repository.

There's still the problem about latest's identifier being out of sync when the default branch changes, but that's fixed after a new build, and this isn't a new problem (now users aren't blocked at least), and changing the default branch of a repo isn't something users do that often, and this only affects projects that aren't linked to a repository, so I think we are okay with that for now.

This replaces #10927
Closes #12231

read-the-docs-community · 2025-09-17T15:45:33Z

Documentation build overview

📚 dev | 🛠️ Build #29616556 | 📁 Comparing 8701e3c against latest (d9570fb)

🔍 Preview build

Show files changed (6 files in total): 📝 6 modified | ➕ 0 added | ➖ 0 deleted

File	Status
contribute.html	📝 modified
install.html	📝 modified
search-integration.html	📝 modified
style-guide.html	📝 modified
design/embed-api.html	📝 modified
design/theme-context.html	📝 modified

read-the-docs-community · 2025-09-17T15:46:13Z

Documentation build overview

📚 docs | 🛠️ Build #29616557 | 📁 Comparing 8701e3c against latest (d9570fb)

🔍 Preview build

Show files changed (4 files in total): 📝 4 modified | ➕ 0 added | ➖ 0 deleted

File	Status
build-customization.html	📝 modified
support.html	📝 modified
about/index.html	📝 modified
tutorial/index.html	📝 modified

stsewd · 2025-09-17T22:20:27Z

readthedocs/projects/models.py

+        latest_version_identifier = (
+            self.versions.filter(slug=LATEST, machine=True)
+            .values_list("identifier", flat=True)
+            .first()
+        )


This is introducing an extra query for each time latest is returned in a response, since we now rely on latest having the correct value of the default branch (similar to get_original_stable_version).

humitos · 2025-09-22T09:53:23Z

What happens if the default branch is main, we detect and save than, and then the user changes the default branch in GitHub to dev? Do we detect that change?

ericholscher · 2025-10-01T09:32:11Z

Use the default branch from the repository for the checkout (we could skip this step, but since we have a new feature that keeps the files from a previous clone, we need to do a checkout to make sure we are building from the given branch/commit).

The user who needed this isn't using it, so we can probably remove the feature flag for now, and simplify this logic.

ericholscher

This looks reasonable, but is a pretty big change to the build process. Hopefully it won't impact too much, but I do worry a bit about how much we're changing here for some edge cases, but I do like the idea of not specifying a branch checkout on initial clone to work around broken latest, which is a bad UX for users.

readthedocs/projects/models.py

readthedocs/vcs_support/backends/git.py

stsewd · 2025-10-09T22:01:36Z

What happens if the default branch is main, we detect and save than, and then the user changes the default branch in GitHub to dev? Do we detect that change?

Yep, if default_branch isn't set we never reference latest for its branch name when cloning, but we update latest to the new default branch after building, so all links work properly.

Ref #12462 (comment)

agjohnson

Looks good, everything here made sense to me 👍

ericholscher · 2025-10-30T07:23:49Z

I think I'm mostly OK with this change, since it is pretty bad UX for users to get stuck here (I just got stuck with a similar issue and it was super annoying...).

@humitos Any more thoughts before we move forward?

humitos

I reviewed this again and I still have the same thoughts.

I don't still understand why we need to keep the Project.default_branch in sync with the "default branch cloned when calling git clone on a repository". What's the reason for this?

My point is "if the Project.default_branch==None we don't need to do anything special when cloning the repository", and that's all. Why we can't do that and avoid all this complexity to keep this branch in sync?

humitos · 2025-11-03T15:27:41Z

readthedocs/doc_builder/director.py

+        # We can skip the checkout step since we just cloned the repository,
+        # and the default branch is already checked out.
+        if not is_latest_without_default_branch:
+            identifier = self.data.build_commit or self.data.version.identifier
+            log.info("Checking out.", identifier=identifier)
+            self.vcs_repository.checkout(identifier)


I would remove this conditional here and always execute this command, for simplification of the workflow. It's better to be explicit here as well.

It seems this won't be executed only at most, once, per project. Starting from the second time, the project will have default_branch set and we will execute this all the times.

I would simplify this and execute this command always to reduce complexity here.

We don't have the name of the branch for those projects, omitting this step is the purpose of this PR.

It seems this won't be executed only at most, once, per project. Starting from the second time, the project will have default_branch set and we will execute this all the times.

No, default_branch is kept empty for those projects.

humitos · 2025-11-03T15:28:45Z

readthedocs/doc_builder/director.py

+        if is_latest_without_default_branch:
+            self.data.default_branch = self.data.build_director.vcs_repository.get_default_branch()
+            log.info(
+                "Default branch for the repository detected.",
+                default_branch=self.data.default_branch,
+            )


I don't understand why we need to save this in the database. Why we are not always cloning the repository without the default branch set if it will clone the correct one?

I think you are confusing the default_branch from the project model. This is the default branch of the repository, which is used to keep the identifier of latest in sync, not the attribute from the project model.

humitos · 2025-11-03T15:33:46Z

readthedocs/projects/tasks/builds.py

+    # Default branch for the repository.
+    # Only set when building the latest version, and the project
+    # doesn't have an explicit default branch.
+    default_branch: str | None = None


Should we call it more explicitly like vcs_default_branch?

humitos · 2025-11-03T15:38:50Z

readthedocs/vcs_support/backends/git.py

+        # and the user hasn't defined a default branch (which means we need to use the default branch from the repo).
+        omit_remote_reference = self.version.is_machine_latest and not self.project.default_branch
+        if not omit_remote_reference:
+            remote_reference = self.get_remote_fetch_refspec()


My understanding is that if we return None from this method get_remote_fetch_refspec when LATEST and not project.default_branch at https://github.com/readthedocs/readthedocs.org/blob/main/readthedocs/vcs_support/backends/git.py#L102-L115 we shouldn't need this PR and keep the project.default_branch in sync.

I guess my reply is the same as #12462 (comment).

stsewd · 2025-11-03T17:52:06Z

I don't still understand why we need to keep the Project.default_branch in sync with the "default branch cloned when calling git clone on a repository". What's the reason for this?

My point is "if the Project.default_branch==None we don't need to do anything special when cloning the repository", and that's all. Why we can't do that and avoid all this complexity to keep this branch in sync?

This is needed to keep all the links for the version working. Latest is an alias in our code, so it needs to point to a valid version, we can't leave it just empty.

stsewd added 5 commits September 4, 2025 16:48

Build: skip checkout for latest when no default version is given

dbae5b4

Merge branch 'main' into skip-checkout-for-latest

0074e5e

Fix tests

24854de

Keep latest in sync with default branch

9572098

Format

082b1be

stsewd added 5 commits September 17, 2025 13:16

Fix tests

77ae232

Test

f636eb9

Fix tests

8701e3c

Merge branch 'main' into skip-checkout-for-latest

9903db6

Format

335fff8

stsewd commented Sep 17, 2025

View reviewed changes

stsewd marked this pull request as ready for review September 17, 2025 22:20

stsewd requested a review from a team as a code owner September 17, 2025 22:20

stsewd requested a review from agjohnson September 17, 2025 22:20

auto-assign bot assigned stsewd Sep 17, 2025

stsewd requested a review from ericholscher September 17, 2025 22:20

stsewd mentioned this pull request Sep 17, 2025

Build: skip checkout step when building latest without an explicit default branch #10927

Closed

stsewd added 3 commits September 17, 2025 17:25

We don't need this

c7e0087

Default branch should be resolved before checkout

c9e7138

Fix order again

1723c27

stsewd changed the title ~~Build: skip checkout for latest when no default version is given~~ Build: better support for empty default branch Sep 17, 2025

ericholscher reviewed Oct 1, 2025

View reviewed changes

readthedocs/projects/models.py Show resolved Hide resolved

readthedocs/vcs_support/backends/git.py Show resolved Hide resolved

stsewd added 2 commits October 9, 2025 16:31

Merge branch 'main' into skip-checkout-for-latest

68b4405

Comment

722617c

Skip checkout

a3374b5

stsewd added a commit that referenced this pull request Oct 9, 2025

Remove DONT_CLEAN_BUILD feature flag

d80cf46

Ref #12462 (comment)

stsewd mentioned this pull request Oct 9, 2025

Remove DONT_CLEAN_BUILD feature flag #12519

Merged

stsewd added a commit that referenced this pull request Oct 27, 2025

Remove DONT_CLEAN_BUILD feature flag (#12519)

0694d42

Ref #12462 (comment)

agjohnson approved these changes Oct 29, 2025

View reviewed changes

stsewd added 2 commits October 30, 2025 11:43

Merge branch 'main' into skip-checkout-for-latest

23da415

Merge branch 'main' into skip-checkout-for-latest

882d98c

humitos reviewed Nov 3, 2025

View reviewed changes

Uh oh!

Build: better support for empty default branch #12462

Are you sure you want to change the base?

Build: better support for empty default branch #12462

Conversation

stsewd commented Sep 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

read-the-docs-community bot commented Sep 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Documentation build overview

Uh oh!

read-the-docs-community bot commented Sep 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Documentation build overview

Uh oh!

Choose a reason for hiding this comment

Uh oh!

humitos commented Sep 22, 2025

Uh oh!

ericholscher commented Oct 1, 2025

Uh oh!

ericholscher left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

stsewd commented Oct 9, 2025

Uh oh!

agjohnson left a comment

Choose a reason for hiding this comment

Uh oh!

ericholscher commented Oct 30, 2025

Uh oh!

humitos left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

stsewd Nov 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

stsewd commented Nov 3, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

stsewd commented Sep 4, 2025 •

edited

Loading

read-the-docs-community bot commented Sep 17, 2025 •

edited

Loading

read-the-docs-community bot commented Sep 17, 2025 •

edited

Loading

stsewd Nov 3, 2025 •

edited

Loading