Skip to content

caching the same repository avoid reload multiple time on the same build #340

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 15 commits into
base: main
Choose a base branch
from

Conversation

arechavarria
Copy link

@arechavarria arechavarria commented Sep 12, 2024

PR Details

Description

It has been observed that the library makes more repository queries than necessary, which is redundant unless the libraries are stored in different remote repositories. However, each repository only needs to be queried once, and its content should be cached and associated with the execution to avoid impacting future or concurrent runs. This will shorten the initialization time for pipelines using the library

How Has This Been Tested

Tests are conducted on different types of pipelines, including multibranch and simple pipelines, to analyze the previous behavior. The results show a significant reduction in pipeline initialization time

  • Java 21
  • Jenkins 2.462.2

This behavior only affects how remote repositories are queried when the libraries come from a source control management (SCM) system.

Types of Changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Added Unit Testing
  • Docs change / refactoring / dependency upgrade
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist

  • I have added the appropriate label for this PR
  • If necessary, I have updated the documentation accordingly.
  • All new and existing tests passed.

@piglowskilukasz
Copy link

I tested it with [2.492.1] and Java 21 but getting this exception when JTE library are loading:

[JTE] Pipeline Configuration Modifications
[JTE] Loading scm: git https://github.com/xyx/test-project.git from cache
java.io.IOException: Closed
at PluginClassLoader for github-branch-source//org.jenkinsci.plugins.github_branch_source.GitHubSCMFile.checkOpen(GitHubSCMFile.java:83)
at PluginClassLoader for github-branch-source//org.jenkinsci.plugins.github_branch_source.GitHubSCMFile.metadata(GitHubSCMFile.java:108)
at PluginClassLoader for github-branch-source//org.jenkinsci.plugins.github_branch_source.GitHubSCMFile.type(GitHubSCMFile.java:166)
at PluginClassLoader for scm-api//jenkins.scm.api.SCMFile.getType(SCMFile.java:265)
at PluginClassLoader for scm-api//jenkins.scm.api.SCMFile.exists(SCMFile.java:224)
at jenkins.scm.api.SCMFile$exists.call(Unknown Source)
at PluginClassLoader for templating-engine//org.boozallen.plugins.jte.util.FileSystemWrapper.getFileContents(FileSystemWrapper.groovy:41)
at org.boozallen.plugins.jte.util.FileSystemWrapper$getFileContents$0.call(Unknown Source)
at PluginClassLoader for templating-engine//org.boozallen.plugins.jte.job.MultibranchTemplateFlowDefinition.getTemplate(MultibranchTemplateFlowDefinition.groovy:90)
at org.boozallen.plugins.jte.job.MultibranchTemplateFlowDefinition$getTemplate$0.call(Unknown Source)
at PluginClassLoader for templating-engine//org.boozallen.plugins.jte.init.PipelineTemplateResolver.resolve(PipelineTemplateResolver.groovy:45)
at org.boozallen.plugins.jte.init.PipelineTemplateResolver$resolve.call(Unknown Source)
at PluginClassLoader for templating-engine//org.boozallen.plugins.jte.job.TemplateFlowDefinition.create(TemplateFlowDefinition.groovy:80)
at PluginClassLoader for workflow-job//org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:311)
at hudson.model.ResourceController.execute(ResourceController.java:101)
at hudson.model.Executor.run(Executor.java:446)
[withMaven] downstreamPipelineTriggerRunListener - Failure to introspect build steps: java.io.IOException: xyz/test-project/discover #67 did not yet start
[withMaven] downstreamPipelineTriggerRunListener - Failure to introspect build steps: java.io.IOException: xyz/test-project/discover #67 did not yet start
Finished: FAILURE

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants