Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unexpected zinc cache invalidation after sbt remote cache hit #966

Open
bjaglin opened this issue Mar 9, 2021 · 1 comment
Open

Unexpected zinc cache invalidation after sbt remote cache hit #966

bjaglin opened this issue Mar 9, 2021 · 1 comment

Comments

@bjaglin
Copy link

bjaglin commented Mar 9, 2021

steps

Unfortunately, I haven't been able to extract a repro for this, as this issue is transient (but consistent for a given remote cache artifact). I decided to open a ticket to start adding information along my investigation to see if I can find a pattern.

Context

  • Zulu OpenJDK 11.0.8 on Linux
  • scala 2.13.4 and 2.13.5
  • sbt 1.4.6 and 1.4.8
  • remote caching activated with an adhoc remoteCacheId
    • remote cache artifact producer & consumer is the same machine (CI Jenkins slave running sbt in Docker)
    • problem is repeatable on another machine consuming the artifact

problem

Zinc runs incrementally after a remote cache artifact hit on IntegrationTest, detecting a change in an untouched marker trait trait ConnectTransactor defined in Compile (on which Zinc had a cache hit after pulling the remote cache).

expectation

Zinc cache hit - "No changes"

notes

Here are incOptions.value.withApiDebug(true) logs:

[debug] [zinc] IncrementalCompile -----------
[debug] IncrementalCompile.incrementalCompile
[debug] previous = Stamps for: 181 products, 50 sources, 27 libraries
[debug] current source = Set(${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/controllers/handlers/CreateManagedBuyerSeatForEntityHandlerSpec.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/controllers/handlers/UpdateBuyerSeatFeeHandlerSpec.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/cucumber/addUrlBasedConversion/AddUrlBasedConversionTestRunner.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/cucumber/ensureVisitsConversionExistsForBuyerPixelAndDomain/EnsureVisitsConversionExistsForBuyerPixelAndDomainSpec.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/cucumber/removeUrlBasedConversion/RemoveUrlBasedConversionSpec.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/controllers/handlers/CreateIoBuyerSeatForEntityHandlerSpec.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/controllers/handlers/TransitionIoBuyerSeatToManagedHandlerSpec.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/controllers/handlers/CreateAdvertiserForPublisherHandlerSpec.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/controllers/handlers/ActivateFeatureForSeatHandlerSpec.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/controllers/handlers/TransitionSelfServedBuyerSeatToManagedHandlerSpec.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/scalatest/Fixtures.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/cucumber/removeUrlBasedConversion/RemoveUrlBasedConversionTestRunner.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/controllers/handlers/CreateSelfServedBuyerSeatForEntityHandlerSpec.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/controllers/handlers/CreateAgencyForPublisherHandlerSpec.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/controllers/handlers/DeleteAdvertiserHandlerSpec.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/models/BuyerSeat.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/controllers/listeners/CleanOutdatedConversionsStatisticsListenerSpec.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/repositories/BuyerSeatRepositorySpec.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/cucumber/OneAdvertiserAppPerCucumberScenario.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/controllers/handlers/DeleteAgencyHandlerSpec.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/controllers/handlers/RenameAgencyHandlerSpec.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/cucumber/addEventConversionToAdvertiser/AddEventConversionToAdvertiserSpec.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/controllers/handlers/MarkClientMsaFilledForSeatHandlerSpec.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/controllers/handlers/EnableBuyerSeatHandlerSpec.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/scalatest/OneAdvertiserAppPerSuite.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/controllers/handlers/MarkClientBillingDetailsFilledForSeatHandlerSpec.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/cucumber/setWebsitesDomainNamesForPixelsAnonymously/SetWebsitesDomainNamesForPixelsAnonymouslySpec.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/controllers/handlers/DisableBuyerSeatHandlerSpec.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/controllers/handlers/TransitionIoBuyerSeatToSelfServedHandlerSpec.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/repositories/StandardEventRepositorySpec.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/cucumber/ensureVisitsConversionExistsForBuyerPixelAndDomain/EnsureVisitsConversionExistsForBuyerPixelAndDomainTestRunner.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/controllers/handlers/RenameAdvertiserHandlerSpec.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/controllers/handlers/SharePixelWithBuyerHandlerSpec.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/scalatest/AdvertiserSpec.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/controllers/handlers/MarkTeadsFinanceApprovalFilledForSeatHandlerSpec.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/repositories/AdvertiserRepositoryFixtures.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/cucumber/WriteThroughDatabaseCache.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/scalatest/AdvertiserFixtures.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/models/Advertiser.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/controllers/handlers/TransitionManagedBuyerSeatToSelfServedHandlerSpec.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/cucumber/StepCommon.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/cucumber/addEventConversionToAdvertiser/AddEventConversionToAdvertiserTestRunner.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/controllers/handlers/DeactivateFeatureForSeatHandlerSpec.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/scalatest/PixelServiceSpec.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/scalatest/ConversionServiceSpec.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/models/Agency.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/cucumber/setWebsitesDomainNamesForPixelsAnonymously/TestRunner.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/repositories/ConversionRepositorySpec.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/cucumber/addUrlBasedConversion/AddUrlBasedConversionSpec.scala, ${BASE}/domains/advertiser/advertiser-commands/src/it/scala/tv/teads/domains/advertiser/commands/services/repositories/PixelRepositorySpec.scala)
[debug] [diff] Detected a change in a public API (tv.teads.domains.advertiser.commands.ConnectTransactor):
[debug] [diff] abstract trait tv.teads.domains.advertiser.commands.ConnectTransactor extends java.lang.this#Object with scala.this#Any {
[debug] [diff] 
[debug] [diff] 
[debug] [diff] 
[debug] [diff] }
[debug] [diff] object tv.teads.domains.advertiser.commands.ConnectTransactor extends  {
[debug] [diff] 
[debug] [diff] 
[debug] [diff] 
[debug] [diff] }
[debug] [diff] Detected a change in a public API (tv.teads.domains.advertiser.commands.AdvertiserCommandsDependencies):
[debug] [diff] abstract trait tv.teads.domains.advertiser.commands.AdvertiserCommandsDependencies extends tv.teads.domains.commons.this#CommonDependencies with java.lang.this#Object with scala.this#Any {
[debug] [diff] 
[debug] [diff] 
[debug] [diff] 
[debug] [diff] }
[debug] [diff] object tv.teads.domains.advertiser.commands.AdvertiserCommandsDependencies extends  {
[debug] [diff] 
[debug] [diff] 
[debug] [diff] 
[debug] [diff] }
[debug] > initialChanges = InitialChanges(Changes(added = Set(), removed = Set(), changed = Set(), unmodified = ...),Set(),Set(),API Changes: Set(NamesChange(tv.teads.domains.advertiser.commands.ConnectTransactor,ModifiedNames(changes = UsedName(finalize,[Default]))), NamesChange(tv.teads.domains.advertiser.commands.AdvertiserCommandsDependencies,ModifiedNames(changes = UsedName(finalize,[Default])))))
[debug] The tv.teads.domains.advertiser.commands.ConnectTransactor has the following regular definitions changed:
[debug] 	UsedName(finalize,[Default]).
...

I have tried to diff the text format representation of inc_compile.bin for Compile & IntegrationTest before and after the recompilation, but I haven't found anything interesting (the diff is hard to read as order changes). I am not including it here as it's massive, but happy to provide extracts.

@bjaglin
Copy link
Author

bjaglin commented Mar 9, 2021

I have compared the relevant part of api_companions.bin (both taken from the remote cache artifacts) between Compile and IntegrationTest, and it confirms that the stamping differs.

I still don't understand why though. Could it be related to https://github.com/AdoptOpenJDK/openjdk-jdk11/blob/19fb8f93c59dfd791f62d41f332db9e306bc1422/src/java.base/share/classes/java/lang/Object.java#L557 ? A classpath change between 1614849670371 & 1614849586071 that would either expose the annotation or not? The same JDK was used for both compilations. The fact that the Compile timestamp comes ater the IntegrationTest one is suspicious.

-  internal {
+  external {
     key: "tv.teads.domains.advertiser.commands.ConnectTransactor"
     value {
-      compilationTimestamp: 1614849670371
+      compilationTimestamp: 1614849586071
       name: "tv.teads.domains.advertiser.commands.ConnectTransactor"
       api {
         classApi {
           name: "tv.teads.domains.advertiser.commands.ConnectTransactor"
           access {
             public {
             }
           }
           modifiers {
             flags: 1
           }
           selfType {
             emptyType {
             }
           }
           structure {
             parents {
               projection {
                 id: "Object"
                 prefix {
                   singleton {
                     path {
                       components {
                         id {
                           id: "java"
                         }
                       }
                       components {
                         id {
                           id: "lang"
                         }
                       }
                       components {
                         this {
                         }
                       }
                     }
                   }
                 }
               }
             }
             parents {
               projection {
                 id: "Any"
                 prefix {
                   singleton {
                     path {
                       components {
                         id {
                           id: "scala"
                         }
                       }
                       components {
                         this {
                         }
                       }
                     }
                   }
                 }
               }
             }
           }
           savedAnnotations: "scala.native"
           savedAnnotations: "jdk.internal.HotSpotIntrinsicCandidate"
           topLevel: true
         }
         objectApi {
           name: "tv.teads.domains.advertiser.commands.ConnectTransactor"
           access {
             public {
             }
           }
           modifiers {
           }
           definitionType: MODULE
           selfType {
             emptyType {
             }
           }
           structure {
           }
           topLevel: true
         }
       }
-      apiHash: 176768709
+      apiHash: -488308903
       nameHashes {
         name: "notify"
         hash: 1514289040
       }
       nameHashes {
         name: "wait"
         hash: 1963066972
       }
       nameHashes {
         name: "equals"
         hash: 1626438048
       }
       nameHashes {
         name: "asInstanceOf"
         hash: -2008440760
       }
       nameHashes {
         name: "synchronized"
         hash: 1581358093
       }
       nameHashes {
         name: "notifyAll"
         hash: 287659096
       }
       nameHashes {
         name: "isInstanceOf"
         hash: -42141924
       }
       nameHashes {
         name: "ConnectTransactor"
         hash: 1351354997
       }
       nameHashes {
         name: "=="
         hash: 1290070777
       }
       nameHashes {
         name: "clone"
         hash: -1872329449
       }
       nameHashes {
         name: "toString"
         hash: -799817619
       }
       nameHashes {
         name: "!="
         hash: -1478124343
       }
       nameHashes {
         name: "getClass"
         hash: 1196203508
       }
       nameHashes {
         name: "ne"
         hash: -1383471943
       }
       nameHashes {
         name: "eq"
         hash: -491545741
       }
       nameHashes {
         name: "##"
         hash: 1393487132
       }
       nameHashes {
         name: "finalize"
-        hash: -245298853
+        hash: 1383774555
       }
       nameHashes {
         name: "hashCode"
         hash: -2114247454
       }
-      extraHash: 176768709
+      extraHash: -488308903
     }
   }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant