Skip to content

Stan using the same seed=1234 | strange performance tests #3

@serban-nicusor-toptal

Description

@serban-nicusor-toptal
method = sample (Default)
  sample
    num_samples = 1000 (Default)
    num_warmup = 1000 (Default)
    save_warmup = 0 (Default)
    thin = 1 (Default)
    adapt
      engaged = 1 (Default)
      gamma = 0.050000000000000003 (Default)
      delta = 0.80000000000000004 (Default)
      kappa = 0.75 (Default)
      t0 = 10 (Default)
      init_buffer = 75 (Default)
      term_buffer = 50 (Default)
      window = 25 (Default)
    algorithm = hmc (Default)
      hmc
        engine = nuts (Default)
          nuts
            max_depth = 10 (Default)
        metric = diag_e (Default)
        metric_file =  (Default)
        stepsize = 1 (Default)
        stepsize_jitter = 0 (Default)
id = 0 (Default)
data
  file = XXX
init = 2 (Default)
random
  seed = 1234
output
  file = XXX
  diagnostic_file =  (Default)
  refresh = 100 (Default)

Tests I found during Jenkins runs that give strange performance reports on the same machine in subsequent runs.

PrevBuildURI90% | CurrentBuildURI90% | RelativeDiff | RelativeDiff(%)

286               40526               40240           14069.93%
example-models/bugs_examples/vol1/pump/pump.data.R
---
443               42348               41905            9459.37%
example-models.bugs_examples.vol1.dyes.dyes  
---
23237                 533              -22704             -97.71%
example-models.bugs_examples.vol1.equiv.equiv  
---
8302               39174               30872             371.86%
example-models.bugs_examples.vol1.leuk.leuk 
---
7807                2274               -5533             -70.87%
example-models.bugs_examples.vol1.mice.mice 
---
163                 365                 202             123.93%
example-models.bugs_examples.vol2.beetles.beetles_cloglog   
---
7807               39129               31322             401.20%
example-models.bugs_examples.vol1.mice.mice
---
23237                 506              -22731             -97.82%
example-models.bugs_examples.vol1.equiv.equiv
---
 153              114870              114717           74978.43%
example-models.bugs_examples.vol2.beetles.beetles_probit

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions