Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build more complete hardware profile in NBench reports #107

Open
Aaronontheweb opened this issue May 1, 2016 · 6 comments
Open

Build more complete hardware profile in NBench reports #107

Aaronontheweb opened this issue May 1, 2016 · 6 comments

Comments

@Aaronontheweb
Copy link
Member

Ran into a fun issue on DotNetty Azure/DotNetty#95 where it looked like a change in the underlying hardware we're running NBench on affected the benchmark significantly with no explanation, thanks to our use of auto-scaling build agents on Windows Azure.

We really need to expand the hardware profile captured by SystemInfo and include much more detail about:

  1. The clock speeds of all CPUs, and model information if possible
  2. Total amount of available memory and any other details we can get about it
  3. BIOS information, data about the underlying hyper visor if there is one
  4. And more that I'm probably forgetting.

That way it'll be easier to tell if it was a code change or a hardware change that resulted in a divergence in a metric, and not cause the developers to panic when it's the latter.

@antondelsink
Copy link

antondelsink commented May 2, 2016

If you're open to a platform-specific option, WMI may yield enough info. An example from PowerShell:
Get-WmiObject -Class Win32_Processor | fl
or
Get-WmiObject -Class Win32_Processor | Get-Member
Some of the available data:

  • CurrentClockSpeed
  • L2CacheSize
  • L2CacheSpeed
  • L3CacheSize
  • L3CacheSpeed
  • NumberOfCores
  • NumberOfEnabledCore
  • NumberOfLogicalProcessors
  • Stepping,
    etc.

We can get WMI data via Microsoft.Management.Infrastructure, if you're open to that...

@Aaronontheweb
Copy link
Member Author

Aaronontheweb commented May 2, 2016

@antondelsink I'm open to platform-specific tooling here, although here's the catch: we have to make it switchable underneath so the core NBench library can appear to be platform-agnostic. This is an absolute must-have for supporting .NET Core in the future.

So here's what I propose: we create a provider model for SysInfo which will lazily load one of three possible SysInfo concrete implementations:

  1. The generalized, "safe" default one we have now;
  2. A Windows-specific WMI provider; and
  3. A to-be-announced Linux / Posix-specific provider.

Provider 1 is included in the default NBench library, and it's always the fallback in the event that implementations 2 or 3 fail to load or aren't available.

Implementations 2 and 3 will be implemented in external DLLs, so NBench.SysInfo.Windows and NBench.SysInfo.Linux perhaps.

At runtime, we'll detect the OS and Assembly.Load the appropriate implementation and then that that SysInfo object as the data provider going forward. If any part of that operation fails, fall back to Provider 1 and continue.

We'll also need to expand the fields on the SysInfo object to include this additional data too. Maybe it's better to just have a dictionary of properties we can populate rather than a bunch of static fields.

What do you think @antondelsink ?

@antondelsink
Copy link

Assembly.Load would work... also consider a Conditional on the reference to the library NBench.SysInfo.Windows so the decision is made at time of build, perhaps with the default build providing only the "safe" default SysInfo.

@antondelsink
Copy link

I'll put together a naïve NBench.SysInfo.Windows.WMI class in a new project under the same solution, but @Aaronontheweb tell me more about how you would like to load it. Perhaps from the current SysInfo.CreateInstance?

@antondelsink
Copy link

antondelsink commented May 3, 2016

@Aaronontheweb: take a look at #109; I've also added:

  • Get-WmiObject -Class Win32_PhysicalMemory
  • Get-WmiObject -Class Win32_BIOS

@antondelsink
Copy link

Hi @Aaronontheweb, still interested in this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants