New build system under Linux

[alexkalmuk]

To speed up the build process under Linux (because of llvm-core and other packages build from sources with Conan), I've found this hint #4481 (comment):

mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_CXX_FLAGS="-march=sandybridge -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer" \
  ..

cmake --build .

Is this still relevant? I'm trying to build like this, but I'm getting some errors.

[beckerhe]

Yes, that should work. You might need to install some dependencies though.

This is what we install on the CI (Ubuntu):

           sudo apt update &&                             \
           sudo apt install --yes --no-install-recommends \
           build-essential                                \
           clang-tidy-14                                  \
           cmake                                          \
           ccache                                         \
           libboost-dev                                   \
           libcapstone-dev                                \
           libgrpc++-dev                                  \
           libssh2-1-dev                                  \
           vulkan-validationlayers-dev                    \
           libz-dev                                       \
           llvm-dev                                       \
           protobuf-compiler-grpc                         \
           pkg-config                                     \
           libvulkan-volk-dev                             \
           libvulkan-dev                                  \
           libopengl-dev                                  \
           libglx-dev                                     \
           mesa-common-dev                                \
           qtbase5-dev                                    \
           libgtest-dev                                   \
           libgmock-dev                                   \
           git                                            \
           libprotobuf-dev

If you have specific error messages that you can't solve, feel free to post them here. I will try to help.

[alexkalmuk]

Thank you @beckerhe, my issue was in libvulkan-volk-dev. There is no such package in Ubuntu 20.04, so Findvolk.cmake failed. On ubuntu 22.04 it works.

By the way, this volk is just two files (.c and .h), maybe in that particular case it makes sense to get the release from their official repo from Github (for backward compatibility with older distributions). Or, did you consider to make Vulkan as optional when building Orbit?

[alexkalmuk]

@beckerhe It's built under ubuntu 22.04, thank you.

[beckerhe]

Thank you @beckerhe, my issue was in libvulkan-volk-dev. There is no such package in Ubuntu 20.04, so Findvolk.cmake failed. On ubuntu 22.04 it works.

By the way, this volk is just two files (.c and .h), maybe in that particular case it makes sense to get the release from their official repo from Github (for backward compatibility with older distributions). Or, did you consider to make Vulkan as optional when building Orbit?

You're right. Only Ubuntu 22.04+ works as dependencies in 20.04 are either too old or not available.

As far as I remember there are more things needed to compile the non-conan build on Ubuntu 20.04 than just Volk, but I didn't look into the details.

Making Vulkan optional is very much an option and shouldn't be too hard. But so far nobody had time to look into that. PRs are welcome. :-)

[alexkalmuk]

@beckerhe I've added WITH_VULKAN option into CMakeLists.txt to exclude Vulkan modules from build in case if the option set to OFF. But while building I'm getting errors such as:

Consolidate compiler generated dependencies of target IntegrationTestCommons
[ 40%] Building CXX object src/LinuxTracingIntegrationTests/CMakeFiles/IntegrationTestCommons.dir/IntegrationTestPuppet.cpp.o
/home/alex/git/orbit/src/LinuxTracingIntegrationTests/IntegrationTestPuppet.cpp:22:10: fatal error: VulkanTutorial/OffscreenRenderingVulkanTutorial.h: No such file or directory
   22 | #include "VulkanTutorial/OffscreenRenderingVulkanTutorial.h"
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
gmake[2]: *** [src/LinuxTracingIntegrationTests/CMakeFiles/IntegrationTestCommons.dir/build.make:90: src/LinuxTracingIntegrationTests/CMakeFiles/IntegrationTestCommons.dir/IntegrationTestPuppet.cpp.o] Error 1

The error is clear, because another Orbit's module tries to use something from Vulkan.

My question is: Would it make sense to put such places under some #ifdef ENABLE_VULKAN (defined in cmake through -DENABLE_VULKAN) to make Orbit more modular and independent on Vulkan?

By the way, you're right about other issues with 20.04. Even after I'd disabled Vulkan I've had issues with protobuf compilation. So disabling Vulkan wouldn't be a solution for 20.04 (but still might be an improvement to overall modularity).

[beckerhe]

My question is: Would it make sense to put such places under some #ifdef ENABLE_VULKAN (defined in cmake through -DENABLE_VULKAN) to make Orbit more modular and independent on Vulkan?

Yes, that sounds reasonable to me. It's what I would have proposed in that case as well!

[alexkalmuk]

OK, I'll give it a try. grep shows me multiple places when Vulkan is used as dependency, so I plan to take a look to these places and ask you if something is unclear :)

[alexkalmuk]

About building under Ubuntu 20.04. It looks that due to some old packages and versions incompatibility (including boost, protobuf, compiler, etc.) it's a big effort to make Orbit working right now. For instance, after manually building and installing googletest and protobuf, I'm still getting problems with boost. So I've decided to completely switch to another solution with Ubuntu 22.04. What do think - does it make sense to somehow fix build for 20.04?

From my perspective, it would be nice to even have some really small CLI version of Orbit as well (maybe current WITH_GUI=NO solves this problem), something like perf but with your nice runtime instrumentation. Maybe I'm completely wrong, but people might like such version.

Alternatively, for those people who's using Ubuntu 20.04 or other Linux distribution with old/deprecated packets, docker image could be provided. But docker is also questionable, because in that case we have to establish remote profiling between docker and host OS (which I think supported by Orbit), also GUI is an open question.

So far, I don't have any questions regarding new build system using Cmake, so this issue can be closed from my perspective. Thank you for your support!

[alexkalmuk]

Alternatively, for those people who's using Ubuntu 20.04 or other Linux distribution with old/deprecated packets, docker image could be provided. But docker is also questionable, because in that case we have to establish remote profiling between docker and host OS (which I think supported by Orbit), also GUI is an open question.

Sorry, after playing a little with non-GUI version, I've realized that even in Docker case we still need OrbitServer to be running directly on host OS.. Or at least it seems to be.

[florian-kuebler]

About building under Ubuntu 20.04. It looks that due to some old packages and versions incompatibility (including boost, protobuf, compiler, etc.) it's a big effort to make Orbit working right now. For instance, after manually building and installing googletest and protobuf, I'm still getting problems with boost. So I've decided to completely switch to another solution with Ubuntu 22.04. What do think - does it make sense to somehow fix build for 20.04?

Yeah we see quite some interest in Ubuntu 20.04. Unfortunately, none of us has the resources right now, to actually go ahead with that.

From my perspective, it would be nice to even have some really small CLI version of Orbit as well (maybe current WITH_GUI=NO solves this problem), something like perf but with your nice runtime instrumentation. Maybe I'm completely wrong, but people might like such version.

As you probably already found out, OrbitClientGgp is a very basic cli client. I haven't used it in a while, but it should do the job (while I it is still using the "old"/kernel-based (uprobes) instrumentation, which adds much more overhead. Should be straightforward to add it, though. In general, a clean-up or even complete rewrite of the CLI client would be a nice addition to Orbit. As said, unfortunately, we can't spend the time on this right now.

Sorry, after playing a little with non-GUI version, I've realised that even in Docker case we still need OrbitServer to be running directly on host OS.. Or at least it seems to be.

Yes, at least OrbitService can not be run inside a Docker container, as it otherwise won't be able to read the perf data from the parent's namespace.

[beckerhe]

Yes, at least OrbitService can not be run inside a Docker container, as it otherwise won't be able to read the perf data from the parent's namespace.

I'm not sure if that's true. Running inside a container should be fine as long as the service has all the permissions that it needs. Docker usually restricts syscalls using seccomp and we at least need to be able to call ptrace. --unprivileged might also be needed to read some of the system performance data.

[alexkalmuk]

I'm not sure if that's true. Running inside a container should be fine as long as the service has all the permissions that it needs. Docker usually restricts syscalls using seccomp and we at least need to be able to call ptrace. --unprivileged might also be needed to read some of the system performance data.

I thought Docker's process has access to host OS, but it isolates us (as Orbit) from the host system, so from inside the Docker even /proc/<id> will represent completely different processes. It means Docker itself can use ptrace, but I doubt that there is a way to do it for Orbit process. But that's really interesting discussion :)

[florian-kuebler]

I'm not sure if that's true. Running inside a container should be fine as long as the service has all the permissions that it needs. Docker usually restricts syscalls using seccomp and we at least need to be able to call ptrace. --unprivileged might also be needed to read some of the system performance data.

Right.

I thought Docker's process has access to host OS, but it isolates us (as Orbit) from the host system, so from inside the Docker even /proc/<id> will represent completely different processes. It means Docker itself can use ptrace, but I doubt that there is a way to do it for Orbit process. But that's really interesting discussion :)

You can certainly run docker without "pid" namespaces. So you should see the other processes from within the docker container.
Unfortunately, I don't know the details here. But it might be doable. Certainly and interesting discussion.

[alexkalmuk]

Thank you guys, I didn't know about running the docker without normal namespaces.

As you probably already found out, OrbitClientGgp is a very basic cli client. I haven't used it in a while, but it should do the job (while I it is still using the "old"/kernel-based (uprobes) instrumentation, which adds much more overhead. Should be straightforward to add it, though. In general, a clean-up or even complete rewrite of the CLI client would be a nice addition to Orbit. As said, unfortunately, we can't spend the time on this right now.

Yes, I've tried it :) I'm not sure about if it's working correctly, because I have 0 events in OrbitServer's output, but I must investigate it prior to asking.
I'm just wondering why there is "client + service" using gRPC in this CLI version. Shouldn't it be just a single ELF which attaches itself to some PID and does the job?

[florian-kuebler]

I'm just wondering why there is "client + service" using gRPC in this CLI version. Shouldn't it be just a single ELF which attaches itself to some PID and does the job?

That has historical reasons. The original use case was a CLI client for remote profiling to a Stadia server. Though, it was never more than a prototype. But there are also valid reasons to split the client and the collector even for local cli profiling. Mostly, as the collector requires root access for most use cases. The client does not.

But anyways, a small and easy to use cli client would be nice to have.

[alexkalmuk]

I'm just wondering if this variant of the build with cmake can be part of the documentation next to the Conan-based build? Also, could these flags -DCMAKE_CXX_FLAGS="-march=sandybridge -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer" be somehow hidden in CMakeLists.txt (maybe, except march)? What do you think?

[florian-kuebler]

I guess that makes sense (both of your suggestions). I won't have time this week, but can probably have a look next week.

[KingDuckZ]

Just wanted to report that I wanted to try Orbit, I'm running Gentoo and it looks like a nightmare. I installed conan, pretty much for nothing because I hit #4861, then I tried running cmake manually, which complains that grpc++_unsecure cannot be found, and in fact I can't spot anything relevant here. It would be nice to have Orbit package for Gentoo, or at least a sane build system that has some fallback mechanism for unusual dependencies. I'm a meson user and it offers a way to deal with that for example.

[pierricgimmig]

@KingDuckZ, thanks for the info. I feel your pain, building Orbit is a massive headache even for the most used platforms (Ubuntu, Windows) as described in "Building Orbit can be extremely painful - How can we make it better?". Hopefully we find time to tackle this issue.