Ship only one C library (picolibc) and drop newlib from the SDK

[nashif]

We currently ship newlib in multiple configurations + picolibc. Picolibc is the default and others are provided mostly for historical and legacy reasons. Proposal is top drop newlib and focus on delivering the SDK only with picolibc with the benefit of reducing complexity and reducing size and build time for the SDK.

[nashif]

for fun, I tried this with some xtensa toolchain and had those build errors:

.build/xtensa-intel_ace15_mtpm_zephyr-elf/src/gdb/gdb/xtensa-tdep.h:  int litbase_regnum = -1;	/* Register number of LITBASE.  */
build.log:[ERROR]    ../../src/picolibc/picocrt/machine/xtensa/crt0.S:59: Error: invalid register 'LITBASE' for 'wsr' instruction
build.log:[ERROR]    ../../src/picolibc/picocrt/machine/xtensa/crt0.S:59: Error: invalid register 'LITBASE' for 'wsr' instruction
build.log:[ERROR]    ../../src/picolibc/picocrt/machine/xtensa/crt0.S:59: Error: invalid register 'LITBASE' for 'wsr' instruction
build.log:[ERROR]    ../../src/picolibc/picocrt/machine/xtensa/crt0.S:59: Error: invalid register 'LITBASE' for 'wsr' instruction

building for some other target seems to work

[keith-packard]

Sounds like the core-isa.h for that target is promising support for a feature which isn't present in that target?

[nashif]

ok, looks like the overlays are not being used correctly for xtensa toolchains in picolibc, if I copy the core-isa.h dmanually, it builds fine

[nashif]

ok, looks like the overlays are not being used correctly for xtensa toolchains in picolibc, if I copy the core-isa.h dmanually, it builds fine

adding overlay for picolibc fixes the issue.

ok, next one :)

/build-workspace/build-aarch64-zephyr-elf/.build/aarch64-zephyr-elf/src/gcc/libgcc/soft-fp/eqtf2.c -fvisibility=hidden -DHIDE_EXPORTS
[ALL  ]    In file included from /home/nashif/zephyrproject/build-workspace/output/aarch64-zephyr-elf/aarch64-zephyr-elf/include/machine/ieeefp.h:12,
[ALL  ]                     from /home/nashif/zephyrproject/build-workspace/output/aarch64-zephyr-elf/aarch64-zephyr-elf/include/sys/config.h:32,
[ALL  ]                     from /home/nashif/zephyrproject/build-workspace/output/aarch64-zephyr-elf/aarch64-zephyr-elf/include/sys/cdefs.h:45,
[ALL  ]                     from /home/nashif/zephyrproject/build-workspace/output/aarch64-zephyr-elf/aarch64-zephyr-elf/include/sys/auxv.h:39,
[ALL  ]                     from /home/nashif/zephyrproject/build-workspace/build-aarch64-zephyr-elf/.build/aarch64-zephyr-elf/src/gcc/libgcc/config/aarch64/cpuinfo.c:28:
[ERROR]    /home/nashif/zephyrproject/build-workspace/output/aarch64-zephyr-elf/aarch64-zephyr-elf/include/sys/features.h:28:10: fatal error: picolibc.h: No such file or directory
[ALL  ]       28 | #include <picolibc.h>
[ALL  ]          |          ^~~~~~~~~~~~
[ALL  ]    compilation terminated.

with aarch64-zephyr-elf

[keith-packard]

I'll have to wait until Sunday to take a serious look at this -- I'm traveling and my build machine has taken itself offline without any remote console configured. It looks like there's some issue with the crosstool-ng configuration that uses picolibc as the primary C library. I haven't tried the Zephyr version of those patches in a while...

[adigie]

picolibc libstd++ is built with exceptions enabled. Removing newlib (nano) would increase size of the C++ applications by ~4kB (#860). I think this should be considered before making such change.

[nashif]

@keith-packard running anything with qemu_xtensa/dc233c fails when using piclibc, works fine with minimal. This might be a general picolibc/xtensa problem (tested with new toolchain using 14.2/picolibc)

[keith-packard]

picolibc libstd++ is built with exceptions enabled. Removing newlib (nano) would increase size of the C++ applications by ~4kB (#860). I think this should be considered before making such change.

We could consider changing the picolibc configuration if we're willing to merge the libstdc++ module for Zephyr. That seems blocked at this point. zephyrproject-rtos/zephyr#80762

[Damian-Nordic]

We could consider changing the picolibc configuration if we're willing to merge the libstdc++ module for Zephyr. That seems blocked at this point. zephyrproject-rtos/zephyr#80762

I understand that we didn't want to include libstdc++ binaries with and without exceptions enabled not to increase the toolchain size, but if newlib is to be removed and so is libstdc++ built for newlib, then maybe two variants of libstdc++ for picolibc wouldn't be a big problem? At least until the support for libstdc++ as a module is merged.

[keith-packard]

I understand that we didn't want to include libstdc++ binaries with and without exceptions enabled not to increase the toolchain size, but if newlib is to be removed and so is libstdc++ built for newlib, then maybe two variants of libstdc++ for picolibc wouldn't be a big problem? At least until the support for libstdc++ as a module is merged.

That's a reasonable idea; we effectively do this today with newlib/newlib-nano -- newlib gets exceptions, newlib-nano does not.

I'm not sure how best to approach this though; we could try adding another multilib selector and see if that 'just works'. Alternatively, installing the two versions in separate directories or using different library names, although that would require some crosstool-ng hacking.

[carlescufi]

@frkv @tomi-font will TF-M build fine without newlib?

[tomi-font]

@Vge0rge any idea?

[frkv]

I find no direct link to newlibc in the core TF-M build-system, beyond some usages of spec-strings in toolchain_GNUARM.cmake (-specs=nano.specs, -specs=nosys.specs) as well as a library dependency for -lc -lnosys in psa-arch-tests

TF-M provides memcmp, memcpy, memset, memmove, printf, vprintf via a runtime library level. Additionally it provides heap support via Mbed TLS (which can defer down to hand-implemented buffers)

I've confirmed with the TF-M code owner that removing dependency on standard C-library is doable similar to how TF-M (main branch) now supports Clang toolchain (see this changeset)

I can't speak for all types of device integration, though. It is possible that device-specific code for hardware accelerated crypto or other types of device-integration may have some innate dependencies

[carlescufi]

I find no direct link to newlibc in the core TF-M build-system, beyond some usages of spec-strings in toolchain_GNUARM.cmake (-specs=nano.specs, -specs=nosys.specs) as well as a library dependency for -lc -lnosys in psa-arch-tests

TF-M provides memcmp, memcpy, memset, memmove, printf, vprintf via a runtime library level. Additionally it provides heap support via Mbed TLS (which can defer down to hand-implemented buffers)

Understood. So, to me this sounds like a "no, TF-M does not use newlib".

[tokangas]

While updating an application from newlib to picolibc, I ran into a linking problem. It seems to be caused by calling mallinfo(). After I had figured that out, I found this bug: zephyrproject-rtos/zephyr#59424

mallinfo() should also work with picolibc, right?

[keith-packard]

mallinfo isn't provided in the Zephyr malloc implementation which is used when you use picolibc. If you reference it, you'll end up pulling in the picolibc version which isn't useful.

[tokangas]

mallinfo isn't provided in the Zephyr malloc implementation which is used when you use picolibc. If you reference it, you'll end up pulling in the picolibc version which isn't useful.

Thanks, now it makes sense. It would be nice, though, that calling mallinfo() would cause an error which would be more informative. Currently, it's quite hard to figure out the root cause. And I guess it should be possible to add mallinfo() support, if desired.

[keith-packard]

Yeah, it might be nice to not include malloc and friends in the SDK.

[tokangas]

One of our applications has used mallinfo() to get statistics of malloc() heap usage. After migrating to picolibc, mallinfo() is not available. So, I tried to use sys_heap_runtime_stats_get() to get statistics from z_malloc_heap instead, but it turns out that it does not work either, because z_malloc_heap is static.

@keith-packard Is there some method to get malloc() memory usage statistics with picolibc? I think there should be.