Race condition/non deterministic behavior of backends

[koffes]

Describe the bug

Scenario:
User wants to use RTT as shell:
CONFIG_SHELL_BACKEND_RTT=y, furthermore, they want printk and LOG_*** to pipe out over UART/SERIAL.

Now, if CONFIG_EARLY_CONSOLE=y is set, these two functions will run:

zephyr/drivers/console/uart_console.c

Line 615 in 6085aad

SYS_INIT(uart_console_init, PRE_KERNEL_1, CONFIG_CONSOLE_INIT_PRIORITY);

zephyr/drivers/console/rtt_console.c

Line 89 in 6085aad

SYS_INIT(rtt_console_init, PRE_KERNEL_1, CONFIG_CONSOLE_INIT_PRIORITY);

both in the end running:
__printk_hook_install(...

Now, it sems that mechanisms such as Link Time Optimizations, will affect which of these run first. So, in some cases, printk ends up on RTT and in other situations on Serial/UART.

Steps to reproduce

Hard to reproduce, but the fact that these two commands are run with the same SYS_INIT command argument leads to this issue.

Impact

Functional Limitation – Some features not working as expected, but system usable.

Additional Context

It is strange that RTT does __printk_hook_install. It should be possible to pipe printk to serial, whilst outputting console information to RTT.

[teburd]

This should be possible to have both the shell on RTT while having the console on a UART, this is a configuration issue.

[koffes]

Suggestion of the top of my head:
Convert all the CONFIG_CONSOLE_INIT_PRIORITY to CONFIG_CONSOLE_INIT_RTT_PRIORITY, CONFIG_CONSOLE_INIT_UART_PRIORITY..

Then these default to CONFIG_CONSOLE_INIT_PRIORITY. That means one can override the order if one wants to, and does not change anything by default.

That does just circumvent some the issue, and does not properly resolve it.

[teburd]

The above setup is quite confusing, it suggests that you've managed to get two chosen consoles setup in some manner by involving the shell. And by doing so end up with printk trying to go to both and racing.

In which case, the DT chosen node for the console should be winning. This could be a bug, but I don't think enough information is being provided here to replicate the issue. One way to show this is create a new case case and send a PR for this along with a fix.