-mcmodel=large option issue

[columbos14927]

Hi,
I'm trying to test the -mcmodel=large option. When I compiled a simple C code program, I received a linker error. Here are the steps to reproduce the issue:

• Riscv-gnu-toochain compilation:

configure --with-arch=rv64imafc --with-abi=lp64f --with-cmodel=large --prefix=/toolchain/gnu-workspace/install-gnu-toolchain-upstream MAKEINFO=true

make MAKEINFO=true -j 8

• C code file(main.c):

#include <string.h>
 
__attribute__((section(".far_dccm"))) const char* G_s1 = "Str1";
__attribute__((section(".far_dccm"))) const char* G_s2 = "Str2";
 
void main()
{
        if(strcmp(G_s1,G_s2)) return;
}

• Linker script file(link.ld):

ENTRY(main)
 
MEMORY
{
    BRAM (rx)  :  ORIGIN = 0x00000000,       LENGTH = 256K
    DCCM (rw)  :  ORIGIN = 0xAA00000000,     LENGTH = 256K
}
 
SECTIONS
{
    .text : 
    {
        *(.text*)  
    } > BRAM
 
    .far_dccm :
    {
        *(.srodata .srodata.*) 
    } > DCCM
}

• Compilation command:

riscv64-unknown-elf-gcc main.c -o main.elf -T link.ld -mabi=lp64f -march=rv64imafc -mcmodel=large -nostartfiles

• Compilation error:
repos/cmodel-demo/toolchains/install-gnu-toolchain-upstream/bin/../lib/gcc/riscv64-unknown-elf/14.2.0/../../../../riscv64-unknown-elf/lib/libc.a(libc_a-strcmp.o): in function strcmp': (.text+0xa): relocation truncated to fit: R_RISCV_PCREL_HI20 against mask'
collect2: error: ld returned 1 exit status

[kito-cheng]

Oh, okay, you hit a really issue that we don't handle right in the newlib side, I am writing patch and testing, will post here and upstream once I finish the test.

[kito-cheng]

Here is the patch, apply that on newlib should fix this issue, let me know if that fix your issue:

From 22438798fad071d6e74bf817353e95724b8bda27 Mon Sep 17 00:00:00 2001
From: Kito Cheng <kito.cheng@sifive.com>
Date: Tue, 1 Apr 2025 21:38:06 +0800
Subject: [PATCH] RISC-V: Fix the asm code for large code model

The large code model assume the data may far away from the code, so we
must put the address of the target data wihin the `.text` section,
normally we will just put within the function or nearby the function to
prevent it out-of-range.

Report from riscv-gnu-toolchain:
https://github.com/riscv-collab/riscv-gnu-toolchain/issues/1699
---
 libgloss/riscv/crt0.S              | 41 +++++++++++++++++++++++++++++-
 newlib/libc/machine/riscv/strcmp.S |  9 ++++++-
 2 files changed, 48 insertions(+), 2 deletions(-)

diff --git a/libgloss/riscv/crt0.S b/libgloss/riscv/crt0.S
index 54443e412..1b90c99ab 100644
--- a/libgloss/riscv/crt0.S
+++ b/libgloss/riscv/crt0.S
@@ -27,8 +27,13 @@ _start:
 .option pop
 
   /* Initialize jvt CSR (reg addr: 0x0017) */
+#ifndef __riscv_cmodel_large
   .weak __jvt_base$
   lla a0, __jvt_base$
+#else
+  la a0, .Laddr_jvt_base
+  ld a0, 0(a0)
+#endif
   beqz a0, .Ljvt_init_end
 .option push
 .option norelax
@@ -38,21 +43,38 @@ _start:
 .Ljvt_init_end:
 
   # Clear the bss segment
+#ifndef __riscv_cmodel_large
   la      a0, __bss_start
   la      a2, _end
+#else
+  la      a0, .Laddr_bss_start
+  ld      a0, 0(a0)
+  la      a2, .Laddr_end
+  ld      a2, 0(a2)
+#endif
   sub     a2, a2, a0
   li      a1, 0
   call    memset
 #ifdef _LITE_EXIT
   # Make reference to atexit weak to avoid unconditionally pulling in
   # support code.  Refer to comments in __atexit.c for more details.
+#ifndef __riscv_cmodel_large
   .weak   atexit
   la      a0, atexit
+#else
+  la      a0, .Laddr_atexit
+  ld      a0, 0(a0)
+#endif
   beqz    a0, .Lweak_atexit
-  .weak   __libc_fini_array
 #endif
 
+#ifndef __riscv_cmodel_large
+  .weak   __libc_fini_array
   la      a0, __libc_fini_array   # Register global termination functions
+#else
+  la      a0, .Laddr_libc_fini_array
+  ld      a0, 0(a0)
+#endif
   call    atexit                  #  to be called upon exit
 #ifdef _LITE_EXIT
 .Lweak_atexit:
@@ -66,4 +88,21 @@ _start:
   add     a2, a2, a1                 # a2 = envp
   call    main
   tail    exit
+#ifdef __riscv_cmodel_large
+.Laddr_gp:
+  .dword __global_pointer$
+.Laddr_jvt_base:
+  .weak __jvt_base$
+  .dword __jvt_base$
+.Laddr_bss_start:
+  .dword __bss_start
+.Laddr_end:
+  .dword _end
+.Laddr_atexit:
+  .weak   atexit
+  .dword atexit
+.Laddr_libc_fini_array:
+  .weak   __libc_fini_array
+  .dword __libc_fini_array
+#endif
   .size  _start, .-_start
diff --git a/newlib/libc/machine/riscv/strcmp.S b/newlib/libc/machine/riscv/strcmp.S
index 12c39db94..cc29b7b8b 100644
--- a/newlib/libc/machine/riscv/strcmp.S
+++ b/newlib/libc/machine/riscv/strcmp.S
@@ -188,9 +188,16 @@ strcmp:
   foundnull 1 3
   foundnull 2 3
 #endif
+#ifdef __riscv_cmodel_large
+  # Put the data within the funciton for large code model to prevent
+  # the data put too far.
+.align 3
+mask:
+.dword 0x7f7f7f7f7f7f7f7f
+#endif
 .size  strcmp, .-strcmp
 
-#if SZREG == 8
+#if SZREG == 8 && !defined(__riscv_cmodel_large)
 .section .srodata.cst8,"aM",@progbits,8
 .align 3
 mask:
-- 
2.34.1

[algrobman]

it is not just strcmp - there are other glib modules too which fail with the same issue when compiling dhrystone benchmark , placing srodata too far away from .text

also I think proposed solution wont work in our case, where CPU can't read /write code memory by Load/store instructions.
why can't you use "li" instruction to get the 'mask' value?

[kito-cheng]

it is not just strcmp - there are other glib modules too which fail with the same issue when compiling dhrystone benchmark , placing srodata too far away from .text

That should not happened if compile with -mcmodel=large, that's a bug if got those problem with -mcmodel=large.

also I think proposed solution wont work in our case, where CPU can't read /write code memory by Load/store instructions. why can't you use "li" instruction to get the 'mask' value?

But the large code model will put address within the text section, so use li in strcmp still can't not fix in that scenario?

[algrobman]

there are two issues here

1. these .srodata sections if placed to far from the text. The issue can be solved by linking srodata with .text, but
2. CPU with Harvard architecture can't access application code with Load/Stores instructions

So if you could solve the issue if you fulfill these following conditions:

1. .text code should not access other section addresses with PC relative instructions (auipc)
2. .text code should not use L/S instructions to access .text content

(although I'm not sure if these conditions fulfillment is possible without adding new linker/as relocation functions for 64 bit "li" or "la" pseudo instructions, to load a GPR with 64 address without use "ld" instructions targeting code)
(Of course this code can't be position independent)

regarding 'li' - li translates to arithmetic instructions (add/shift /or etc)
why can't mask be loaded as?:
li a0, 0x7f7f7f7f7f7f ... should load the"mask" value 0x7f .. to a GRP with arithmetic instructions ...

[algrobman]

following functions used in dhryshone have the same issue:

/fac/proj_wy/toolchain/gnu-workspace/install-gnu-toolchain-gcc14-newlib4.4.0-binutils2.43.1-gdb15.2_final_rh8/bin/../lib/gcc/riscv32-swerv-elf/14.2.0/../../../../riscv32-swerv-elf/lib/rv64imafc/lp64f/libc.a(libc_a-vfprintf.o): in function _vfprintf_r': vfprintf.c:(.text._vfprintf_r+0x256): relocation truncated to fit: R_RISCV_PCREL_HI20 against .L25'
/fac/proj_wy/toolchain/gnu-workspace/install-gnu-toolchain-gcc14-newlib4.4.0-binutils2.43.1-gdb15.2_final_rh8/bin/../lib/gcc/riscv32-swerv-elf/14.2.0/../../../../riscv32-swerv-elf/lib/rv64imafc/lp64f/libc.a(libc_a-vfiprintf.o): in function _vfiprintf_r': vfiprintf.c:(.text._vfiprintf_r+0x162): relocation truncated to fit: R_RISCV_PCREL_HI20 against .L25'
/fac/proj_wy/toolchain/gnu-workspace/install-gnu-toolchain-gcc14-newlib4.4.0-binutils2.43.1-gdb15.2_final_rh8/bin/../lib/gcc/riscv32-swerv-elf/14.2.0/rv64imafc/lp64f/libgcc.a(divdf3.o): in function __divdf3': /fac/proj_wy/toolchain/gnu-workspace/build-gnu-toolchain-gcc14-newlib4.4.0-binutils2.43.1-gdb15.2/build-gcc-newlib-stage2/riscv32-swerv-elf/rv64imafc/lp64f/libgcc/../../../../../../../riscv-gnu-toolchain-gcc14-newlib4.4.0-binutils2.43.1-gdb15.2/riscv-gcc/libgcc/soft-fp/divdf3.c:44:(.text+0xd8): relocation truncated to fit: R_RISCV_PCREL_HI20 against .L10'
collect2: error: ld returned 1 exit status

I suspect there are some other modules too.

(again, our linker file places data more than 2GB apart from text )

[kito-cheng]

@algrobman Do you mind provide more information for how to reproduce that?

[columbos14927]

@kito-cheng I will provide more data and try on how to reproduce it, giving as many details as possible.
It will take me some time, though.

[columbos14927]

Hi @kito-cheng ,
Here is the reproduction of other large model code issues:

C code file(main.c):

#include <stdio.h>
 
void main()
{
        printf("Hello");
}

Linker script file(link.ld):

ENTRY(main)
 
MEMORY
{
    BRAM (rx)  :  ORIGIN = 0x00000000,       LENGTH = 256K
    DCCM (rw)  :  ORIGIN = 0xAA00000000,     LENGTH = 256K
}
 
SECTIONS
{
    .text : 
    {
        *(.text*)  
    } > BRAM
 
    .far_dccm :
    {
        *(.srodata .srodata.*) 
    } > DCCM
}

Compilation command:

riscv64-unknown-elf-gcc main.c -o main.elf -T link.ld -mabi=lp64f -march=rv64imafc -mcmodel=large -nostartfiles

Compilation error:

install-gnu-toolchain-gcc14-newlib4.4.0-binutils2.43.1-gdb15.2_final_rh8/bin/../lib/gcc/riscv32-swerv-elf/14.2.0/../../../../riscv32-swerv-elf/lib/rv64imafc/lp64f/libc.a(libc_a-vfprintf.o): in function _vfprintf_r': vfprintf.c:(.text._vfprintf_r+0x256): relocation truncated to fit: R_RISCV_PCREL_HI20 against .L25'

install-gnu-toolchain-gcc14-newlib4.4.0-binutils2.43.1-gdb15.2_final_rh8/bin/../lib/gcc/riscv32-swerv-elf/14.2.0/../../../../riscv32-swerv-elf/lib/rv64imafc/lp64f/libc.a(libc_a-strcmp.o): in function strcmp': (.text+0xa): relocation truncated to fit: R_RISCV_PCREL_HI20 against mask'

toolchains/install-gnu-toolchain-gcc14-newlib4.4.0-binutils2.43.1-gdb15.2_final_rh8/bin/../lib/gcc/riscv32-swerv-elf/14.2.0/../../../../riscv32-swerv-elf/lib/rv64imafc/lp64f/libc.a(libc_a-vfiprintf.o): in function `_vfiprintf_r':
vfiprintf.c:(.text._vfiprintf_r+0x162): relocation truncated to fit: R_RISCV_PCREL_HI20 against `.L25'

install-gnu-toolchain-gcc14-newlib4.4.0-binutils2.43.1-gdb15.2_final_rh8/bin/../lib/gcc/riscv32-swerv-elf/14.2.0/rv64imafc/lp64f/libgcc.a(divdf3.o): in function `__divdf3':
build-gnu-toolchain-gcc14-newlib4.4.0-binutils2.43.1-gdb15.2/build-gcc-newlib-stage2/riscv32-swerv-elf/rv64imafc/lp64f/libgcc/../../../../../../../riscv-gnu-toolchain-gcc14-newlib4.4.0-binutils2.43.1-gdb15.2/riscv-gcc/libgcc/soft-fp/divdf3.c:44:(.text+0xd8): relocation truncated to fit: R_RISCV_PCREL_HI20 against `.L10'
collect2: error: ld returned 1 exit status

[kito-cheng]

@columbos14927 I can't reproduce the issue with the toolchain configure with --with-arch=rv64imafc --with-abi=lp64f --with-cmodel=large, it can only reproduce with the toolchain configure without --with-cmodel=large

[columbos14927]

Hi @kito-cheng ,
Yes, you are right. I made a mistake here; sorry for that.
Here is the correct reproduction of a relocation error:

main.c file:

#include <stdio.h>

void main()
{
        printf("Hello");
}

linker.ld file:

ENTRY(main)

MEMORY
{
    BRAM (wx)  :      ORIGIN = 0x00000000,       LENGTH = 256K
    DCCM (rw)  :      ORIGIN = 0xAA00000000,     LENGTH = 256K
    BRAM_FAR (wx)  :  ORIGIN = 0xFF00000000,       LENGTH = 256K
}

SECTIONS
{
    .text : 
    {
	EXCLUDE_FILE(*libc_a-vfiprintf.o)
        *(.text*)  
    } > BRAM

    .far_dccm :
    {
       *(.srodata .srodata.*) 
    } > DCCM

    .bram_far :
    {
	*libc_a-vfiprintf.o (.text .text*)
    } > BRAM_FAR
}

Compilation line:

riscv32-swerv-elf-gcc main.c -o main.elf -T link.ld -march=rv64imac -mabi=lp64 -mcmodel=large -Wl,-Map=diag.map -nostartfiles

As a result we got one warning and several errors:

Warning:

/bin/../lib/gcc/riscv32-swerv-elf/14.2.0/../../../../riscv32-swerv-elf/bin/ld: warning: main.elf has a LOAD segment with RWX permissions

Errors:

/bin/../lib/gcc/riscv32-swerv-elf/14.2.0/../../../../riscv32-swerv-elf/lib/rv64imac/lp64/libc.a(libc_a-vfprintf.o): in function _vfprintf_r': vfprintf.c:(.text._vfprintf_r+0x256): relocation truncated to fit: R_RISCV_PCREL_HI20 against .L25'
/bin/../lib/gcc/riscv32-swerv-elf/14.2.0/rv64imac/lp64/libgcc.a(divdf3.o): in function __divdf3': /fac/proj_wy/toolchain/gnu-workspace/build-gnu-toolchain-wyouming-multil-lib-enable-with-kito-fix/build-gcc-newlib-stage2/riscv32-swerv-elf/rv64imac/lp64/libgcc/../../../../../../../riscv-gnu-toolchain-gcc14-newlib4.4.0-binutils2.43.1-gdb15.2/riscv-gcc/libgcc/soft-fp/divdf3.c:44:(.text+0x8c): relocation truncated to fit: R_RISCV_PCREL_HI20 against .L10'
collect2: error: ld returned 1 exit status

[oferShinaar]

Hi Kito,
The warning of LOAD segment with RWX message that we see from @columbos14927 message is related to the message from @algrobman
.text code should not use L/S instructions to access .text content.

The liker invoked this warning because @columbos14927 assigns memory (bram) to the text segment and stated it is rx type. That caused the linker to warn him that he should be careful when doing loads from RWX. Some cores can not do that... As @algrobman mentioned.

I think that the initiative behind newlib/strcmp implementation on putting "mask" on rodata (or small RO) is just for that. The user can choose where to place this memory.

Since mask is always 0x7f7f7f7f7f7f 7f7f , what is the downside of using li?
This way, we don't access the memory (on 64-bit machines, it's pseudo for lui, Addi,slli,addi...), which is impossible in some cases.
Is it because it will just invoke more instructions? (perf and size)

Here is a dump for the fix (a5 is holding the mask):

0000000000000058 <strcmp>:
  58:	00b56733        or	a4,a0,a1
  5c:	53fd               	li	t2,-1
  5e:	8b1d               	andi	a4,a4,7
  60:	eb4d               	bnez	a4,112 <strcmp+0xba>
  62:	00000797        auipc	a5,0x0
  66:	0e67b783        ld	a5,230(a5) # 148 <mask>
  6a:	6110               	ld	a2,0(a0)
  6a:	6110               	ld	a2,0(a0)
  6c:	6194               	ld	a3,0(a1)
  6e:	00f672b3        and	t0,a2,a5
  72:	00f66333         or	t1,a2,a5
.
.
.
0000000000000148 <mask>:
 148: 7f7f7f7f          	.word	0x7f7f7f7f
 14c: 7f7f7f7f          	.word	0x7f7f7f7f

and we see that basically, a5 could get its value from li and do the same logic

[aswaterman]

It's an optimization for RV64 cores in the case that the load hits in the cache. This is probably more of a Dhrystone optimization than anything else, truth be told...

[oferShinaar]

Do you mean the load will hit the d$?
If so, initially the load should be from a physical memory that allows loads. That is not always the case for memories that hold the text