Problem when using the ‘hdr_info’ property of ‘to_markdown()’.

[Fianax]

Hi, I'm using the ‘hdr_info’ property of the pdf4llm.to_markdown() method to customize the sections (add X amount of ‘#’).

But when I use the following method(test version to see what results it returned):

group_size_sentence: dict[int, list[dict[int, tuple[str, int]]]] = {}

def my_custom_section(span, page):
    """
    """
    key_1: int = int(page.number)
    size: int = int(float(span['size']))
    text: str = str(span['text'])
    print(text)
    total_len: int = sum(len(value) for value in group_size_sentence.values())

    if(key_1 not in group_size_sentence):
        group_size_sentence[key_1] = []

    group_size_sentence[key_1].append({
        total_len: (text, size)
    })

    return ""

there are sentences that never come back to me. That is, when the markdown is created, the sentences appear but when I print all the ‘span’ it doesn't return all of them (there are some lines that it doesn't "return").

In the following pdf for example it does not return the full title of the section 1. Andrew Lee - Goldman Sachs:, it only returns the 1.

and the first sentence of the first list, for example, is also not printed (My first question is on leases. So, there's a clear kind of distinction between).

I'm using pdf4llm version==0.0.9
prueba_section_no_detected.pdf

[JorjMcKie]

I suggest you make a list of the spans that actually are created to see what the situation is:

blocks = page.get_text("dict",flags=pymupdf.TEXTFLAGS_TEXT)["blocks"]
spans = [s for b in blocks for l in b["lines"] for s in l["spans"]]
for s in spans:
    print(f"{s['text']=}")

[Fianax]

thank you