j>VdZddlmZmZmZmZmZmZmZgdZ dZ dZ dZ dZ d d ZdS) uCanonical text source for chunking: render one LayoutBox to markdown. This module is the single place where the chunking package calls the markdown renderers in ``document_layout``. ``box_to_markdown`` mirrors the per-box dispatch of ``ParsedDocument.to_markdown`` so that chunk text is the same markdown the document renderer emits for that box — upstream renderer signature drift is absorbed here and nowhere else. Documented deviations from ``to_markdown``: - picture/formula boxes without text are not rendered here at all; the chunker represents them with a placeholder (see ``figure_placeholder``). - formula boxes *with* textlines are rendered via ``text_to_md``. ``to_markdown`` drops that text (it only emits the image reference), but dropping it from chunks would lose indexable content. - image files / base64 payloads are never inlined into chunk text; images travel in metadata. )create_list_item_levelsfootnote_to_mdlist_item_to_mdpicture_text_to_mdsection_hdr_to_md text_to_md title_to_md)box_to_markdownrextract_table_headersfigure_placeholder table_contentc t|jtr|jni}|d}|8|d}|r!dd|Dpd}|d|fS|ddfS)uaAdapter for ``box.table`` — the only place chunking reads its schema. Returns ``(markdown, html)`` where at most one side is a string, mirroring the ``to_markdown`` branching: when the parse ran with HTML table rendering, the table dict carries the canonical rendering under ``"html"`` and ``markdown`` is ``None``; otherwise ``markdown`` is the canonical rendering (possibly ``""`` for a degenerate table) and ``html`` is ``None``. Routing uses ``is not None`` on purpose: a degenerate table carries ``markdown == ""`` and must still route as markdown, never as html. htmlN html_tablesz c3lK|]/}t|t|ddV0dS)rN) isinstancedictget).0ts /Users/tedpeters/Documents/Projects/Claude Projects/Misc Tools/notename-sheets/.venv/lib/python3.11/site-packages/pymupdf4llm/helpers/chunking/text_source.py z table_content..;sS&' 1d8K8Kfb!!markdown)rtablerrjoin)boxrr per_tables rr r 's$CIt44 `` elements emitted by the table engine — no local header heuristics. Returns ``[]`` when the rendering carries no ```` (including all markdown-mode parses). ) HTMLParserc0eZdZfdZdZdZdZxZS)+extract_table_headers.._THCollectorcrtd|_g|_g|_dS)Nr!)super__init___depth_partsheaders)self __class__s rr'z4extract_table_headers.._THCollector.__init__Ps2 GG     DKDKDLLLrcD|dkr|xjdz c_g|_dSdS)Nth)r(r))r+tagattrss rhandle_starttagz;extract_table_headers.._THCollector.handle_starttagVs-d{{ q   {rc|dkr]|jrX|xjdzc_|jd|jdSdSdS)Nr.r/r)r(r*appendrr)strip)r+r0s r handle_endtagz9extract_table_headers.._THCollector.handle_endtag[sbd{{t{{ q  ##BGGDK$8$8$>$>$@$@AAAAA{{{rcL|jr|j|dSdS)N)r(r)r4)r+datas r handle_dataz7extract_table_headers.._THCollector.handle_data`s2{ ) ""4((((( ) )r)__name__ __module__ __qualname__r'r2r6r9 __classcell__)r,s@r _THCollectorr$Osh       ! ! !  B B B  ) ) ) ) ) ) )rr>) html.parserr"feedr*)rr"r> collectors rr r Csu  &&&&&&)))))z)))* I NN4  rcd|DS)uCopy textlines deep enough to isolate renderer side effects. Some renderers mutate their input in place (``footnote_to_md`` extends the first line's span list, ``list_item_to_md`` rewrites/pops spans), so rendering the same box twice — e.g. ``to_chunks`` followed by ``to_markdown`` — would duplicate text. Rendering from a copy keeps ``box_to_markdown`` free of side effects on the parsed document. c Zg|](}i|dd|dgDi)S)spansc,g|]}t|S)r)rss r z._copy_textlines...ss>>>Qa>>>r)r)rtls rrHz#_copy_textlines..rsU     @2?w>>"&&"*=*=>>>??   rrF) textliness r_copy_textlinesrKis#     rct|j|jz t|j|jz }}d|d|d|dS)z>Stable text placeholder for a figure/formula box without text.z [Figure fz: x])intx1x0y1y0) fig_numberrwhs rr r xsL sv  SVcf_!5!5qA -z - -Q - - - - --rNFc|j}|p|j}|dvrt|\}}||S|pd}|jr|dd}|r|St |dd} | rt t| |SdSt |dd} | sdSt| } |dkrt| |S|dkrt | |S|d krt|j | S|d krt|j | S|d kr:|t|j }t| ||d S|d krt!| St | |S)aRender one LayoutBox to markdown, mirroring ``to_markdown`` dispatch. Args: page: the PageLayout owning the box (``full_ocred`` is honored). box: the LayoutBox to render. box_idx: index of the box on its page (list-item level lookup). list_item_levels: optional precomputed ``create_list_item_levels`` result for the page; computed on demand when omitted. ignore_code: suppress code-block styling (as in ``to_markdown``). Returns the markdown string for the box ("" when there is nothing to render). Trailing block separators ("\n\n") are preserved as emitted by the renderers; callers strip as needed. )rztable-fallbackNr`rJ) ignore_codepictureformulatitlezsection-headerz list-itemr/footnote)boxclass full_ocredr replacer5getattrrrKrr header_levelrrboxesrrr) pagerbox_idxlist_item_levelsrYbtyperr table_textrJs rr r ~s LE0K +++&s++$  K^ ? 5#++C44J       Cd33  Soi88kRRR Rr[$//I r **I !)EEEE )==== 3+Y777     !19===   #6tzBB y*:*>*>w*J*JKKK i((( i[ 9 9 99r)NF)__doc__document_layoutrrrrrrr __all__r r rKr r rFrrrls&   '''8###L   ... 6:6:6:6:6:6:r