WjV UdZddlmZddlmZddlmZddlZddl Z ddl m Z m Z dd lmZmZdd l mZd2dZGddZedkr0ddlZddlZejdZeddeddededded ed!d"ed#d$gd%d&'eZeejejej(Z eej!Z!gZ"d)e#d*<ej$D]|Z%e"&e'e!(d+e%e"&e'e!(d+e%)}e'e*e"Z"e"se+d,e!ne+d-e,e"d.e"D]Z-ej.e/e-Z0e0e+d/e-j11e 2e0Z3e+e-j1d0e,e3j4d1e,e3j5e 6e0e3dSdS)3a TableGridExtractor.py Loads an exported GridModelV1 ONNX model and predicts horizontal / vertical grid boundary positions from a cropped table image. Post-processing pipeline ------------------------ 1. Resize input image to model input size. 2. Run ONNX inference -> h_heatmap (H,), v_heatmap (W,). 3. Apply threshold to each 1D heatmap. 4. Run 1D Connected Component Labeling (CCL) on the thresholded signal. 5. Compute weighted centroid of each component -> boundary coordinate. ) annotations)Path)OptionalN) resize_imageto_gray)GridPredictionCellInfo) make_sessionheatmap np.ndarray thresholdfloatreturn list[float]c||k}g}d}g}g}tt||D]\}\}} | r:d}|t|||D|rt |} | dkr+t dt||D| z } n,tt |t |z } || g}g}d}|r|rt |} | dkr+t dt||D| z } n,tt |t |z } || t |S)aK Apply threshold to a 1D heatmap, find connected components, and return the weighted centroid of each component. Parameters ---------- heatmap : 1D float32 array threshold : minimum value to consider active Returns ------- Sorted list of centroid positions (float, 0-based index space). FTrc3&K|] \}}||zV dSN.0vis /Users/tedpeters/Documents/Projects/Claude Projects/TPC Houses/house-viewer/.venv/lib/python3.11/site-packages/pymupdf/layout/onnx/TableGridExtractor.py z$_ccl_1d_centroids..Cs*"Q"QTQ1q5"Q"Q"Q"Q"Q"Qc3&K|] \}}||zV dSrrrs rrz$_ccl_1d_centroids..Os*IITQ1q5IIIIIIr) enumeratezipappendrsumlensorted) r ractive centroidsin_group group_vals group_idxsrval is_activetotalcentroids r_ccl_1d_centroidsr.!s" !FIH J J(Wf)=)=>>!! C  !H   eCjj ) ) )   a  !J199""Q"QSZ5P5P"Q"Q"QQQTYYHH$S__55JGH  ***   #J#J 199IISZ-H-HIIIIIEQHHS__--J?H""" )  rcheZdZdZ d.d/dZd0dZed1dZd2d Z d3d4d&Z d5d6d-Z dS)7TableGridExtractorz ONNX-based table grid boundary extractor. Usage ----- extractor = TableGridExtractor("grid_model_v1.onnx") pred = extractor.predict(image_bgr) print(pred.h_lines, pred.v_lines) ffffff??N onnx_path str | Pathh_on_thresholdrv_on_threshold providersOptional[list[str]]rNonect||_||_||_|dg}t t |j||_|jd}t|j d|_ t|j d|_ tj dd|j |j ftj}|jdd|i}t|dj d|_t|dj d|_dS) ae Parameters ---------- onnx_path : path to the exported .onnx model file hmap_threshold : activation threshold for h_heatmap CCL vmap_threshold : activation threshold for v_heatmap CCL providers : ONNX Runtime execution providers (default: ["CPUExecutionProvider"]) NCPUExecutionProviderrrr dtypeimage)rr3r5r6r str_sess get_inputsintshape_input_h_input_wnpzerosfloat32run_out_h_out_w)selfr3r5r6r7inpdummyoutputss r__init__zTableGridExtractor.__init__gs #9oo,,  /0I!#dn"5"5yAA  --//2CIaL)) CIaL)) (Aq$-?rzRRR*..'788'!**1-.. '!**1-.. r image_bgrrr c|jdd\t|jjf}|dddddddft j}||}}||kr ||z ||z z }nt j |}| dddt j ddf}j dd|i}|dd}|dd} t|j} t| j} fd| D} fd| D} t#t%| t%| || S) z Run inference on a single BGR image. Parameters ---------- image_bgr : H x W x 3 uint8 BGR image (any size) Returns ------- GridPrediction with h_lines and v_lines in input image space. Nrrr r?c*g|]}|zjz Sr)rK)rcorig_hrMs r z3TableGridExtractor.predict_grid..$EEE1v: +EEErc*g|]}|zjz Sr)rL)rrVorig_wrMs rrXz3TableGridExtractor.predict_grid..rYrh_linesv_lines h_heatmap v_heatmap)rDrrFrEastyperGrIminmax zeros_like transposenewaxisrArJr.r5r6r r$)rMrR img_resizedimg_rgbmnmxrNrPr_r`h_centroids_outv_centroids_outr]r^rWr[s` @@r predict_gridzTableGridExtractor.predict_grids#!,#  ]DM *  aaaDDbDj)00<<++--B 77|R0GGmG,,G1a((QQQ7JNN4'388AJqM AJqM ,It7JKK+It7JKKFEEEE_EEEEEEEE_EEE7OO7OO     rlinesrcenters image_sizec |s|St|}d}|r/d}|sn)dg|zt|gz t dz } fdt|D}|dr|ddkrd|d<d}j|dr|r|d}g}d}tt|D]t} |rd}|| dz} | rE| dzt|kr/||| || dzzdz d}d}Y||| ut|}|/|S) a Remove adjacent line pairs that have no bbox center between them, and handle empty outer gaps: - If the gap between 0 and the first line contains no bbox center, the first line is moved to 0.0 (collapsed to the image edge). - If the gap between the last line and image_size contains no bbox center, the last line is removed entirely. - For inner lines: when two adjacent lines share an empty gap between them, they are replaced by their midpoint. The process repeats until stable. Parameters ---------- lines : sorted list of boundary coordinates centers : list of bbox center coordinates (along same axis) image_size : image dimension along this axis (H or W) Returns ------- Filtered and possibly merged list of boundary lines (sorted). TFr cLg|]tfdD  S)c3VK|]#}|cxkodzkncV$dS)r Nr)rrVedgesrs rrzDTableGridExtractor._filter_empty_lines...sKEEa13333uQU|3333EEEEEEr)any)rrrorus @rrXz:TableGridExtractor._filter_empty_lines..sMEEEEEWEEEEEErrrT@)r$rr#rangepopr!) rnrorpcurrentchangedn_gaps gap_emptymerged skip_nextrgap_right_emptyrus ` @r_filter_empty_linesz&TableGridExtractor._filter_empty_liness: L--- %G EGOuZ'8'8&99EZZ!^FvI |  c 1 1  }   #%FI3w<<(( . . %I"+AE"2".q1us7||';';MM71:A#>#"EFFF $I $GGMM'!*----VnnG[- %^r bboxes_pagegridcrop_x0crop_y0crop_hcrop_wspan_thresholdlist[CellInfo]c t|j}t|j} dg|z|gz} dg| z|gz} ttt | } ttt | } t | dkrd|g} t | dkrd|g} dd } g} t |D]6\}}t|d t|d t|dt|d f\}}}}||z }||z }||z }||z }||zd z }||zd z }| || }| || }|}|d z}|}|d z}|} | d kr@| | }!| | | | d z z }"|"d kr|!|z |"z |kr | d z }| d z} nn| d k@|} | t | dz krS| | d z}#| | d z| | z }"|"d kr||#z |"z |kr | dz}| d z } nn| t | dz kS|}$|$d kr@| |$}%| |$| |$d z z }&|&d kr|%|z |&z |kr |$d z }|$d z}$nn|$d k@|}$|$t | dz krS| |$d z}'| |$d z| |$z }&|&d kr||'z |&z |kr |$dz}|$d z }$nn|$t | dz kS| t|||||||8| S)a Assign each bbox to a grid cell (row, col) and compute span. Steps ----- 1. Convert each bbox from page space to crop space. 2. Locate the cell that contains the bbox center point. 3. If the bbox extends beyond the center cell by more than span_threshold of that cell's dimension, expand the span. Parameters ---------- bboxes_page : (N, 4) array of [x0, y0, x1, y1] in page space grid : GridPrediction with h_lines / v_lines in crop space crop_x0 : left edge of the table crop in page space crop_y0 : top edge of the table crop in page space crop_h : height of the crop image in pixels crop_w : width of the crop image in pixels span_threshold : fractional overlap required to expand span (e.g. 0.1) Returns ------- List of CellInfo, one per bbox. rrrposrrurrrCctt|dz D]%}|||cxkr||dzkrn!|cS&tdt|dz S)z4Return 0-based index of the interval containing pos.r rr)rxr#rc)rrurs r find_cell_idxz.find_cell_idxJsy3u::>**  8s1111U1q5\11111HHH2q#e**q.)) )rrr r<rw)bbox_idx row_startrow_end col_startcol_endrowcol)rrrurrrC) r$r]r^listsetr#rrr!r )(rMrrrrrrr_h_lines_sorted_v_lines_sorted row_edges col_edgesrresultsrbboxx0_py0_px1_py1_px0y0x1y1cxcybase_rowbase_colrrrrrcell_top cell_height cell_bottomrV cell_left cell_width cell_rights( r_post_process_gridz%TableGridExtractor._post_process_gridsD!.. ..EO+vh6 EO+vh6 4I//00 4I//00 y>>A  f I y>>A  f I * * * *#% --J J GAt%*47^^U47^^',T!W~~uT!W~~&F "D$dBBBBr'SBr'SB%}R33H$}R33H I 1 G I 1 GAa%%'l'lYq1u-== ??2 'D~'U'U !AIFAAa%%Ac)nnq((('A. 'A.1= ??[(8K'G.'X'X!eGFAAc)nnq(((Aa%%&q\ &q\Ia!e,<< >>y2~&Cn&T&T !AIFAAa%%Ac)nnq(((&q1u- &q1u- ! < >>rJ*&D~&U&U!eGFAAc)nnq((( NN8%#%#$$    r皙?bboxesOptional[np.ndarray]texts%tuple[GridPrediction, list[CellInfo]]c ||}|t|dkr|gfStj|tj}t |jd}t |jd}td|D} td|D} ||j | |} ||j | |} t| | |j |j }|||dd||| } |9| D]6}d|jcxkrt|krn$||j|_7|| fS) a Predict grid boundaries and optionally assign bboxes to grid cells. Parameters ---------- image_bgr : cropped table BGR image (any size) bboxes : (N, 4) array of [x0, y0, x1, y1] in crop space. If None, only grid prediction is performed. texts : list of text strings aligned with bboxes. If provided, each CellInfo.text is populated. span_threshold : fractional overlap to trigger span expansion (default 0.1) Returns ------- (GridPrediction, list[CellInfo]) CellInfo list is empty when bboxes is None. Nrr=r c3rK|]2}t|dt|dzdz V3dS)rrrwNrrbs rrz-TableGridExtractor.predict..@OOq%!++ad 3s:OOOOOOrc3rK|]2}t|dt|dzdz V3dS)r r<rwNrrs rrz-TableGridExtractor.predict..rrr\rr)rrrrrrr)rmr#rGasarrayrIrrDr$rr]r^r r_r`rrtext)rMrRrrrr bboxes_arrrrcx_listcy_list filtered_h filtered_vcellscells rpredictzTableGridExtractor.predicts0  ++ >S[[A--8OZbj999 yq)**yq)**OOJOOOOOOOJOOOOO--dlGVLL --dlGVLL nn    ''")(     5 5 2222E 22222 %dm 4DIU{rpred window_titler@max_dimrCc ddl}|jdd\}}|}|jD]6} t t | } |j|d| f|| fdd7|jD]6} t t | } |j|| df| |fdd7|j|j ddtj f||f|j } |j | dzddtj|j}|j|d |jd d |jd d d|j|jtj ddf||f|j }|j |dzddtj|j}|j|d|jd d |jd d dtj||dftj}dt1|jdt1|jg}t3|D].\}}|j||dd|dzzf|jddd|j/tj||gd}tj||gd}tj||gd}|jdd\}}||ks||kri|t9||z }|j|t9dt ||zt9dt ||zf|j}|j|||jd|j ||S)a Draw predicted grid lines on the image and display via OpenCV. Layout (2x2 grid, all panels same size) ---------------------------------------- (1,1) input image with predicted grid lines overlaid (1,2) h_heatmap tiled to 2D (rows = H axis) (2,1) v_heatmap tiled to 2D (cols = W axis) (2,2) summary info panel Parameters ---------- image_bgr : original BGR image pred : GridPrediction from predict() window_title : OpenCV window title max_dim : maximum panel dimension before downscaling Returns ------- The composed BGR visualization array. rNr)rrr )rrr) interpolationrzh_thr=z.2f)皙?)rrrzv_thr=r<r=z h_lines : z v_lines : g?)rr)axis)!cv2rDcopyr]rCroundliner^resizer_rGrf INTER_LINEAR applyColorMapcliprauint8 COLORMAP_JETputTextr5FONT_HERSHEY_SIMPLEXr`r6rHr#rLINE_AA concatenaterc INTER_AREAimshowwaitKey destroyWindow)rMrRrrrrimg_himg_woverlayyy_pxxx_pxh_heatmap_resizedh_panelv_heatmap_resizedv_panel info_panelsummaryrtxttop_rowbot_rowcomposedchcwscales r visualizezTableGridExtractor.visualizes8   rr* u.."" H HAuQxx==D CHWq$i% Q G G G G H HAuQxx==D CHWtQi$ Q G G G G'CJ N111bj= )E5>*   $##  $ * *1c 2 2 9 9"( C CSEU    G?d&9???S5sOQ P P P'CJ N2:qqq= )E5>*   $##  $ * *1c 2 2 9 9"( C CSEU    G?d&9???S5sOQ P P PXueQ/rx@@@ ,T\** , , ,T\** , ,  (( % %FAs CK C"b1r6k):0$  % % % % >7G"41===>7J"7CCC>7G"41===#B <<2<<R,E!szQBJ((#aR%Z*A*AB!nH  <*** A,'''r)r1r2N) r3r4r5rr6rr7r8rr9)rRrrr )rnrrorrprrr)rrrr rrrrrrrrrrrr)NNr) rRrrrrr8rrrr)r0r) rRrrr rr@rrCrr) __name__ __module__ __qualname____doc__rQrm staticmethodrrrrrrrr0r0\s!% $)- "/"/"/"/"/P1 1 1 1 nQQQ\QfEEEET(,%) # AAAAAV1 ]]]]]]]rr0__main__z4Run TableGridExtractor on all images in a directory.) descriptionr3z!Path to the exported .onnx model.)help image_dirz"Directory containing input images.z--hmap_thresholdg333333?z6Activation threshold for h_heatmap CCL (default: 0.3).)typedefaultrz--vmap_thresholdrz6Activation threshold for v_heatmap CCL (default: 0.4).z --extensions*)pngjpgjpegbmptiftiffzImage file extensions to scan.)nargsrr)r3hmap_thresholdvmap_thresholdz list[Path] img_pathsz*.zNo images found in: zFound z$ image(s). Press any key to advance.z Cannot read: z h_lines=z v_lines=)r rrrrr)7r __future__rpathlibrtypingrnumpyrG onnxruntimeort common_utilrrtable_grid_typesr r r r.r0rargparserArgumentParserparser add_argumentr parse_argsargsr3rr extractorrr__annotations__ extensionsextextendr$globupperrprintr#img_pathimreadr@img_bgrnamermrr]r^rrrrr-s #"""""////////66666666%%%%%% 3333viiiiiiii` zOOOJJJ $X $JF  +NOOO  +OPPP  E  E c<<< -     D"".**I T^$$IIEE zCzz : :;;<<< /ACIIKK/A/A B BCCDDDDss9~~&&I M 0Y001111 Kss9~~KKKLLL + +#*SS]]++ ? E1(-11 2 2 2 %%g.. } H Hs4<(( H H47C 4E4E H H    GT****kT + +r