Wj$GdZddlZddlmZmZddlmZdZdZ dZ dZ e e e e zZ d ejd ejfd Zd ejd ejfd Zd ejd ejfdZdejd ejfdZdZdZdZdZdZdZdZ d"dejdeeeeeefdedededed ejfdZGd d!ZdS)#a? Adaptive RoI-pooling feature extraction. Automatically dispatches between: - Naive per-bbox loop: zero setup cost, best for small B - SAT (integral image) path: O(C*H*W) setup, O(1) per-bbox mean/min/max query, best for large B Calibrated for ~300x300 feature maps with C~10. Crossover points (B * avg_patch_area / H*W): - entropy/margin requested: ~2.0 (SAT setup amortized across the extra maps) - otherwise: ~50.0 (naive loop is extremely fast without probs) Pooling ops (pooling_ops parameter) ------------------------------------ Two families of ops are supported, and they behave differently: - Per-channel ops: 'mean', 'min', 'max' Each produces one value per input channel (C dims). - Per-region ops: 'entropy', 'margin' Each collapses the channel axis into a single confidence/uncertainty scalar per bbox (1 dim). Always computed from channel-softmax probabilities regardless of `apply_softmax` (they are only meaningful over a probability distribution). The output is the concatenation of all requested per-channel ops (in the order given, each C-wide) followed by all requested per-region ops (in the order given, each 1-wide). For example pooling_ops=('mean', 'min', 'max', 'entropy', 'margin') on a C-channel input produces a (B, 3*C + 2) array. N)ListTuple) defaultdictmeanmax)rminrentropymargin)rr r)r r fmapreturnc0|j\}}}||d}||ddz}t j||||dddzz}||||S)z8Channel-wise softmax along axis=0 of a (C, H, W) tensor.rT)axiskeepdims)out-q=)shapereshapecopyrnpexpsum)r CHWfs /Users/tedpeters/Documents/Projects/Claude Projects/TPC Houses/house-viewer/.venv/lib/python3.11/site-packages/pymupdf/layout/roi_pooling.py_softmax_channelsr5sjGAq! Q  ""AA % %%AF1!A % % --A 99Q1  probscdd}|tj||zzd S)zAPer-pixel entropy of a (C, H, W) probability map. Returns (H, W).rrr)rlogr)r!epss r_pixel_entropyr&?s4 C RVECK(( ( - -1 - 5 5 55r c|jd}|dkr|dStj|dddd}tj||d}|d|d|dz S)zJPer-pixel top1-top2 margin of a (C, H, W) probability map. Returns (H, W).rr#N)rr argpartitiontake_along_axissort)r!ridxtop2s r _pixel_marginr0Es| AA1uuQx /%! , , ,RSS 1C eSq 1 1 1DII1I 7T!W r arrc \|jdd}|jd|jd}}tjg||dz|dzRtj}tj|d|dddddftj|dddddfd|dddddf|S)zBSummed Area Table over the last two axes. Returns (..., H+1, W+1).Nr)rr*dtype.)rr)rrzerosfloat64cumsum)r1prefixrrsats r _build_satr:Ps Yss^F 9R=#)B-qA (*V*QU*AE**"* = = =CIcCQRRK 01111Ic#qrr122+RSabb!""-=>>>> Jr cx|d|dz|dzf|d||dzfz |d|dz|fz |d||fzS)z8Vectorized rectangle sum on a SAT. Coords are inclusive..r*)r9y1x1y2x2s r _sat_rect_sumrAZsc Caa  c2rAvo   c262o   c2rk  r c^|t|z }|t|z }tj|dddf|ztjd|dz }tj|dddf|ztjd|dz }tj|dddf|ztjdz d|dz } tj|dddf|ztjdz d|dz } tj| |} tj| |} ||| | fS)zz_split_ops..ts"BBB"r\/A/A2/A/A/Ar c$g|] }|tv |Sr<) _REGION_OPSrXs rr[z_split_ops..us"@@@bK.?.?".?.?.?r r<) pooling_ops channel_ops region_opss r _split_opsraqs6CB BBBK@@{@@@J  ""r cF|t|zt|zSN)len)rr_r`s r _output_dimreys s; #j// 11r c t|\}}|jd} t|} t| ||} t j| | ftj} t| D]} || || || || f\}}}}|dd||dz||dzf}|jdkrIg}|D]}|dkr*| | d2|dkr*| | db|dkr)| | d|r|dd||dz||dzf}|D]}|d krH| t j t| gP|d krG| t j t| g|rt j|nt jdtj}|tj| | d|jdf<| S) Nrr3r*r)r*r(r#rr r r )rarrdrerr5float32rangesizeappendrrr arrayr&r0 concatenaterF) fmap_usedr!rPrQrRrSr^r_r`rBout_dimriy1iy2ix1ix2ipatchpartsrZpppooleds r _path_naiverysq(55KA CA![*55G (Aw"+=+=+B+B+D+D*E!F!FGGGG8^^LL=+<+<+A+A+C+C*D!E!EFFF*/R&&&RXarz5R5R5R#)==#<#<A Q   Jr c|jd}t|}|r"tj||ftjnd} |r"tj||ftjnd} ||z dz} ||z dz} t t } t|D]E}| t| |t| |f |F| D]\\}}}tj |tj }t|}tj||||f|j }t|D]7\}}|dd|||||z|||||zf||<8|r|d| |<|r|d| |<| | fS)a Compute per-bbox min and/or max via grouped batch slicing: bboxes with the same (height, width) in feature-grid units are stacked into one array and reduced together, avoiding a per-bbox python loop. min/max cannot be expressed via a summed-area table (unlike mean), so this always has to run against the actual bboxes -- there is nothing here that can be precomputed once and reused across different bbox sets (contrast with RoiPoolingSession's sat_mean / sat_region, which ARE bbox-independent and shareable). Returns: (max_all, min_all) -- each either an (B, C) float32 array or None depending on whether it was requested. rr3Nr*)r(rCr#)rrdremptyrgrlistrhintrjitemsrkrGr4 enumeraterr )rmrPrQrRrSneed_maxneed_minrrnmax_allmin_allph_arrpw_arrgroupsrpphpwindicesr.npatchesjbis r_grouped_minmaxrs A CA4<Fbh1vRZ0000$G4<Fbh1vRZ0000$G 3Y]F 3Y]F   F 1XX;;F1IF1I/077::::#\\^^ 4 4R'hwbg... HH(Aq"b>AAAs^^ R REAr"111c"gc"gl&:CGCGbL.s 7 7 7R_R 7 7 7r cHg|]}|ddtjfSrc)rnewaxis)rYrZregion_resultss rr[z_path_sat..s- E E EBnR BJ/ E E Er rr3r#)rarrdrFrr6r:rArTrgrr&r0r5rl)rmr!rPrQrRrSr^r_r`rrrrnareasmean_allsat_frrrrrZ region_mapsat_r region_meanrvrrs @@r _path_satrs(55KoGAq! CACi!mc A . 6 6rz B BEH 9%%!%c3<<BJM*+,-ffRZ.@.@  #H #H  4  4 3S#xJJJ'3GW (HHON)) ??'..JJ 8^^&u--J:&&$UCc3?? &rz 2 2 (r 7 7 7 7; 7 7 7E E E E E* E E EEE 2xAbj1111 >%a ( ( ((r FfeaturesbboxesrLrMr^ apply_softmaxc |jdksJ|j\}}}} |dksJt|tz } | r5t dt | dt tdt |\} } t| } t|}|dkr1tj dt|| | ftj S|d}tj |tj}t|| |||\}}}}t!||z dz||z dzz}||zt!|| zz }| rdnd }||k}|r1|s| rt%|}nd }|r|n|}t'|||||||S|rt%|}| r|nd }n|}| rt%|nd }t)|||||||S) al Extract per-bbox features via configurable pooling ops. Args: features: (1, C, H, W) feature map (raw activations or logits). bboxes: list of [x1, y1, x2, y2] in the same coordinate space as page_width/page_height (NOT necessarily the feature map's own H/W; this function rescales internally). page_width, page_height: dimensions of the coordinate space the bboxes are expressed in (e.g. the page raster used to run the model, not the model's internal H/W). pooling_ops: subset of {'mean', 'min', 'max', 'entropy', 'margin'}. - 'mean'/'min'/'max' are per-channel (C dims each) and are computed on `features` directly, or on softmax(features) if apply_softmax=True. - 'entropy'/'margin' are per-region scalars (1 dim each) and are ALWAYS computed on softmax(features), regardless of apply_softmax -- they are only meaningful over a probability distribution, so this is not user-configurable. Output is the concatenation of per-channel ops (in the order given) followed by per-region ops (in the order given). apply_softmax: whether the per-channel ops ('mean'/'min'/'max') pool over raw `features` or over channel-softmax probabilities. Has no effect on 'entropy'/'margin'. Returns: np.ndarray of shape (B, C*len(channel_ops) + len(region_ops)). Adaptively dispatches between a naive per-bbox loop and a SAT-based vectorized path based on the estimated computational cost ratio. r*Unknown pooling_ops: . Valid ops are .rr3g@gI@N)ndimrset _VALID_OPS ValueErrorsortedraboolrdrr5rergasarrayr6rTrDrrrry)rrrLrMr^rNrrr unknown_opsr_r` needs_probsrnr rKrPrQrRrSavg_patch_area cost_metric thresholduse_satr!rms r$extract_bbox_features_by_roi_poolingrsKN =A    JAq!Q 6666k""Z/KA@ 1D1D@@*0*<*<@@@AA A)55Kz""K F AAvvxK; CCDBJWWWW A;D F"* - - -B)"aJ LLCc3S3Y]sSy1}=CCEEFFNn$uQU||3K",II%GN  K %d++EEE*4EE E3S#{KKK  E)$//I!,6II$EEI/:D%d+++E9eS#sCMMMr cxeZdZdZ ddejdedefdZde e e e e e fd e d e d ejfd Z d S)RoiPoolingSessionaC Precomputes the bbox-INDEPENDENT parts of ROI pooling once for a given (features, pooling_ops, apply_softmax) combination, then lets you query() it against multiple different bbox sets cheaply. Motivating case: BoxRFDGNN pools the same feature_map/class_logits tensor twice per page -- once for node bboxes, once for edge-union bboxes (when use_image_edge=True). Calling extract_bbox_features_by_roi_pooling() twice rebuilds the channel softmax and the 'mean'/'entropy'/'margin' summed-area tables (SATs) twice, even though those depend only on the tensor + pooling_ops, not on which bboxes are being queried. This class builds them once and reuses them for every query() call. 'min'/'max' are NOT expressible via a SAT (unlike mean), so they are always recomputed per query() call directly from that call's bboxes (grouped batch slicing) -- there is nothing to share for those ops. When to use this vs. extract_bbox_features_by_roi_pooling(): - Querying a tensor ONCE (e.g. YF's per-page feature map): use the plain function. It dispatches naive-vs-SAT based on bbox count, so small queries stay cheap; a session would force the SAT build cost even for a single small query. - Querying the SAME tensor MULTIPLE times (e.g. node bboxes then edge bboxes against the same feature_map): use a session. The one-time SAT build cost is paid once and amortized across all query() calls, which is strictly cheaper than paying it (or the naive-path cost) separately per call. rFrr^rc|jdksJ|j\}}}}|dksJt|tz }|r5t dt |dt td|||c|_|_|_t|\|_ |_ |d} t|j } |s| rt| } nd} |r| n| |_d|j vrt|jnd|_i|_|j D]=} | dkrt%| nt'| } t| |j| <>dS) Nrr*rrrrrr )rrrrrr_C_H_Wra _channel_ops _region_opsrr _fmap_usedr: _sat_mean _sat_regionr&r0)selfrr^rrrrrrr rr!rZrs r__init__zRoiPoolingSession.__init__ws}!!!!^ 1aAvvvv+&&3  EDVK5H5HDD.4Z.@.@DDDEE E%&q!!$'.8.E.E+4+{4+,,  K %d++EEE#0:%%d$t'888%T_555>B " : :B24 //...}UZG[G[J#-j#9#9D R  : :r rrLrMr c 4t|}|dkr@tjdt|j|j|jftjStj|tj }t||j |j ||\}}}} | |z dz||z dzz tj } d} |jQt|j||| || tjddfz j tj} d|jv} d|jv} | s| rt%|j|||| | | nd\}}| ||dfd |jD}|jD]i}t|j|||| || z tj}||ddtjfj|s"tj|dftjStj|d S) aE Pool this session's tensor over a new set of bboxes. Cheap to call repeatedly: 'mean'/'entropy'/'margin' are O(1) per bbox (SAT lookup, built once in __init__); 'min'/'max' still cost a grouped batch slicing pass over these specific bboxes (unavoidable, see class docstring). rr3r*Nrr rrc g|] }| Sr<r<rs rr[z+RoiPoolingSession.query..sAAA$AAAr r#)rdrr5rerrrrgrr6rTrrrFrrArrrrrrjrl)rrrLrMrnrKrPrQrRrSrrrrrrrvrZrrs @rqueryzRoiPoolingSession.querys- KK 668Q DGT5FHX Y YZ"$*... .Zbj 1 1 1-b$'47JP[\\S#s)a-C#IM2::2:FF > %%dnc3SII AAA ./01&&2D2D D--D-- 8$ 8ODOS#sC8 T T T+7  $,GGLLAAAAt/@AAA" 5 5B()9")=sCcRR!"#)6"*#5#5  LLQQQ ]3 4 4 4 4 68QF"*555 5~e!,,,,r NrF)__name__ __module__ __qualname____doc__rndarraytuplerrrrrDr}rr<r rrrXs<CR&+:::%:#::::<(-DueUE'A!BC(-(-+.(-35:(-(-(-(-(-(-r rr) rnumpyrtypingrr collectionsrDEFAULT_FEATURE_MAP_POOLING_OPS DEFAULT_CLASS_LOGITS_POOLING_OPSrWr]rrrrr&r0r:rArTrareryrrrDr}rrrrr<r rrsQ@###### #2#N % # S  [!1!1 1 BJ2:6"*66666  BJ2:   ###222   N$$$N%)%)%)b) VNVNjVN ueUE12 3VNVN VN  VN  VNZVNVNVNVNze-e-e-e-e-e-e-e-e-e-r