j^dZddlmZddlmZddlmZddlmZddl m Z dZ d d Z d Z d S)z PDF feature extraction utilities - Orchestrator Purpose: Coordinate feature extractors (RF, IMF, YF, JF) Each extractor lives in its own module for independent maintenance. )make_custom_feature)extract_rf_features)image_feature_extraction_task)extract_yf_features)extract_jf_featuresc4td|DS)a Round bbox coords to integer (pixel-level) granularity for cheap, tolerance-aware dedup. Detected bboxes (e.g. from IMF, which rescales from model resolution back to page pixel space) rarely match an existing bbox to the exact float, so exact-equality dedup effectively never fires; integer rounding treats near-identical boxes as the same box, which is what "already detected this region" is meant to capture. c3NK|] }tt|V!dS)N)intround).0cs /Users/tedpeters/Documents/Projects/Claude Projects/Misc Tools/notename-sheets/.venv/lib/python3.11/site-packages/pymupdf/layout/pymupdf_util_ext.py z"_bbox_dedup_key..s.--1U1XX------)tuple)bboxs r_bbox_dedup_keyrs! ----- - --r rf+imf+yftextNcd|vrd|vrtd|dd|vrt|d|vr||d}t|||\}}} } ||d<||d <d |d D} t| | D]\} } t | }|| vr| ||d | |d d d|vr|d| t| d }|d|d|vr'|d}|t||d|vr||t|||nCd|vr?|=t|||d}|d}t||d|\}}} } ||d<||d <d|vrt||S)a Apply experimental feature extractors to base data. Args: data_dict: Base data dictionary from extract_base_elements feature_set_name: String indicating which features to extract (e.g., 'rf+imf+yf') feature_extractor: ONNX model for image feature extraction page: PyMuPDF page object (required for YF features) page_dict: Page dictionary (required for YF features) Returns: data_dict: Enhanced with custom_features Raises: ValueError: if feature_set_name requests both 'imf' and 'ymf'. These are mutually exclusive image-input strategies -- 'imf' runs the model on the plain page image, 'ymf' runs it on the page image with an extra YF-derived channel appended (aug_fetmap) -- not features meant to be combined. Declaring both would leave data_dict['feature_map']/['class_logits'] ambiguous (silently overwritten by whichever block runs second) and would run inference twice for no benefit. imfymfzfeature_set_name=z requests both 'imf' and 'ymf', which are mutually exclusive image-input strategies (plain image vs. YF-augmented image channel). Choose one.custom_featuresNimage feature_map class_logitsc,h|]}t|S)r)r bs r z+apply_feature_extractors..Us NNNQoa00NNNrbboxesrbox_typerf stext_pageyfT)return_fet_map)r") aug_fetmapjf) ValueError_init_custom_featuresrzipraddappendrgetrrr) data_dictfeature_set_name input_typefeature_extractorpage page_dictpage_imgrr bboxes_to_addbox_types_to_addexisting_bbox_keysrtype_valkeycustom_featurer%yfms rapply_feature_extractorsr>s2    U.>%>%> H 0 H H H    ))i(((    %6%BW%Eb 'F F B \=2B$/ - $0 .!ON)H:MNNN!-1ABB D DND(!$''C,,,"&&s+++(#**4000&!((,,,**j)00:::!4Xr!B!B+,33NCCC ]]<00  !  : 6 6 6    5  4 ; ; ; " " "'8'D")T9TRRRW%Eb '3F F F B \=2B$/ - $0 .! I&&& rc.g|d<|dg}tt|dD]V}|t|kr||nd}|d|}|dt ||WdS)z.Initialize custom_features from box_type list.rr#r!unknownrN)r/rangelenr.r)r0r#row_idxbtrs rr+r+~s#%I }}Z,,HYx01122KK")CMM"9"9Xg  y )#$++,?D,I,IJJJJKKr)rrNNN)__doc__pymupdf_util_baserpymupdf_util_rfrpymupdf_util_imfrpymupdf_util_yfrpymupdf_util_jfrrr>r+rrrrKs 322222000000;;;;;;000000000000 . . .:E_caaaaHKKKKKr