|jdZddlZddlmZdZdZdZdZee dZ ej d Z d e d e fd Zd ed edzfdZd ed dfdZdS)ajStage 1d: UTF-8 structural validation. Validation is decode-based: CPython's strict UTF-8 decoder enforces exactly the rules the old hand-rolled loop checked (overlong encodings, surrogates, codepoints above U+10FFFF) at C speed on every build flavor, where a per-byte loop runs an order of magnitude slower even compiled. See ADR-0006. The input is fed to an incremental decoder in chunks so a large input never allocates a matching ``str``; decoded output is discarded. ``final=False`` reproduces the truncated-tail tolerance (an incomplete multi-byte sequence at the very end is fine, the input is usually a prefix of a larger whole), the same trick :func:`chardet._utils.decodes_without_error` uses. Note: ``from __future__ import annotations`` is intentionally omitted because this module is compiled with mypyc, which does not support PEP 563 string annotations. N)DetectionResultg?gGz?iutf-8bytereturncrd|cxkrdkrnndSd|cxkrdkrnndSd|cxkrdkrnnd Sd S) zSequence length a UTF-8 lead byte declares, or 0 for a non-lead. 0xC0-0xC1 are overlong 2-byte encodings of ASCII, so leads start at 0xC2. r)rs /Users/tedpeters/Documents/Projects/Claude Projects/Misc Tools/notename-sheets/.venv/lib/python3.11/site-packages/chardet/pipeline/utf8.py_expected_seq_lenr*s~  ttq ttq ttq 1datac,t|dS)aValidate UTF-8 byte structure. Returns a result only if multi-byte sequences are found (pure ASCII is handled by the ASCII stage). :param data: The raw byte data to examine. :returns: A :class:`DetectionResult` for UTF-8, or ``None``. ) scan_utf8)rs r detect_utf8r8s T??1 rz#tuple[bool, DetectionResult | None]c |sdS|rdSt|}t}d}d}d}|}d}||krd|||tz}|t|z} | }|dkr|r| }J|t|dt z } ||dn#t$r} d} ttd|dz |D]&} t|| } | r | | z|kr| } n'||z t| ddz}tt|d|d z }| dks|| krYd} ~ d S| }Yd} ~ n=d} ~ wwxYwt| d}| }||kd||z }||kr5|||}|t|dt z}|dkrdS||z }ttz }ttt|t|t zd zz}d t#d |dfS)aValidate UTF-8 structure, separating validity from evidence. ``None`` from :func:`detect_utf8` is two different verdicts: the data is *not* UTF-8, or it is valid UTF-8 carrying no multi-byte evidence (pure ASCII, ASCII plus control bytes, ASCII plus a truncated tail). Only the first justifies ruling UTF-8 out downstream, so callers that act on a rejection need both halves of the answer. :param data: The raw byte data to examine. :returns: ``(window_is_valid_utf8, result_or_None)``. The flag covers the whole of *data*; the result is present only when complete multi-byte sequences were found. )TNrNF)finalrstartr)FNg?Tr)encoding confidencelanguage)isasciilen _utf8_decoder _CHUNK_SIZE translate _ASCII_DELETEdecode UnicodeErrorrangemaxrgetattrmingetstate_MAX_CONFIDENCE_BASE_CONFIDENCE_MB_RATIO_SCALEr)rlengthdecodermultibyte_bytes pending_lenpos tail_start counted_endchunk chunk_endexc tail_leadkseq_len err_starttailmb_ratioconfidence_ranger!s rrrDs | ||~~| YYFooGOK C JK ,,S3,,-#e**$  !    C 3ut]CCDDD  NN5N . . . .   I3q&1*--v66  +DG44q7{V33 !IEk)GC!,D,DDIC 1--vz::I1}} I 5 5$}}}}}#J EEEE1 2'**,,Q/00 M ,,Tk) KJ{*+3t~~dMBBCCC !|'H&)99+c(_2Lc.R.RRRJ Z$OOO s8C E:BE5.E55E:)__doc__codecschardet.pipelinerr1r0r2r&bytesr+r(getincrementaldecoderr%intrrrrrrrJs$ ,,,,,, eeDkk"" ,,W55   C  C      e $ 6    hEhChhhhhhr