o
MK&hE @ s" d Z ddlmZ dZdgZddlmZ ddlmZm Z m
Z
m Z m Z m
Z
mZmZmZmZmZ ddlmZmZmZmZmZmZ ddlmZm Z dd l m Z m Z m Z m!Z! dd
l"m#Z# erodd l$m%Z% dd lm&Z& dd
l'm(Z(m)Z)m*Z* dZ+e e e,e,f e,e,gdf Z-G dd dee Z.G dd de Z/dS )zCUse the HTMLParser library to parse HTML files that aren't too bad. ) annotationsMITHTMLParserTreeBuilder)
HTMLParser) AnyCallablecastDictIterableListOptional
TYPE_CHECKINGTupleTypeUnion)
AttributeDictCDataComment DeclarationDoctypeProcessingInstruction)EntitySubstitution
UnicodeDammit)DetectsXMLParsedAsHTMLHTMLHTMLTreeBuilderSTRICTParserRejectedMarkup)
BeautifulSoup)NavigableString) _Encoding
_Encodings
_RawMarkupz html.parserNc @ s e Zd ZU dZded< dZded< edd4ddZd ed < ded< ded< d5ddZd6ddZ d7d8d d Z d7d9d"d#Z
d:d%d&Z d;d'd(Z d;d)d*Z
d:d+d,Zd:d-d.Zd:d/d0Zd:d1d2Zd3S )<BeautifulSoupHTMLParserreplacestrREPLACEignoreIGNOREon_duplicate_attributesoupr argsr r+ &Union[str, _DuplicateAttributeHandler]kwargsc O s@ || _ || _|jj| _tj| g|R i | g | _| d S N)r, r+ builderattribute_dict_classr __init__ already_closed_empty_element_initialize_xml_detector)selfr, r+ r- r/ r7 J/var/www/html/venv/lib/python3.10/site-packages/bs4/builder/_htmlparser.pyr3 T s
z BeautifulSoupHTMLParser.__init__z List[str]r4 messagereturnNonec C s t |r0 r )r6 r9 r7 r7 r8 erroro s z BeautifulSoupHTMLParser.errornameattrs List[Tuple[str, Optional[str]]]c C s | j ||dd | | dS )zHandle an incoming empty-element tag.
html.parser only calls this method when the markup looks like
.
F)handle_empty_elementN)handle_starttag
handle_endtag)r6 r= r> r7 r7 r8 handle_startendtag s z*BeautifulSoupHTMLParser.handle_startendtagTr@ boolc C s | }|D ]3\}}|du rd}||v r5| j}|| jkr q|d| jfv r)|||< qtt|}|||| q|||< q| jjjrF| \}} nd }} | jj
|dd||| d}
|
rj|
j rj|rj| j |dd | j
| | jdu rv| | dS dS )zHandle an opening tag, e.g. ''
:param handle_empty_element: True if this tag is known to be
an empty-element tag (i.e. there is not expected to be any
closing tag).
N )
sourceline sourceposF)check_already_closed)r2 r+ r) r' r _DuplicateAttributeHandlerr, r1 store_line_numbersgetposrA is_empty_elementrB r4 append_root_tag_name_root_tag_encountered) r6 r= r> r@ attr_dictkeyvalueon_duperF rG tagr7 r7 r8 rA s2
z'BeautifulSoupHTMLParser.handle_starttagrH c C s. |r|| j v r| j | dS | j| dS )zHandle a closing tag, e.g. ''
:param name: A tag name.
:param check_already_closed: True if this tag is expected to
be the closing portion of an empty-element tag,
e.g. ''.
N)r4 remover, rB )r6 r= rH r7 r7 r8 rB s z%BeautifulSoupHTMLParser.handle_endtagdatac C s | j | dS )z4Handle some textual data that shows up between tags.N)r, handle_datar6 rV r7 r7 r8 rW s z#BeautifulSoupHTMLParser.handle_datac C s | drt|dd}n| dr t|dd}nt|}d}|dk rE| jjdfD ]}|s1q,z
t|g|}W q, tyD Y q,w |sYzt|}W n t t
fyX Y nw |p\d}| | dS )zHandle a numeric character reference by converting it to the
corresponding Unicode character and treating it as textual
data.
:param name: Character number, possibly in hexadecimal.
x XN z windows-1252u �)
startswithintlstripr, original_encoding bytearraydecodeUnicodeDecodeErrorchr
ValueError
OverflowErrorrW )r6 r= real_namerV encodingr7 r7 r8 handle_charref s.
z&BeautifulSoupHTMLParser.handle_charrefc C s0 t j|}|dur
|}nd| }| | dS )zHandle a named entity reference by converting it to the
corresponding Unicode character(s) and treating it as textual
data.
:param name: Name of the entity reference.
Nz&%s)r HTML_ENTITY_TO_CHARACTERgetrW )r6 r= characterrV r7 r7 r8 handle_entityref
s
z(BeautifulSoupHTMLParser.handle_entityrefc C s& | j | j | | j t dS )zOHandle an HTML comment.
:param data: The text of the comment.
N)r, endDatarW r rX r7 r7 r8 handle_comment s
z&BeautifulSoupHTMLParser.handle_commentc C s6 | j |tdd }| j | | j t dS )zYHandle a DOCTYPE declaration.
:param data: The text of the declaration.
zDOCTYPE N)r, rn lenrW r rX r7 r7 r8 handle_decl& s
z#BeautifulSoupHTMLParser.handle_declc C sN | drt}|tdd }nt}| j | j| | j| dS )z{Handle a declaration of unknown type -- probably a CDATA block.
:param data: The text of the declaration.
zCDATA[N)upperr] r rp r r, rn rW )r6 rV clsr7 r7 r8 unknown_decl0 s
z$BeautifulSoupHTMLParser.unknown_declc C s0 | j | j | | | | j t dS )z\Handle a processing instruction.
:param data: The text of the instruction.
N)r, rn rW _document_might_be_xmlr rX r7 r7 r8 handle_pi? s
z!BeautifulSoupHTMLParser.handle_piN)r, r r- r r+ r. r/ r )r9 r& r: r; )r= r& r> r? r: r; )T)r= r& r> r? r@ rD r: r; )r= r& rH rD r: r; )rV r& r: r; )r= r& r: r; )__name__
__module__ __qualname__r' __annotations__r) r3 r< rC rA rB rW ri rm ro rq rt rv r7 r7 r7 r8 r$ = s*
>
(
r$ c s e Zd ZU dZdZded< dZded< eZded< ee e
gZ d ed
< d ed < dZ ded
< d$d% fdd
Z
d&d'd d Zd(d"d#Z ZS ))r zA Beautiful soup `bs4.builder.TreeBuilder` that uses the
:py:class:`html.parser.HTMLParser` parser, found in the Python
standard library.
FrD is_xmlT picklabler& NAMEz
Iterable[str]featuresz$Tuple[Iterable[Any], Dict[str, Any]] parser_argsTRACKS_LINE_NUMBERSNOptional[Iterable[Any]]
parser_kwargsOptional[Dict[str, Any]]r/ r c sp t }dD ]}||v r||}|||< qtt| jdi | |p#g }|p'i }|| d|d< ||f| _dS )a Constructor.
:param parser_args: Positional arguments to pass into
the BeautifulSoupHTMLParser constructor, once it's
invoked.
:param parser_kwargs: Keyword arguments to pass into
the BeautifulSoupHTMLParser constructor, once it's
invoked.
:param kwargs: Keyword arguments for the superclass constructor.
r* Fconvert_charrefsNr7 )dictpopsuperr r3 updater )r6 r r r/ extra_parser_kwargsargrR __class__r7 r8 r3 [ s
z HTMLParserTreeBuilder.__init__markupr# user_specified_encodingOptional[_Encoding]document_declared_encodingexclude_encodingsOptional[_Encodings]r: DIterable[Tuple[str, Optional[_Encoding], Optional[_Encoding], bool]]c c s t |tr|dddfV dS g }|r|| g }|r!|| t|||d|d}|jdu r3td|j|j|j|jfV dS )a2 Run any preliminary steps necessary to make incoming markup
acceptable to the parser.
:param markup: Some markup -- probably a bytestring.
:param user_specified_encoding: The user asked to try this encoding.
:param document_declared_encoding: The markup itself claims to be
in this encoding.
:param exclude_encodings: The user asked _not_ to try any of
these encodings.
:yield: A series of 4-tuples: (markup, encoding, declared encoding,
has undergone character replacement)
Each 4-tuple represents a strategy for parsing the document.
This TreeBuilder uses Unicode, Dammit to convert the markup
into Unicode, so the ``markup`` element of the tuple will
always be a string.
NFT)known_definite_encodingsuser_encodingsis_htmlr zPCould not convert input to Unicode, and html.parser will not accept bytestrings.)
isinstancer& rM r unicode_markupr r` declared_html_encoding contains_replacement_characters)r6 r r r r r r dammitr7 r7 r8 prepare_markupy s4
z$HTMLParserTreeBuilder.prepare_markupr; c
C s | j \}}t|ts J | jd usJ t| jg|R i |}z || | W n ty: } zt|d }~ww g |_ d S r0 )
r r r& r, r$ feedcloseAssertionErrorr r4 )r6 r r- r/ parserer7 r7 r8 r s
zHTMLParserTreeBuilder.feed)NN)r r r r r/ r )NNN)
r r# r r r r r r r: r )r r# r: r; )rw rx ry __doc__r{ rz r|
HTMLPARSERr} r r r~ r r3 r r
__classcell__r7 r7 r r8 r J s
! H)0r
__future__r __license____all__ html.parserr typingr r r r r
r r r
r r r bs4.elementr r r r r r
bs4.dammitr r bs4.builderr r r r bs4.exceptionsr bs4r r bs4._typingr! r" r# r r& rI r$ r r7 r7 r7 r8 s* 4