PPQc@sdZdZddlZddlZddlmZmZddlmZm Z m Z ddlm Z m Z ddl mZmZddlmZdd lmZddlZdd lmZmZmZmZdd lmZdd lmZmZmZdd lm Z m!Z!m"Z"de fdYZ#de fdYZ$de fdYZ%de fdYZ&de'fdYZ(dfdYZ)defdYZ*defdYZ+defdYZ,e-d Z.d!fd"YZ/e0d#d$d%Z1e0d&d$d'Z2d(Z3d)e,fd*YZ4d+e4fd,YZ5d-e4fd.YZ6d/e6fd0YZ7d1e6fd2YZ8d3e6fd4YZ9d5e6fd6YZ:d7e6fd8YZ;d9e6e5fd:YZ<d;e:fd<YZ=d=e6fd>YZ>d?e6fd@YZ?dAe4fdBYZ@dCe,fdDYZAdEeAfdFYZBdGeBfdHYZCdIeBfdJYZDdKe,fdLYZEe4e7e8e9e:e;e>e=e?eAeCeDe@e5e<fZFdS(Ms This is the ``docutils.parsers.rst.states`` module, the core of the reStructuredText parser. It defines the following: :Classes: - `RSTStateMachine`: reStructuredText parser's entry point. - `NestedStateMachine`: recursive StateMachine. - `RSTState`: reStructuredText State superclass. - `Inliner`: For parsing inline markup. - `Body`: Generic classifier of the first line of a block. - `SpecializedBody`: Superclass for compound element members. - `BulletList`: Second and subsequent bullet_list list_items - `DefinitionList`: Second+ definition_list_items. - `EnumeratedList`: Second+ enumerated_list list_items. - `FieldList`: Second+ fields. - `OptionList`: Second+ option_list_items. - `RFC2822List`: Second+ RFC2822-style fields. - `ExtensionOptions`: Parses directive option fields. - `Explicit`: Second+ explicit markup constructs. - `SubstitutionDef`: For embedded directives in substitution definitions. - `Text`: Classifier of second line of a text block. - `SpecializedText`: Superclass for continuation lines of Text-variants. - `Definition`: Second line of potential definition_list_item. - `Line`: Second line of overlined section title or transition marker. - `Struct`: An auxiliary collection class. :Exception classes: - `MarkupError` - `ParserError` - `MarkupMismatch` :Functions: - `escape2null()`: Return a string, escape-backslashes converted to nulls. - `unescape()`: Return a string, nulls removed or restored to backslashes. :Attributes: - `state_classes`: set of State classes used with `RSTStateMachine`. Parser Overview =============== The reStructuredText parser is implemented as a recursive state machine, examining its input one line at a time. To understand how the parser works, please first become familiar with the `docutils.statemachine` module. In the description below, references are made to classes defined in this module; please see the individual classes for details. Parsing proceeds as follows: 1. The state machine examines each line of input, checking each of the transition patterns of the state `Body`, in order, looking for a match. The implicit transitions (blank lines and indentation) are checked before any others. The 'text' transition is a catch-all (matches anything). 2. The method associated with the matched transition pattern is called. A. Some transition methods are self-contained, appending elements to the document tree (`Body.doctest` parses a doctest block). The parser's current line index is advanced to the end of the element, and parsing continues with step 1. B. Other transition methods trigger the creation of a nested state machine, whose job is to parse a compound construct ('indent' does a block quote, 'bullet' does a bullet list, 'overline' does a section [first checking for a valid section header], etc.). - In the case of lists and explicit markup, a one-off state machine is created and run to parse contents of the first item. - A new state machine is created and its initial state is set to the appropriate specialized state (`BulletList` in the case of the 'bullet' transition; see `SpecializedBody` for more detail). This state machine is run to parse the compound element (or series of explicit markup elements), and returns as soon as a non-member element is encountered. For example, the `BulletList` state machine ends as soon as it encounters an element which is not a list item of that bullet list. The optional omission of inter-element blank lines is enabled by this nested state machine. - The current line index is advanced to the end of the elements parsed, and parsing continues with step 1. C. The result of the 'text' transition depends on the next line of text. The current state is changed to `Text`, under which the second line is examined. If the second line is: - Indented: The element is a definition list item, and parsing proceeds similarly to step 2.B, using the `DefinitionList` state. - A line of uniform punctuation characters: The element is a section header; again, parsing proceeds as in step 2.B, and `Body` is still used. - Anything else: The element is a paragraph, which is examined for inline markup and appended to the parent element. Processing continues with step 1. treStructuredTextiN(t FunctionTypet MethodType(tnodest statemachinetutils(tApplicationErrort DataError(tStateMachineWStStateWS(tfully_normalize_name(twhitespace_normalize_name(t directivest languagest tableparsertroles(ten(t escape2nulltunescapet column_width(tpunctuation_charstromant urischemest MarkupErrorcBseZRS((t__name__t __module__(((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRystUnknownInterpretedRoleErrorcBseZRS((RR(((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRzst"InterpretedRoleNotImplementedErrorcBseZRS((RR(((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyR{st ParserErrorcBseZRS((RR(((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyR|stMarkupMismatchcBseZRS((RR(((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyR}stStructcBseZdZdZRS(s3Stores data attributes for dotted-attribute access.cKs|jj|dS(N(t__dict__tupdate(tselft keywordargs((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyt__init__s(RRt__doc__R#(((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRstRSTStateMachinecBs eZdZdeddZRS(sy reStructuredText's master StateMachine. The entry point to reStructuredText parsing is the `run()` method. icCstj|jj|_||_|d kr9t}n|j|jt d|d|j d|jdgdddt d||_ ||_ |j|j|j j |_ ||_tj|||d |d }d |_|_ d S( s Parse `input_lines` and modify the `document` node in place. Extend `StateMachineWS.run()`: set up parse-global data and run the StateMachine. tdocumenttreportertlanguaget title_stylest section_levelitsection_bubble_up_kludgetinlinert input_sourcetsourceN(R t get_languagetsettingst language_codeR(t match_titlestNonetInlinertinit_customizationsRR'tFalsetmemoR&tattach_observert note_sourcetnodeRtrun(R!t input_linesR&t input_offsetR2R,tresults((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyR;s(          N(RRR$tTrueR3R;(((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyR%stNestedStateMachinecBseZdZedZRS(sh StateMachine run from within other StateMachine runs, to parse nested document structures. cCsk||_||_|j|_|j|jj|j|_|j|_||_tj |||}|S(s Parse `input_lines` and populate a `docutils.nodes.document` instance. Extend `StateMachineWS.run()`: set up document-wide data. ( R2R7R&R8R9R'R(R:RR;(R!R<R=R7R:R2R>((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyR;s      (RRR$R?R;(((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyR@stRSTStatecBseZdZeZgZedZdZdZ dZ dZ edddZ diedddZdZd Zd Zd Zd Zd ZdZRS(s` reStructuredText State superclass. Contains methods used by all State subclasses. cCs.itd6dd6|_tj|||dS(Nt state_classestBodyt initial_state(RBtnested_sm_kwargsR R#(R!t state_machinetdebug((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyR#s  cCstj||jj}||_|j|_|j|_|j|_|jj|_t |jds||jj |j_ ndS(Ntget_source_and_line( R t runtime_initRFR7R'R,R&R:tparentthasattrRH(R!R7((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRIs      cCs,y|jj|Wntk r'nXdS(sT Jump to input line `abs_line_offset`, ignoring jumps past the end. N(RFt goto_linetEOFError(R!tabs_line_offset((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRLs cCs9|jjd|jj|||jjf|dgfS(ss Override `StateWS.no_match` to generate a system message. This code should never be run. sjInternal error: no transition pattern match. State: "%s"; transitions: %s; context: %s; current line: %r.N(R'tseveret __class__RRFtlineR3(R!tcontextt transitions((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pytno_matchs  cCs ggfS(sCalled at beginning of file.((R!RR((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pytbofsc CsJd}|dkr(|j}|d7}n|dkrJ|j}|d7}nt|}d} |dkry|jj} Wqtk rqXn| s|d|j|} n| j||d|j d|d||dkr|jj | n | j | j } |j rFt||dkrF|jjt||n| S( sg Create a new StateMachine rooted at `node` and run it over the input `block`. iiiRGR7R:R2N(R3t nested_smREtlentnested_sm_cachetpopt IndexErrorRGR;R7tappendtunlinkRNRJRFt next_line( R!tblockR=R:R2tstate_machine_clasststate_machine_kwargst use_defaultt block_lengthRFt new_offset((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyt nested_parses4               c  Cs| dkr|j} n| dkr6|jj} n|| d<| d|j| } |dkrj|}n|| j|_x1|jD]#\} } t| j|| | qW| j ||d|j d|d|| j|j}| j | j |fS(s Create a new StateMachine rooted at `node` and run it over the input `block`. Also keep track of optional intermediate blank lines and the required final one. RDRGR7R:R2N( R3RVREtcopyRGtstatest blank_finishtitemstsetattrR;R7R\RN(R!R^R=R:RDRgtblank_finish_statetextra_settingsR2R_R`RFtkeytvalue((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pytnested_list_parse&s"          cCs/|j|||r+|j|||ndS(s=Check for a valid subsection and create one if it checks out.N(tcheck_subsectiontnew_subsection(R!ttitleR.tstyletlinenotmessages((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pytsectionDscCs|j}|j}|j}y|j|d}WnVtk rt||jkrh|j|dS|j|j||7_dSnX||kr||_t|dkrt |_ n|j j t|dtn||dkrdS|j|j||7_dSdS(s Check for a valid subsection header. Return 1 (true) or None (false). When a new section is reached that isn't a subsection of the current section, back up the line count (use ``previous_line(-x)``), then ``raise EOFError``. The current StateMachine will finish, then the calling StateMachine can re-examine the title. This will work its way back up the calling chain until the correct section level isreached. @@@ Alternative: Evaluate the title, store the title info & level, and back up the chain until that level is reached. Store in memo? Or return in results? :Exception: `EOFError` when a sibling or supersection encountered. iiN(R7R)R*tindext ValueErrorRWR[RJttitle_inconsistentR3R?R+RFt previous_lineRM(R!R.RrRsR7R)tmyleveltlevel((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRoIs*         cCs+|jjdtjd|d|}|S(NsTitle level inconsistent:tRQ(R'RORt literal_block(R!t sourcetextRsterror((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRxrs  cCs=|j}|j}|jd7_tj}|j|7_|j||\}}tj|d|} t| j} |dj | || 7}||7}||7}|j j |||j j d} |j jd} |j|j j| d| d|dt} |j| |j|kr0tn||_dS(s?Append new subsection to document tree. On return, check level.iR|tnamesR=R:R2N(R7R*RRuRJt inline_textRqtnormalize_nametastextR[R&tnote_implicit_targetRFt line_offsetRNRdR<R?RLRM(R!RqRsRtR7Rzt section_nodet textnodesttitle_messagest titlenodetnametoffsett absoffsett newabsoffset((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRpxs,        c Csdj|j}tjd|ryt|dkrCgdfS|ddkrf|d j}n |d }d}n |}d}|j||\}}tj|d |}|jj |\|_ |_ |g||fS( sW Return a list (paragraph & messages) & a boolean: literal_block next? s s(?}t|tkrP|jt|dq|j|qWdj|}dt} |rtj| tj S| SdS(s! Build, compile and return a regular expression based on `definition`. :Parameter: `definition`: a 4-tuple (group name, prefix, suffix, parts), where "parts" is a list of regular expressions and/or regular expression definitions to be joined into an or-group. t|s.%(prefix)s(?P<%(name)s>%(or_group)s)%(suffix)sN( ttypettupleR[t build_regexpR3RtlocalsRtcompiletUNICODE( t definitionRRtprefixtsuffixtpartst part_stringstparttor_grouptregexp((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRs  R4cBseZdZdZdZdZdejejfZ dej ejej fZ dZ dZdZd Zd Zd Zd Zd ZdeZdZdZde dddedddddgfdde deddddded d!egfgfd"d#eed$gfgfZed%eed&ejed'e ejd(ejed)e ejd*ejd+eejejBd,ejd-eejejBd.eje d/e d0ejed1e d2ejed3e d4ejeed5ejejBd6ejd7ed8eejejBd9ejd:eejejBd;ejd<eejejB Z d=Z!e"d>Z#d?Z$d@Z%dAZ&dBZ'dCZ(dDZ)dEZ*dFZ+dGZ,dHZ-dIZ.e"dJZ/dKZ0dLZ1dMZ2dNZ3dOZ4dPZ5i e%dQ6e&dR6e'dS6e+d6e,d6e.dT6e-dU6e/dV6e0dW6Z6RS(Xs9 Parse inline markup; call the `parse()` method. cCs|jj|jfg|_dS(N(tpatternsturitstandalone_uritimplicit_dispatch(R!((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyR#scCsZ|jr+|jj|jj|jfn|jrV|jj|jj|jfndS(s6Setting-based customizations; run when parsing begins.N( tpep_referencesRR[Rtpept pep_referencetrfc_referencestrfct rfc_reference(R!R0((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyR5s   cCsg|j|_|j|_|j|_||_|jjj}|j}t|}g}g} g} x|r*||} | r&| j } || dp| dp| dp| d} | || |\}}}}| j || |7} |r'||j dj | |7}||7}g} q'qfPqfWdj | |}|r]||j ||7}n|| fS(s Return 2 lists: nodes (text and inline elements), and system_messages. Using `self.patterns.initial`, a pattern which matches start-strings (emphasis, strong, interpreted, phrase reference, literal, substitution reference, and inline target) and complete constructs (simple reference, footnote reference), search for a candidate. When one is found, check for validity (e.g., not a quoted '*' character). If valid, search for the corresponding end string if applicable, and check it for validity. If not found or invalid, generate a warning and ignore the start-string. Implicit inline markup (e.g. standalone URIs) is found last. tstartt backquotetrefendtfnendR|( R'R&R(RJRtinitialRtdispatchRt groupdictR[timplicit_inlineR(R!RRsR7RJtpattern_searchRt remainingt processedt unprocessedRttmatchtgroupstmethodtbeforetinlinest sysmessages((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRs<               u(^|(?<=\s|[%s%s]))u($|(?=\s|[%s%s%s]))s (?]s[_~*/=+a-zA-Z0-9]s-(?:%(urilast)s|%(uric)s(?=%(uri_end_delim)s))s"[-_!~*'{|}/#?^`&=+$%a-zA-Z0-9\x00]s %(emailc)s+(?:\.%(emailc)s+)* # name (?%s)(?P__?)t footnotelabels\[s(?P\]_)s[0-9]+s\#(%s)?s\*s(?P%s)Rs(?P(:%s:)?)s`(?!`)Rtemphasiss(\*)tstrongs(\*\*)tinterpreted_or_phrase_refs %(non_unescaped_whitespace_escape_before)s ( ` (?P (?P:%(simplename)s:)? (?P__?)? ) ) %(end_string_suffix)s t embedded_links% ( (?:[ \n]+|^) # spaces or beginning of line/string < # open bracket %(non_whitespace_after)s ([^<>\x00]+(\x00_)?) # anything but angle brackets & nulls # except escaped trailing low line %(non_whitespace_before)s > # close bracket w/o whitespace before ) $ # end of string tliterals(``)ttargets(`)tsubstitution_refs (\|_{0,2})temailt$Rs5 %(start_string_prefix)s (?P (?P # absolute URI (?P # scheme (http, ftp, mailto) [a-zA-Z][a-zA-Z0-9.+-]* ) : ( ( # either: (//?)? # hierarchical URI %(uric)s* # URI characters %(uri_end)s # final URI char ) ( # optional query \?%(uric)s* %(uri_end)s )? ( # optional fragment \#%(uric)s* %(uri_end)s )? ) ) | # *OR* (?P # email address s] ) ) %(end_string_suffix)s Rs %(start_string_prefix)s ( (pep-(?P\d+)(.txt)?) # reference to source file | (PEP\s+(?P\d+)) # reference by name ) %(end_string_suffix)sRs{ %(start_string_prefix)s (RFC(-|\s+)?(?P\d+)) %(end_string_suffix)scCsl|j}|j}|dkr%tS||d}y||j}Wntk r[tSXtj||S(sTest if inline markup start-string is 'quoted'. 'Quoted' in this context means the start-string is enclosed in a pair of matching opening/closing delimiters (not necessarily quotes) or at the end of the match. ii(tstringRR6tendRZR?Rt match_chars(R!RRRtprestartt poststart((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyt quoted_starts    cCsr|j}|jd}|jd}|j|rQ|| g||gdfS|j||} | r| jdrt| j| jd |} || jd} t||| !d} || || | g|| g| jdfS|jjd|j d|} t|||!d} t|||!d} |j | | | }|| |g||| gdfS(NRR|is*Inline %s start-string without end-string.RQ( RRRRRRtgroupR'RRt problematic(R!RRst end_patternt nodeclasstrestore_backslashesRt matchstarttmatchendtendmatchRttextendt rawsourcetmsgtprb((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyt inline_objs(   cCsS|jj||j}tj||d|}|jj|}|j||S(Ntrefid(R&tset_idRJRRt add_backref(R!RRtmessagetmsgidRtprbid((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRs  cCs@|j|||jjtj\}}}}}||||fS(N(RRRR(R!RRsRRRRt endstring((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRs*cCs@|j|||jjtj\}}}}}||||fS(N(RRRR(R!RRsRRRRR((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRs*cCs|jj}|j}|jd}|jd}|jd}|jd}d} |rs|dd!}d} n'|j|r|| g||gfS|j||} | r| jdr|| j} | jdrg|rH|jj dd|} t ||| !d} |j | | | }|| |g|| | gfS| jd dd!}d } n| j| jd }t ||| !d}|dd kr5|r|jj d | d|} t ||| !d} |j | | | }|| |g|| | gfS|j || || ||t |St ||| !d}|j ||||\}}|| ||| |fSn|jj d d|} t |||!d} |j | | | }|| |g||| gfS( NRtroleR|iiRsVMultiple roles in interpreted text (both prefix and suffix present; only one allowed).RQRt_s=Mismatch: both interpreted text role %s and reference suffix.sLInline interpreted text or phrase reference start-string without end-string.(RRRRRRRRR'RRRt phrase_reft interpreted(R!RRsRRRRt rolestartRtpositionRRRRRtescapedRtnodelistRt((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRs`           cCs|jjj|}|rct||jd }t|jddt}|jdr|jdp|jjj | rd}t |d } t j |jdd | } |d | _ nqd }d j|j} |j| } | jdr| d d} nt j |jdd | } d| _|sQtd|n|si| }qind} t |} t j||dt|} | g} |d dkr| r|dkr| | d <|jj| q| r|d kr| | d 6ed?6d@dA6dBdC6dDedE6d dF6Z!dZ"dGZ#dHZ$ejdIejZ%dJZ&dKZ'dLZ(dMZ)dNZ*dOZ+ddPZ-dQZ.dRZ/dSZ0dTZ1dUZ2dVZ3dWZ4dXZ5dYZ6dZZ7d[Z8d\Z9d]Z:d^Z;d_Z<d`Z=daZ>dbZ?dcZ@ddZAd d deZBd dfZCdgZDeZEedhejdieFeGejHejBdjejdkeFeGejHejBdlejdmeFeGejHejBeE_!dnZIdoZJdpZKdqZLdrZMdsZNdtZOduZPdvZQdwZRdxZSdyZTdzZUd{ZVd|ZWd}ZXd~ZYeIejdeGjZejHejBfeJejdeGjZejHejBfeKejdejHejBfePejdejHejBfeRejdeGjZejHejBfgeE_[dZ\dZ]dZ^dZ_dZ`dZadZbRS(s: Generic classifier of the first line of a block. Rt(Rt)RiRitparensR|itrparent.tperiodtarabict loweralphat upperalphat lowerromant upperromans[0-9]+s[a-z]s[A-Z]s [ivxlcdm]+s [IVXLCDM]+Rs\+-[-+]+-\+ *$s =+( +=+)+ *$s=+[ =]*$s[!-/:-@[-`{-~]tnonalphanum7bits[a-zA-Z]talphas [a-zA-Z0-9]talphanums [a-zA-Z0-9_-]t alphanumplussJ(%(arabic)s|%(loweralpha)s|%(upperalpha)s|%(lowerroman)s|%(upperroman)s|#)tenums%(alphanum)s%(alphanumplus)s*toptnames%(%(alpha)s%(alphanumplus)s*|<[^<>]+>)toptargs!(-|\+)%(alphanum)s( ?%(optarg)s)?tshortopts"(--|/)%(optname)s([ =]%(optarg)s)?tlongopts(%(shortopt)s|%(longopt)s)toptions(?P<%s>%s%s%s)u[-+*•‣⁃]( +|$)tbullets((%(parens)s|%(rparen)s|%(period)s)( +|$)t enumerators#:(?![: ])([^:\\]|\\.)*(?>>( +|$)tdoctests\|( +|$)t line_blocktgrid_table_toptsimple_table_tops \.\.( +|$)texplicit_markups__( +|$)Rs(%(nonalphanum7bit)s)\1* *$RQRc Csj|jj\}}}}|j||}|j|7_|s]|j|jd7_n||gfS(s Block quote.s Block quote(RFt get_indentedt block_quoteRJR( R!RRRt next_statetindentedtindentRRgtelements((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRtks c Csg}x|r|j||\}}}}}tj}|j||||j||r|j||\} } || 7}|| 7}n|}x)|r|d r|d}|d7}qWq W|S(Nii(tsplit_attributionRRqRdR[tparse_attribution( R!RsRRutblockquote_linestattribution_linestattribution_offsettnew_line_offsett blockquotet attributionRt((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRqus  !     u(---?(?!-)|—) *(?=[^ \n])c Csd}t}xtt|D]}||j}|r|r||dkr|jj|}|r|j||\}} |r|||!} | j|j dd| j| dd|| | |||||fSqnt }q|}qW|ddddfSdS(s Check for a block quote attribution and split it off: * First line after a blank line must begin with a dash ("--", "---", em-dash; matches `self.attribution_pattern`). * Every line after that must have consistent indentation. * Attributions must be preceded by block quote content. Return a tuple of: (block quote content lines, content offset, attribution lines, attribution offset, remaining indented lines). iRRN( R3R6trangeRWRtattribution_patternRtcheck_attributiont trim_leftRR?( R!RsRtblankt nonblank_seentiRQRtattribution_endRtta_lines((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRvs(      cCsd}|d}xt|dt|D]q}||j}|sJPn|dkrut|t|j}q*t|t|j|kr*dSq*W|d7}||pdfS(st Check attribution shape. Return the index past the end of the attribution, and the indent. iiN(NN(R3R~RWRtlstrip(R!Rstattribution_startRtRRQ((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRs   " cCs}dj|j}|jj|}|j||\}}tj|d|}|jj|\|_|_ ||fS(Ns R|( RRRFRRRR}RHR.RQ(R!RsRRRsRRtR:((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRws c Cstj}|j|7_|jd|d<|j|j\}}||7}|jjd}|j|jj |d|jj dd|ddd|\}}|j ||s|j|j d 7_ng|gfS( sBullet list item.iRgiR=R:RDt BulletListRgs Bullet list( Rt bullet_listRJRt list_itemRRFRRnR<RNRLR( R!RRRRrt bulletlistRRgRR{((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRgs    cCs|jj|r.|jj|\}}}n|jj|\}}}}tjdj|}|r|j|d|d|n||fS(Ns R=R:(RFRQtget_known_indentedtget_first_known_indentedRRRRd(R!RtRsRRgtlistitem((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRs cCs|j|\}}}}|j|||sBtjdntj}|j|7_|dkrvd|d           cCs|j}d}x.|jjD]}||rPqqWtd|||jj|j|jj|j!}|dkrd}n}|ry&|jj|j|r|}nWqt k rtd|qXn*|dkrd}n|dkrd}n|sIx@|jj D]#}|jj|j|rPqqWtd n|dkr^d }n7y|jj ||}Wnt j k rd }nX||||fS( sA Analyze an enumerator and return the results. :Return: - the enumerator format ('period', 'parens', or 'rparen'), - the sequence used ('arabic', 'loweralpha', 'upperroman', etc.), - the text of the enumerator, stripped of formatting, and - the ordinal value of the enumerator ('a' -> 1, 'ii' -> 2, etc.; ``None`` is returned for invalid enumerator text). The enumerator format has already been determined by the regular expression match. If `expected_sequence` is given, that sequence is tried first. If not, we check for Roman numeral 1. This way, single-character Roman numerals (which are also alphabetical) can be matched. If no sequence has been matched, all sequences are checked in order. R|senumerator format not matchedRsunknown enumerator sequence: %sRR[tIR\senumerator sequence not matchediN(RRatformatsRRRRtsequenceregexpsRtKeyErrort sequencest convertersRtInvalidRomanNumeralErrorR3(R!Rtexpected_sequenceRRRRR((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRsB              cCs|dkrdSy|jj}Wntk rD|jjdSX|jj|d jsfdS|j|d||}|r|\}}y&|j|s|j|rdSWqtk rqXndS(s Check validity based on the ordinal value and the second line. Return true if the ordinal is valid and the second line is blank, indented, or starts with the next enumerator or an auto-enumerator. iN( R3RFR]RMRytstriptmake_enumeratorR*t TypeError(R!RRRR]tresulttnext_enumeratortauto_enumerator((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRGs(      cCsO|dkrd}n|dkr0t|}n|jdrl|dkrOd St|tdd}nM|jdrytj|}Wqtjk rd SXntd||j d r|j }n.|j d r|j }ntd||j j |}|j||jd }|jd|jd }||fS( s Construct and return the next enumerated list item marker, and an auto-enumerator ("#" instead of the regular enumerator). Return ``None`` for invalid (out of range) ordinals. RRXR^iRHiRs!unknown enumerator sequence: "%s"R#RPt N(tstrRR3tchrRIRttoRomant RomanErrorRR*R#RPRaRRR(R!RRRRhRRR((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRds2      c Cstj}|j|7_|j|\}}||7}|jjd}|j|jj|d|jjdd|ddd|\}}|j ||s|j|j d7_ng|gfS(sField list item.iR=R:RDt FieldListRgs Field list( Rt field_listRJtfieldRFRRnR<RNRLR( R!RRRRrRRRgRR((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRis    cCs|j|}|jj\}}|jj}|jj|j\}}}} tj} || _|| _ |j ||\} } | tj |d| 7} tj dj || } | | 7} |r|j||| n| | fS(NR|s (tparse_field_markerRFRHRRRRRR.RQRt field_namet field_bodyRtparse_field_body(R!RRtsrctsrclineRsRsRtRRgt field_nodet name_nodest name_messagesR((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRs$    cCs'|jd}||jd }|S(s6Extract & return field name from a field marker match.it:(Rtrfind(R!RR((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRscCs|j|d|d|dS(NR=R:(Rd(R!RsRR:((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRsc Csxtj}y|j|\}}Wntk r}|jjd|}|j|7_|jj|j \} } } }|j | | } |j| 7_|s|j|j d7_ng|gfSX|j|7_||7}|jj d} |j |jj| d|jjdd|ddd|\}}|j||sk|j|j d7_ng|gfS( sOption list item.uInvalid option list marker: %ss Option listiR=R:RDt OptionListRg(Rt option_listtoption_list_itemRR'RRJRFRRRqRRRnR<RNRL(R!RRRRrt optionlistRRgRRRsRtRRuRR((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRjs2   $   c Cs|jj}|j|}|jj|j\}}}}|sg|j|tjdntj d|}tj dj |} tj d|| } |r|j |d|d| n| |fS(NRR|s R=R:(RFRNtparse_option_markerRRRLRRRt option_groupt descriptionRRRd( R!RRtoptionsRsRtRRgRRR((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRs$    c Csg}|jjjd}x|D]}|j}d}|djdd}t|dkr{||d*d}nyt|ddkr|djdr|djd s|djd r|dd |ddg|d*d }nt|dkrI|djd rI|d jd rIdj|dg|d)ndt|kofdknrtj|}|tj |d|d7}t|dkr|tj |d|dd|7}n|j |q(t dt||fq(W|S(s Return a list of `node.option` and `node.option_argument` objects, parsed from an option marker match. :Exception: `MarkupError` for invalid option markers. s, Rit=iit-s--t+R|tt delimiters;wrong number of option tokens (=%s), should be 1 or 2: "%s"( RRRRWR*RRRRft option_stringtoption_argumentR[R( R!Rtoptlistt optionstringst optionstringttokensRtfirstoptRf((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRs:      %" cCs@dj|jj}|jtj||7_g|gfS(Ns (RRFtget_text_blockRJRt doctest_block(R!RRRRrR((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRk sc CsBtj}|j|7_|jj}|j||\}}}||7}|j|7_|s|jjd} |j|jj| d|jj dd|dddd\} }|j | n|s|j|j j dd |d7_nt |r5|djd kr%d|d_n|j|ng|gfS( sFirst line of a line block.iR=R:RDt LineBlockRgis%Line block ends without a blank line.RQN(RRlRJRFRtline_block_lineRRnR<RNRLR'RRWRtR3tnest_line_block_lines( R!RRRRrR^RsRQRtRgRR{((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRls.    c Cs|jj|jdt\}}}}dj|}|j||\}} tj|d|} |jj dkrt |j dd| _ n| | |fS(s(Return one line element of a line_block.t until_blanku R|Ri( RFRRR?RRRRQRRRWRRt( R!RRsRsRtRRgRt text_nodesRtRQ((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyR+scCshxTtdt|D]=}t||dddkr||dj||_qqW|j|dS(NiRt(R~RWtgetattrR3Rttnest_line_block_segment(R!R^Rv((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyR7scCsg|D]}|j^q}t|}g}tj}xo|D]g}|j|krf|j|qAt|r|j||j|tj}n|j|qAWt|r|j||j|n||(dS(N(RttminRRlR[RWR(R!R^titemtindentstleastt new_itemst new_block((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyR=s         cCs|j||||jtjS(sTop border of a full table.(t table_toptisolate_grid_tableRtGridTableParser(R!RRRRr((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRmPscCs|j||||jtjS(sTop border of a simple table.(Rtisolate_simple_tableRtSimpleTableParser(R!RRRRr((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRnVsc Csq|j||\}}|j|7_|sd|jjdd|jjd}|j|7_ng|gfS(sTop border of a generic table.s Blank line required after table.RQi(ttableRJR'RRFR( R!RRRRrtisolate_functiont parser_classRRgR((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyR\s c Cs|\}}}|ryX|}|j|}|jjt|d}|j||} | g|} Wqtjk r} |j|dj| j d| j |} qXn|} | |fS(sParse a table.iRR( RRFRRWt build_tableRtTableMarkupErrortmalformed_tableRtargsR( R!RRR^RtRgtparsert tabledatat tablelineRRterr((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRhs c Cs>g}d}y|jjdt}WnStjk rz}|j\}}}|j|jjdd|d|d}nX|j |j |j t |dj }xmtt |D]Y}||j ||<||ddkrd}|jjt ||||3PqqW|jj|dsd}xtt |d ddD]I}|jj||rT|jjt ||d||d3PqTqTW|j|j|g||fSnxgtt |D]S}t |||ks ||ddkr|j|j|g||fSqW|||fS( Nit flush_leftsUnexpected indentation.R.RQis+|ii(RFRR?RtUnexpectedIndentationErrorRR[R'Rt disconnecttpad_double_widthtdouble_width_pad_charRWRR~Rytgrid_table_top_patRtextendR( R!RtRgR^RRRtwidthR((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRzsB  # *cCs1|jj}|jj}t|d}t||j}|jj}d}d}|d}xq||krS||} || } | rFt| j|kr|jj|||j |||d!d} g| ||kp||dj fS|d7}|}|dks9||ks9||dj rF|} PqFn|d7}qcW|rd} |jj|||||d!}n(d} |jj||d||}|j |d| } g| | fS|jj| |||| d!}|j |j |g| |kp-|| dj fS(Niis5Bottom/header table border does not match top border.is$ or no blank line after table bottomR|sNo bottom table border found%s.( RFRR<RWRtsimple_table_border_patRR3R]RRR(R!RRtlimitttoplent pattern_matchtfoundtfound_atRRQRRtRtextraR^((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRsJ      ( - cCs|j|jddj|}d}|jjt|d}|r\|d|7}n|jj|tj ||d||}|gS(NR|s sMalformed table.iRQ( treplaceRRRFRRWR'RRR}(R!R^tdetailRRRt startlineR((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRs cCs|\}}}tj}tjdt|}||7}xJ|D]B} tjd| } |r|d| jd<|d8}n|| 7}qDW|rtj} || 7}x'|D]} | |j| |7} qWntj} || 7}x$|D]} | |j| |7} qW|S(Ntcolstcolwidthitstub( RRttgroupRWtcolspect attributesttheadtbuild_table_rowttbody(R!RRt stub_columnst colwidthstheadrowstbodyrowsRRRR R trowR ((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRs(           c Cstj}x|D]}|dkr+qn|\}}}}i} |rV|| d`?) # optional open quote (?![ `]) # first char. not space or # backquote (?P # reference name .+? ) %(non_whitespace_escape_before)s (?P=quote) # close quote if open quote used ) (?%(simplename)s)_ | # *OR* ` # open backquote (?![ ]) # not space (?P.+?) # hyperlink phrase %(non_whitespace_escape_before)s `_ # close backquote, # reference mark ) $ # end of string t substitutions ( (?![ ]) # first char. not space (?P.+?) # substitution text %(non_whitespace_escape_before)s \| # close delimiter ) ([ ]+|$) # followed by whitespace c Cs|jj\}}|jj|j\}}}}|jd}t|} tjdj|} || _ || _ | ddkr| d} d| d<| r| dj | n|j j | nf| dkrd} d| d<|j j| n7| tjd|7} | dj | |j j| | rQ|j j| | n|j j| | |r|j|d |d | n| g|fS( Nis iRRRRR|R=R:(RFRHRRRRRtfootnoteRR.RQR[R&tnote_autofootnotetnote_symbol_footnoteRt note_footnoteRRRd( R!RRRRsRtRRgRRR((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyR/s4$       c Cs|jj\}}|jj|j\}}}}|jd}t|} tjdj|} || _ || _ | tj d|7} | dj | |j j| |j j| | |r|j|d|d| n| g|fS(Nis R|RR=R:(RFRHRRRRRtcitationRR.RQRR[R&t note_citationRRd( R!RRRRsRtRRgRRR((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRNs$   cCsd|jjj}|jj}|jj|jdtdt\}}}}|j |j dj |}g|D]} t | ^qu}|d} d} x_tr|j | } | rPn| d7} y| || 7} Wqt k rtdqXqW|| 4|dd| jt| dj|d<|j|||| jd} | g|fS( NRt strip_indents iismalformed hyperlink target.RR(texplicitRRRFRRRR?R6RRRRRZRRWRt make_targetR(R!RR4RsR^RtRRgt blocktextRQRt blockindext targetmatchR((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pythyperlink_target_s, '     0cCs|j|||\}}|dkrxtj|ddt|}||_|j|d|||jj||S|dkrtj|d}|j|||||S|SdS(NRR|R(t parse_targetRRRRt add_targetR&R(R!R^t block_textRst target_namet target_typeRR((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyR"xs   cCs|rj|djddkrjdjg|D]}|j^q-}|j|}|rjd|fSndjg|D]}dj|j^qw}dt|fS(s Determine the type of reference of a target. :Return: A 2-tuple, one of: - 'refname' and the indirect reference name - 'refuri' and the URI - 'malformed' and a system_message node iRRRR|R(RRt is_referenceRR(R!R^R)RsRQRR((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyR's ( 1cCsJ|jjjjt|}|s(dSt|jdpF|jdS(Ntsimpletphrase(R!RRRR R3RR(R!RR((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyR,s cCs||_|rtt|}|dj||rp|jj|}|r]||d|j|\}}|j|7_|j|g|gfS(s3Footnotes, hyperlink targets, directives, comments.(texplicit_constructRJt explicit_list(R!RRRRrRRg((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRos c Csg}x|jjD]\}}|j|j}|ry|||SWqtk r}|jj}dj|j}|j |j j |d|PqXqqW|j |\} } | || fS(s>Determine which explicit construct this is, parse & return it.RRQ( R!t constructsRRRRFRRRR[R'RRk( R!RterrorsRR4texpmatchRRsRRRg((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRl s c Cs|jjd}|j|jj|d|jjdd|jddd|d|jj\}}|j||s|j|jd7_nd S( s Create a nested state machine for a series of explicit markup constructs (including anonymous hyperlink targets). iR=R:RDtExplicitRgR2sExplicit markupN( RFRRnR<RNRJR2RLR(R!RgRR((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRm s cCs>|j|\}}|j|7_|j|g|gfS(sAnonymous hyperlink targets.(tanonymous_targetRJRm(R!RRRRrRRg((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyR s c Cs|jj}|jj|jdt\}}}}|j|j dj|}g|D]}t|^q`}|j|||d} | g|fS(NRs R|( RFRRRR?RRRR"( R!RRsR^RtRRgR#RQR((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRr' s cCs|jjr|jgdgfS|jjdkrFtjdnt|jjdkr|jjdd|jj }|j |7_ tjdnX|jj }|jj dt j||d|jj }|j |7_ g|gfSdS( s,Section title overline or transition marker.tLines::RiseUnexpected possible title overline or transition. Treating it as ordinary text because it's so short.RQs'Unexpected section title or transition.N(RFR2RRRRRWR'RRRJRQRORR}(R!RRRRrRR#((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRQ1 s"    cCs|jgdgfS(s%Titles, definition lists, paragraphs.R(R(R!RRRRr((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRG s( sbullets enumerators field_markers option_markersdoctests line_blocksgrid_table_topssimple_table_topsexplicit_markups anonymousslinestextN(cRRR$Rt TableParserRRRaRtkeysRRt sequencepatsR+RLRNRQRRORRRRRRRtsimple_table_top_patRtpatsRtescapeRRRtinitial_transitionsRtRqRRvRRwRgRRhR3RRRRiRRRRjRRRkRlRRRRmRnRRRRRRR R!tvarsR4RGRRR&R"R'R,R(R=R6R@RARIR]R^R`RBRkR@RnRoRlRmRRrRQR(((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRCs,  ""(     %    ,        #    # 7  %       )        & +              D  < .   ! &     t RFC2822BodycBskeZdZejjZded|jjd}|d}|j}d}y|jj}Wntk r|d|}t|jdkr|j|||dq|jj dt j ||d|} |j | 7_ gd gfSnXd |||f} |j}|j}|j d dj|s|d|d|}t|jdkra|j|||dq@|jj d t j | | d|} |j | 7_ gd gfSn||kr@|d|d|}t|jdkr|j|||dq@|jj d t j | | d|} |j | 7_ gd gfSn|j}g} t|t|kr|d|d|}t|jdkr|j|||dq|jjdt j | | d|} | j| n|d|df} d|_|j|j| | |d| d|_gd gfS(s#Potential over- & underlined title.iiR|s iisIncomplete section title.RQRCs%s %s %sRs6Missing matching underline for section title overline.s$Title overline & underline mismatch.sTitle overline too short.(RFRRR]RMRWRtshort_overlineR'RORR}RJRSRRRR[RRuR( R!RRRRrRstoverlineRqRR#RR.RtRr((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRW sn                # cCs|d}|d|jj}|jjd}t|jdkrb|j|||dn|jjdtj ||d|}|j |7_ gdgfS(Nis iis+Invalid section title or transition marker.RQRC( RFRQRRWRRR'RRR}RJ(R!RRRRrRR#RsR((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyR s   cCs;|jjdd|}|j|7_|j||dS(Ns`Possible incomplete section title. Treating the overline as ordinary text because it's so short.RQ(R'RRJR(R!RRR#RsRR((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyR s   cCs-|jj|g|(tjdddS(NRCR(RFRyRtStateCorrection(R!RRR((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyR s( RRR$RRRRRtRRR(((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRs3 s  > RcBsqeZdZidejd6dd6Zd ZedZdZ dZ dZ d Z d Z d ZRS( s Nested parse handler for quoted (unindented) literal blocks. Special-purpose. Not for inclusion in `state_classes`. s(%(nonalphanum7bit)s)tinitial_quotedR|RcCs)tj|||g|_d|_dS(N(RAR#RtR3tinitial_lineno(R!RFRG((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyR# s cCs |rtn ||gfSdS(N(RM(R!RRRRr((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyR s cCs|rf|jj|j\}}dj|}tj||}||_||_|j|7_n7|j|j j dd|jj 7_|jj |j|j 7_gS(Ns s#Literal block expected; none found.RQ(RFRHRRRR}R.RQRJR'RRRyRt(R!RRRRRR}((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyR s    cCsB|jj|jjdd|jj|jjtdS(NsUnexpected indentation.RQ(RtR[R'RRFRRyRM(R!RRRRr((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyRt s   cCs|jd|jd}tjtj|tj}|jd||j|jj f|j j |_ |jg|gfS(s7Match arbitrary quote character on the first line only.Ritquoted( tremove_transitionRRRRyRtadd_transitionRRPRRFRR(R!RRRRrtquoteR4((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyR s   cCs|j|j||gfS(s,Match consistent quotes on subsequent lines.(R[R(R!RRRRr((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyR scCsK|rA|jj|jjdd|jj|jjntdS(Ns#Inconsistent literal block quoting.RQ(RtR[R'RRFRRyRM(R!RRRRr((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyR s  (sinitial_quotedstext(RRR$RCRxRRzR6R#RRRtRRR(((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pyR s     (GR$t __docformat__tsysRttypesRRRJRRRRRtdocutils.statemachineRR tdocutils.nodesR RR RHR R RRtdocutils.parsers.rst.languagesRt_fallback_language_moduletdocutils.utilsRRRRRRRRRRt ExceptionRRR%R@RAR?RR4RIRLRNRQRCR|RRRRRRRRgRRqR1RRRRsRRB(((sE/usr/local/lib/python2.7/site-packages/docutils/parsers/rst/states.pytesr   "' ; B.,  K