Y-Qc @s dZddlZddlZddlmZmZddlmZddlm Z m Z m Z m Z ddl mZmZmZmZddd d d d d dddg Zd1d2d3d4d5gZedZdefdYZdefdYZd efdYZd efd YZd!efd"YZeZd#efd$YZ d%efd&YZ!d'Z"d(efd)YZ#e#Z$d*Z%d+efd,YZ&defd-YZ'd efd.YZ(d e'fd/YZ)d0Z*dS(6s pygments.lexer ~~~~~~~~~~~~~~ Base lexer classes. :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. iN(t apply_filterstFilter(tget_filter_by_name(tErrortTexttOthert _TokenType(t get_bool_optt get_int_optt get_list_opttmake_analysatortLexert RegexLexertExtendedRegexLexertDelegatingLexert LexerContexttincludetinherittbygroupstusingtthisssutf-8ssutf-32tsutf-32bessutf-16ssutf-16becCsdS(Ng((tx((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pytst LexerMetacBseZdZdZRS(s This metaclass automagically converts ``analyse_text`` methods into static methods which always return float values. cCs9d|kr#t|d|ds(R*t __class__R (R/((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyt__repr__es   cKs5t|ts!t||}n|jj|dS(s8 Add a new stream filter to this lexer. N(t isinstanceRRR)tappend(R/R0R*((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyR.lscCsdS(s~ Has to return a float between ``0`` and ``1`` that indicates if a lexer wants to highlight this text. Used by ``guess_lexer``. If this method returns ``0`` it won't highlight it in any case, if it returns ``1`` highlighting with this lexer is guaranteed. The `LexerMeta` metaclass automatically wraps this function so that it works like a static method (no ``self`` or ``cls`` parameter) and the return value is automatically converted to `float`. If the return value is an object that is boolean `False` it's the same as if the return values was ``0.0``. N((ttext((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyRtsc skttskjdkryy5jdjdrRtdnWqhtk rujdqhXqjdkrVyddl}Wntk rtdnXd}xFt D]>\}}j|rtt||d d }PqqW|dkrM|j d }t|j d p>dd d }n|qjjn"jdrtdnj d dj ddjrjnjrjdnjdkrjjnjr.jd r.d7nfd}|} |sgt| j} n| S(s= Return an iterable of (tokentype, value) pairs generated from `text`. If `unfiltered` is set to `True`, the filtering mechanism is bypassed even if filters are defined. Also preprocess the text, i.e. expand tabs and strip it if wanted and applies registered filters. tguesssutf-8uR(tchardetiNskTo enable chardet encoding guessing, please install the chardet library from http://chardet.feedparser.org/terrorstreplaceiR's s s ic3s2x+jD]\}}}||fVqWdS(N(tget_tokens_unprocessed(titttv(R/R6(s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pytstreamers(R4tunicodeR'tdecodet startswithtlentUnicodeDecodeErrorR8t ImportErrortNonet _encoding_maptdetectR-R:R$tstripR#R&t expandtabsR%tendswithRR)( R/R6t unfilteredR8tdecodedtbomR'tencR?tstream((R/R6s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyt get_tokenssT           cCs tdS(s Return an iterable of (tokentype, value) pairs. In subclasses, implement this method as a generator to maximize effectiveness. N(tNotImplementedError(R/R6((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyR;sN(R R!R"RFRtaliasest filenamestalias_filenamest mimetypestpriorityRt __metaclass__R1R3R.RR,RQR;(((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyR -s    ?cBs#eZdZedZdZRS(s  This lexer takes two lexer as arguments. A root lexer and a language lexer. First everything is scanned using the language lexer, afterwards all ``Other`` tokens are lexed using the root lexer. The lexers from the ``template`` lexer package use this base lexer. cKs;|||_|||_||_tj||dS(N(t root_lexertlanguage_lexertneedleR R1(R/t _root_lexert_language_lexert_needleR*((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyR1s cCsd}g}g}x}|jj|D]i\}}}||jkrx|rk|jt||fg}n||7}q%|j|||fq%W|r|jt||fnt||jj|S(Nt(RZR;R[R5RCt do_insertionsRY(R/R6tbufferedt insertionst lng_bufferR<R=R>((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyR;s"  (R R!R"RR1R;(((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyRs cBseZdZRS(sI Indicates that a state should include rules from another state. (R R!R"(((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyRst_inheritcBseZdZdZRS(sC Indicates the a state should inherit from its superclass. cCsdS(NR((R/((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyR3s(R R!R"R3(((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyRdstcombinedcBs eZdZdZdZRS(s: Indicates a state combined from multiple states. cGstj||S(N(ttupleR(Rtargs((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyRscGsdS(N((R/Rg((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyR1 s(R R!R"RR1(((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyRes t _PseudoMatchcBsMeZdZdZddZddZddZdZdZ RS(s: A pseudo match object constructed from a string. cCs||_||_dS(N(t_textt_start(R/tstartR6((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyR1s cCs|jS(N(Rj(R/targ((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyRkscCs|jt|jS(N(RjRCRi(R/Rl((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pytendscCs|rtdn|jS(Ns No such group(t IndexErrorRi(R/Rl((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pytgroupscCs |jfS(N(Ri(R/((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pytgroups#scCsiS(N((R/((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyt groupdict&sN( R R!R"R1RFRkRmRoRpRq(((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyRhs     csdfd}|S(sL Callback that yields multiple actions for each group in the match. c3sxtD]\}}|dkr+q q t|tkrw|j|d}|r|j|d||fVqq |j|d}|dk r |r|j|d|_nxA||t|j|d||D]}|r|VqqWq q W|r|j|_ndS(Ni( t enumerateRFRRRoRktposRhRm(tlexertmatchtctxR<tactiontdatatitem(Rg(s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pytcallback.s" ! N(RF(RgRz((Rgs8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyR*st_ThiscBseZdZRS(sX Special singleton used for indicating the caller class. Used by ``using``. (R R!R"(((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyR{Dsc sidkrVjd}t|ttfrC|d(t gt_kwargstkwargs(s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyRzbs (c3sj|j}|j}x;|j|jD]!\}}}||||fVqAW|r~|j|_ndS(N(RR*RkR;RoRmRs(RtRuRvRRR<R=R>(t_otherRR(s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyRzqs  (N(tpopR4tlistRfRRF(RRRRz((RRRs8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyRLs     tRegexLexerMetacBsPeZdZdZdZdZdZddZdZ dZ RS( sw Metaclass for RegexLexer, creates the self._tokens attribute from self.tokens on the first instantiation. cCstj||jS(sBPreprocess the regular expression component of a token definition.(tretcompileRu(Rtregextrflags((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyt_process_regexscCs|S(s5Preprocess the token component of a token definition.((Rttoken((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyt_process_tokenscCst|trd|dkrdS||kr2|fS|dkrB|S|d dkrt|d Snt|trd|j}|jd7_g}x*|D]"}|j|j|||qW|||<|fSt|trx|D]}qW|SdS( s=Preprocess the state transition action of a token definition.s#popis#pushis#pop:s_tmp_%diN(R4tstrtintRet_tmpnametextendt_process_stateRf(Rt new_statet unprocessedt processedt tmp_statetitokenstistate((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyt_process_new_states0       c Cs=||kr||Sg}||<|j}x ||D]}t|trs|j|j||t|q6nt|trq6ny|j|d|}Wn3tk r}t d|d|||fnX|j |d} t |dkrd} n|j |d||} |j|| | fq6W|S(s%Preprocess a single state definition.is+uncompilable regex %r in state %r of %r: %siiN(tflagsR4RRRRRdRt Exceptiont ValueErrorRRCRFRR5( RRRR|ttokensRttdeftrexterrRR((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyRs.     cCsRi}|j|<|p!|j|}x'|jD]}|j|||q1W|S(s-Preprocess a dictionary of token definitions.(t _all_tokensRtkeysR(RRt tokendefsRR|((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pytprocess_tokendefs c Cs.i}i}xtj|f|jD]}|jjdi}x|jD]\}}|j|}|dkr|||toldittmpvaltit_indextit_tokentit_valuetp((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyR`sL            (ssutf-8(ssutf-32(ssutf-32be(ssutf-16(ssutf-16be(+R"RRtpygments.filterRRtpygments.filtersRtpygments.tokenRRRRt pygments.utilRRR R t__all__RGt staticmethodt_default_analyseRRtobjectR RRRRdRRfReRhRR{RRRR RR R`(((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyt s<""   '     2UD