ó Y-Qc @s dZddlZddlZddlmZmZddlmZddlm Z m Z m Z m Z ddl mZmZmZmZddd d d d d dddg Zd1d2d3d4d5gZed„ƒZdefd„ƒYZdefd„ƒYZd efd„ƒYZd efd „ƒYZd!efd"„ƒYZeƒZd#efd$„ƒYZ d%efd&„ƒYZ!d'„Z"d(efd)„ƒYZ#e#ƒZ$d*„Z%d+efd,„ƒYZ&defd-„ƒYZ'd efd.„ƒYZ(d e'fd/„ƒYZ)d0„Z*dS(6s“ pygments.lexer ~~~~~~~~~~~~~~ Base lexer classes. :copyright: Copyright 2006-2013 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. i’’’’N(t apply_filterstFilter(tget_filter_by_name(tErrortTexttOthert _TokenType(t get_bool_optt get_int_optt get_list_opttmake_analysatortLexert RegexLexertExtendedRegexLexertDelegatingLexert LexerContexttincludetinherittbygroupstusingtthissļ»æsutf-8s’žsutf-32tž’sutf-32bes’žsutf-16sž’sutf-16becCsdS(Ng((tx((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pytst LexerMetacBseZdZd„ZRS(s‚ This metaclass automagically converts ``analyse_text`` methods into static methods which always return float values. cCs9d|kr#t|dƒ|ds(R*t __class__R (R/((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyt__repr__es   cKs5t|tƒs!t||}n|jj|ƒdS(s8 Add a new stream filter to this lexer. N(t isinstanceRRR)tappend(R/R0R*((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyR.lscCsdS(s~ Has to return a float between ``0`` and ``1`` that indicates if a lexer wants to highlight this text. Used by ``guess_lexer``. If this method returns ``0`` it won't highlight it in any case, if it returns ``1`` highlighting with this lexer is guaranteed. The `LexerMeta` metaclass automatically wraps this function so that it works like a static method (no ``self`` or ``cls`` parameter) and the return value is automatically converted to `float`. If the return value is an object that is boolean `False` it's the same as if the return values was ``0.0``. N((ttext((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyRtsc sktˆtƒskˆjdkryy5ˆjdƒ‰ˆjdƒrRˆtdƒ‰nWqhtk ruˆjdƒ‰qhXqˆjdkrVyddl}Wntk r·tdƒ‚nXd}xFt D]>\}}ˆj|ƒrÅtˆt|ƒ|d d ƒ}PqÅqÅW|dkrM|j ˆd ƒ}tˆ|j d ƒp>dd d ƒ}n|‰qˆjˆjƒ‰n"ˆjdƒrˆtdƒ‰nˆj d dƒ‰ˆj ddƒ‰ˆjrɈjƒ‰nˆjräˆjdƒ‰nˆjdkrˆjˆjƒ‰nˆjr.ˆjdƒ r.ˆd7‰n‡‡fd†}|ƒ} |sgt| ˆjˆƒ} n| S(s= Return an iterable of (tokentype, value) pairs generated from `text`. If `unfiltered` is set to `True`, the filtering mechanism is bypassed even if filters are defined. Also preprocess the text, i.e. expand tabs and strip it if wanted and applies registered filters. tguesssutf-8uļ»æR(tchardeti’’’’NskTo enable chardet encoding guessing, please install the chardet library from http://chardet.feedparser.org/terrorstreplaceiR's s s ic3s2x+ˆjˆƒD]\}}}||fVqWdS(N(tget_tokens_unprocessed(titttv(R/R6(s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pytstreamer¹s(R4tunicodeR'tdecodet startswithtlentUnicodeDecodeErrorR8t ImportErrortNonet _encoding_maptdetectR-R:R$tstripR#R&t expandtabsR%tendswithRR)( R/R6t unfilteredR8tdecodedtbomR'tencR?tstream((R/R6s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyt get_tokens‚sT           cCs t‚dS(s  Return an iterable of (tokentype, value) pairs. In subclasses, implement this method as a generator to maximize effectiveness. N(tNotImplementedError(R/R6((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyR;ĮsN(R R!R"RFRtaliasest filenamestalias_filenamest mimetypestpriorityRt __metaclass__R1R3R.RR,RQR;(((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyR -s    ?cBs#eZdZed„Zd„ZRS(s  This lexer takes two lexer as arguments. A root lexer and a language lexer. First everything is scanned using the language lexer, afterwards all ``Other`` tokens are lexed using the root lexer. The lexers from the ``template`` lexer package use this base lexer. cKs;|||_|||_||_tj||dS(N(t root_lexertlanguage_lexertneedleR R1(R/t _root_lexert_language_lexert_needleR*((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyR1Ōs cCsĶd}g}g}x}|jj|ƒD]i\}}}||jkrx|rk|jt|ƒ|fƒg}n||7}q%|j|||fƒq%W|r“|jt|ƒ|fƒnt||jj|ƒƒS(Nt(RZR;R[R5RCt do_insertionsRY(R/R6tbufferedt insertionst lng_bufferR<R=R>((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyR;Śs"  (R R!R"RR1R;(((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyRŹs cBseZdZRS(sI Indicates that a state should include rules from another state. (R R!R"(((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyRńst_inheritcBseZdZd„ZRS(sC Indicates the a state should inherit from its superclass. cCsdS(NR((R/((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyR3üs(R R!R"R3(((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyRdųstcombinedcBs eZdZd„Zd„ZRS(s: Indicates a state combined from multiple states. cGstj||ƒS(N(ttupleR(Rtargs((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyRscGsdS(N((R/Rg((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyR1 s(R R!R"RR1(((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyRes t _PseudoMatchcBsMeZdZd„Zdd„Zdd„Zdd„Zd„Zd„Z RS(s: A pseudo match object constructed from a string. cCs||_||_dS(N(t_textt_start(R/tstartR6((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyR1s cCs|jS(N(Rj(R/targ((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyRkscCs|jt|jƒS(N(RjRCRi(R/Rl((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pytendscCs|rtdƒ‚n|jS(Ns No such group(t IndexErrorRi(R/Rl((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pytgroupscCs |jfS(N(Ri(R/((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pytgroups#scCsiS(N((R/((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyt groupdict&sN( R R!R"R1RFRkRmRoRpRq(((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyRhs     csd‡fd†}|S(sL Callback that yields multiple actions for each group in the match. c3sxśtˆƒD]ģ\}}|dkr+q q t|ƒtkrw|j|dƒ}|rł|j|dƒ||fVqłq |j|dƒ}|dk r |rµ|j|dƒ|_nxA||t|j|dƒ|ƒ|ƒD]}|rŽ|VqŽqŽWq q W|r|jƒ|_ndS(Ni( t enumerateRFRRRoRktposRhRm(tlexertmatchtctxR<tactiontdatatitem(Rg(s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pytcallback.s" ! N(RF(RgRz((Rgs8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyR*st_ThiscBseZdZRS(sX Special singleton used for indicating the caller class. Used by ``using``. (R R!R"(((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyR{Dsc s–i‰dˆkrVˆjdƒ}t|ttfƒrC|ˆd(t gt_kwargstkwargs(s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyRzbs (c3s‚ˆj|jƒˆˆ}|jƒ}x;|j|jƒˆD]!\}}}||||fVqAW|r~|jƒ|_ndS(N(RR*RkR;RoRmRs(RtRuRvR€RR<R=R>(t_otherR‚Rƒ(s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyRzqs  (N(tpopR4tlistRfRRF(R„RƒRRz((R„R‚Rƒs8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyRLs     tRegexLexerMetacBsPeZdZd„Zd„Zd„Zd„Zdd„Zd„Z d„Z RS( sw Metaclass for RegexLexer, creates the self._tokens attribute from self.tokens on the first instantiation. cCstj||ƒjS(sBPreprocess the regular expression component of a token definition.(tretcompileRu(Rtregextrflags((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyt_process_regex„scCs5t|ƒtks1t|ƒs1td|fƒ‚|S(s5Preprocess the token component of a token definition.s2token type must be simple type or callable, not %r(RRtcallabletAssertionError(Rttoken((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyt_process_tokenˆs!cCskt|tƒrz|dkrdS||kr2|fS|dkrB|S|d dkrat|dƒ Stsgtd|ƒ‚nķt|tƒrd|j}|jd7_g}xF|D]>}||ksŌtd |ƒ‚|j|j|||ƒƒq²W|||<|fSt|t ƒrQx6|D].}||ks|d kstd |ƒ‚qW|Stsgtd |ƒ‚d S(s=Preprocess the state transition action of a token definition.s#popi’’’’s#pushis#pop:sunknown new state %rs_tmp_%discircular state ref %rsunknown new state sunknown new state def %rN(s#pops#push( R4tstrtintR,RŽRet_tmpnametextendt_process_stateRf(Rt new_statet unprocessedt processedt tmp_statetitokenstistate((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyt_process_new_stateŽs6        c Cs½t|ƒtks"td|ƒ‚|ddksBtd|ƒ‚||krV||Sg}||<|j}xI||D]=}t|tƒrŃ||ks©td|ƒ‚|j|j||t|ƒƒƒqxnt|tƒręqxnt|ƒt kstd|ƒ‚y|j |d|ƒ}Wn3t k rW}t d|d|||fƒ‚nX|j |dƒ} t|ƒd kr†d } n|j|d ||ƒ} |j|| | fƒqxW|S( s%Preprocess a single state definition.swrong state name %rit#sinvalid state name %rscircular state reference %rswrong rule def %rs+uncompilable regex %r in state %r of %r: %siiN(RR‘RŽtflagsR4RR”R•RdRfRŒt Exceptiont ValueErrorRRCRFRœR5( RR—R˜R|ttokensR‹ttdeftrexterrRR–((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyR•±s6"   "   cCsRi}|j|<|p!|j|}x'|jƒD]}|j|||ƒq1W|S(s-Preprocess a dictionary of token definitions.(t _all_tokensR”tkeysR•(RRt tokendefsR˜R|((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pytprocess_tokendef×s c Cs.i}i}xtj|f|jƒD]}|jjdiƒ}xć|jƒD]Õ\}}|j|ƒ}|dkrø|||toldittmpvaltit_indextit_tokentit_valuetp((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyR`ĮsL            (sļ»æsutf-8(s’žsutf-32(sž’sutf-32be(s’žsutf-16(sž’sutf-16be(+R"RˆR©tpygments.filterRRtpygments.filtersRtpygments.tokenRRRRt pygments.utilRRR R t__all__RGt staticmethodt_default_analyseRRtobjectR RR‘RRdRRfReRhRR{RRR‡R RR R`(((s8/usr/local/lib/python2.7/site-packages/pygments/lexer.pyt s<""   '     2šUD