U  W[\@sdZddlmZmZddlmZddlmZddlm Z ddl m Z m Z m Z ddlmZddlmZmZmZmZd ZGd d d eZGd d d eZGdddeZGdddeZGdddee Zddd d dgZdS)ap Memcache client protocol. Memcached is a caching server, storing data in the form of pairs key/value, and memcache is the protocol to talk with it. To connect to a server, create a factory for L{MemCacheProtocol}:: from twisted.internet import reactor, protocol from twisted.protocols.memcache import MemCacheProtocol, DEFAULT_PORT d = protocol.ClientCreator(reactor, MemCacheProtocol ).connectTCP("localhost", DEFAULT_PORT) def doSomething(proto): # Here you call the memcache operations return proto.set("mykey", "a lot of data") d.addCallback(doSomething) reactor.run() All the operations of the memcache protocol are present, but L{MemCacheProtocol.set} and L{MemCacheProtocol.get} are the more important. See U{http://code.sixapart.com/svn/memcached/trunk/server/doc/protocol.txt} for more information about the protocol. )absolute_importdivision)deque) LineReceiver) TimeoutMixin)Deferredfail TimeoutError)log) intToBytes iteritems nativeString networkStringi+c@seZdZdZdS) NoSuchCommandzA Exception raised when a non existent command is called. N__name__ __module__ __qualname____doc__rrZ!d?d@Z"dAdBZ#dYdCdDZ$dZdEdFZ%dGdHZ&d[dJdKZ'dLdMZ(dNdOZ)dPdQZ*dIS)\MemCacheProtocola1 MemCache protocol: connect to a memcached server to store/retrieve values. @ivar persistentTimeOut: the timeout period used to wait for a response. @type persistentTimeOut: L{int} @ivar _current: current list of requests waiting for an answer from the server. @type _current: L{deque} of L{Command} @ivar _lenExpected: amount of data expected in raw mode, when reading for a value. @type _lenExpected: L{int} @ivar _getBuffer: current buffer of data, used to store temporary data when reading in raw mode. @type _getBuffer: L{list} @ivar _bufferLength: the total amount of bytes in C{_getBuffer}. @type _bufferLength: L{int} @ivar _disconnected: indicate if the connectionLost has been called or not. @type _disconnected: L{bool} F<cCs*t|_d|_d|_d|_||_|_dS)z Create the protocol. @param timeOut: the timeout to wait before detecting that the connection is dead and close it. It's expressed in seconds. @type timeOut: L{int} N)r_current _lenExpected _getBuffer _bufferLengthpersistentTimeOuttimeOut)rr/rrrr"s zMemCacheProtocol.__init__cCs |jr|j}||qdS)zW Cancel all the outstanding commands, making them fail with C{reason}. N)r*popleftr)rreasoncmdrrr_cancelCommandss z MemCacheProtocol._cancelCommandscCs|td|jdS)z: Close the connection in case of timeout. zConnection timeoutN)r3r Z transportZloseConnectionrrrrtimeoutConnectionsz"MemCacheProtocol.timeoutConnectioncCs d|_||t||dS)z9 Cause any outstanding commands to fail. TN) _disconnectedr3rconnectionLost)rr1rrrr7s zMemCacheProtocol.connectionLostcCs"|js||jt||dS)zA Override sendLine to add a timeout to response. N)r* setTimeoutr.rsendLine)rlinerrrr9s zMemCacheProtocol.sendLinecCs||j||jt|7_|j|jdkrd|j}|d|j}||jdd}|}d|_d|_d|_|jd}|jr|j |j \}}|||f|j |j <n||_ | |dS)z) Collect data for a get. Nr) resetTimeoutr,appendr-lenr+joinr*multiplevalues currentKeyr$Z setLineMode)rdataZbufZremvalr2flagscasrrrrawDataReceiveds"   z MemCacheProtocol.rawDataReceivedcCs|jddS)z? Manage a success response to a set operation. TNr*r0r%r4rrr cmd_STOREDszMemCacheProtocol.cmd_STOREDcCs|jddS)z Manage a specific 'not stored' response to a set operation: this is not an error, but some condition wasn't met. FNrIr4rrrcmd_NOT_STOREDszMemCacheProtocol.cmd_NOT_STOREDcCs|j}|jdkrN|jr:ddt|jD}||q||j|jfnb|jdkr|jrl||jq||j|j |jfn,|jdkr||jnt dt |jfdS)zB This the end token to a get or a stat operation. getcSs i|]\}}||dddqS)Nr;r).0keyrErrr sz,MemCacheProtocol.cmd_END..getsstatsz%Unexpected END response to %s commandN) r*r0rrAr rBr%rFr$rG RuntimeErrorr )rr2rBrrrcmd_ENDs"      zMemCacheProtocol.cmd_ENDcCs|jddS)z= Manage error response for incr/decr/delete. FNrIr4rrr cmd_NOT_FOUNDszMemCacheProtocol.cmd_NOT_FOUNDcCs|jd}|jdkr(|\}}}d}n|\}}}}t||_g|_d|_|jr||jkrft d||_ t||g|j |<n"|j |krt dt||_ ||_|dS)z: Prepare the reading a value after a get. rrLr<zUnexpected commands answer.N)r*rsplitintr+r,r-rAkeysrRrCrBrNrFrGZ setRawMode)rr:r2rNrFlengthrGrrr cmd_VALUEs$      zMemCacheProtocol.cmd_VALUEcCs(|jd}|dd\}}||j|<dS)z- Reception of one stat line. r N)r*rUrB)rr:r2rNrErrrcmd_STATs zMemCacheProtocol.cmd_STATcCs|j|dS)z% Read version token. NrI)rZ versionDatarrr cmd_VERSION%szMemCacheProtocol.cmd_VERSIONcCs$td|j}|tdS)z7 A non-existent command has been sent. zNon-existent command sent.N)r errr*r0rr)rr2rrr cmd_ERROR,s  zMemCacheProtocol.cmd_ERRORcCs2t|}td||j}|t|dS)z0 An invalid input as been sent. zInvalid input: N)reprr r^r*r0rrrZerrTextr2rrrcmd_CLIENT_ERROR5s z!MemCacheProtocol.cmd_CLIENT_ERRORcCs2t|}td||j}|t|dS)z4 An error has happened server-side. zServer error: N)r`r r^r*r0rrrarrrcmd_SERVER_ERROR?s z!MemCacheProtocol.cmd_SERVER_ERRORcCs|jddS)z> A delete command has completed successfully. TNrIr4rrr cmd_DELETEDIszMemCacheProtocol.cmd_DELETEDcCs|jddS)z6 The last command has been completed. TNrIr4rrrcmd_OKPszMemCacheProtocol.cmd_OKcCs|jddS)z5 A C{checkAndSet} update has failed. FNrIr4rrr cmd_EXISTSWszMemCacheProtocol.cmd_EXISTScCs||ddd}t|dt|d}|dk rb|dddd}|rZ||dq|nL|dd}t|dt|d}|dk r|n|j}t|}|||js| ddS)z8 Receive line commands from the server. rZr[rZcmd_N_) r=rUgetattrr replacer*r0rVr%r8)rr:tokenr2argsrErrr lineReceived^s"   zMemCacheProtocol.lineReceivedr[cCs|d||S)a Increment the value of C{key} by given value (default to 1). C{key} must be consistent with an int. Return the new value. @param key: the key to modify. @type key: L{bytes} @param val: the value to increment. @type val: L{int} @return: a deferred with will be called back with the new value associated with the key (after the increment). @rtype: L{Deferred} sincr _incrdecrrrNrErrr increment|szMemCacheProtocol.incrementcCs|d||S)a Decrement the value of C{key} by given value (default to 1). C{key} must be consistent with an int. Return the new value, coerced to 0 if negative. @param key: the key to modify. @type key: L{bytes} @param val: the value to decrement. @type val: L{int} @return: a deferred with will be called back with the new value associated with the key (after the decrement). @rtype: L{Deferred} sdecrrmrorrr decrementszMemCacheProtocol.decrementcCs|jrttdSt|ts2ttdt|fSt||jkrLttdSd ||t t |g}| |t ||d}|j||jS)z1 Internal wrapper for incr/decr. not connected)Invalid type for key: %s, expecting bytes Key too longrZrN)r6rrR isinstancebytesrtyper?MAX_KEY_LENGTHr@r rVr9rr*r>r)rr2rNrEfullcmdcmdObjrrrrns       zMemCacheProtocol._incrdecrrcCs|d||||dS)a Replace the given C{key}. It must already exist in the server. @param key: the key to replace. @type key: L{bytes} @param val: the new value associated with the key. @type val: L{bytes} @param flags: the flags to store with the key. @type flags: L{int} @param expireTime: if different from 0, the relative time in seconds when the key will be deleted from the store. @type expireTime: L{int} @return: a deferred that will fire with C{True} if the operation has succeeded, and C{False} with the key didn't previously exist. @rtype: L{Deferred} sreplacer<_setrrNrErF expireTimerrrriszMemCacheProtocol.replacecCs|d||||dS)a Add the given C{key}. It must not exist in the server. @param key: the key to add. @type key: L{bytes} @param val: the value associated with the key. @type val: L{bytes} @param flags: the flags to store with the key. @type flags: L{int} @param expireTime: if different from 0, the relative time in seconds when the key will be deleted from the store. @type expireTime: L{int} @return: a deferred that will fire with C{True} if the operation has succeeded, and C{False} with the key already exists. @rtype: L{Deferred} saddr<r|r~rrraddszMemCacheProtocol.addcCs|d||||dS)a9 Set the given C{key}. @param key: the key to set. @type key: L{bytes} @param val: the value associated with the key. @type val: L{bytes} @param flags: the flags to store with the key. @type flags: L{int} @param expireTime: if different from 0, the relative time in seconds when the key will be deleted from the store. @type expireTime: L{int} @return: a deferred that will fire with C{True} if the operation has succeeded. @rtype: L{Deferred} ssetr<r|r~rrrsetszMemCacheProtocol.setcCs|d|||||S)am Change the content of C{key} only if the C{cas} value matches the current one associated with the key. Use this to store a value which hasn't been modified since last time you fetched it. @param key: The key to set. @type key: L{bytes} @param val: The value associated with the key. @type val: L{bytes} @param cas: Unique 64-bit value returned by previous call of C{get}. @type cas: L{bytes} @param flags: The flags to store with the key. @type flags: L{int} @param expireTime: If different from 0, the relative time in seconds when the key will be deleted from the store. @type expireTime: L{int} @return: A deferred that will fire with C{True} if the operation has succeeded, C{False} otherwise. @rtype: L{Deferred} scasr|)rrNrErGrFrrrr checkAndSetszMemCacheProtocol.checkAndSetc Cs|jrttdSt|ts2ttdt|fSt||jkrLttdSt|tslttdt|fS|rxd|}t|}d ||t d|||fg|}| || |t ||||d} |j | | jS)z6 Internal wrapper for setting values. rrrsrtz+Invalid type for value: %s, expecting bytesrZz%d %d %d)rNrFrX)r6rrRrvrwrrxr?ryr@rr9rr*r>r) rr2rNrErFrrGrXrzr{rrrr}s:        zMemCacheProtocol._setcCs|d||dddS)a Append given data to the value of an existing key. @param key: The key to modify. @type key: L{bytes} @param val: The value to append to the current value associated with the key. @type val: L{bytes} @return: A deferred that will fire with C{True} if the operation has succeeded, C{False} otherwise. @rtype: L{Deferred} sappendrr<r|rorrrr>4szMemCacheProtocol.appendcCs|d||dddS)a Prepend given data to the value of an existing key. @param key: The key to modify. @type key: L{bytes} @param val: The value to prepend to the current value associated with the key. @type val: L{bytes} @return: A deferred that will fire with C{True} if the operation has succeeded, C{False} otherwise. @rtype: L{Deferred} sprependrr<r|rorrrprependGszMemCacheProtocol.prependcCs||g|dS)a Get the given C{key}. It doesn't support multiple keys. If C{withIdentifier} is set to C{True}, the command issued is a C{gets}, that will return the current identifier associated with the value. This identifier has to be used when issuing C{checkAndSet} update later, using the corresponding method. @param key: The key to retrieve. @type key: L{bytes} @param withIdentifier: If set to C{True}, retrieve the current identifier along with the value and the flags. @type withIdentifier: L{bool} @return: A deferred that will fire with the tuple (flags, value) if C{withIdentifier} is C{False}, or (flags, cas identifier, value) if C{True}. If the server indicates there is no value associated with C{key}, the returned value will be L{None} and the returned flags will be C{0}. @rtype: L{Deferred} F_get)rrNwithIdentifierrrrgetZszMemCacheProtocol.getcCs|||dS)a Get the given list of C{keys}. If C{withIdentifier} is set to C{True}, the command issued is a C{gets}, that will return the identifiers associated with each values. This identifier has to be used when issuing C{checkAndSet} update later, using the corresponding method. @param keys: The keys to retrieve. @type keys: L{list} of L{bytes} @param withIdentifier: If set to C{True}, retrieve the identifiers along with the values and the flags. @type withIdentifier: L{bool} @return: A deferred that will fire with a dictionary with the elements of C{keys} as keys and the tuples (flags, value) as values if C{withIdentifier} is C{False}, or (flags, cas identifier, value) if C{True}. If the server indicates there is no value associated with C{key}, the returned values will be L{None} and the returned flags will be C{0}. @rtype: L{Deferred} @since: 9.0 Tr)rrWrrrr getMultiplesszMemCacheProtocol.getMultiplec Cst|}|jrttdS|D]F}t|tsFttdt|fSt||j krttdSq|rpd}nd}d |g|}| ||rt dd|D}t |||d d }nt ||d d d d dd}|j||jS)z> Helper method for C{get} and C{getMultiple}. rrrsrtrPrLrZcSsg|] }|dfqS))rr<Nr)rMrNrrr sz)MemCacheProtocol._get..T)rWrBrArNr<F)rNr$rFrGrA)listr6rrRrvrwrrxr?ryr@r9dictrr*r>r) rrWrrArNr2rzrBr{rrrrs2     zMemCacheProtocol._getNcCsL|rd|}nd}|jr$ttdS||tdid}|j||jS)a Get some stats from the server. It will be available as a dict. @param arg: An optional additional string which will be sent along with the I{stats} command. The interpretation of this value by the server is left undefined by the memcache protocol specification. @type arg: L{None} or L{bytes} @return: a deferred that will fire with a L{dict} of the available statistics. @rtype: L{Deferred} sstats rQrr)rBr6rrRr9rr*r>r)rargr2r{rrrstatss     zMemCacheProtocol.statscCs6|jrttdS|dtd}|j||jS)z Get the version of the server. @return: a deferred that will fire with the string value of the version. @rtype: L{Deferred} rrsversionrrr{rrrversions    zMemCacheProtocol.versioncCs^|jrttdSt|ts2ttdt|fS|d|td|d}|j ||j S)a Delete an existing C{key}. @param key: the key to delete. @type key: L{bytes} @return: a deferred that will be called back with C{True} if the key was successfully deleted, or C{False} if not. @rtype: L{Deferred} rrrssdelete sdeleteru) r6rrRrvrwrrxr9rr*r>r)rrNr{rrrdeletes      zMemCacheProtocol.deletecCs6|jrttdS|dtd}|j||jS)z Flush all cached values. @return: a deferred that will be called back with C{True} when the operation has succeeded. @rtype: L{Deferred} rrs flush_allrrrrrflushAlls    zMemCacheProtocol.flushAll)r))r[)r[)rr)rr)rr)rr)F)F)N)+rrrrryr6r"r3r5r7r9rHrJrKrSrTrYr\r]r_rbrcrdrerfrlrprqrnrirrrr}r>rrrrrrrrrrrrr'nsP                 r' DEFAULT_PORTN)rZ __future__rr collectionsrZtwisted.protocols.basicrZtwisted.protocols.policiesrZtwisted.internet.deferrrr Ztwisted.pythonr Ztwisted.python.compatr r r rr Exceptionrrrobjectrr'__all__rrrrs*    ,