twisted.protocols.basic.NetstringReceiver

Known subclasses: twisted.internet.test.test_endpoints.NetstringTracker, twisted.protocols.test.test_basic.TestNetstring

A protocol that sends and receives netstrings.

See http://cr.yp.to/proto/netstrings.txt for the specification of netstrings. Every netstring starts with digits that specify the length of the data. This length specification is separated from the data by a colon. The data is terminated with a comma.

Override stringReceived to handle received netstrings. This method is called with the netstring payload as a single argument whenever a complete netstring is received.

Security features:

Messages are limited in size, useful if you don't want someone sending you a 500MB netstring (change self.MAX_LENGTH to the maximum length you wish to accept).
The connection is lost if an illegal message is received.

Instance Variable	MAX_LENGTH	Defines the maximum length of netstrings that can be received. (type: `int`)
Instance Variable	brokenPeer	Indicates if the connection is still functional (type: `int`)
Method	makeConnection	Initializes the protocol.
Method	sendString	Sends a netstring.
Method	dataReceived	Receives some characters of a netstring.
Method	stringReceived	Override this for notification when each complete string is received.
Instance Variable	_LENGTH	A pattern describing all strings that contain a netstring length specification. Examples for length specifications are `b'0:'`, `b'12:'`, and `b'179:'`. `b'007:'` is not a valid length specification, since leading zeros are not allowed. (type: `re.Match`)
Instance Variable	_LENGTH_PREFIX	A pattern describing all strings that contain the first part of a netstring length specification (without the trailing comma). Examples are '0', '12', and '179'. '007' does not start a netstring length specification, since leading zeros are not allowed. (type: `re.Match`)
Instance Variable	_PARSING_LENGTH	Indicates that the `NetstringReceiver` is in the state of parsing the length portion of a netstring. (type: `int`)
Instance Variable	_PARSING_PAYLOAD	Indicates that the `NetstringReceiver` is in the state of parsing the payload portion (data and trailing comma) of a netstring. (type: `int`)
Instance Variable	_state	Indicates if the protocol is consuming the length portion (`PARSING_LENGTH`) or the payload (`PARSING_PAYLOAD`) of a netstring (type: `int`)
Instance Variable	_remainingData	Holds the chunk of data that has not yet been consumed (type: `string`)
Instance Variable	_payload	Holds the payload portion of a netstring including the trailing comma (type: `BytesIO`)
Instance Variable	_expectedPayloadSize	Holds the payload size plus one for the trailing comma. (type: `int`)
Method	_maxLengthSize	Calculate and return the string size of `self.MAX_LENGTH`.
Method	_consumeData	Consumes the content of `self._remainingData`.
Method	_consumeLength	Consumes the length portion of `self._remainingData`.
Method	_checkPartialLengthSpecification	Makes sure that the received data represents a valid number.
Method	_processLength	Processes the length definition of a netstring.
Method	_extractLength	Attempts to extract the length information of a netstring.
Method	_checkStringSize	Checks the sanity of lengthAsString.
Method	_prepareForPayloadConsumption	Sets up variables necessary for consuming the payload of a netstring.
Method	_consumePayload	Consumes the payload portion of `self._remainingData`.
Method	_extractPayload	Extracts payload information from `self._remainingData`.
Method	_payloadComplete	Checks if enough data have been received to complete the netstring.
Method	_processPayload	Processes the actual payload with `stringReceived`.
Method	_checkForTrailingComma	Checks if the netstring has a trailing comma at the expected position.
Method	_handleParseError	Terminates the connection and sets the flag `self.brokenPeer`.

Inherited from Protocol:

Method	logPrefix	Return a prefix matching the class name, to identify log messages related to this protocol instance.
Method	connectionLost	Called when the connection is shut down.

Inherited from BaseProtocol (via Protocol):

Method

connectionMade

Called when a connection is made.

MAX_LENGTH =

Defines the maximum length of netstrings that can be received. (type: int)

_LENGTH =

A pattern describing all strings that contain a netstring length specification. Examples for length specifications are b'0:', b'12:', and b'179:'. b'007:' is not a valid length specification, since leading zeros are not allowed. (type: re.Match)

_LENGTH_PREFIX =

A pattern describing all strings that contain the first part of a netstring length specification (without the trailing comma). Examples are '0', '12', and '179'. '007' does not start a netstring length specification, since leading zeros are not allowed. (type: re.Match)

_PARSING_LENGTH =

Indicates that the NetstringReceiver is in the state of parsing the length portion of a netstring. (type: int)

_PARSING_PAYLOAD =

Indicates that the NetstringReceiver is in the state of parsing the payload portion (data and trailing comma) of a netstring. (type: int)

brokenPeer =

Indicates if the connection is still functional (type: int)

_state =

Indicates if the protocol is consuming the length portion (PARSING_LENGTH) or the payload (PARSING_PAYLOAD) of a netstring (type: int)

_remainingData =

Holds the chunk of data that has not yet been consumed (type: string)

_payload =

Holds the payload portion of a netstring including the trailing comma (type: BytesIO)

_expectedPayloadSize =

Holds the payload size plus one for the trailing comma. (type: int)

def makeConnection(self, transport):

from twisted.internet.interfaces.IProtocol

overrides twisted.internet.protocol.BaseProtocol.makeConnection

Initializes the protocol.

def sendString(self, string):

Sends a netstring.

Wraps up string by adding length information and a trailing comma; writes the result to the transport.

Parameters string The string to send. The necessary framing (length prefix, etc) will be added. (type: bytes)

def dataReceived(self, data):

from twisted.internet.interfaces.IProtocol

overrides twisted.internet.protocol.Protocol.dataReceived

Receives some characters of a netstring.

Whenever a complete netstring is received, this method extracts its payload and calls stringReceived to process it.

Parameters data A chunk of data representing a (possibly partial) netstring (type: bytes)

def stringReceived(self, string):

overridden in twisted.internet.test.test_endpoints.NetstringTracker, twisted.protocols.test.test_basic.TestNetstring

Override this for notification when each complete string is received.

Parameters	string	The complete string which was received with all framing (length prefix, etc) removed. (type: `bytes`)
Raises	NotImplementedError	because the method has to be implemented by the child class.

def _maxLengthSize(self):

Calculate and return the string size of self.MAX_LENGTH.

Returns The size of the string representation for self.MAX_LENGTH (type: float)

def _consumeData(self):

Consumes the content of self._remainingData.

Raises	IncompleteNetstring	if `self._remainingData` does not contain enough data to complete the current netstring.
	NetstringParseError	if the received data do not form a valid netstring.

def _consumeLength(self):

Consumes the length portion of self._remainingData.

Raises	IncompleteNetstring	if `self._remainingData` contains a partial length specification (digits without trailing comma).
	NetstringParseError	if the received data do not form a valid netstring.

def _checkPartialLengthSpecification(self):

Makes sure that the received data represents a valid number.

Checks if self._remainingData represents a number smaller or equal to self.MAX_LENGTH.

Raises NetstringParseError if self._remainingData is no number or is too big (checked by _extractLength).

def _processLength(self, lengthMatch):

Processes the length definition of a netstring.

Extracts and stores in self._expectedPayloadSize the number representing the netstring size. Removes the prefix representing the length specification from self._remainingData.

Parameters	lengthMatch	A regular expression match object matching a netstring length specification (type: `re.Match`)
Raises	NetstringParseError	if the received netstring does not start with a number or the number is bigger than `self.MAX_LENGTH`.

def _extractLength(self, lengthAsString):

Attempts to extract the length information of a netstring.

Parameters	lengthAsString	A chunk of data starting with a length specification (type: `bytes`)
Returns	The length of the netstring (type: `int`)
Raises	NetstringParseError	if the number is bigger than `self.MAX_LENGTH`.

def _checkStringSize(self, lengthAsString):

Checks the sanity of lengthAsString.

Checks if the size of the length specification exceeds the size of the string representing self.MAX_LENGTH. If this is not the case, the number represented by lengthAsString is certainly bigger than self.MAX_LENGTH, and a NetstringParseError can be raised.

This method should make sure that netstrings with extremely long length specifications are refused before even attempting to convert them to an integer (which might trigger a MemoryError).

def _prepareForPayloadConsumption(self):

Sets up variables necessary for consuming the payload of a netstring.

def _consumePayload(self):

Consumes the payload portion of self._remainingData.

If the payload is complete, checks for the trailing comma and processes the payload. If not, raises an IncompleteNetstring exception.

Raises	IncompleteNetstring	if the payload received so far contains fewer characters than expected.
	NetstringParseError	if the payload does not end with a comma.

def _extractPayload(self):

Extracts payload information from self._remainingData.

Splits self._remainingData at the end of the netstring. The first part becomes self._payload, the second part is stored in self._remainingData.

If the netstring is not yet complete, the whole content of self._remainingData is moved to self._payload.

def _payloadComplete(self):

Checks if enough data have been received to complete the netstring.

Returns True iff the received data contain at least as many characters as specified in the length section of the netstring (type: bool)

def _processPayload(self):

Processes the actual payload with stringReceived.

Strips self._payload of the trailing comma and calls stringReceived with the result.

def _checkForTrailingComma(self):

Checks if the netstring has a trailing comma at the expected position.

Raises NetstringParseError if the last payload character is anything but a comma.

def _handleParseError(self):

Terminates the connection and sets the flag self.brokenPeer.

API Documentation for twisted, generated by pydoctor at 2020-03-25 17:34:30.

twisted.protocols.basic.NetstringReceiver(protocol.Protocol) class documentation

`twisted.protocols.basic.NetstringReceiver(protocol.Protocol)` class documentation