twisted.web.proxy.Proxy(HTTPChannel) class documentation

Part of twisted.web.proxy (View In Hierarchy)

This class implements a simple web proxy.

Since it inherits from twisted.web.http.HTTPChannel, to use it you should do something like this:

   from twisted.web import http
   f = http.HTTPFactory()
   f.protocol = Proxy

Make the HTTPFactory a listener on a port as per usual, and you have a fully-functioning web proxy!

Inherited from HTTPChannel:

Instance Variable MAX_LENGTH Maximum length for initial request line and each line from the header.
Instance Variable maxHeaders Maximum number of headers allowed per request. (type: int)
Instance Variable totalHeadersSize Maximum bytes for request line plus all headers from the request. (type: int)
Instance Variable abortTimeout The number of seconds to wait after we attempt to shut the transport down cleanly to give up and forcibly terminate it. This is only used when we time a connection out, to prevent errors causing the FD to get leaked. If this is None, we will wait forever. (type: int)
Method __init__ Undocumented
Method connectionMade Called when a connection is made.
Method lineReceived Called for each line from request until the end of headers when it enters binary mode.
Method headerReceived Do pre-processing (for content-length) and store this header away. Enforce the per-request header limit.
Method allContentReceived Undocumented
Method dataReceived Data was received from the network. Process it.
Method rawDataReceived Override this for when raw data is received.
Method allHeadersReceived Undocumented
Method checkPersistence Check if the channel should close or not.
Method requestDone Called by first request in queue when it is done.
Method timeoutConnection Called when the connection times out.
Method forceAbortClient No summary
Method connectionLost Called when the connection is shut down.
Method isSecure Return True if this channel is using a secure transport.
Method writeHeaders Called by Request objects to write a complete set of HTTP headers to a transport.
Method write Called by Request objects to write response data.
Method writeSequence Write a list of strings to the HTTP response.
Method getPeer Get the remote address of this connection.
Method getHost Get the local address of this connection.
Method loseConnection Closes the connection. Will write any data that is pending to be sent on the network, but if this response has not yet been written to the network will not write anything.
Method registerProducer Register to receive data from a producer.
Method unregisterProducer Stop consuming data from a producer, without disconnecting.
Method stopProducing Stop producing data.
Method pauseProducing Pause producing data.
Method resumeProducing Resume producing data.
Instance Variable _transferDecoder None or a decoder instance if the request body uses the chunked Transfer-Encoding. (type: _ChunkedTransferDecoder)
Instance Variable _receivedHeaderSize Bytes received so far for the header. (type: int)
Instance Variable _handlingRequest Whether a request is currently being processed. (type: bool)
Instance Variable _dataBuffer Any data that has been received from the connection while processing an outstanding request. (type: list of bytes)
Instance Variable _networkProducer Either the transport, if it provides interfaces.IPushProducer, or a null implementation of interfaces.IPushProducer. Used to attempt to prevent the transport from producing excess data when we're responding to a request. (type: interfaces.IPushProducer)
Instance Variable _requestProducer If the Request object or anything it calls registers itself as an interfaces.IProducer, it will be stored here. This is used to create a producing pipeline: pause/resume producing methods will be propagated from the transport, through the HTTPChannel instance, to the c{_requestProducer}.

The reason we proxy through the producing methods rather than the old behaviour (where we literally just set the Request object as the producer on the transport) is because we want to be able to exert backpressure on the client to prevent it from sending in arbitrarily many requests without ever reading responses. Essentially, if the client never reads our responses we will eventually stop reading its requests.

(type: interfaces.IPushProducer)
Instance Variable _requestProducerStreaming A boolean that tracks whether the producer on the Request side of this channel has registered itself as a interfaces.IPushProducer or an interfaces.IPullProducer. (type: bool or None)
Instance Variable _waitingForTransport A boolean that tracks whether the transport has asked us to stop producing. This is used to keep track of what we're waiting for: if the transport has asked us to stop producing then we don't want to unpause the transport until it asks us to produce again. (type: bool)
Instance Variable _abortingCall The twisted.internet.base.DelayedCall that will be used to forcibly close the transport if it doesn't close cleanly. (type: twisted.internet.base.DelayedCall)
Instance Variable _optimisticEagerReadSize When a resource takes a long time to answer a request (via twisted.web.server.NOT_DONE_YET, hopefully one day by a Deferred), we would like to be able to let that resource know about the underlying transport disappearing as promptly as possible, via Request.notifyFinish, and therefore via self.requests[...].connectionLost() on this HTTPChannel.

However, in order to simplify application logic, we implement head-of-line blocking, and do not relay pipelined requests to the application until the previous request has been answered. This means that said application cannot dispose of any entity-body that comes in from those subsequent requests, which may be arbitrarily large, and it may need to be buffered in memory.

To implement this tradeoff between prompt notification when possible (in the most frequent case of non-pipelined requests) and correct behavior when not (say, if a client sends a very long-running GET request followed by a PUT request with a very large body) we will continue reading pipelined requests into self._dataBuffer up to a given limit.

_optimisticEagerReadSize is the number of bytes we will accept from the client and buffer before pausing the transport.

This behavior has been in place since Twisted 17.9.0 .

(type: int)
Method _finishRequestBody Undocumented
Method _maybeChooseTransferDecoder If the provided header is content-length or transfer-encoding, choose the appropriate decoder if any.
Method _send100Continue Sends a 100 Continue response, used to signal to clients that further processing will be performed.
Method _respondToBadRequestAndDisconnect This is a quick and dirty way of responding to bad requests.

Inherited from LineReceiver (via HTTPChannel):

Class Variable delimiter The line-ending delimiter to use. By default this is b'\r\n'.
Method clearLineBuffer Clear buffered data.
Method setLineMode Sets the line-mode of this receiver.
Method setRawMode Sets the raw mode of this receiver. Further data received will be sent to rawDataReceived rather than lineReceived.
Method sendLine Sends a line to the other end of the connection.
Method lineLengthExceeded Called when the maximum line length has been reached. Override if it needs to be dealt with in some special way.

Inherited from Protocol (via HTTPChannel, LineReceiver):

Method logPrefix Return a prefix matching the class name, to identify log messages related to this protocol instance.

Inherited from BaseProtocol (via HTTPChannel, LineReceiver, Protocol):

Method makeConnection Make a connection to a transport and a server.

Inherited from TimeoutMixin (via HTTPChannel):

Class Variable timeOut The number of seconds after which to timeout the connection.
Method callLater Wrapper around reactor.callLater for test purpose.
Method resetTimeout Reset the timeout count down.
Method setTimeout Change the timeout period
Method __timedOut Undocumented
API Documentation for twisted, generated by pydoctor at 2020-03-25 17:34:30.