Posts Tagged ‘xmlrpc’

Transport class for Python's XML-RPC lib

Thursday, June 21st, 2007

Given that the xmlrpclib.Transport class can be derived, it is perhaps easier to define a new transport class that implements the patch shown in the facelift post, though I still believe Python’s XML-RPC library is due a much needed update.

Thus, I present the HTTPTransport class:

Update: It seems I forgot to parse the resulting payload. this is now fixed in the updated code below.

from xmlrpclib import Transport
from xmlrpclib import ProtocolError

class HTTPTransport(Transport):
    ##
    # Connect to server.
    #
    # @param host Target host.
    # @return A connection handle.

    def make_connection(self, host):
        # create a HTTP connection object from a host descriptor
        import httplib
        host, extra_headers, x509 = self.get_host_info(host)
        return httplib.HTTPConnection(host)
    ##
    # Send a complete request, and parse the response.
    #
    # @param host Target host.
    # @param handler Target PRC handler.
    # @param request_body XML-RPC request body.
    # @param verbose Debugging flag.
    # @return XML response.

    def request(self, host, handler, request_body, verbose=0):
        # issue XML-RPC request

        h = self.make_connection(host)
        if verbose:
            h.set_debuglevel(1)

        self.send_request(h, handler, request_body)
        self.send_host(h, host)
        self.send_user_agent(h)
        self.send_content(h, request_body)

        response = h.getresponse()

        if response.status != 200:
          raise ProtocolError(host + handler, response.status, response.reason, response.msg.headers)

        payload = response.read()
        parser, unmarshaller = self.getparser()
        parser.feed(payload)
        parser.close()

        return unmarshaller.close()

Python XML-RPC needs a facelift.

Friday, June 15th, 2007

I have been experimenting with the Python XML-RPC implementation for a while now, and yesterday, I came across what is most accurately described as a bug. Let’s consider a nice figure to illustrate how the XML-RPC implementation handles things in the Python 2.5 release.

Python xmlrpc madness

So, basically the XML-RPC ServerProxy files a request with the Transport class to deliver the XML goodies to the remote server. However, in the current implementation, Transport uses httplib.HTTP. This is a wrapper class that uses HTTPConnection for most things, but not for receiving responses from the server. And that is exactly where the problem lies. The HTTP.getreply function fetches the HTTP status, reason and headers. But the XML-RPC Transport class does not check the headers for any indication of a content length. When they get the response, things really turn haywire. No matter what, they ask a socket (or a file imposing as a socket) to read 1024 bytes. The socket library tries to comply, but obviously when either the content is shorter, or the connection is closed after the content has been read, an error is raised.

So what are the options to correct this behaviour. I think one can do two things. First of all, fix the Transport function that asks the socket for data to use an extra argument indicating the expected payload size. Obviously, once the headers are received they should be chacked for the presence of a Content-Length field and the requested size should correspond to the value in this length field. I’ve implemented that and it works.

However, I think a second option is perhaps better. Why remain with the HTTP class when a nice and shiny HTTPConnection class is available that does all we need and more? Let’s move the XML-RPC HTTP connection object to that class, and voila! Fixed.

In unified diff format, it boils down to this:

--- /sw/lib/python2.5/xmlrpclib.py      2006-11-29 02:46:38.000000000 +0100
+++ xmlrpclib.py        2007-06-15 15:59:02.000000000 +0200
@@ -1182,23 +1182,13 @@
         self.send_user_agent(h)
         self.send_content(h, request_body)

-        errcode, errmsg, headers = h.getreply()
+        response = h.getresponse()
+
+        if response.status != 200:
+          raise ProtocolError(host + handler, response.status, response.reason, response.msg.headers)

-        if errcode != 200:
-            raise ProtocolError(
-                host + handler,
-                errcode, errmsg,
-                headers
-                )
-
-        self.verbose = verbose
-
-        try:
-            sock = h._conn.sock
-        except AttributeError:
-            sock = None
-
-        return self._parse_response(h.getfile(), sock)
+        payload = response.read()
+        return payload

     ##
     # Create parser.
@@ -1250,7 +1240,7 @@
         # create a HTTP connection object from a host descriptor
         import httplib
         host, extra_headers, x509 = self.get_host_info(host)
-        return httplib.HTTP(host)
+        return httplib.HTTPConnection(host)