History

add hg and python

cinap_lenrek committed 15 years ago

Commit 4bb9e221 parent 987265c3706 files · +1228772 −-233Browse files

Showing the changes to sys/lib/python/SocketServer.py (1 of 3706 files). Show all

Unified · Split

sys/lib/python/SocketServer.py+588−0
@@ -0,0 +1,588 @@
1+"""Generic socket server classes.
2+
3+This module tries to capture the various aspects of defining a server:
4+
5+For socket-based servers:
6+
7+- address family:
8+ - AF_INET{,6}: IP (Internet Protocol) sockets (default)
9+ - AF_UNIX: Unix domain sockets
10+ - others, e.g. AF_DECNET are conceivable (see <socket.h>
11+- socket type:
12+ - SOCK_STREAM (reliable stream, e.g. TCP)
13+ - SOCK_DGRAM (datagrams, e.g. UDP)
14+
15+For request-based servers (including socket-based):
16+
17+- client address verification before further looking at the request
18+ (This is actually a hook for any processing that needs to look
19+ at the request before anything else, e.g. logging)
20+- how to handle multiple requests:
21+ - synchronous (one request is handled at a time)
22+ - forking (each request is handled by a new process)
23+ - threading (each request is handled by a new thread)
24+
25+The classes in this module favor the server type that is simplest to
26+write: a synchronous TCP/IP server. This is bad class design, but
27+save some typing. (There's also the issue that a deep class hierarchy
28+slows down method lookups.)
29+
30+There are five classes in an inheritance diagram, four of which represent
31+synchronous servers of four types:
32+
33+ +------------+
34+ | BaseServer |
35+ +------------+
36+ |
37+ v
38+ +-----------+ +------------------+
39+ | TCPServer |------->| UnixStreamServer |
40+ +-----------+ +------------------+
41+ |
42+ v
43+ +-----------+ +--------------------+
44+ | UDPServer |------->| UnixDatagramServer |
45+ +-----------+ +--------------------+
46+
47+Note that UnixDatagramServer derives from UDPServer, not from
48+UnixStreamServer -- the only difference between an IP and a Unix
49+stream server is the address family, which is simply repeated in both
50+unix server classes.
51+
52+Forking and threading versions of each type of server can be created
53+using the ForkingMixIn and ThreadingMixIn mix-in classes. For
54+instance, a threading UDP server class is created as follows:
55+
56+ class ThreadingUDPServer(ThreadingMixIn, UDPServer): pass
57+
58+The Mix-in class must come first, since it overrides a method defined
59+in UDPServer! Setting the various member variables also changes
60+the behavior of the underlying server mechanism.
61+
62+To implement a service, you must derive a class from
63+BaseRequestHandler and redefine its handle() method. You can then run
64+various versions of the service by combining one of the server classes
65+with your request handler class.
66+
67+The request handler class must be different for datagram or stream
68+services. This can be hidden by using the request handler
69+subclasses StreamRequestHandler or DatagramRequestHandler.
70+
71+Of course, you still have to use your head!
72+
73+For instance, it makes no sense to use a forking server if the service
74+contains state in memory that can be modified by requests (since the
75+modifications in the child process would never reach the initial state
76+kept in the parent process and passed to each child). In this case,
77+you can use a threading server, but you will probably have to use
78+locks to avoid two requests that come in nearly simultaneous to apply
79+conflicting changes to the server state.
80+
81+On the other hand, if you are building e.g. an HTTP server, where all
82+data is stored externally (e.g. in the file system), a synchronous
83+class will essentially render the service "deaf" while one request is
84+being handled -- which may be for a very long time if a client is slow
85+to reqd all the data it has requested. Here a threading or forking
86+server is appropriate.
87+
88+In some cases, it may be appropriate to process part of a request
89+synchronously, but to finish processing in a forked child depending on
90+the request data. This can be implemented by using a synchronous
91+server and doing an explicit fork in the request handler class
92+handle() method.
93+
94+Another approach to handling multiple simultaneous requests in an
95+environment that supports neither threads nor fork (or where these are
96+too expensive or inappropriate for the service) is to maintain an
97+explicit table of partially finished requests and to use select() to
98+decide which request to work on next (or whether to handle a new
99+incoming request). This is particularly important for stream services
100+where each client can potentially be connected for a long time (if
101+threads or subprocesses cannot be used).
102+
103+Future work:
104+- Standard classes for Sun RPC (which uses either UDP or TCP)
105+- Standard mix-in classes to implement various authentication
106+ and encryption schemes
107+- Standard framework for select-based multiplexing
108+
109+XXX Open problems:
110+- What to do with out-of-band data?
111+
112+BaseServer:
113+- split generic "request" functionality out into BaseServer class.
114+ Copyright (C) 2000 Luke Kenneth Casson Leighton <lkcl@samba.org>
115+
116+ example: read entries from a SQL database (requires overriding
117+ get_request() to return a table entry from the database).
118+ entry is processed by a RequestHandlerClass.
119+
120+"""
121+
122+# Author of the BaseServer patch: Luke Kenneth Casson Leighton
123+
124+# XXX Warning!
125+# There is a test suite for this module, but it cannot be run by the
126+# standard regression test.
127+# To run it manually, run Lib/test/test_socketserver.py.
128+
129+__version__ = "0.4"
130+
131+
132+import socket
133+import sys
134+import os
135+
136+__all__ = ["TCPServer","UDPServer","ForkingUDPServer","ForkingTCPServer",
137+ "ThreadingUDPServer","ThreadingTCPServer","BaseRequestHandler",
138+ "StreamRequestHandler","DatagramRequestHandler",
139+ "ThreadingMixIn", "ForkingMixIn"]
140+if hasattr(socket, "AF_UNIX"):
141+ __all__.extend(["UnixStreamServer","UnixDatagramServer",
142+ "ThreadingUnixStreamServer",
143+ "ThreadingUnixDatagramServer"])
144+
145+class BaseServer:
146+
147+ """Base class for server classes.
148+
149+ Methods for the caller:
150+
151+ - __init__(server_address, RequestHandlerClass)
152+ - serve_forever()
153+ - handle_request() # if you do not use serve_forever()
154+ - fileno() -> int # for select()
155+
156+ Methods that may be overridden:
157+
158+ - server_bind()
159+ - server_activate()
160+ - get_request() -> request, client_address
161+ - verify_request(request, client_address)
162+ - server_close()
163+ - process_request(request, client_address)
164+ - close_request(request)
165+ - handle_error()
166+
167+ Methods for derived classes:
168+
169+ - finish_request(request, client_address)
170+
171+ Class variables that may be overridden by derived classes or
172+ instances:
173+
174+ - address_family
175+ - socket_type
176+ - allow_reuse_address
177+
178+ Instance variables:
179+
180+ - RequestHandlerClass
181+ - socket
182+
183+ """
184+
185+ def __init__(self, server_address, RequestHandlerClass):
186+ """Constructor. May be extended, do not override."""
187+ self.server_address = server_address
188+ self.RequestHandlerClass = RequestHandlerClass
189+
190+ def server_activate(self):
191+ """Called by constructor to activate the server.
192+
193+ May be overridden.
194+
195+ """
196+ pass
197+
198+ def serve_forever(self):
199+ """Handle one request at a time until doomsday."""
200+ while 1:
201+ self.handle_request()
202+
203+ # The distinction between handling, getting, processing and
204+ # finishing a request is fairly arbitrary. Remember:
205+ #
206+ # - handle_request() is the top-level call. It calls
207+ # get_request(), verify_request() and process_request()
208+ # - get_request() is different for stream or datagram sockets
209+ # - process_request() is the place that may fork a new process
210+ # or create a new thread to finish the request
211+ # - finish_request() instantiates the request handler class;
212+ # this constructor will handle the request all by itself
213+
214+ def handle_request(self):
215+ """Handle one request, possibly blocking."""
216+ try:
217+ request, client_address = self.get_request()
218+ except socket.error:
219+ return
220+ if self.verify_request(request, client_address):
221+ try:
222+ self.process_request(request, client_address)
223+ except:
224+ self.handle_error(request, client_address)
225+ self.close_request(request)
226+
227+ def verify_request(self, request, client_address):
228+ """Verify the request. May be overridden.
229+
230+ Return True if we should proceed with this request.
231+
232+ """
233+ return True
234+
235+ def process_request(self, request, client_address):
236+ """Call finish_request.
237+
238+ Overridden by ForkingMixIn and ThreadingMixIn.
239+
240+ """
241+ self.finish_request(request, client_address)
242+ self.close_request(request)
243+
244+ def server_close(self):
245+ """Called to clean-up the server.
246+
247+ May be overridden.
248+
249+ """
250+ pass
251+
252+ def finish_request(self, request, client_address):
253+ """Finish one request by instantiating RequestHandlerClass."""
254+ self.RequestHandlerClass(request, client_address, self)
255+
256+ def close_request(self, request):
257+ """Called to clean up an individual request."""
258+ pass
259+
260+ def handle_error(self, request, client_address):
261+ """Handle an error gracefully. May be overridden.
262+
263+ The default is to print a traceback and continue.
264+
265+ """
266+ print '-'*40
267+ print 'Exception happened during processing of request from',
268+ print client_address
269+ import traceback
270+ traceback.print_exc() # XXX But this goes to stderr!
271+ print '-'*40
272+
273+
274+class TCPServer(BaseServer):
275+
276+ """Base class for various socket-based server classes.
277+
278+ Defaults to synchronous IP stream (i.e., TCP).
279+
280+ Methods for the caller:
281+
282+ - __init__(server_address, RequestHandlerClass)
283+ - serve_forever()
284+ - handle_request() # if you don't use serve_forever()
285+ - fileno() -> int # for select()
286+
287+ Methods that may be overridden:
288+
289+ - server_bind()
290+ - server_activate()
291+ - get_request() -> request, client_address
292+ - verify_request(request, client_address)
293+ - process_request(request, client_address)
294+ - close_request(request)
295+ - handle_error()
296+
297+ Methods for derived classes:
298+
299+ - finish_request(request, client_address)
300+
301+ Class variables that may be overridden by derived classes or
302+ instances:
303+
304+ - address_family
305+ - socket_type
306+ - request_queue_size (only for stream sockets)
307+ - allow_reuse_address
308+
309+ Instance variables:
310+
311+ - server_address
312+ - RequestHandlerClass
313+ - socket
314+
315+ """
316+
317+ address_family = socket.AF_INET
318+
319+ socket_type = socket.SOCK_STREAM
320+
321+ request_queue_size = 5
322+
323+ allow_reuse_address = False
324+
325+ def __init__(self, server_address, RequestHandlerClass):
326+ """Constructor. May be extended, do not override."""
327+ BaseServer.__init__(self, server_address, RequestHandlerClass)
328+ self.socket = socket.socket(self.address_family,
329+ self.socket_type)
330+ self.server_bind()
331+ self.server_activate()
332+
333+ def server_bind(self):
334+ """Called by constructor to bind the socket.
335+
336+ May be overridden.
337+
338+ """
339+ if self.allow_reuse_address:
340+ self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
341+ self.socket.bind(self.server_address)
342+ self.server_address = self.socket.getsockname()
343+
344+ def server_activate(self):
345+ """Called by constructor to activate the server.
346+
347+ May be overridden.
348+
349+ """
350+ self.socket.listen(self.request_queue_size)
351+
352+ def server_close(self):
353+ """Called to clean-up the server.
354+
355+ May be overridden.
356+
357+ """
358+ self.socket.close()
359+
360+ def fileno(self):
361+ """Return socket file number.
362+
363+ Interface required by select().
364+
365+ """
366+ return self.socket.fileno()
367+
368+ def get_request(self):
369+ """Get the request and client address from the socket.
370+
371+ May be overridden.
372+
373+ """
374+ return self.socket.accept()
375+
376+ def close_request(self, request):
377+ """Called to clean up an individual request."""
378+ request.close()
379+
380+
381+class UDPServer(TCPServer):
382+
383+ """UDP server class."""
384+
385+ allow_reuse_address = False
386+
387+ socket_type = socket.SOCK_DGRAM
388+
389+ max_packet_size = 8192
390+
391+ def get_request(self):
392+ data, client_addr = self.socket.recvfrom(self.max_packet_size)
393+ return (data, self.socket), client_addr
394+
395+ def server_activate(self):
396+ # No need to call listen() for UDP.
397+ pass
398+
399+ def close_request(self, request):
400+ # No need to close anything.
401+ pass
402+
403+class ForkingMixIn:
404+
405+ """Mix-in class to handle each request in a new process."""
406+
407+ active_children = None
408+ max_children = 40
409+
410+ def collect_children(self):
411+ """Internal routine to wait for died children."""
412+ while self.active_children:
413+ if len(self.active_children) < self.max_children:
414+ options = os.WNOHANG
415+ else:
416+ # If the maximum number of children are already
417+ # running, block while waiting for a child to exit
418+ options = 0
419+ try:
420+ pid, status = os.waitpid(0, options)
421+ except os.error:
422+ pid = None
423+ if not pid: break
424+ self.active_children.remove(pid)
425+
426+ def process_request(self, request, client_address):
427+ """Fork a new subprocess to process the request."""
428+ self.collect_children()
429+ pid = os.fork()
430+ if pid:
431+ # Parent process
432+ if self.active_children is None:
433+ self.active_children = []
434+ self.active_children.append(pid)
435+ self.close_request(request)
436+ return
437+ else:
438+ # Child process.
439+ # This must never return, hence os._exit()!
440+ try:
441+ self.finish_request(request, client_address)
442+ os._exit(0)
443+ except:
444+ try:
445+ self.handle_error(request, client_address)
446+ finally:
447+ os._exit(1)
448+
449+
450+class ThreadingMixIn:
451+ """Mix-in class to handle each request in a new thread."""
452+
453+ # Decides how threads will act upon termination of the
454+ # main process
455+ daemon_threads = False
456+
457+ def process_request_thread(self, request, client_address):
458+ """Same as in BaseServer but as a thread.
459+
460+ In addition, exception handling is done here.
461+
462+ """
463+ try:
464+ self.finish_request(request, client_address)
465+ self.close_request(request)
466+ except:
467+ self.handle_error(request, client_address)
468+ self.close_request(request)
469+
470+ def process_request(self, request, client_address):
471+ """Start a new thread to process the request."""
472+ import threading
473+ t = threading.Thread(target = self.process_request_thread,
474+ args = (request, client_address))
475+ if self.daemon_threads:
476+ t.setDaemon (1)
477+ t.start()
478+
479+
480+class ForkingUDPServer(ForkingMixIn, UDPServer): pass
481+class ForkingTCPServer(ForkingMixIn, TCPServer): pass
482+
483+class ThreadingUDPServer(ThreadingMixIn, UDPServer): pass
484+class ThreadingTCPServer(ThreadingMixIn, TCPServer): pass
485+
486+if hasattr(socket, 'AF_UNIX'):
487+
488+ class UnixStreamServer(TCPServer):
489+ address_family = socket.AF_UNIX
490+
491+ class UnixDatagramServer(UDPServer):
492+ address_family = socket.AF_UNIX
493+
494+ class ThreadingUnixStreamServer(ThreadingMixIn, UnixStreamServer): pass
495+
496+ class ThreadingUnixDatagramServer(ThreadingMixIn, UnixDatagramServer): pass
497+
498+class BaseRequestHandler:
499+
500+ """Base class for request handler classes.
501+
502+ This class is instantiated for each request to be handled. The
503+ constructor sets the instance variables request, client_address
504+ and server, and then calls the handle() method. To implement a
505+ specific service, all you need to do is to derive a class which
506+ defines a handle() method.
507+
508+ The handle() method can find the request as self.request, the
509+ client address as self.client_address, and the server (in case it
510+ needs access to per-server information) as self.server. Since a
511+ separate instance is created for each request, the handle() method
512+ can define arbitrary other instance variariables.
513+
514+ """
515+
516+ def __init__(self, request, client_address, server):
517+ self.request = request
518+ self.client_address = client_address
519+ self.server = server
520+ try:
521+ self.setup()
522+ self.handle()
523+ self.finish()
524+ finally:
525+ sys.exc_traceback = None # Help garbage collection
526+
527+ def setup(self):
528+ pass
529+
530+ def handle(self):
531+ pass
532+
533+ def finish(self):
534+ pass
535+
536+
537+# The following two classes make it possible to use the same service
538+# class for stream or datagram servers.
539+# Each class sets up these instance variables:
540+# - rfile: a file object from which receives the request is read
541+# - wfile: a file object to which the reply is written
542+# When the handle() method returns, wfile is flushed properly
543+
544+
545+class StreamRequestHandler(BaseRequestHandler):
546+
547+ """Define self.rfile and self.wfile for stream sockets."""
548+
549+ # Default buffer sizes for rfile, wfile.
550+ # We default rfile to buffered because otherwise it could be
551+ # really slow for large data (a getc() call per byte); we make
552+ # wfile unbuffered because (a) often after a write() we want to
553+ # read and we need to flush the line; (b) big writes to unbuffered
554+ # files are typically optimized by stdio even when big reads
555+ # aren't.
556+ rbufsize = -1
557+ wbufsize = 0
558+
559+ def setup(self):
560+ self.connection = self.request
561+ self.rfile = self.connection.makefile('rb', self.rbufsize)
562+ self.wfile = self.connection.makefile('wb', self.wbufsize)
563+
564+ def finish(self):
565+ if not self.wfile.closed:
566+ self.wfile.flush()
567+ self.wfile.close()
568+ self.rfile.close()
569+
570+
571+class DatagramRequestHandler(BaseRequestHandler):
572+
573+ # XXX Regrettably, I cannot get this working on Linux;
574+ # s.recvfrom() doesn't return a meaningful client address.
575+
576+ """Define self.rfile and self.wfile for datagram sockets."""
577+
578+ def setup(self):
579+ try:
580+ from cStringIO import StringIO
581+ except ImportError:
582+ from StringIO import StringIO
583+ self.packet, self.socket = self.request
584+ self.rfile = StringIO(self.packet)
585+ self.wfile = StringIO()
586+
587+ def finish(self):
588+ self.socket.sendto(self.wfile.getvalue(), self.client_address)

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t