History

add hg and python

cinap_lenrek committed 15 years ago

Commit 4bb9e221 parent 987265c3706 files · +1228772 −-233Browse files

Showing the changes to sys/lib/python/SocketServer.py (1 of 3706 files). Show all

Unified · Split

sys/lib/python/SocketServer.py+588−0
@@ -0,0 +1,588 @@
1"""Generic socket server classes.
2
3This module tries to capture the various aspects of defining a server:
4
5For socket-based servers:
6
7- address family:
8 - AF_INET{,6}: IP (Internet Protocol) sockets (default)
9 - AF_UNIX: Unix domain sockets
10 - others, e.g. AF_DECNET are conceivable (see <socket.h>
11- socket type:
12 - SOCK_STREAM (reliable stream, e.g. TCP)
13 - SOCK_DGRAM (datagrams, e.g. UDP)
14
15For request-based servers (including socket-based):
16
17- client address verification before further looking at the request
18 (This is actually a hook for any processing that needs to look
19 at the request before anything else, e.g. logging)
20- how to handle multiple requests:
21 - synchronous (one request is handled at a time)
22 - forking (each request is handled by a new process)
23 - threading (each request is handled by a new thread)
24
25The classes in this module favor the server type that is simplest to
26write: a synchronous TCP/IP server. This is bad class design, but
27save some typing. (There's also the issue that a deep class hierarchy
28slows down method lookups.)
29
30There are five classes in an inheritance diagram, four of which represent
31synchronous servers of four types:
32
33 +------------+
34 | BaseServer |
35 +------------+
36 |
37 v
38 +-----------+ +------------------+
39 | TCPServer |------->| UnixStreamServer |
40 +-----------+ +------------------+
41 |
42 v
43 +-----------+ +--------------------+
44 | UDPServer |------->| UnixDatagramServer |
45 +-----------+ +--------------------+
46
47Note that UnixDatagramServer derives from UDPServer, not from
48UnixStreamServer -- the only difference between an IP and a Unix
49stream server is the address family, which is simply repeated in both
50unix server classes.
51
52Forking and threading versions of each type of server can be created
53using the ForkingMixIn and ThreadingMixIn mix-in classes. For
54instance, a threading UDP server class is created as follows:
55
56 class ThreadingUDPServer(ThreadingMixIn, UDPServer): pass
57
58The Mix-in class must come first, since it overrides a method defined
59in UDPServer! Setting the various member variables also changes
60the behavior of the underlying server mechanism.
61
62To implement a service, you must derive a class from
63BaseRequestHandler and redefine its handle() method. You can then run
64various versions of the service by combining one of the server classes
65with your request handler class.
66
67The request handler class must be different for datagram or stream
68services. This can be hidden by using the request handler
69subclasses StreamRequestHandler or DatagramRequestHandler.
70
71Of course, you still have to use your head!
72
73For instance, it makes no sense to use a forking server if the service
74contains state in memory that can be modified by requests (since the
75modifications in the child process would never reach the initial state
76kept in the parent process and passed to each child). In this case,
77you can use a threading server, but you will probably have to use
78locks to avoid two requests that come in nearly simultaneous to apply
79conflicting changes to the server state.
80
81On the other hand, if you are building e.g. an HTTP server, where all
82data is stored externally (e.g. in the file system), a synchronous
83class will essentially render the service "deaf" while one request is
84being handled -- which may be for a very long time if a client is slow
85to reqd all the data it has requested. Here a threading or forking
86server is appropriate.
87
88In some cases, it may be appropriate to process part of a request
89synchronously, but to finish processing in a forked child depending on
90the request data. This can be implemented by using a synchronous
91server and doing an explicit fork in the request handler class
92handle() method.
93
94Another approach to handling multiple simultaneous requests in an
95environment that supports neither threads nor fork (or where these are
96too expensive or inappropriate for the service) is to maintain an
97explicit table of partially finished requests and to use select() to
98decide which request to work on next (or whether to handle a new
99incoming request). This is particularly important for stream services
100where each client can potentially be connected for a long time (if
101threads or subprocesses cannot be used).
102
103Future work:
104- Standard classes for Sun RPC (which uses either UDP or TCP)
105- Standard mix-in classes to implement various authentication
106 and encryption schemes
107- Standard framework for select-based multiplexing
108
109XXX Open problems:
110- What to do with out-of-band data?
111
112BaseServer:
113- split generic "request" functionality out into BaseServer class.
114 Copyright (C) 2000 Luke Kenneth Casson Leighton <lkcl@samba.org>
115
116 example: read entries from a SQL database (requires overriding
117 get_request() to return a table entry from the database).
118 entry is processed by a RequestHandlerClass.
119
120"""
121
122# Author of the BaseServer patch: Luke Kenneth Casson Leighton
123
124# XXX Warning!
125# There is a test suite for this module, but it cannot be run by the
126# standard regression test.
127# To run it manually, run Lib/test/test_socketserver.py.
128
129__version__ = "0.4"
130
131
132import socket
133import sys
134import os
135
136__all__ = ["TCPServer","UDPServer","ForkingUDPServer","ForkingTCPServer",
137 "ThreadingUDPServer","ThreadingTCPServer","BaseRequestHandler",
138 "StreamRequestHandler","DatagramRequestHandler",
139 "ThreadingMixIn", "ForkingMixIn"]
140if hasattr(socket, "AF_UNIX"):
141 __all__.extend(["UnixStreamServer","UnixDatagramServer",
142 "ThreadingUnixStreamServer",
143 "ThreadingUnixDatagramServer"])
144
145class BaseServer:
146
147 """Base class for server classes.
148
149 Methods for the caller:
150
151 - __init__(server_address, RequestHandlerClass)
152 - serve_forever()
153 - handle_request() # if you do not use serve_forever()
154 - fileno() -> int # for select()
155
156 Methods that may be overridden:
157
158 - server_bind()
159 - server_activate()
160 - get_request() -> request, client_address
161 - verify_request(request, client_address)
162 - server_close()
163 - process_request(request, client_address)
164 - close_request(request)
165 - handle_error()
166
167 Methods for derived classes:
168
169 - finish_request(request, client_address)
170
171 Class variables that may be overridden by derived classes or
172 instances:
173
174 - address_family
175 - socket_type
176 - allow_reuse_address
177
178 Instance variables:
179
180 - RequestHandlerClass
181 - socket
182
183 """
184
185 def __init__(self, server_address, RequestHandlerClass):
186 """Constructor. May be extended, do not override."""
187 self.server_address = server_address
188 self.RequestHandlerClass = RequestHandlerClass
189
190 def server_activate(self):
191 """Called by constructor to activate the server.
192
193 May be overridden.
194
195 """
196 pass
197
198 def serve_forever(self):
199 """Handle one request at a time until doomsday."""
200 while 1:
201 self.handle_request()
202
203 # The distinction between handling, getting, processing and
204 # finishing a request is fairly arbitrary. Remember:
205 #
206 # - handle_request() is the top-level call. It calls
207 # get_request(), verify_request() and process_request()
208 # - get_request() is different for stream or datagram sockets
209 # - process_request() is the place that may fork a new process
210 # or create a new thread to finish the request
211 # - finish_request() instantiates the request handler class;
212 # this constructor will handle the request all by itself
213
214 def handle_request(self):
215 """Handle one request, possibly blocking."""
216 try:
217 request, client_address = self.get_request()
218 except socket.error:
219 return
220 if self.verify_request(request, client_address):
221 try:
222 self.process_request(request, client_address)
223 except:
224 self.handle_error(request, client_address)
225 self.close_request(request)
226
227 def verify_request(self, request, client_address):
228 """Verify the request. May be overridden.
229
230 Return True if we should proceed with this request.
231
232 """
233 return True
234
235 def process_request(self, request, client_address):
236 """Call finish_request.
237
238 Overridden by ForkingMixIn and ThreadingMixIn.
239
240 """
241 self.finish_request(request, client_address)
242 self.close_request(request)
243
244 def server_close(self):
245 """Called to clean-up the server.
246
247 May be overridden.
248
249 """
250 pass
251
252 def finish_request(self, request, client_address):
253 """Finish one request by instantiating RequestHandlerClass."""
254 self.RequestHandlerClass(request, client_address, self)
255
256 def close_request(self, request):
257 """Called to clean up an individual request."""
258 pass
259
260 def handle_error(self, request, client_address):
261 """Handle an error gracefully. May be overridden.
262
263 The default is to print a traceback and continue.
264
265 """
266 print '-'*40
267 print 'Exception happened during processing of request from',
268 print client_address
269 import traceback
270 traceback.print_exc() # XXX But this goes to stderr!
271 print '-'*40
272
273
274class TCPServer(BaseServer):
275
276 """Base class for various socket-based server classes.
277
278 Defaults to synchronous IP stream (i.e., TCP).
279
280 Methods for the caller:
281
282 - __init__(server_address, RequestHandlerClass)
283 - serve_forever()
284 - handle_request() # if you don't use serve_forever()
285 - fileno() -> int # for select()
286
287 Methods that may be overridden:
288
289 - server_bind()
290 - server_activate()
291 - get_request() -> request, client_address
292 - verify_request(request, client_address)
293 - process_request(request, client_address)
294 - close_request(request)
295 - handle_error()
296
297 Methods for derived classes:
298
299 - finish_request(request, client_address)
300
301 Class variables that may be overridden by derived classes or
302 instances:
303
304 - address_family
305 - socket_type
306 - request_queue_size (only for stream sockets)
307 - allow_reuse_address
308
309 Instance variables:
310
311 - server_address
312 - RequestHandlerClass
313 - socket
314
315 """
316
317 address_family = socket.AF_INET
318
319 socket_type = socket.SOCK_STREAM
320
321 request_queue_size = 5
322
323 allow_reuse_address = False
324
325 def __init__(self, server_address, RequestHandlerClass):
326 """Constructor. May be extended, do not override."""
327 BaseServer.__init__(self, server_address, RequestHandlerClass)
328 self.socket = socket.socket(self.address_family,
329 self.socket_type)
330 self.server_bind()
331 self.server_activate()
332
333 def server_bind(self):
334 """Called by constructor to bind the socket.
335
336 May be overridden.
337
338 """
339 if self.allow_reuse_address:
340 self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
341 self.socket.bind(self.server_address)
342 self.server_address = self.socket.getsockname()
343
344 def server_activate(self):
345 """Called by constructor to activate the server.
346
347 May be overridden.
348
349 """
350 self.socket.listen(self.request_queue_size)
351
352 def server_close(self):
353 """Called to clean-up the server.
354
355 May be overridden.
356
357 """
358 self.socket.close()
359
360 def fileno(self):
361 """Return socket file number.
362
363 Interface required by select().
364
365 """
366 return self.socket.fileno()
367
368 def get_request(self):
369 """Get the request and client address from the socket.
370
371 May be overridden.
372
373 """
374 return self.socket.accept()
375
376 def close_request(self, request):
377 """Called to clean up an individual request."""
378 request.close()
379
380
381class UDPServer(TCPServer):
382
383 """UDP server class."""
384
385 allow_reuse_address = False
386
387 socket_type = socket.SOCK_DGRAM
388
389 max_packet_size = 8192
390
391 def get_request(self):
392 data, client_addr = self.socket.recvfrom(self.max_packet_size)
393 return (data, self.socket), client_addr
394
395 def server_activate(self):
396 # No need to call listen() for UDP.
397 pass
398
399 def close_request(self, request):
400 # No need to close anything.
401 pass
402
403class ForkingMixIn:
404
405 """Mix-in class to handle each request in a new process."""
406
407 active_children = None
408 max_children = 40
409
410 def collect_children(self):
411 """Internal routine to wait for died children."""
412 while self.active_children:
413 if len(self.active_children) < self.max_children:
414 options = os.WNOHANG
415 else:
416 # If the maximum number of children are already
417 # running, block while waiting for a child to exit
418 options = 0
419 try:
420 pid, status = os.waitpid(0, options)
421 except os.error:
422 pid = None
423 if not pid: break
424 self.active_children.remove(pid)
425
426 def process_request(self, request, client_address):
427 """Fork a new subprocess to process the request."""
428 self.collect_children()
429 pid = os.fork()
430 if pid:
431 # Parent process
432 if self.active_children is None:
433 self.active_children = []
434 self.active_children.append(pid)
435 self.close_request(request)
436 return
437 else:
438 # Child process.
439 # This must never return, hence os._exit()!
440 try:
441 self.finish_request(request, client_address)
442 os._exit(0)
443 except:
444 try:
445 self.handle_error(request, client_address)
446 finally:
447 os._exit(1)
448
449
450class ThreadingMixIn:
451 """Mix-in class to handle each request in a new thread."""
452
453 # Decides how threads will act upon termination of the
454 # main process
455 daemon_threads = False
456
457 def process_request_thread(self, request, client_address):
458 """Same as in BaseServer but as a thread.
459
460 In addition, exception handling is done here.
461
462 """
463 try:
464 self.finish_request(request, client_address)
465 self.close_request(request)
466 except:
467 self.handle_error(request, client_address)
468 self.close_request(request)
469
470 def process_request(self, request, client_address):
471 """Start a new thread to process the request."""
472 import threading
473 t = threading.Thread(target = self.process_request_thread,
474 args = (request, client_address))
475 if self.daemon_threads:
476 t.setDaemon (1)
477 t.start()
478
479
480class ForkingUDPServer(ForkingMixIn, UDPServer): pass
481class ForkingTCPServer(ForkingMixIn, TCPServer): pass
482
483class ThreadingUDPServer(ThreadingMixIn, UDPServer): pass
484class ThreadingTCPServer(ThreadingMixIn, TCPServer): pass
485
486if hasattr(socket, 'AF_UNIX'):
487
488 class UnixStreamServer(TCPServer):
489 address_family = socket.AF_UNIX
490
491 class UnixDatagramServer(UDPServer):
492 address_family = socket.AF_UNIX
493
494 class ThreadingUnixStreamServer(ThreadingMixIn, UnixStreamServer): pass
495
496 class ThreadingUnixDatagramServer(ThreadingMixIn, UnixDatagramServer): pass
497
498class BaseRequestHandler:
499
500 """Base class for request handler classes.
501
502 This class is instantiated for each request to be handled. The
503 constructor sets the instance variables request, client_address
504 and server, and then calls the handle() method. To implement a
505 specific service, all you need to do is to derive a class which
506 defines a handle() method.
507
508 The handle() method can find the request as self.request, the
509 client address as self.client_address, and the server (in case it
510 needs access to per-server information) as self.server. Since a
511 separate instance is created for each request, the handle() method
512 can define arbitrary other instance variariables.
513
514 """
515
516 def __init__(self, request, client_address, server):
517 self.request = request
518 self.client_address = client_address
519 self.server = server
520 try:
521 self.setup()
522 self.handle()
523 self.finish()
524 finally:
525 sys.exc_traceback = None # Help garbage collection
526
527 def setup(self):
528 pass
529
530 def handle(self):
531 pass
532
533 def finish(self):
534 pass
535
536
537# The following two classes make it possible to use the same service
538# class for stream or datagram servers.
539# Each class sets up these instance variables:
540# - rfile: a file object from which receives the request is read
541# - wfile: a file object to which the reply is written
542# When the handle() method returns, wfile is flushed properly
543
544
545class StreamRequestHandler(BaseRequestHandler):
546
547 """Define self.rfile and self.wfile for stream sockets."""
548
549 # Default buffer sizes for rfile, wfile.
550 # We default rfile to buffered because otherwise it could be
551 # really slow for large data (a getc() call per byte); we make
552 # wfile unbuffered because (a) often after a write() we want to
553 # read and we need to flush the line; (b) big writes to unbuffered
554 # files are typically optimized by stdio even when big reads
555 # aren't.
556 rbufsize = -1
557 wbufsize = 0
558
559 def setup(self):
560 self.connection = self.request
561 self.rfile = self.connection.makefile('rb', self.rbufsize)
562 self.wfile = self.connection.makefile('wb', self.wbufsize)
563
564 def finish(self):
565 if not self.wfile.closed:
566 self.wfile.flush()
567 self.wfile.close()
568 self.rfile.close()
569
570
571class DatagramRequestHandler(BaseRequestHandler):
572
573 # XXX Regrettably, I cannot get this working on Linux;
574 # s.recvfrom() doesn't return a meaningful client address.
575
576 """Define self.rfile and self.wfile for datagram sockets."""
577
578 def setup(self):
579 try:
580 from cStringIO import StringIO
581 except ImportError:
582 from StringIO import StringIO
583 self.packet, self.socket = self.request
584 self.rfile = StringIO(self.packet)
585 self.wfile = StringIO()
586
587 def finish(self):
588 self.socket.sendto(self.wfile.getvalue(), self.client_address)

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t