Thursday, April 30, 2009

Basic python threads - host pinger

Just a basic script modified to be useful. I stole the bulk of it from http://stackoverflow.com/questions/316866/ping-a-site-in-python (sorry!).

As identified, the problem with basic shell scripts or sequential code to ping dozens of hosts is that there's no inherent threading. You could background the commands, log to files, concatenate the results and generate a report, but that's annoying.

Python has pretty easy threading let's have a go of that.

Create a file with a list of host names or IP addresses called "serverlist.txt".

Put the following code in a python script:

from threading import Thread
import subprocess
from Queue import Queue

listfile = "serverlist.txt"

num_threads = 4
queue = Queue()

ips = []
hostfptr = open(listfile)
while True:
hostname = hostfptr.readline().strip()
if not hostname: break
ips.append(hostname)

#wraps system ping command
def pinger(i, q):
"""Pings subnet"""
while True:
ip = q.get()
##print "Thread %s: Pinging %s" % (i, ip)
ret = subprocess.call("ping -c 1 %s" % ip,
shell=True,
stdout=open('/dev/null', 'w'),
stderr=subprocess.STDOUT)
if ret == 0:
print "%-10s Alive" % ip
else:
print "%-10s No response" % ip
q.task_done()

#Spawn thread pool
for i in range(num_threads):

worker = Thread(target=pinger, args=(i, queue))
worker.setDaemon(True)
worker.start()

#Place work in queue
for ip in ips:
queue.put(ip)

#Wait until worker threads are done to exit
queue.join()


You may need to adjust the ping parameters if "ping -c 1 " doesn't work on your OS. And assuming a unix-like ping that returns an exit code of 0 if the ping was a success.

Run the script and you should find it quickly prints the status of each host in your server list.

If you like, you can uncomment the "##print" in the pinger function, it will show which thread it's using (out of the 4 we've made available).

One thing I've noticed after running it a few dozen times is some thread errors when the interpreter is exiting:

Exception in thread Thread-2 (most likely raised during interpreter shutdown):
Traceback (most recent call last):
File "/usr/lib/python2.5/threading.py", line 486, in __bootstrap_inner
File "/usr/lib/python2.5/threading.py", line 446, in run
File "./pingle2.py", line 23, in pinger
File "/usr/lib/python2.5/Queue.py", line 165, in get
File "/usr/lib/python2.5/threading.py", line 209, in wait
<type 'exceptions.TypeError'>: 'NoneType' object is not callable


Sounds like the threads aren't really complete before the script ends?

Just to test that theory I added an explicit test after the queue.join() which is supposed to wait until they're done:

if queue.empty(): print "good!!"
else: print "onoes!!"


Result: It reports "good!!" even when the error appears. So not really sure what's going on here. Possibly some buffering from the os call?

Probably should use python native code to build its own pings - will look into it.

Monday, April 20, 2009

Python 2.6.2 on SCO OpenServer 5.0.6a

Just a basic notice regarding Python 2.6.2 on SCO OpenServer 5.0.6a in case people are doing a search for it. Haven't seen any other reports.

The build was done based on my previous instructions on creating a working development environment.

Had to run --without-threads to start with.

/bin/bash
/usr/gnu/bin/tar zxvf Python-2.6.2.tgz
cd Python-2.6.2
CFLAGS=" -O2 -s " LIBS=" -lsocket " ./configure --without-threads
make
make install


Result is a python build as follows:

Failed to find the necessary bits to build these modules:
_sqlite3 bsddb185 linuxaudiodev
ossaudiodev sunaudiodev zlib
To find the necessary bits, look in setup.py in detect_modules() for the module's name.


Failed to build these modules:
_ctypes _curses _curses_panel
_multiprocessing _socket _ssl
_tkinter mmap readline
select spwd



Test.


Python-2.6.2 >./python --version
Python 2.6.2
Python-2.6.2 > make test
...
./python -E -tt ./Lib/test/regrtest.py -l
Traceback (most recent call last):
File "./Lib/test/regrtest.py", line 168, in
from test import test_support
File "/u/he61041/source/Python-2.6.2/Lib/test/test_support.py", line 8, in
import socket
File "/u/he61041/source/Python-2.6.2/Lib/socket.py", line 46, in
import _socket
ImportError: No module named _socket
make: [test] Error 1 (ignored)
./python -E -tt ./Lib/test/regrtest.py -l
Traceback (most recent call last):
File "./Lib/test/regrtest.py", line 168, in
from test import test_support
File "/u/he61041/source/Python-2.6.2/Lib/test/test_support.py", line 8, in
import socket
File "/u/he61041/source/Python-2.6.2/Lib/socket.py", line 46, in
import _socket
ImportError: No module named _socket
make: *** [test] Error 1


Yep I would call it a showstopper if the socket lib fails.