Basic python threads - host pinger

Just a basic script modified to be useful. I stole the bulk of it from http://stackoverflow.com/questions/316866/ping-a-site-in-python (sorry!).

As identified, the problem with basic shell scripts or sequential code to ping dozens of hosts is that there's no inherent threading. You could background the commands, log to files, concatenate the results and generate a report, but that's annoying.

Python has pretty easy threading let's have a go of that.

Create a file with a list of host names or IP addresses called "serverlist.txt".

Put the following code in a python script:

from threading import Thread
import subprocess
from Queue import Queue

listfile = "serverlist.txt"

num_threads = 4
queue = Queue()

ips = []
hostfptr = open(listfile)
while True:
   hostname = hostfptr.readline().strip()
   if not hostname: break
   ips.append(hostname)

#wraps system ping command
def pinger(i, q):
    """Pings subnet"""
    while True:
        ip = q.get()
        ##print "Thread %s: Pinging %s" % (i, ip)
        ret = subprocess.call("ping -c 1 %s" % ip,
                        shell=True,
                        stdout=open('/dev/null', 'w'),
                        stderr=subprocess.STDOUT)
        if ret == 0:
            print "%-10s Alive" % ip
        else:
            print "%-10s No response" % ip
        q.task_done()

#Spawn thread pool
for i in range(num_threads):

    worker = Thread(target=pinger, args=(i, queue))
    worker.setDaemon(True)
    worker.start()

#Place work in queue
for ip in ips:
    queue.put(ip)

#Wait until worker threads are done to exit
queue.join()

You may need to adjust the ping parameters if "ping -c 1 " doesn't work on your OS. And assuming a unix-like ping that returns an exit code of 0 if the ping was a success.

Run the script and you should find it quickly prints the status of each host in your server list.

If you like, you can uncomment the "##print" in the pinger function, it will show which thread it's using (out of the 4 we've made available).

One thing I've noticed after running it a few dozen times is some thread errors when the interpreter is exiting:

Exception in thread Thread-2 (most likely raised during interpreter shutdown):
Traceback (most recent call last):
  File "/usr/lib/python2.5/threading.py", line 486, in __bootstrap_inner
  File "/usr/lib/python2.5/threading.py", line 446, in run
  File "./pingle2.py", line 23, in pinger
  File "/usr/lib/python2.5/Queue.py", line 165, in get
  File "/usr/lib/python2.5/threading.py", line 209, in wait
<type 'exceptions.TypeError'>: 'NoneType' object is not callable

Sounds like the threads aren't really complete before the script ends?

Just to test that theory I added an explicit test after the queue.join() which is supposed to wait until they're done:

if queue.empty(): print "good!!"
else: print "onoes!!"

Result: It reports "good!!" even when the error appears. So not really sure what's going on here. Possibly some buffering from the os call?

Probably should use python native code to build its own pings - will look into it.