stillhq.com : Mikal, a geek from Canberra living in Silicon Valley (no blather posts)

paramiko exec_command timeout

Sun, 05 Oct 2008 12:20:00 GMT

I have a paramiko program which sshs to a large number of machines, and sometimes it hits a machine where Channel.exec_command() doesn't return. I know this is a problem with the remote machine, because the same thing happens when I try to ssh to the machine from the command line. However, I don't have any way of determining which machines are broken beforehand.

Paramiko doesn't support a timeout for exec_command(), so I am looking for a generic way of running a function call with a timeout. I can see sample code which does this using threads, but that's pretty ugly. I can't use SIGALARM because I am not running on the main thread.

Can anyone think of a better way of doing this?

Tags for this post: python() paramiko()

Comment

MythNetTV release 6

Fri, 03 Oct 2008 13:09:00 GMT

New things in this release:

Better testing:

r114: Unit tests for video.py
r148: Updated unit test now that we create more tables
r151: Start work on syndication unit tests
r153: Unit test for bad syndication dates, as well as more flexibility in db connections needed for this unit test
r157: Nicer unit test failure output, and refactor GetVideoDir() into the utility module
r181: Slightly improve syndication unit tests
r183: Add flag parsing to unit tests, and fix some more flag name changes

Better documentation:

r117: Added a man page for video.py -- I'm not 100% happy with its name though

Bug fixes:

r114: Fixed a bug where the new filename for video transcode could be the same as the input filename, resulting in video corruption. This was found with one of the new video.py unit tests
r116: The logic for the --prompt flag was the wrong way around. Fixed.
r119: Nicer download status messages
r121: Handle 404s in feed updates better
r129: Slight tweak to SVN submit script
r131: More accurate tracking of proxy usage (update during download, instead of just at the end)
r137: Proxy budget being exceeded doesn't count as a failed download attempt
r143: Subscribe now renables inactive subscriptions
r146: Add support to decimals to utility byte printer, fix a bug in the check for video directories
r155: Have users send problems to the mailing list, instead of me personally
r161: Don't throw exceptions for the videodir command line
r167, 169: Display friendly sizes in records_tool output
r171: Move verbose update arg into a flag
r173: Add "-vo null" to mplayer invocation per Ryan Lutz. This improves support on machines without X, and speeds up the identify operation
r175: Import patch from Thomas Mashos which fixes subscription removal, started work on syndication unit test improvements
r177: Fix character escaping bug in show subtitles during import
r179: Renamed --datadirdefault to --datadir. If set this will now change your data directory, regardless of if there was a previous value set.
r190: Recording_tool now prompts for deletes
r192: Improved explainvideodir output
r194: Don't crash in explainvideodir if there is no video directory
r197: Order results by subtitle in recordings_tool output

New features:

r115: Upgraded schema to version 15 to support http_proxies for subscriptions. Added http_proxy command line, which allows you to use HTTP proxies for specified URLs. Moved HTTP accesses to use the proxy code.
r127: Bump schema to v17, and add proxy use tracking including the "proxyusage" command
r133: Allow daily budgets for proxy usage
r115: Provide a user agent for HTTP requests, instead of just Python-urllib/1.17
r117: Users will now be prompted to subscribe to an announcements video feed. This will happen exactly once. This behavior may be disabled with the --nopromptforannounce command line flag.
r125: Add a full info dump command to video.py's command line interface
r139: Bump schema to 19, and implement categories with the "category" command
r141: Implement recording group support, and clarify category support
r151: Implement a helper (recordings_tool) for handling the MythTV recordings table, this is useful for testing.
r159: Add videodir and explainvideodir debugging commands, and update man page
r163: Add title list feature to recordings_tool
r165: Include recording count in title list
r185: Add the resetattempts command

Development changes:

r123: Added a submit script to automate putting the revision number into the ChangeLog
r135: Tweak to new ChangeLog auto logging formatting

Release 6 continues the tradition of better testing, improves the documentation (a little, there is more work to be done there), fixes a bunch of bugs, and implements some new features which will hopefully be useful to others. Please grab your copy here.

Tags for this post: mythtv() mythnettv()

Comment

The Malloreon

Thu, 02 Oct 2008 20:34:00 GMT

This is the followup series to the Belgariad. Its mostly a sequel series, but is quite readable and worth the effort.

Year	Title
1988	Guardians of the West
1989	King of the Murgos
1989	Demon Lord of Karanda
1990	Sorceress of Darshiva
1992	Seeress of Kell

Tags for this post: book() David_Eddings()

Comment

The Seeress of Kell

Thu, 02 Oct 2008 20:20:00 GMT

The conclusion to the Malloreon. Quite a good read, although the plot twists are predictable.

LibraryThing link for ISBN 0345377591.

Tags for this post: book() David_Eddings()

Comment

Books read in September 2008

Tue, 30 Sep 2008 20:01:00 GMT

Enchanters' End Game
Robot Visions
Forward the Foundation
Foundation
Foundation and Empire
Guardians of the West
The Stainless Steel Rat for President
King of the Murgos
Second Foundation
Sticklers, Sideburns and Bikinis (not fully read yet, its a reference book)
Foundation's Fear (didn't finish cause it sucked)
Demon Lord of Karanda
Sorceress of Darshiva

Tags for this post: book() read()

Comment

Sorceress of Darshiva

Tue, 30 Sep 2008 19:55:00 GMT

This is book four of the Malloreon. It feels like not much happens in this book, although there is plot development happening. I guess it more feels like this book is a connector between the book before and the book after, and not much more. On the other hand, it was an engaging read.

LibraryThing link for ISBN 0345369351.

Tags for this post: book() David_Eddings()

Comment

Demon Lord of Karanda

Sun, 28 Sep 2008 18:00:00 GMT

I needed something to cheer me up after the previous sucky book. This one did the trick. Again its pretty standard Fantasy stuff, and there were no surprises, but its fun to read.

LibraryThing link for ISBN 0345363310.

Tags for this post: book() David_Eddings()

Comment

Foundation's Fear

Fri, 26 Sep 2008 21:16:00 GMT

This book is a solid zero stars in my mind. I got to page 372, but simply couldn't wade through the chore any longer. The plot meanders, and its not clear to me where the story is going. Worse still, basically nothing has happened yet. I am a little surprised, given the generally positive LibraryThing reviews. I should have read the Amazon reviews instead. Some examples:

Normally, I do a lot of my reading on the train (BART for those of you familiar with San Francisco), getting to and from work. An engrossing book keeps me awake and I read it relatively quickly. "Foundation's Fear", especially the first half of it, set a record for putting me to sleep. There were days in when I only managed to read a couple of pages. A paragraph or two and I'd be out, even before the train started moving. As others here have pointed out, there is a lot of boring dialogue and description and much of it focuses around the Voltaire and Joan of Arc artificial entities. Hundreds of pages of philosophical noodling and descriptions of imaginary scenes conjured up in cyberspace become numbing.

And another:

This book is not good, not because it's not Asimov but because it's simply not good. I had the luxury of reading it within the context of the other two "new" books and while that helps in hindsight, it doesn't while you're slogging through Benford's weighty prose.

Don't expect Asimov but then the reader shouldn't. As Bear and to a lesser extent Brin show, authors can bring a fresh perspective on the topic and do it fairly well. Benford never seems to make up his mind which of his myriad little sub plots will be the main plot and thus, nothing really happens that expands our understanding of the Foundation Galaxy. Moreover, instead of fleshing out some of Asimov's admittedly skimpy ideas in the Foundation galaxy or introducing new themes that build upon previous concepts, instead, we take a quantum leap into a muddled unknown with concepts (aliens and tiktoks being the two most egregious examples) that clearly don't belong in the Foundation setting.

This book differs from Asimov's view of the Foundation universe in important ways:

This book is much more explicit about Dors' nature than Asimov ever was. There was some element of doubt in Forward the Foundation right up until Dors' death. That is not the case with this book.
This book reworks Hari's entry into the First Minister position, which I found annoying. Especially because the discussion around that entry is slow, and lacks action. Basically the new version was kinda boring.
Worm holes are a major part of the economic makeup of the galactic empire in this book, but somehow Asimov never mentioned them in his books.
This book dwells on computers, robots, artificial intelligence, and aliens -- all things Asimov left out of his books (except for robots of course). Its not like Asimov was unaware of these things, he just didn't use them in this universe.
This book is really long (600 pages), but nothing much seems to actually happen in the first several hundred. The Sims sequence is the first really interesting part of the book, and even that drags on into long boring descriptions of polygons waving in the virtual wind.

LibraryThing link for ISBN 0061056383.

Tags for this post: book() Gregory_Benford()

Comment

Sticklers, Sideburns and Bikinis

Wed, 24 Sep 2008 13:48:00 GMT

This is the second book I have got from the LibraryThing early reviewer program (the previous book was The Spy Within).

I'm going to have to be honest with this book and say that I haven't read all of it yet. Its basically a dictionary and I don't tend to read those cover to cover. On the other hand, I have read about 10% of this book so far, and do intend to read it all.

This book is great. The content is interesting, well laid out, and excellently written. That's probably made a lot easier by how interesting the content is -- the stories behind various works tend to be short and novel, which is fun. The book is also useful -- it's already ended an office debate over the origin of the word "hooker".

LibraryThing link for ISBN 9781846033001.

Tags for this post: book() Graeme_Donald()

Comment

Please help me test trunk

Tue, 23 Sep 2008 20:42:00 GMT

I'm getting ready to do another MythNetTV release, as I've fixed quite a few things recently. I'd appreciate people testing the new code before I do an official release. You can get the current code like this:

svn co http://www.stillhq.com/mythtv/mythnettv/svn

That will create a directory called mythnettv, with a subdirectory named trunk, which is the latest development version of the code. You should probably create that directory somewhere where you don't mind a new directory being created.

This release has lots of small changes, which are listed in the ChangeLog which will be downloaded with the code.

Thanks.

Tags for this post: mythtv() mythnettv()

Comment

Factory reset

Tue, 23 Sep 2008 07:34:00 GMT

Huh. I just had to factory reset the roku netflix box after suffering from the forever downloading my instant queue problem. In my case the device was in this state for over a week (it was acting weird, so I watched a different device and completely forgot about it). Its disappointing that a factory reset is needed, and that power cycling the device does just work.

I wonder if reactivating the device counts in the four device limit that netflix imposes?

Tags for this post: netflix() roku()

Comment

Second Foundation

Mon, 22 Sep 2008 20:58:00 GMT

I am starting to think that there is something missing in the Foundation trilogy. Specifically, there isn't much action. Most of it is just people talking at each other -- with pages and pages of dialog. This makes these three books much harder to read (and therefore less compelling) than those elsewhere in the extended Foundation Series.

I did enjoy this book, I just feel that I could have done with some more action to make it less hard work.

LibraryThing link for ISBN 0345336291.

Tags for this post: book() Isaac_Asimov()

Comment

King of the Murgos

Mon, 22 Sep 2008 15:44:00 GMT

This is book two of the Malloreon, the sequel series to the Belgariad. Some people complain that this series is just a repeat of the previous series with some of the names changed, and I certainly used to think that myself. Now that I've read this book again I think that's unfair -- the story line is actually quite different, and the characters are generally older and wiser (which makes them feel more developed). I enjoyed this book, probably more than I did Guardians of the West.

LibraryThing link for ISBN 0345358805.

Tags for this post: book() David_Eddings()

Comment

Does anyone else think its odd

Mon, 22 Sep 2008 14:06:00 GMT

Does anyone else think it is odd that my kid's school had a "code red alert" drill this morning? This is where they have the whole school pretend that there is a gun toting maniac on campus, and the class goes to a corner and hides. They practise being quiet while someone beats on the door, that sort of thing.

I find the whole thing just a little disturbing.

Tags for this post: blog()

Comment

The Stainless Steel Rat for President

Wed, 17 Sep 2008 19:57:00 GMT

I need a break from all this serious reading, and the Stainless Steel Rat books are always amusing. In fact, this book was much better than the previous two I have read (The Stainless Steel Rat Saves The World and The Stainless Steel Rat Wants You) as it didn't suffer from the incredibly weak plot lines that books involving time travel seem to cause for Harrison.

This book is back to the classic rat -- a simple criminal romp through an unsuspecting society, his hottie wife in tow, and with his two now grown kids along for the fun. I enjoyed this book a lot.

LibraryThing link for ISBN 0553227599.

Tags for this post: book() Harry_Harrison()

Comment

Fixed LibraryThing links

Wed, 17 Sep 2008 13:09:00 GMT

I only just realized that the links to LibraryThing that I automatically insert into book posts only work if you have a LibraryThing account. I've now fixed that so that there is a link to a public page, as well as a link to your personal library if you use LibraryThing and also have a book that I mention.

Tags for this post: book()

Comment

Guardians of the West

Tue, 16 Sep 2008 13:35:00 GMT

I read this book as a child, and must admit I didn't like it. That's interesting to me, because it quite liked it this time. Its not the greatest literature ever written, but it is a good read. You need to have read the Belgariad first though, because this book (the first of the Mallorean) reuses the characters without spending a lot of time introducing them to you. I think I prefer it that way because it would be annoying to have to go through all that character development again as someone who has read the first series.

LibraryThing link for ISBN 0345352661.

Tags for this post: book() David_Eddings()

Comment

Weird paramiko problem

Tue, 16 Sep 2008 11:41:00 GMT

I had a strange paramiko problem the other day. Sometimes executing a command through a channel (via the exec_command() call) would result in an exit code being returned, but no stdout or stderr. This was for a command I was absolutely sure always returns output, and it wasn't consistent -- I'd run batches of commands and about 10% of them would fail, but not always on the same machine and not always at the same time. I spent ages looking at my code, and the code for the command running at the other end of the channel.

Then it occurred to me that this seemed a lot like a race condition. I started looking at the code for the paramiko Channel class, and ended up deciding that the answer was to check that the eof_received member variable was true before trying to close the channel.

It turns out this just works. I've my code running commands for a couple of days now and have had zero more instances of the "no output, but did exit" error. So, there you go. Its a shame that member variable doesn't have accessors and isn't documented though. I guess that makes my code a little more fragile than I would be happy with.

Tags for this post: python() paramiko()

Comment

Couldn't determine the video directory?

Sun, 14 Sep 2008 10:32:00 GMT

A couple of people have contacted me in the last couple of days with an error where MythNetTV can't determine the directory to put videos in. The error would look something like this:


Importing data/tekzilla--0050--tehbunniez--hd.h264.mp4
Traceback (most recent call last):
  File "./mythnettv", line 453, in 
    main(sys.argv)
  File "./mythnettv", line 212, in main
    mythnettvcore.DownloadAndImport(db, guid, out=out)
  File "/home/mythbox/Scripts/mythnettv/mythnettvcore.py", line 92, in 
  DownloadAndImport
    prog.Import(out=out)
  File "/home/mythbox/Scripts/mythnettv/program.py", line 472, in Import
    raise FilenameException(self.db, 'Could not determine the video '
program.FilenameException: Could not determine the video directory for this 
machine. Please report this to mythnettv@stillhq.com

The stack trace is mostly irrelevant. The problem here is that MythNetTV couldn't decide what directory to put the video in once downloaded. Please execute the following SQL statements against your MythTV database:

select * from storagegroup where groupname="MythNetTV";
select * from storagegroup where groupname="Default";
select * from settings where value="RecordFilePrefix";

This will dump all of the possible places MythNetTV will look for a video directory.

Once you've dumped this information, perform some simple checks:

Is there anything listed at all? If not, you need to configure storage groups with the MythTV user interface
Is there anything listed with the hostname that MythNetTV is running on? If not, you need to add configuration entries for your current hostname.

If you're still having problems, please send email to mythnettv@stillhq.com, with the output of those select commands, and the output of the hostname command.

Tags for this post: mythtv() mythnettv() tips()

Comment

Foundation and Empire

Sat, 13 Sep 2008 18:22:00 GMT

This is the second book in the original foundation trilogy, which I am reading as part of the the extended Foundation series that I am working my way slowly though. This book contains two stories -- both of them Seldon crises, although one of them unpredicted by Hari. As Hari had always said in the series -- his techniques can only predict broad social trends, and the not the work on individuals. What happens if a single person who could not be predicted appears? This story covers that scenario.

I found this book harder to read than the previous one, but that might have been because I've had a pretty distracted week. Once I actually sat down to read without too many interruptions, I enjoyed it.

The comments from others on LibraryThing are fair though -- the character names are odd, and the writing does feel a little awkward.

LibraryThing link for ISBN 0553293370.

(LibraryThing for some reason gets the ISBN mapping for this book wrong. The above link's ISBN is right, but this link goes to the right place).

Tags for this post: book() Isaac_Asimov()

Comment