stillhq.com : Mikal, a geek from Canberra living in Silicon Valley

MythNetTV release 6

Fri, 03 Oct 2008 13:09:00 GMT

New things in this release:

Better testing:

r114: Unit tests for video.py
r148: Updated unit test now that we create more tables
r151: Start work on syndication unit tests
r153: Unit test for bad syndication dates, as well as more flexibility in db connections needed for this unit test
r157: Nicer unit test failure output, and refactor GetVideoDir() into the utility module
r181: Slightly improve syndication unit tests
r183: Add flag parsing to unit tests, and fix some more flag name changes

Better documentation:

r117: Added a man page for video.py -- I'm not 100% happy with its name though

Bug fixes:

r114: Fixed a bug where the new filename for video transcode could be the same as the input filename, resulting in video corruption. This was found with one of the new video.py unit tests
r116: The logic for the --prompt flag was the wrong way around. Fixed.
r119: Nicer download status messages
r121: Handle 404s in feed updates better
r129: Slight tweak to SVN submit script
r131: More accurate tracking of proxy usage (update during download, instead of just at the end)
r137: Proxy budget being exceeded doesn't count as a failed download attempt
r143: Subscribe now renables inactive subscriptions
r146: Add support to decimals to utility byte printer, fix a bug in the check for video directories
r155: Have users send problems to the mailing list, instead of me personally
r161: Don't throw exceptions for the videodir command line
r167, 169: Display friendly sizes in records_tool output
r171: Move verbose update arg into a flag
r173: Add "-vo null" to mplayer invocation per Ryan Lutz. This improves support on machines without X, and speeds up the identify operation
r175: Import patch from Thomas Mashos which fixes subscription removal, started work on syndication unit test improvements
r177: Fix character escaping bug in show subtitles during import
r179: Renamed --datadirdefault to --datadir. If set this will now change your data directory, regardless of if there was a previous value set.
r190: Recording_tool now prompts for deletes
r192: Improved explainvideodir output
r194: Don't crash in explainvideodir if there is no video directory
r197: Order results by subtitle in recordings_tool output

New features:

r115: Upgraded schema to version 15 to support http_proxies for subscriptions. Added http_proxy command line, which allows you to use HTTP proxies for specified URLs. Moved HTTP accesses to use the proxy code.
r127: Bump schema to v17, and add proxy use tracking including the "proxyusage" command
r133: Allow daily budgets for proxy usage
r115: Provide a user agent for HTTP requests, instead of just Python-urllib/1.17
r117: Users will now be prompted to subscribe to an announcements video feed. This will happen exactly once. This behavior may be disabled with the --nopromptforannounce command line flag.
r125: Add a full info dump command to video.py's command line interface
r139: Bump schema to 19, and implement categories with the "category" command
r141: Implement recording group support, and clarify category support
r151: Implement a helper (recordings_tool) for handling the MythTV recordings table, this is useful for testing.
r159: Add videodir and explainvideodir debugging commands, and update man page
r163: Add title list feature to recordings_tool
r165: Include recording count in title list
r185: Add the resetattempts command

Development changes:

r123: Added a submit script to automate putting the revision number into the ChangeLog
r135: Tweak to new ChangeLog auto logging formatting

Release 6 continues the tradition of better testing, improves the documentation (a little, there is more work to be done there), fixes a bunch of bugs, and implements some new features which will hopefully be useful to others. Please grab your copy here.

Tags for this post: mythtv() mythnettv() Comment

Please help me test trunk

Tue, 23 Sep 2008 20:42:00 GMT

I'm getting ready to do another MythNetTV release, as I've fixed quite a few things recently. I'd appreciate people testing the new code before I do an official release. You can get the current code like this:

svn co http://www.stillhq.com/mythtv/mythnettv/svn

That will create a directory called mythnettv, with a subdirectory named trunk, which is the latest development version of the code. You should probably create that directory somewhere where you don't mind a new directory being created.

This release has lots of small changes, which are listed in the ChangeLog which will be downloaded with the code.

Thanks.

Tags for this post: mythtv() mythnettv() Comment

Couldn't determine the video directory?

Sun, 14 Sep 2008 10:32:00 GMT

A couple of people have contacted me in the last couple of days with an error where MythNetTV can't determine the directory to put videos in. The error would look something like this:


Importing data/tekzilla--0050--tehbunniez--hd.h264.mp4
Traceback (most recent call last):
  File "./mythnettv", line 453, in 
    main(sys.argv)
  File "./mythnettv", line 212, in main
    mythnettvcore.DownloadAndImport(db, guid, out=out)
  File "/home/mythbox/Scripts/mythnettv/mythnettvcore.py", line 92, in 
  DownloadAndImport
    prog.Import(out=out)
  File "/home/mythbox/Scripts/mythnettv/program.py", line 472, in Import
    raise FilenameException(self.db, 'Could not determine the video '
program.FilenameException: Could not determine the video directory for this 
machine. Please report this to mythnettv@stillhq.com

The stack trace is mostly irrelevant. The problem here is that MythNetTV couldn't decide what directory to put the video in once downloaded. Please execute the following SQL statements against your MythTV database:

select * from storagegroup where groupname="MythNetTV";
select * from storagegroup where groupname="Default";
select * from settings where value="RecordFilePrefix";

This will dump all of the possible places MythNetTV will look for a video directory.

Once you've dumped this information, perform some simple checks:

Is there anything listed at all? If not, you need to configure storage groups with the MythTV user interface
Is there anything listed with the hostname that MythNetTV is running on? If not, you need to add configuration entries for your current hostname.

If you're still having problems, please send email to mythnettv@stillhq.com, with the output of those select commands, and the output of the hostname command.

Tags for this post: mythtv() mythnettv() tips() Comment

MythNetTV release 5

Sat, 30 Aug 2008 15:11:00 GMT

New things in this release:

There is now a users mailing list at http://lists.stillhq.com/listinfo.cgi/mythnettv-stillhq.com
Moved to a public SVN server at http://www.stillhq.com/mythtv/mythnettv/svn/
Added the 'justone' syntax to the download command
Another try at using gflags. This means that all the command lines have changed slightly.
Moved the core of MythTV out of the user interface file.
Started writing unit tests
Changed user output code so that it doesn't insist on writing to stdout. You can now write to other file descriptors, which makes things like unit tests much easier to write.
Added video/msvideo to the enclosure whitelist
Added HTTP download progress information
Added a flag which turns off the prompts for markread (--noprompt)
Patches from Thomas Mashos
- Search ~/.mythtv/mysql.txt, /usr/share/mythtv/mysql.txt and /etc/mythtv/mysql.txt in that order for MySQL connection information
- A manpage
- setup.py
video.py now has a simple command line interface to let you query it
Fix update of inactive programs bug per http://ubuntuforums.org/showpost.php?p=5580005&postcount=4
Better DB error handling
Included a COPYING file with the right version of the GPL (it was missing before)
Fixed a bug where programs would be downloaded more than once (found with a unit test!)
Started raising exceptions instead of just sys.exit(1). This should make life easier for user interfaces in the future
Default to using storage groups for storing recordings before falling back to the RecordFilePrefix. This makes the behaviour: use a storage group named "MythNetTV" if it exists; use the default storage group if it exists; use the value of RecordFilePrefix.
Transcode avc1 videos, because some need it
Force ASCII encoding of title, subtitle, and all fields in the database to get around feeds which use unicode which python / MySQL can't store correctly
If there is only one attachment to an item, and its not in our whitelist of video formats, then warn the user that you're assuming its a video file and then add it to the todo list
Slight tweak to the signature of video.MythNetTvVideo.Transcode()
Fix buf in RepairMissingDates which caused it to consistently crash
Fix typo in date warning code
Better handling of videos where the length of the video cannot be determined by mplayer

Release 5 is by far the best tested release of MythNetTV yet, with both unit tests and several users working quite closely with me to resolve problems found in the wild. You can grab your copy here.

Tags for this post: mythtv() mythnettv() Comment

Please help me test trunk

Sun, 24 Aug 2008 14:42:00 GMT

I'm preparing a new release of MythNetTV, and would like some help testing the code, as I have re-factored how the user interface works and there is some risk that I have broken things in the process. You can get the code like this:

svn co http://www.stillhq.com/mythtv/mythnettv/svn

That will create a directory called mythnettv, with a subdirectory named trunk, which is the latest development version of the code. You should probably create that directory somewhere where you don't mind a new directory being created.

The biggest change is that the command line syntax has changed slightly -- the dashes have been removed from the commands. Therefore, to update your list of feeds, you now use:

  mythnettv update

Instead of:

  mythnettv --update

And so on. This was done so that I could add "real" flags, which are used to change default values like where the database configuration is read from, as well as what the default location for the temporary data directory is.

"Real" flags which are currently supported are:

  --datadirdefault:  The default location of the data directory
    (default: 'data')

  --db_host:  The name of the host the MySQL database is on,
              don't define if you want to parse
              ~/.mythtv/mysql.txt instead
    (default: '')
  --db_name:  The name of the database which MythNetTV uses,
              don't define if you want to parse 
              ~/.mythtv/mysql.txt instead
    (default: '')
  --db_password:  The password for the database user, don't
              define if you want to parse 
              ~/.mythtv/mysql.txt instead
    (default: '')
  --db_user:  The name of the user to connect to the database
              with, don't define if you want to parse 
              ~/.mythtv/mysql.txt instead
    (default: '')

  --[no]commflag:  Run the mythcommflag command on new videos
    (default: 'true')

(These are the result of adding the gflags module back into the implementation).

I am hoping to release this version in the next few days, so if you find any bugs please send email to the mailing list.

Tags for this post: mythtv() mythnettv() Comment

MythNetTV mailing list

Wed, 13 Aug 2008 16:56:00 GMT

I get a few support requests for MythNetTV, and I must admit that I sometimes get quite behind on my personal email, especially when I'm travelling. I've therefore created a mailing list for announcements and discussion in the hope that people will chat and come up with great ideas, as well as being able to support each other.

Tags for this post: mythtv() mythnettv() Comment

MythNetTV release 4

Wed, 13 Aug 2008 16:47:00 GMT

New things in this release:

Removed gflags because people found it hard to find
Bug fix patch from David Linville applied
- Fixed broken imports caused by refactoring
- Transcode not needed for avc1 and m4v
Another bug fix patch from David Linville applied
- Store filesize in the db
Removed some namespace imports I am not a fan of
Made verbosity optional for --update
Small code cleanups
Let the user know of repeated attempts to download a program
Documented the need for bittornado for bittorrent to work
Abandon downloading after 3 failed attempts (you can configure the number)
Detect stuck bittorrent downloads

Release 3 had a few bugs folks reported, this release should fix those problems. Let me know if you still see any. You can grab your copy here.

Tags for this post: mythtv() mythnettv() Comment

MythNetTV release 3

Sun, 13 Jul 2008 21:36:00 GMT

New things in this release:

Started work on an RSS exporter for MythTV recordings
DX50 doesn't need transcode
Tweaked supported video mime types so "Tikibar TV" and "Ask a ninja" work
First cut of Bittorrent support
Schema upgrades
Archive recordings as well as importing them
Improved --list output
Subtitle restrictions on download as well
Make subscriptions inactive instead of deleting them (for unsubscribe)
Better filename safening
More markread options
Refactored code to be more sane
Don't archive things imported from the archive location
Bulk import (--importmanylocal)

This version was a long time coming. Sorry about that. You can grab your copy here.

Tags for this post: mythtv() mythnettv() Comment

MythNetTV release 2

Mon, 03 Sep 2007 20:24:00 GMT

This release has bug fixes (including code from "nickca" for FLV support), and some new minor features. Enjoy!

Started working harder to ensure video filenames are unique once imported into MythTV, I now prepend video files with epoch seconds at time of import
DIVX (not divx) doesn't need transcode, added. Check for the existence of the video directory, and return an error if it needs creating.
Handle storage groups, check for the existence of the data directory
Updated docs
Complained about the poor state of SWF support
WMV support
FLV support

Get it here if you care: http://www.stillhq.com/mythtv/mythnettv/

Tags for this post: mythtv() mythnettv() Comment

MythNetTV release 1

Fri, 22 Jun 2007 11:49:00 GMT

I figured it was time for another release of MythNetTV, especially since I've fixed some bugs recently and rolled in code generously provided by George Nassas. New things in this release:

Initial work on transcoding smaller files to something else
Better error checking for MySQL configuration and accessibility
Fixed bug where MythNetTV was unhappy if there were no channels configured in the MySQL database
Found, and hopefully fixed, a bug where program dates were not always being tracked correctly. Also added a check to make sure this doesn't happen again
Added --markread, which lets you mark old shows as already downloaded

The code has seen a fair bit of use now, so it should be fairly stable. Let me know if you find any problems...

$ ./mythnettv --statistics
Programs tracked: 735
  Google EngEdu: 350
  Scoble Show: 85
  linux.conf.au 2007: 74
  Mobuzz TV: 82
  dl.tv: 32
  Net Cafe: 2
  Foo: 1
  ABC News: 2
  Australia: 1
  Play School: 1
  Diggnation: 72

Programs still to download: 50
  Google EngEdu: 49
  Diggnation: 1

Data transferred: 103 gb (111017264341 bytes)

Get it here if you care: http://www.stillhq.com/mythtv/mythnettv/

Tags for this post: mythtv() mythnettv() Comment

MythNetTV beta 4

Sat, 17 Mar 2007 18:42:00 GMT

As it says in the email:

To: Discussion about mythtv <mythtv-users@mythtv.org>
Subject: MythNetTV beta 4

Hi again.

MythNetTv is my video downloader / aggregator / importer thingie for MythTV.

I've spent a little longer testing this beta, and I think it's getting
kinda ready for a real release number or something. New things in this
version (from the change log):

  - Made --update more terse
  - Made --update more liberal about what it considers to be a video,
    specifically added video/quick time and text/html (to work around the
    rather broken http://www.mobuzztv.com/uk/rss/quicktime
  - Added a filter option to --download to constrain it to only
    downloading shows with a specific title
  - Added transcoding for mov and wmv2 files
  - If there is a straggling import, and it causes as error, just skip
    it and mark it as imported
  - Don't reset transferred data statistic when we start downloading a
    show again
  - Started implementing moniker support for downloads
  - Unsupported monikers will result in an attempt to download the URL
    using mplayer (which works for RTSP and MMS at the least)

Yes, that's right. You can now download only shows with a specific
title, and you can also download any protocol which mplayer supports --
specifically, RTSP and MMS are now supported.

Now, if only there was a non-sucky way to work out what streams
abc.net.au were offering at any given time...

Get it here if you care: http://www.stillhq.com/mythtv/mythnettv/

Mikal

Tags for this post: mythtv() mythnettv() Comment

Now, about the video backlog

Sun, 25 Feb 2007 20:12:00 GMT

$ ./mythnettv --statistics
Programs tracked: 385
  Google EngEdu: 279
  Scoble Show: 26
  linux.conf.au 2007: 74

Programs still to download: 283
  Google EngEdu: 202
  Scoble Show: 8
  linux.conf.au 2007: 73

Data transferred: 22 gb

This is why I want to queue video, and then have it download to Myth a little at a time.

Tags for this post: mythtv() mythnettv() Comment

MythNetTv Beta 3

Sun, 25 Feb 2007 20:03:00 GMT

Beta 3 is out! It includes:

Renamed to mythnettv
todoremote bug fixed (bad column name)
statistics bug fixed (no transfer stats caused crash)
It is now possible to do a --todoremote, --importremote, or --importlocal and provide all the needed information on the command line

Get it from here.

Tags for this post: mythtv() mythnettv() Comment

MythIPTV Beta 2

Mon, 19 Feb 2007 10:11:00 GMT

Just announced. It was either do the washing, or this:

Date: Mon, 19 Feb 2007 10:11:09 -0800
From: Michael Still 
To: Discussion about mythtv 
Subject: MythIPTV Beta 2

Hi again.

There's a new version of MythIPTV now at http://www.stillhq.com/mythtv/mythiptv/source/beta-2/

This version adds the following:

 - subscription management: you can add RSS URLs to watch,
   list them, update them (download new versions of the XML and
   import the shows), and remove them

 - ad hoc video management: add a remote URL to the TODO list,
   download a remote URL and import it now, import a local file

 - transcoding: it's quite probable that you'll encounter a
   video format which isn't know. Please report it if you do

 - statistics: how many programs do I have? how many are waiting
   to download? how much data have I downloaded so far?

 - reporting: what shows would be downloaded next? what does the
   internal log say about the state of MythIPTV?

Once again, this is a beta, so good backups are needed. I would love more feedback.

Oh, and you _really_ need mplayer and mencoder installed now. Really.

Cheers,
Mikal

Tags for this post: mythtv() mythnettv() Comment

MythIpTV Beta 1

Sat, 17 Feb 2007 09:41:00 GMT

Some folk at LCA expressed interest in my video blog aggregator for MythTV. I've cleaned it up, and just sent out the first release:

Date: Sat, 17 Feb 2007 09:38:04 -0800
From: Michael Still 
To: Discussion about mythtv 
Subject: MythIpTv Beta 1

Hi.

I've been recently wanting a video blog aggregator which integrates with
MythTV. Specifically, I wanted the videos to appear as recordings. I've
had a horrible script for that for a few months, but ended up cleaning
it up when it occurred to me that Akimbo (http://www.akimbo.com) is just
a per user RSS feed, and something a lot like MythTV.

So, I wrote MythIpTv. It takes RSS feeds, imports the entries in the
feed and builds a TODO list. It then downloads shows and imports them
into the MythTV recordings menu.

The code has only been tested by me, so if you're interested in giving
it a try, please take good backups. You can find the code here:
http://www.stillhq.com/mythtv/mythiptv/source/beta-1/mythiptv

Before use, you need to make a temporary data directory in your current
working directory:

mkdir data

You use mythiptv like this:

./mythiptv --url <url to RSS feed> <show title>

or

./mythiptv --file <RSS file on local disk> <show title>

I've been using Google Video to test this, as the AVI format they use is
natively supported by the MythTV player. This isn't true for a lot of
RSS feeds and transcoding isn't implemented yet, so if you want
something to test with, you might want to use the same command line as me:

./mythiptv --url
"http://video.google.com/videofeed?type=search&q=engedu&so=1&num=20&output=rss"
"Goole EngEdu"

That command just builds a TODO list. You then need to download some of
the shows. You do that like this:

./mythiptv --download <number of shows to download>

Once downloaded, the shows will appear in the recordings menu.

I would be really interested in people's comments on the code, and
suggestions for improvement. There are a couple of things I know about
already:

 - it really needs transcoding
 - the is no graphical UI yet

Cheers,
Mikal

Tags for this post: mythtv() mythnettv() Comment