download output plugin can't handle gzipped nzbs from nzbmatrix

[ejstacey@…]

I'm running r722.

When using the download output plugin with the nzbmatrix rss feeds (in download mode, VIP account on site required), the accepted file is downloaded as [file name here].nzb. The file is a gzipped version though, so maybe it should be properly labeled as [file name here].nzb.gz? That's what hellanzb expects, at least.

I "fixed" this by going to ./flexget/plugins/output_download.py and changing:

# combine to full path + filename, replace / from filename (#208)
            destfile = os.path.join(path, entry.get('filename', entry['title']).replace('/', '_'))

to

# combine to full path + filename, replace / from filename (#208)
            destfile = os.path.join(path, entry.get('filename', entry['title']).replace('/', '_') + '.gz')

This is obviously a stupid hack, but I don't know python well enough to do something better ;) It works for me because I only use one rss feed on flexget.

Suggested fixes: - Use the *nix file command equivalent for python to detect gzipped/zipped/etc files and give an appropriate extension. - Add another option to the download output plugin to add an extension to all files downloaded from that rss feed (or whatever the input plugin is).

Not great ideas, but the only things I could think of offhand.

Thanks!

[paranoidi]

Thanks for the report .. we'll fix this soonish :)

Ps. instead of hack see extension plugin .. =)

[paranoidi]

Quick question, if you save the nzb from the feed with browser do you get .nzb.gz or uncompressed .nzb ?

[flexget]

Very sorry for the delay.. I thought I'd get emails when I got updates (not sure why, I just assumed ;) ), but I guess I don't.

I get nzb (uncompressed) when saved from the feed with a browser (FF 3.5.2).

I'll check out the extension plugin in the meantime. It seems like it's exactly what I want!

Thanks!

[flexget]

NZBMatrix sends the .nzb with a Content-Encoding HTTP header. Firefox knows how to handle this, output_download.py ignores it and so ends up with a .nzb file which is actually gzipped. The attached patch teaches the plugin about Content-Encoding. It also removes the 1k buffer; sorry, I couldn't get it to work with it. This has been working for me for a couple of weeks and dozens of NZB files. I suppose this also fixes #345.

[paranoidi]

Hmm, looks good otherwise but the content must be buffered or downloading large files will be impossible :(

[flexget]

I took another shot at it, using the buffer again. Works for me for all recognized encodings as well as uncompressed streams. Adding 32 to windowsizeBits makes zlib determine if it should use zlib (deflate) or gzip (gzip, x-gzip) decoding, so no need for gzip.GzipFile? any more. It'd be nice to also support Content-Encoding: compress ( http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.5).

[paranoidi]

Nice work! I give my thumbs up and green light for this :)

[gazpachoking]

(In [1043]) Implements gzip handling from #292 fixes #292 #345