Available formats: content-negotiated html turtle (see SIOC for the vocabulary)
Back to channel and daily index: content-negotiated html turtle
These logs are provided as an experiment in indexing discussions using IRCHub.py, Irc2RDF.hs, and SIOC.
| 05:28:53 | <DrFrasierCrane> | tuukkah, hi |
| 05:28:59 | <DrFrasierCrane> | I was thinking... |
| 05:29:35 | <DrFrasierCrane> | maybe it's normal when programming UI with threads, I mean one thread takes a lot of CPU so the process doesn't get enough CPU time overall to feed all threads |
| 05:29:48 | <DrFrasierCrane> | of course no thread will starve, but freezes can occur |
| 05:30:05 | <DrFrasierCrane> | but still... even when it uses only 20%, freezes are there ehh... |
| 06:00:02 | <DrFrasierCrane> | nope, I'm wrong |
| 06:00:23 | <DrFrasierCrane> | when I put while 1: pass in the thread code and straw uses 100%, UI does NOT freeze |
| 07:57:25 | <DrFrasierCrane> | sgmllib has the same effect btw |
| 07:57:39 | <DrFrasierCrane> | after work today I'll chase this bastard bug further |
| 07:58:01 | <DrFrasierCrane> | I'll dump all feeds to disk and write simple gtk program and launch parsing in background thread |
| 07:58:06 | <DrFrasierCrane> | we'll see then |
| 07:58:47 | <tuukkah> | you could also run the simpler test of using file: uris first |
| 07:59:38 | <DrFrasierCrane> | why? |
| 07:59:48 | <DrFrasierCrane> | it's not a network bug, only parser |
| 08:01:12 | <tuukkah> | the program logic with file: uris should be close to http: uris so the bug shouldn't manifest. but the timing should be close to passing a string, so a bug related to threading might manifest |
| 08:02:32 | <DrFrasierCrane> | this bug happens when I pass string or url or whatever |
| 08:02:45 | <tuukkah> | oh? |
| 08:02:46 | <DrFrasierCrane> | but I guess with threads you never know |
| 08:03:23 | <DrFrasierCrane> | simplest form is to invoke parse method from feedparser either with feed content or url as argument |
| 08:03:48 | <DrFrasierCrane> | when call to saxparser.parse(source) is commented out it's all ok |
| 08:03:48 | <tuukkah> | i thought the situation was still what you said earlier: passing uris to feed parser doesn't block, passing content strings to feed parser blocks? |
| 08:03:58 | <DrFrasierCrane> | same with commenting out call to sgmllib.feed(data) or something like that |
| 08:04:09 | <DrFrasierCrane> | it turns out it's both... |
| 08:04:21 | <DrFrasierCrane> | at least now it's both |
| 08:04:26 | <tuukkah> | ok sorry, i wasn't up-to-date |
| 08:04:38 | <DrFrasierCrane> | I wasn't either until yesterday |
| 08:04:49 | <DrFrasierCrane> | I thought one of those invocations was ok |
| 08:04:57 | <DrFrasierCrane> | but apparently not |
| 08:05:40 | <DrFrasierCrane> | well we could use IPC after all :-) |
| 08:05:53 | <DrFrasierCrane> | do parsing or maybe even downloading in the separate process |
| 08:06:01 | <DrFrasierCrane> | but I think we should try to avoid that |
| 08:06:25 | <tuukkah> | me too |
| 08:08:47 | <DrFrasierCrane> | well it wouldn't be end of the world I guess |
| 08:09:04 | <DrFrasierCrane> | surely separate process is better than unusable UI |
| 08:09:14 | <tuukkah> | it wouldn't. it's definitely an option to keep in mind |
| 08:09:42 | <tuukkah> | did you ask any python people about this? |
| 08:09:56 | <DrFrasierCrane> | yeah, but I think for now my description was too specific for them |
| 08:10:04 | <DrFrasierCrane> | I'll try to find something more generic today |
| 08:10:12 | <tuukkah> | right |
| 08:10:43 | <DrFrasierCrane> | this bug pissed me off that's for sure |
| 08:10:59 | <DrFrasierCrane> | and that usually doesn't end up nicely for the bug |
| 08:11:01 | <DrFrasierCrane> | :) |
| 08:11:05 | <tuukkah> | =) |
| 08:15:31 | <tuukkah> | so saxparser.parse(source) and sgmllib.SGMLParser.feed(self, data) both cause blocking, or they together cause blocking? |
| 08:16:04 | <DrFrasierCrane> | feedparser uses only one of them, depending on the state of XML_AVAILABLE variable |
| 08:16:12 | <DrFrasierCrane> | but both cause blocking |
| 08:16:28 | <tuukkah> | oh, ok |
| 08:20:39 | <DrFrasierCrane> | would be nice to find some other python app that uses feedparser |
| 08:20:53 | <DrFrasierCrane> | but now I only know penguintv |
| 08:21:45 | <tuukkah> | http://google.com/codesearch?q=lang%3Apython+%22import+feedparser%22 |
| 08:22:20 | <DrFrasierCrane> | oh cool I didn't know that google had code search |
| 08:22:47 | <tuukkah> | yeah it's precisely the tool for the job :-) |
| 08:23:06 | <DrFrasierCrane> | I've been using koders.com for some time |
| 08:25:11 | <tuukkah> | also the tool for the job |
| 08:25:33 | <DrFrasierCrane> | http://www.csh.rit.edu/~synack/repos/index.cgi/pynewspaper/ |
| 08:25:39 | <DrFrasierCrane> | I'll try that later |
| 08:25:50 | <DrFrasierCrane> | looks like the perfecgt test case for feedparser :) |
| 08:26:05 | <DrFrasierCrane> | class feedGrabber(Thread): |
| 08:26:05 | <DrFrasierCrane> | def __init__(self, url): |
| 08:26:05 | <DrFrasierCrane> | Thread.__init__(self) |
| 08:26:05 | <DrFrasierCrane> | self.url = url |
| 08:26:05 | <DrFrasierCrane> | self.feed = 0 |
| 08:26:06 | <DrFrasierCrane> | def run(self): |
| 08:26:08 | <DrFrasierCrane> | self.feed = feedparser.parse(self.url) |
| 08:26:34 | <tuukkah> | there's also a cookbook recipe which uses parsed = feedparser.parse(_StringIO.StringIO(feed)) |
| 08:27:07 | <DrFrasierCrane> | yeah I'll take a look at all this after work |
| 08:27:31 | <DrFrasierCrane> | after this problem is solved I think it's all piece of cake from there :) |
| 08:29:43 | <DrFrasierCrane> | *** today's feed munching is sponsored by Blam newsreader *** |
| 08:29:56 | <tuukkah> | :-) |
| 08:50:13 | <tuukkah> | newspaper.py has a very simple nntp backend too |
| 08:50:36 | <tuukkah> | like, 30 lines |
| 08:51:34 | <DrFrasierCrane> | sounds cool |
| 08:59:37 | <tuukkah> | there's a couple of news groups i might follow if i didn't have to use a separate app |
| 08:59:51 | <DrFrasierCrane> | yeah me too |
| 08:59:56 | <DrFrasierCrane> | and mailing lists, too |
| 09:20:14 | <tuukkah> | hello janm |
| 09:21:00 | <DrFrasierCrane> | hey |
| 09:22:18 | <janm> | hello |
| 09:23:05 | <DrFrasierCrane> | janm, I pushed TODO.work as you requested to the fork repo |
| 09:26:17 | <janm> | DrFrasierCrane: great. thanks! |
| 09:28:42 | <DrFrasierCrane> | no problem, I hope you'll find it useful |
| 09:32:02 | <janm> | ACTION curses at opera for being so crashy |
| 09:32:27 | <DrFrasierCrane> | hehe |
| 09:32:32 | <DrFrasierCrane> | some dev build perhaps? |
| 09:33:09 | <tuukkah> | what's opera ;-) |
| 09:33:14 | <janm> | nope. eversince debian xorg got updated |
| 09:33:50 | <janm> | heh rather use that than FF |
| 09:34:31 | <DrFrasierCrane> | how about Midori? :) |
| 09:34:35 | <DrFrasierCrane> | browser based on webkit |
| 09:34:45 | <tuukkah> | serious alternatives to FF, i only know webbrowser.py =) |
| 09:35:28 | <tuukkah> | DrFrasierCrane, i use midori on openmoko |
| 09:36:25 | <DrFrasierCrane> | cool, I tried it some time ago and was actually surprised to see how mature it is |
| 09:36:27 | <janm> | tuukkah: lol right =) |
| 09:36:40 | <janm> | webkit gtk+ is looking good these days |
| 09:36:55 | <DrFrasierCrane> | yeah especially those python bindings |
| 09:36:58 | <DrFrasierCrane> | great stuff :) |
| 11:21:03 | <janm> | tuukkah: do you mind putting up a logbot for webkit? =) |
| 11:29:04 | <tuukkah> | not at all, but i'll have to ask the channel whether they're fine with that |
| 11:38:31 | <tuukkah> | met some opposion |
| 11:40:08 | <tuukkah> | opposition |
| 12:45:48 | <janm> | tuukkah: thanks for asking. appreciate it |
| 12:55:38 | <tuukkah> | could running irchub on your server be a solution? |
| 18:37:35 | <DrFrasierCrane> | tuukkah, sorry, but I won't have time for squashing that bug today after all :-( and it seems my weekend is doomed by work too :( |
| 18:38:11 | <tuukkah> | no hurry, but a doomed weekend sounds bad :-( |
| 18:38:54 | <DrFrasierCrane> | well it's not that bad, because I have been assigned some interesting stuff, but still I feel bad that there's another delay with Straw and Newton :-( |
| 18:39:28 | <DrFrasierCrane> | I guess I should stop treating these projects as products that need to be shipped on some deadline... |
Back to channel and daily index: content-negotiated html turtle