MythLogBot@irc.freenode.net :: #mythtv

Daily chat history

Current users (86):

aloril, amejia, amessina, Anssi, anykey_, autopatch, brfransen, CaCtus491, Captain_Murdoch, cattelan_away, Chutt, clever, coling, Cougar, damaltor, danielk22, David_Mi1ler, dblain, dinamic|screen, dlblog, eharris_, ElmerFudd, foobum, foxbuntu, ghoti, GreyFoxx, highzeth, idl0r, J-e-f-f-A, jams, jarle, jcarlos, joe____, joki, jpabq, jstenback, jwh, jwhite, kc, knightr__, kormoc, kurre2, kwmonroe, laga, mag0o, markcerv, MaverickTech, mrand, mrec, MythBuild, MythLogBot, mzanetti, NightMonkey, peitolm, Peps, petefunk, pheld, poptix, purserj, rsiebert, seld, Sharky112065, skd5aner, Slasher`, SmallR2002, sphery, sraue, stichnot_, sutula, tgm4883, ThisNewGuy, toeb, tomimo, tris, Unhelpful, vallor, Vernon_at_work, VManiac16, wagnerrp, wahrhaft, whoDat, xavierh, XDS2010, yb0t, _charly_, _Techie_-_AFK_
Wednesday, June 20th, 2012, 00:48 UTC
[00:48:13] stichnot (stichnot!~chatzilla@192.55.55.37) has joined #mythtv
[00:48:13] stichnot (stichnot!~chatzilla@mythtv/developer/stichnot) has joined #mythtv
[00:48:13] stichnot (stichnot!~chatzilla@192.55.55.37) has quit (Changing host)
[01:30:25] NightMonkey (NightMonkey!~NightrMon@pdpc/supporter/professional/nightmonkey) has quit (Quit: Body blow! Body blow!)
[01:32:47] stichnot: hmm... updated to latest Master, and now the PlaybackBox basically hangs at the "Loading..." dialog while the progress animation goes, consuming 100% CPU
[01:33:01] stichnot: happens on all 3 ION frontends
[01:33:45] stichnot: I take it no one else sees this?
[01:37:26] stichnot: the MythVideo screen works fine. PBB loads fine when I run on the backend and display remotely on my laptop
[01:45:47] gregL_ (gregL_!~greg@cpe-74-76-125-87.nycap.res.rr.com) has quit (Read error: Connection reset by peer)
[01:50:49] zombor (zombor!~zombor_@kohana/developer/zombor) has quit (Remote host closed the connection)
[02:01:14] gregL_ (gregL_!~greg@cpe-74-76-125-87.nycap.res.rr.com) has joined #mythtv
[02:14:11] zombor (zombor!~zombor_@kohana/developer/zombor) has joined #mythtv
[02:42:29] zombor (zombor!~zombor_@kohana/developer/zombor) has quit (Remote host closed the connection)
[02:55:15] zombor (zombor!~zombor_@kohana/developer/zombor) has joined #mythtv
[03:05:26] danielk22: stichnot: I haven't seen that.
[03:08:45] stichnot: danielk22: I rolled back to an earlier version and it's working again. I'll run a git bisect tomorrow.
[03:10:01] stichnot: stuartm: remind me — do you expect your recent scheduler lock commit to have any effect on the deadlock I reported? (i.e., should I try testing?)
[03:29:14] zombor (zombor!~zombor_@kohana/developer/zombor) has quit (Remote host closed the connection)
[04:00:23] zombor (zombor!~zombor_@65.29.231.135) has joined #mythtv
[04:00:23] zombor (zombor!~zombor_@65.29.231.135) has quit (Changing host)
[04:00:23] zombor (zombor!~zombor_@kohana/developer/zombor) has joined #mythtv
[04:06:04] zombor (zombor!~zombor_@kohana/developer/zombor) has quit (Remote host closed the connection)
[04:58:12] stichnot: danielk22: git bisect said the culprit is your commit [6b04160ac3ad1666cd034aa809ed5bb939a2028c]. Reverting that from HEAD made the problem go away. Then I un-reverted and basically did "rm $LIBDIR/*-0.25.*" and it works again. blah.
[05:06:18] cattelan is now known as cattelan_away
[05:11:18] seld (seld!~seld@h170n6-rny-a12.ias.bredband.telia.com) has quit (Ping timeout: 245 seconds)
[05:14:01] seld (seld!~seld@h170n6-rny-a12.ias.bredband.telia.com) has joined #mythtv
[05:29:18] Czar_Away (Czar_Away!~Unknown@69.4.155.83) has joined #mythtv
[05:32:15] VManiac16 (VManiac16!~Unknown@69.4.155.83) has quit (Ping timeout: 246 seconds)
[07:20:18] natanojl (natanojl!~jonatan@c83-252-237-63.bredband.comhem.se) has joined #mythtv
[07:24:47] rsiebert (rsiebert!~quassel@g229052016.adsl.alicedsl.de) has joined #mythtv
[07:25:21] rsiebert_ (rsiebert_!~quassel@g226062164.adsl.alicedsl.de) has quit (Ping timeout: 246 seconds)
[07:32:53] natanojl (natanojl!~jonatan@c83-252-237-63.bredband.comhem.se) has quit (Read error: Operation timed out)
[07:57:22] Chutt (Chutt!~ijr@cpe-24-29-225-175.neo.res.rr.com) has quit (Read error: Connection reset by peer)
[07:58:32] Chutt (Chutt!~ijr@cpe-24-29-225-175.neo.res.rr.com) has joined #mythtv
[08:01:57] stuartm: stichnot: no I don't
[08:17:22] Chutt (Chutt!~ijr@cpe-24-29-225-175.neo.res.rr.com) has quit (Read error: Connection reset by peer)
[08:17:45] Chutt (Chutt!~ijr@cpe-24-29-225-175.neo.res.rr.com) has joined #mythtv
[08:18:02] pheld (pheld!~heldal@cl-5.osl-01.no.sixxs.net) has quit (Quit: Leaving.)
[08:23:48] MythBuild: build #1227 of master-vista-mingw-32bit is complete: Exception [6exception git] Build details are at http://code.mythtv.org/buildbot/builders/mast . . . /builds/1227 blamelist: Raymond Wagner <rwagner@mythtv.org >
[08:25:49] wagnerrp: bah...
[08:26:09] ** wagnerrp ignores the error and goes to bed **
[08:34:43] stuartm: MythBuild: force build master-vista-mingw-32bit now
[08:34:44] MythBuild: build forced [ETA 52m59s]
[08:34:44] MythBuild: I'll give a shout when the build finishes
[08:52:46] stuartm: stichnot: fwiw if you need to reproduce your deadlock it appears to occur if you bring up the guide or maybe are in the guide at a livetv transition, the guide is requesting data from the scheduler just as the recorder thread has locked the scheduler via GetLiveTVDirectory() (may not be the exact name, I'm going from memory)
[08:54:02] stuartm: anything combination of events that locks the scheduler in both the recorder thread and the main thread at the same time will result in a deadlock since the recorder thread depends on events being processed in the main thread
[08:56:24] stuartm: it actually raises the question of whether some of these protocol/internal events should be handled asynchronously instead
[09:07:27] amessina (amessina!~amessina@2001:470:1f11:a4:d6be:d9ff:fe8d:7c1e) has joined #mythtv
[09:17:40] Goga777 (Goga777!~Goga777@128-71-81-77.broadband.corbina.ru) has joined #mythtv
[09:22:50] stuartm: hows this for interesting – the CPU usage of the backend gets worse over time, it's now hitting 140%
[09:29:38] MythBuild: Hey! build master-vista-mingw-32bit #1228 is complete: Success [3build successful]
[09:29:38] ** MythLogBot http://code.mythtv.org/trac/ticket/1228 **
[09:29:38] MythBuild: Build details are at http://code.mythtv.org/buildbot/builders/mast . . . /builds/1228
[09:32:48] xavierh (xavierh!~xavier@cpc1-swin3-0-0-cust274.3-1.cable.virginmedia.com) has quit (Ping timeout: 245 seconds)
[11:38:20] zombor (zombor!~zombor_@kohana/developer/zombor) has joined #mythtv
[11:42:49] zombor (zombor!~zombor_@kohana/developer/zombor) has quit (Remote host closed the connection)
[12:29:56] xavierh (xavierh!~xavier@cpc1-swin3-0-0-cust274.3-1.cable.virginmedia.com) has joined #mythtv
[12:57:31] dmfrey (dmfrey!~dmfrey@webdefence.cluster-h.websense.net) has joined #mythtv
[13:07:43] Jordack (Jordack!~jordack@h69-131-44-221.plmomi.dedicated.static.tds.net) has joined #mythtv
[13:20:05] Goga777 (Goga777!~Goga777@128-71-81-77.broadband.corbina.ru) has quit (Remote host closed the connection)
[13:28:03] danielk22: stichnot: "sudo rm -rf /usr/local/lib/mythtv" and a make distclean in both mythtv and mythplugins should fix it then. ABI change
[13:34:38] stichnot: danielk22: yeah, I generally do "make distclean" before every build, and especially when weird problems like this arise. I'm just surprised that the presence of stale *-0.25.* libraries would cause problems like this.
[13:36:03] danielk22: I think we have some breakage in the build scripts.
[13:36:30] danielk22: Maybe we even need to delete /usr/local/include/mythtv for a safe build atm.
[13:43:58] danielk22: stichnot: Actually, cleaning out /usr/local/include/mythtv is a good idea. Since we put includes in subdirectories of /usr/local/include/mythtv whenever an includes moves and then changes we end up with two inconsistent copies.
[13:44:58] danielk22: I didn't move anything, but if any header has moved since the last time you cleaned out /usr/local/include/mythtv you'll be generating internally inconsistent code.
[13:47:13] stichnot: yeah, we've seen a number of false bug reports based on header files moving
[13:47:48] danielk22: Yeah, we should get rid of those subdirectories they cause nothing but grief.
[13:49:29] danielk22: The incident that made me thing our build scripts were broken was in retrospect probably due to this.. the actual problem can occur months after the header moved just because some header it includes changes.
[13:57:23] VManiac16 (VManiac16!~Unknown@69.4.155.83) has joined #mythtv
[14:01:24] Czar_Away (Czar_Away!~Unknown@69.4.155.83) has quit (Ping timeout: 255 seconds)
[14:14:05] zombor (zombor!~zombor_@50-73-122-41-ip-static.hfc.comcastbusiness.net) has joined #mythtv
[14:14:06] zombor (zombor!~zombor_@50-73-122-41-ip-static.hfc.comcastbusiness.net) has quit (Changing host)
[14:14:06] zombor (zombor!~zombor_@kohana/developer/zombor) has joined #mythtv
[14:25:54] danielk22: stuartm: I've noticed that with mythlogserver running under valgrind on the backend machine even a remote frontend runs slowly. The only thing I can think of that would cause that is if the logging is somehow blocked by the log server. If that is the case it would explain why recordings aren't making it to disk.
[14:26:51] danielk22: s/if the logging is somehow blocked by the log server/if the mythbackend logging is somehow blocked by the local log server/
[14:27:59] zombor is now known as MickZombor
[14:30:39] dekarl-too (dekarl-too!51c8c614@gateway/web/freenode/ip.81.200.198.20) has joined #mythtv
[14:33:50] MickZombor is now known as zombor
[14:34:15] dekarl-too: danielk22: just a heads up, you can close #10781 as duplicate of #10495. And I'm running pre-utc master with the three patches from #10495 and #10496 as production backend. So they should be ok. (need to look at lossless transcode, it started to leak like a sieve, but I think its unrelated, didn't have time to retest without the patches yet)
[14:34:15] ** MythLogBot http://code.mythtv.org/trac/ticket/10781 **
[14:34:15] ** MythLogBot http://code.mythtv.org/trac/ticket/10495 **
[14:34:15] ** MythLogBot http://code.mythtv.org/trac/ticket/10495 **
[14:34:15] ** MythLogBot http://code.mythtv.org/trac/ticket/10496 **
[14:40:49] stichnot: stuartm: I'm also observing ever-increasing mythbackend CPU usage during remote frontend playback (though I ought to do the full cleanup described by danielk22 just to be sure)
[14:57:36] taylorr (taylorr!~taylorr@unaffiliated/elmojo) has quit (Remote host closed the connection)
[14:59:04] stichnot (stichnot!~chatzilla@mythtv/developer/stichnot) has quit (Ping timeout: 244 seconds)
[14:59:56] danielk22: The full cleanup seems to have worked on a frontend (memory usage down to more sane levels). But not on a the master backend/frontend machine.
[15:00:50] danielk22: I don't observe ever increasing mythbackend CPU usage.. how long should I observe it for? It seems to stay around 0 if I
[15:01:02] taylorr (taylorr!~taylorr@cpe-173-095-145-027.nc.res.rr.com) has joined #mythtv
[15:01:02] taylorr (taylorr!~taylorr@unaffiliated/elmojo) has joined #mythtv
[15:01:02] taylorr (taylorr!~taylorr@cpe-173-095-145-027.nc.res.rr.com) has quit (Changing host)
[15:01:09] danielk22: m watching an existing recording and 3% if I'm watching Live TV
[15:08:02] zombor (zombor!~zombor_@kohana/developer/zombor) has quit (Read error: Operation timed out)
[15:11:31] NightMonkey (NightMonkey!~NightrMon@pdpc/supporter/professional/nightmonkey) has joined #mythtv
[15:13:01] zombor (zombor!~zombor_@kohana/developer/zombor) has joined #mythtv
[15:13:40] stuartm: danielk22: this was over a period of several hours and I had been watching some recordings during that time, last night it was around 70% by this morning it was hitting 140%
[15:14:06] stuartm: both at the same clock speed before anyone wonders
[15:15:10] stuartm: it was variable, not constant, 140% was the max with a minimum around 115%, scheduling was not occurring at that time
[15:16:48] stuartm: both cpu and memory have settled down since I restarted earlier, but I've not watched anything since then and I was starting to see a connection with watching a recording, the time spent watching and the rise in memory and I guess CPU usage
[15:17:07] dmfrey (dmfrey!~dmfrey@webdefence.cluster-h.websense.net) has quit (Remote host closed the connection)
[15:17:33] stuartm: so maybe something in the file streaming or ringbuffer code
[15:17:47] dmfrey (dmfrey!~dmfrey@webdefence.cluster-h.websense.net) has joined #mythtv
[15:24:26] NightMonkey (NightMonkey!~NightrMon@pdpc/supporter/professional/nightmonkey) has quit (Remote host closed the connection)
[15:26:41] andreax (andreax!~Andreaz@tmo-106-21.customers.d1-online.com) has joined #mythtv
[15:41:37] dmfrey (dmfrey!~dmfrey@webdefence.cluster-h.websense.net) has quit (Remote host closed the connection)
[15:42:22] dmfrey (dmfrey!~dmfrey@webdefence.cluster-h.websense.net) has joined #mythtv
[15:44:23] andreax (andreax!~Andreaz@tmo-106-21.customers.d1-online.com) has quit (Read error: Connection reset by peer)
[15:50:33] danielk22: stuartm: Were you watching close to the end of a file being recorded?
[15:52:30] stuartm: no, pre-recorded stuff mostly
[15:54:09] kormoc (kormoc!~kormoc@mythtv/developer/kormoc) has quit (Quit: kormoc)
[15:57:27] zombor (zombor!~zombor_@kohana/developer/zombor) has quit (Ping timeout: 244 seconds)
[15:59:30] zombor (zombor!~zombor_@50-73-122-41-ip-static.hfc.comcastbusiness.net) has joined #mythtv
[15:59:31] zombor (zombor!~zombor_@50-73-122-41-ip-static.hfc.comcastbusiness.net) has quit (Changing host)
[15:59:31] zombor (zombor!~zombor_@kohana/developer/zombor) has joined #mythtv
[16:03:28] stichnot (stichnot!~chatzilla@192.55.55.37) has joined #mythtv
[16:03:28] stichnot (stichnot!~chatzilla@mythtv/developer/stichnot) has joined #mythtv
[16:03:28] stichnot (stichnot!~chatzilla@192.55.55.37) has quit (Changing host)
[16:04:29] NightMonkey (NightMonkey!~NightrMon@pdpc/supporter/professional/nightmonkey) has joined #mythtv
[16:07:17] zombor (zombor!~zombor_@kohana/developer/zombor) has quit (Read error: Connection reset by peer)
[16:07:24] zombor_ (zombor_!~zombor_@kohana/developer/zombor) has joined #mythtv
[16:13:29] zombor (zombor!~zombor_@50-73-122-41-ip-static.hfc.comcastbusiness.net) has joined #mythtv
[16:13:30] zombor (zombor!~zombor_@50-73-122-41-ip-static.hfc.comcastbusiness.net) has quit (Changing host)
[16:13:30] zombor (zombor!~zombor_@kohana/developer/zombor) has joined #mythtv
[16:16:07] zombor_ (zombor_!~zombor_@kohana/developer/zombor) has quit (Ping timeout: 276 seconds)
[16:18:21] stuartm: running the backend under strace to see if it sheds any light
[16:27:09] Mousey (Mousey!~r0dent_@ross154.net) has joined #mythtv
[16:27:14] joki (joki!~joki@p5486579E.dip.t-dialin.net) has quit (Ping timeout: 250 seconds)
[16:27:31] SteveGoodey (SteveGoodey!~steve@host86-160-43-123.range86-160.btcentralplus.com) has joined #mythtv
[16:27:33] joki (joki!~joki@p54862192.dip.t-dialin.net) has joined #mythtv
[16:32:07] Czar_Away (Czar_Away!~Unknown@69.4.155.83) has joined #mythtv
[16:35:12] VManiac16 (VManiac16!~Unknown@69.4.155.83) has quit (Ping timeout: 246 seconds)
[16:38:27] stuartm: seems to me that the backend is using excessive cpu when simply streaming, with a freshly restarted backend it's 3–4% idle but that jumps to 30% to play back an old recording
[16:38:45] danielk22: doh! FYI In case you are considering installing ubuntu 12.04, oprofile has been removed..
[16:38:51] dmfrey (dmfrey!~dmfrey@webdefence.cluster-h.websense.net) has quit (Remote host closed the connection)
[16:39:36] dmfrey (dmfrey!~dmfrey@webdefence.cluster-h.websense.net) has joined #mythtv
[16:40:53] stichnot: stuartm: danielk22: after cleaning out my lib and include dirs and reinstalling, backend CPU usage seems back to normal during remote frontend playback
[16:42:27] stichnot: hmm, or maybe it is very very slowly increasing, not sure yet
[16:43:19] stuartm: stichnot: the increase is pretty slow from what I've seen, I'll see how it goes though
[16:44:27] stuartm: I did a clear out but I'm still seeing what I'd consider to be excessive cpu while streaming, although it does drop again when I stop, the gradual rise in idle usage will take a bit longer to observe
[16:45:09] dekarl-too (dekarl-too!51c8c614@gateway/web/freenode/ip.81.200.198.20) has quit (Quit: Page closed)
[16:45:20] danielk22: stuartm: stichnot: We'll really need something like an oprofile run or VTune to identify where the time is being spent.
[16:46:38] stichnot (stichnot!~chatzilla@mythtv/developer/stichnot) has quit (Ping timeout: 240 seconds)
[16:49:20] stuartm: seeing a lot of "stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=3661, ...}) = 0 <0.000010>"
[16:50:18] danielk22: stuartm: That's always done when we create do QDateTime::currentDateTime(); it's annoying but its not new.
[16:50:33] stuartm: ok
[16:53:46] stuartm: that's really the only thing which stands out in strace, spends most of it's time stating /etc/localtime and a few read/writes but nothing unexpected
[17:02:47] stichnot (stichnot!chatzilla@nat/intel/x-onzubmsjlgauyozm) has joined #mythtv
[17:02:47] stichnot (stichnot!chatzilla@mythtv/developer/stichnot) has joined #mythtv
[17:02:47] stichnot (stichnot!chatzilla@nat/intel/x-onzubmsjlgauyozm) has quit (Changing host)
[17:06:25] stichnot: danielk22: I reinstalled the latest VTune, I'll see if it works (and doesn't crash the machine...) if the CPU usage starts going up again
[17:14:02] stichnot: stuartm: stat /etc/localtime reminds me of http://www.gossamer-threads.com/lists/mythtv/users/406613#406613
[17:18:25] sphery: stichnot: hehe, thanks for finding that one--I planned to look it up but hadn't gotten a chance, yet
[17:18:50] stichnot: sphery: easier when it's your own post :)
[17:19:21] sphery: true
[17:19:44] sphery: I remembered the who, but hadn't yet come up with the search terms
[17:26:56] wagnerrp: stichnot: #10848 is very interesting... been hoping for such a thing for years to help properly identify the cutoff point between shows and ads when there is a fade-out
[17:26:56] ** MythLogBot http://code.mythtv.org/trac/ticket/10848 **
[17:27:50] stichnot: Same here. Even better would be for mythcommflag to use this to refine the blank interval
[17:27:53] andreax (andreax!~andreaz@p5089E390.dip.t-dialin.net) has joined #mythtv
[17:27:58] kormoc (kormoc!~kormoc@mythtv/developer/kormoc) has joined #mythtv
[17:28:43] sphery: stichnot: that's what Beirdo is doing
[17:29:07] stichnot: cool
[17:29:22] stuartm: stichnot: interesting
[17:29:40] dekarl (dekarl!~dekarl@p4FE85C3A.dip.t-dialin.net) has joined #mythtv
[17:30:36] dekarl1 (dekarl1!~dekarl@p4FE85178.dip.t-dialin.net) has quit (Ping timeout: 265 seconds)
[17:31:32] wagnerrp: stichnot: there actually is a little script on the wiki that uses some music trac splitter to identify cutpoints
[17:31:46] stuartm: I'd like to see just where we're checking the current time so often in the backend, I've a suspicion that it's the eit scanner checking whether received events are in the past and can be discarded
[17:31:58] wagnerrp: supposedly works better for british tv... or at least less bad than our current tuning for mythcommflag
[17:32:18] wagnerrp: it was suggested someone do it properly and put it in mythcommflag rather than some external script
[17:32:38] wagnerrp: but mythcommflag does no audio decoding at current, so thats one of the things beirdo is adding in his rewrite
[17:32:39] stuartm: if that's the case then we could probably be a little less aggressive, we don't need to know that to the second
[17:33:41] stuartm: stichnot: there's an ffmpeg 'filter' that identifies audio silence, a video blank detector too
[17:34:21] stuartm: worth a look maybe, rather than re-inventing it from scratch
[17:35:12] stichnot: I'm watching the mythbackend process while 3 frontends are playing (the same recording, as it happens). The CPU and MEM % are slowly increasing in equal proportion. To me that suggests a memory leak plus heap fragmentation.
[17:35:34] Beirdo: here we go again :)
[17:35:41] Beirdo: don't make me call you all Udo
[17:36:02] stichnot: Beirdo: not trying to blame the logging system :)
[17:36:07] Beirdo: if you suspect memory leaks, please get a valgrind log to support it :)
[17:36:15] Beirdo: regardless of what code it is.
[17:36:49] Beirdo: I'm digging through the one from #10846
[17:36:49] ** MythLogBot http://code.mythtv.org/trac/ticket/10846 **
[17:37:24] stuartm: Beirdo: it's a memory growth, but a nasty one that needs to be fixed, but you're right calling it a leak is inaccurate
[17:37:28] Beirdo: so far, no real evidence of a serious leak anywhere
[17:37:40] Beirdo: yeah, growth is an issue
[17:38:08] stichnot: I will look into this "potential plumbing deficiency" since it's reproducible, but for now I'm just giving a color commentary
[17:38:19] stuartm: as in the backend uses 1–2GB of resident memory after a day or so
[17:39:10] stuartm: stichnot: commentary and/or stream of conciousness ramblings are my thing ;)
[17:40:09] stichnot: at this point I'm more suspicious of reference counting than logging
[17:42:09] stuartm: there have been a lot of changes, it's hard to know which one is ultimately responsible, it might even be a combination of things – unlikely, but nothing can be ruled out
[17:45:30] Captain_Murdoch: #10793 by Mark Spieth supposedly adds audio silence checking to mythcommflag, but I haven't had a chance to look at it yet.
[17:45:30] ** MythLogBot http://code.mythtv.org/trac/ticket/10793 **
[17:46:05] Captain_Murdoch: as well as audio thresholds I think.
[17:53:52] stichnot: Beirdo: the patch in 10848 does apply cleanly to master (haven't tested it though)
[17:58:00] stichnot (stichnot!chatzilla@mythtv/developer/stichnot) has quit (Ping timeout: 246 seconds)
[17:59:32] natanojl (natanojl!~jonatan@c83-252-237-63.bredband.comhem.se) has joined #mythtv
[18:02:16] stuartm: danielk22: any closer to figuring out the 'Real-time signal 0' problem?
[18:05:15] stuartm: http://pastebin.com/jZKQBn0K
[18:05:43] stuartm: if I keep trying the frontend will eventually start and run without issue
[18:06:36] stuartm: without exhibiting the problem I mean
[18:17:28] kc (kc!~Casper@unaffiliated/kc) has quit (Ping timeout: 244 seconds)
[18:20:08] Beirdo: what kernels are you guys using?
[18:25:59] danielk22: Linux t61 3.2.0-23-lowlatency #31-Ubuntu SMP PREEMPT
[18:28:23] Beirdo: hmmm, I'm thinking that the real-time signal stuff is related to the kernel
[18:32:16] kc (kc!~Casper@unaffiliated/kc) has joined #mythtv
[18:32:41] aloril (aloril!~aloril@dsl-tkubrasgw3-fe7ef900-153.dhcp.inet.fi) has quit (Ping timeout: 260 seconds)
[18:33:09] gigem: stuartm: I've got a plan of attack for the remaining deadlocks. It might not be the best long term solution, but it's fairly simple and I think it will work.
[18:33:11] gigem: My main concern has always been the possibility of an invalidated iterator due to a slave connecting and new entries getting added to reclist. Consequently, I always held the schedLock when the scheduler was iterating through reclist. I think that can be mitigated by adding any new entries to a temp list which the scheduler can then move to reclist when it's convenient.
[18:33:13] gigem: That means schedLock can be dropped more often. I think the first places where it makes sense to drop the lock are when the scheduler calls the recorder. There might be a few other places to consider, but they're probably not immediate needs.
[18:33:15] gigem: I'll try to make the changes this weekend as I probably won't have much time before then. I also don't want to get in the way of all the fun you guys are having with master right now! :)
[18:33:37] Beirdo: hehe
[18:33:47] Beirdo: yeah, it's being a "fun" time for sure
[18:38:07] stuartm: gigem: ok cool
[18:38:15] stichnot (stichnot!~chatzilla@mythtv/developer/stichnot) has joined #mythtv
[18:40:59] stuartm: gigem: certainly the deadlocks we've seen so far are the ones involving recording e.g. HandleRecording() and the stuff which establishes the directory for the next programme in the livetv chain, if we can remove the locks from those it will go a long way to solving the problem
[18:43:07] stuartm: the livetv one (GetNextLiveTVDir) actually seems out of place, I think it's only in the scheduler to begin with so it can be protected by schedLock?
[18:48:23] gigem: stuartm: Yes. Dunno about the GetNextLiveTVDir. I think that's Captain_Murdoch's doing.
[18:49:40] Captain_Murdoch: it's in the scheduler because the scheduler is the one who schedules disk space normally when it schedules where a new recording gets put at.
[18:50:41] Captain_Murdoch: probably could be elsewhere, but the scheduler knows where it will put future recordings so I think that was on of my main reasons for putting the nextlivetvdir code in there.
[18:50:49] stuartm: so it's in the scheduler because it schedules stuff? :)
[18:51:18] Captain_Murdoch: the recorder falls back to using the dir with the most disk space if it doesnt' get a response quickly.
[18:51:19] stuartm: that came out wrong ...
[18:51:27] Captain_Murdoch: :) no problem.
[18:52:55] stuartm: discussion for a later date anyway, it's not relevant to fixing the deadlock and we've got enough work to do already
[18:53:23] gigem: Captain_Murdoch: Does it call anything that might directly or indirectly call the scheduler? That's my main concern and what I should have been clearer about. I remember when you put the directory selection in and probably still makes sense to do it in the scheduler.
[18:53:28] Captain_Murdoch: strangely enough, my TODO list didn't get any shorter while I was driving cross-country the past couple weeks.
[18:54:30] Captain_Murdoch: gigem, it does touch RecList since it considers upcoming recordings when it decides where to put the LiveTV 'recording' at.
[18:54:57] stuartm: Captain_Murdoch: yeah, I'm not helping any, I should just keep some thoughts to myself
[18:55:02] Captain_Murdoch: that's in Scheduler::FillRecordingDir() which is what gets called anytime we need to decide where to put a recording
[19:08:15] Captain_Murdoch: I would be fine with saying that LiveTV always uses the dir with the most free space and we could get rid of all that logic. I never use LiveTV except for testing so I don't worry about I/O from that. dynamics have changed a bit since I added Storage Groups orignally as well.
[19:19:27] danielk22: Beirdo: A patch for fixing the QTimer bug in Qt 5.0.0 has been submitted for code review: https://codereview.qt-project.org/#change,29063 :)
[19:19:59] sphery: so, Ron Frazier on -users list says he's getting memory growth in 0.25-fixes: http://www.mythtv.org/pipermail/mythtv-users/ . . . /335750.html
[19:20:14] Beirdo: YAY
[19:20:19] aloril (aloril!~aloril@dsl-tkubrasgw3-fe7ef900-153.dhcp.inet.fi) has joined #mythtv
[19:20:25] sphery: about a gigabyte per day while idling at menu
[19:20:40] Beirdo: yeah, we will find that bugger.
[19:21:04] sphery: yeah, but that means that it was a "fix" that was backported and not one of these new features
[19:22:04] skd5aner: well, I just checked out of curiosity and 0.25-fixes mythfrontend is running at 20.3% memory for me and 405M resident
[19:22:12] skd5aner: (idle)
[19:22:19] skd5aner: I haven't attempted to watch for growth
[19:22:22] sphery: recent -fixes?
[19:22:31] skd5aner: within ~2 weeks or so
[19:22:57] danielk22: hmm, ron says he's running a 1 month old version..
[19:23:37] skd5aner: master backend (different machine) is running 4.4% @ 151M resident
[19:24:00] skd5aner: I'm running – v0.25-128-ga9edae2
[19:24:06] skd5aner: fixes branch
[19:24:38] skd5aner: I guess that's from 22 days ago?
[19:26:38] stuartm: hmm, I dunno, I would have expected to notice if the backend was using that much memory for an entire month or more, but I'd happily be wrong if it means the problem commit is easier to find
[19:30:39] skd5aner: oh wow – my slave backend is taking 40.4% memory / 809M resident
[19:31:23] skd5aner: same machine as my fe
[19:31:45] pheld (pheld!~heldal@cl-5.osl-01.no.sixxs.net) has joined #mythtv
[19:36:22] zombor (zombor!~zombor_@kohana/developer/zombor) has quit (Remote host closed the connection)
[19:39:18] Beirdo: danielk22: any reason you used write(STDERR_FILENO...) instead of cerr <<?
[19:39:32] Beirdo: in the signal handling changes, to be more specific
[19:40:08] aloril (aloril!~aloril@dsl-tkubrasgw3-fe7ef900-153.dhcp.inet.fi) has quit (Ping timeout: 244 seconds)
[19:40:15] Beirdo: not to mention, reinvented strsignal :)
[19:40:23] danielk22: yep, I had to chose a signal safe functions. write is in that category.. printf and cout isn't.
[19:40:43] danielk22: heh, yeah, well the strsignal() man page didn't inspire confidence ;]
[19:40:48] Beirdo: how is printf not signal safe?
[19:41:02] danielk22: it's not on the list of signal safe functions
[19:42:37] Beirdo: which list? I've seen several, not sure which one you are referencing in particular
[19:43:22] danielk22: The POSIX one. Basically the issue is that stderr may be in an inconsistent state. STDERR_FILENO is much lower level..
[19:43:36] Beirdo: K.
[19:44:08] dmfrey (dmfrey!~dmfrey@webdefence.cluster-h.websense.net) has quit (Quit: Ex-Chat)
[19:44:49] danielk22: Its also the reason I had to replace usleep() with sleep(), usleep() is not required to be signal safe, but sleep() is.
[19:45:00] Beirdo: yeah, gotcha
[19:45:16] Beirdo: I think strsignal() would still do the trick, but whatever
[19:45:30] Beirdo: as in in the init function
[19:46:03] Beirdo: we shouldn't need to ever pass the name of the signal, libc already has that for us (POSIX says so)
[19:46:34] Beirdo: obviously, in the signal handler, reading it from a locally-controlled location is the safest
[19:47:31] Beirdo: anyways, I'll ponder on it...
[19:47:39] Beirdo: gonna be moving sighup in there today
[19:48:27] danielk22: Feel free to convert to strsignal(), it can return null but that is easy enough to check for.
[19:48:44] danielk22: oh, but strlen is not on the list of safe functions..
[19:48:53] danielk22: I'm sure it's safe, but it's not on the list..
[19:49:04] Beirdo: heh, yeah. gotcha
[19:49:32] Beirdo: I may end up tweaking some, but the core of it's good
[19:49:58] Beirdo: and since I know WHY, I'll be sure not to mess up what you carefully put in place :)
[19:50:19] Beirdo: rather than "WTF, this looks dumb!" and falling into a trap
[19:50:20] Beirdo: heh
[19:50:34] Beirdo: it looks dumb, but it's for a good reason.
[19:51:04] danielk22: Reading up on the memory stuff I learned that some Solaris str* functions weren't signal safe. They used word reads that could lead to SIGBUS errors if the reason for the signal was that memory was unaligned...
[19:51:25] Beirdo: yeah
[19:51:33] Beirdo: that makes perfect sense
[19:51:53] Beirdo: makes for butt-ugly code in good signal handlers
[19:52:02] aloril (aloril!~aloril@dsl-tkubrasgw3-fe7ef900-153.dhcp.inet.fi) has joined #mythtv
[19:52:02] Jordack (Jordack!~jordack@h69-131-44-221.plmomi.dedicated.static.tds.net) has quit ()
[19:52:15] Beirdo: but as the whole purpose was to do it safely, we'll deal with it
[19:53:04] danielk22: Yeah, and it's another reason to do it all in one place. "There be dragons here"
[19:53:13] Beirdo: yup
[19:55:57] Sharky-Sleep (Sharky-Sleep!~Sharky112@c-24-19-57-28.hsd1.wa.comcast.net) has joined #mythtv
[19:56:08] Sharky112065 (Sharky112065!~Sharky112@c-24-19-57-28.hsd1.wa.comcast.net) has quit (Ping timeout: 245 seconds)
[19:57:55] Sharky-Sleep is now known as Sharky112065
[20:04:13] stuartm: Beirdo: if it looks that way, then maybe it should have some comments explaining it so that it doesn't get reworked to use unsafe functions in future
[20:04:30] Beirdo: yeah, we should do so :)
[20:04:51] Beirdo: I'll put some in in addition to what's already there
[20:08:13] stichnot_ (stichnot_!~chatzilla@mythtv/developer/stichnot) has joined #mythtv
[20:10:07] stichnot (stichnot!~chatzilla@mythtv/developer/stichnot) has quit (Ping timeout: 276 seconds)
[20:10:08] stichnot_ is now known as stichnot
[20:24:05] andreax (andreax!~andreaz@p5089E390.dip.t-dialin.net) has quit (Ping timeout: 246 seconds)
[20:27:14] stoffel (stoffel!~quassel@pD9E4329D.dip.t-dialin.net) has joined #mythtv
[20:30:26] NightMonkey (NightMonkey!~NightrMon@pdpc/supporter/professional/nightmonkey) has quit (Read error: Connection reset by peer)
[20:35:05] NightMonkey (NightMonkey!~NightrMon@pdpc/supporter/professional/nightmonkey) has joined #mythtv
[20:44:04] zombor (zombor!~zombor_@50-73-122-41-ip-static.hfc.comcastbusiness.net) has joined #mythtv
[20:44:04] zombor (zombor!~zombor_@50-73-122-41-ip-static.hfc.comcastbusiness.net) has quit (Changing host)
[20:44:04] zombor (zombor!~zombor_@kohana/developer/zombor) has joined #mythtv
[20:47:05] zombor (zombor!~zombor_@kohana/developer/zombor) has quit (Remote host closed the connection)
[20:56:25] danielk22: Beirdo: for communicating signals was there a reason you didn't use setup_pipe() from mythbaseutil.h ? That sets the non-blocking flag which is usually a good thing for these types of things..
[20:56:46] Beirdo: I used what they had in the example code
[20:56:54] Beirdo: I'll take a look at that though
[20:57:33] Beirdo: and I don't think we want nonblocking
[20:58:05] Beirdo: can't see how that would be helpful, just makes the write loop more difficult
[20:58:30] Beirdo: we keep trying to write until it succeeds anyways, what's the point?
[20:59:00] stoffel (stoffel!~quassel@pD9E4329D.dip.t-dialin.net) has quit (Remote host closed the connection)
[20:59:11] Beirdo: and they do suggest using a UNIX socketpair rather than using pipe
[20:59:57] Beirdo: beats me why :)
[20:59:59] Beirdo: but it works
[21:04:11] danielk22: The reason we don't use socketpair() is because SOCK_NONBLOCK is Linux specific. And it can be very convenient on the read(). i.e. you can safely read whether or not there is data. But In this case blocking on read shouldn't be an issue.
[21:04:49] Beirdo: yeah
[21:05:05] danielk22: In the other places we use a socket pair it is just a wakeup signal so we actually empty the buffer when we respond to writes.
[21:05:12] Beirdo: I think in this case, nonblocking will just make life more complicated.
[21:05:33] danielk22: got it
[21:10:49] zombor (zombor!~zombor_@kohana/developer/zombor) has joined #mythtv
[21:11:06] Beirdo: let's see if this compiles
[21:11:07] zombor (zombor!~zombor_@kohana/developer/zombor) has quit (Remote host closed the connection)
[21:13:14] Beirdo: fun stuff. Re-abstracted some of the signal handling as it was a lot of needlessly repeated code
[21:13:29] Vernon_at_work (Vernon_at_work!~singv003@lightcloud.verns.net) has quit (Remote host closed the connection)
[21:14:54] Beirdo: will make it easier to add the SIGHUP handler cleanly too
[21:22:18] SteveGoodey (SteveGoodey!~steve@host86-160-43-123.range86-160.btcentralplus.com) has quit (Read error: Connection reset by peer)
[21:28:01] Vernon_at_work (Vernon_at_work!~singv003@lightcloud.verns.net) has joined #mythtv
[21:47:43] danielk22: Beirdo: Are you going to add handlers for SIGBUS, SIGFPE, SIGILL ? These probably should be handled like SEGFAULT.
[21:48:22] amejia (amejia!~andres@xbmc/staff/amejia) has joined #mythtv
[21:48:30] Beirdo: sure
[21:48:33] Beirdo: if we want them
[21:48:49] Beirdo: would be easy enough to do :)
[21:52:05] Beirdo: but since you asked, I shall
[22:17:04] gregL_ (gregL_!~greg@cpe-74-76-125-87.nycap.res.rr.com) has quit (Quit: Leaving)
[22:20:08] natanojl (natanojl!~jonatan@c83-252-237-63.bredband.comhem.se) has quit (Ping timeout: 240 seconds)
[22:20:17] Beirdo: danielk22: http://www.pastebin.ca/2163367
[22:20:31] Beirdo: that's the log segment now on startup
[22:20:37] Beirdo: look reasonable?
[22:21:01] Beirdo: startup of the backend in this case
[22:22:36] Beirdo: now to TEST the SIGHUP in logserver again after this change :)
[22:23:24] Beirdo: YAY
[22:53:57] Sharky112065 (Sharky112065!~Sharky112@c-24-19-57-28.hsd1.wa.comcast.net) has quit (Ping timeout: 265 seconds)
[22:58:05] Sharky112065 (Sharky112065!~Sharky112@c-24-19-57-28.hsd1.wa.comcast.net) has joined #mythtv
[23:05:33] dekarl (dekarl!~dekarl@p4FE85C3A.dip.t-dialin.net) has quit (Ping timeout: 265 seconds)
[23:13:54] VManiac16 (VManiac16!~Unknown@69.4.155.83) has joined #mythtv
[23:14:13] MythBuild: build #1229 of master-vista-mingw-32bit is complete: Failure [4failed compile] Build details are at http://code.mythtv.org/buildbot/builders/mast . . . /builds/1229 blamelist: Gavin Hurlbut <ghurlbut@mythtv.org >
[23:17:31] Czar_Away (Czar_Away!~Unknown@69.4.155.83) has quit (Ping timeout: 246 seconds)
[23:17:52] Beirdo: fine.
[23:17:56] Beirdo: I'll get right on it
[23:19:16] anykey__ (anykey__!~anykey@46-126-245-147.dynamic.hispeed.ch) has quit (Ping timeout: 246 seconds)
[23:19:27] Beirdo: God, I hate Windows
[23:19:29] anykey_ (anykey_!~anykey@46-126-245-147.dynamic.hispeed.ch) has joined #mythtv
[23:27:26] Beirdo: need a brain... if only I had a brain... or something
[23:29:42] stichnot (stichnot!~chatzilla@mythtv/developer/stichnot) has quit (Ping timeout: 250 seconds)
[23:45:35] Mousey (Mousey!~r0dent_@ross154.net) has quit (Quit: Leaving)
[23:50:25] zombor (zombor!~zombor_@kohana/developer/zombor) has joined #mythtv
[23:53:27] stichnot (stichnot!~chatzilla@192.55.54.42) has joined #mythtv
[23:53:27] stichnot (stichnot!~chatzilla@mythtv/developer/stichnot) has joined #mythtv
[23:53:27] stichnot (stichnot!~chatzilla@192.55.54.42) has quit (Changing host)

IRC Logs collected by BeirdoBot.
Please use the above link to report any bugs.