The cntlmon API on gateway at Caltech-Test

Legend:
green ball Normal status or debugging message
yellow ball Notable condition which may be a non-fatal error
orange ball Error condition not fatal to job
red ball Error condition fatal to job
blue ball Notable condition which is not an error
purple ball Currently undefined
email Condition requires email notification of the responsible administrator of this API
telephone Condition requires phone notification of the responsible administrator of this API

Link: API Status Page for Caltech-Test

09/05/08-16:52:51 PDT 09/05/08-23:52:51 GMT 904693985 IDLE archiveLog /ldas_outgoing/logs/LDAScntlmon.log.html (file13) closed (archived as /ldas_outgoing/logs/archive/cntlmonAPI/LDAScntlmon.904693965)
09/05/08-16:54:24 PDT 09/05/08-23:54:24 GMT 904694078 LDAS-TEST31571281 operator_callback begin processing LDAS-TEST31571281
09/05/08-16:54:26 PDT 09/05/08-23:54:26 GMT 904694080 LDAS-TEST31571171 mgr::rmJobFiles response has 5878 bytes
09/05/08-16:54:26 PDT 09/05/08-23:54:26 GMT 904694080 LDAS-TEST31571281 reattach done processing LDAS-TEST31571281, closing sock13
09/05/08-17:04:38 PDT 09/06/08-00:04:38 GMT 904694692 IDLE ::client22::GlobusChannelHandler read 0 bytes globus channel ''
09/05/08-17:04:38 PDT 09/06/08-00:04:38 GMT 904694692 IDLE client22::delete closed socket gtxiosock13
09/05/08-17:04:38 PDT 09/06/08-00:04:38 GMT 904694692 IDLE client22::delete client handler "client22" destroyed.
09/05/08-17:06:35 PDT 09/06/08-00:06:35 GMT 904694809 IDLE ::getPid appending pids 784 obtained from ps
09/05/08-17:06:35 PDT 09/06/08-00:06:35 GMT 904694809 IDLE ::getPid appending pids 810 obtained from ps
09/05/08-17:06:35 PDT 09/06/08-00:06:35 GMT 904694809 IDLE ::getPid appending pids 848 obtained from ps
09/05/08-17:06:35 PDT 09/06/08-00:06:35 GMT 904694809 IDLE cntlmon::cleanupLDASoutput next cleanup at 1220663195 secs
09/05/08-17:08:43 PDT 09/06/08-00:08:43 GMT 904694937 IDLE ::getPid appending pids 23569 obtained from ps
09/05/08-17:12:45 PDT 09/06/08-00:12:45 GMT 904695179 LDAS-TEST31571444 operator_callback begin processing LDAS-TEST31571444
09/05/08-17:12:50 PDT 09/06/08-00:12:50 GMT 904695184 LDAS-TEST31571439 mgr::rmJobFiles response has 11231 bytes
09/05/08-17:12:50 PDT 09/06/08-00:12:50 GMT 904695184 LDAS-TEST31571444 reattach done processing LDAS-TEST31571444, closing sock5
09/05/08-17:31:21 PDT 09/06/08-00:31:21 GMT 904696295 LDAS-TEST31571570 operator_callback begin processing LDAS-TEST31571570
09/05/08-17:31:22 PDT 09/06/08-00:31:22 GMT 904696296 LDAS-TEST31571565 mgr::rmJobFiles response has 6080 bytes
09/05/08-17:31:22 PDT 09/06/08-00:31:22 GMT 904696296 LDAS-TEST31571570 reattach done processing LDAS-TEST31571570, closing sock5
09/05/08-17:43:46 PDT 09/06/08-00:43:46 GMT 904697040 IDLE emergency_callback In the last hour, 0 emails were sent out, 0 emails blocked; so far 26393 emails were sent, 1090 unsent (throttle exceeded), 0 emails were blocked, 271 emails generated by ldas since 09/02/08-04:10:16 PDT
09/05/08-17:43:46 PDT 09/06/08-00:43:46 GMT 904697040 IDLE emergency_callback In the last hour, 0 emails were sent out, 0 emails blocked; so far 26393 emails were sent, 1090 unsent (throttle exceeded), 0 emails were blocked, 271 emails generated by ldas since 09/02/08-04:10:16 PDT
09/05/08-17:54:21 PDT 09/06/08-00:54:21 GMT 904697675 LDAS-TEST31571694 operator_callback begin processing LDAS-TEST31571694
09/05/08-17:54:23 PDT 09/06/08-00:54:23 GMT 904697677 LDAS-TEST31571680 mgr::rmJobFiles response has 7393 bytes
09/05/08-17:54:23 PDT 09/06/08-00:54:23 GMT 904697677 LDAS-TEST31571694 reattach done processing LDAS-TEST31571694, closing sock5
09/05/08-18:00:13 PDT 09/06/08-01:00:13 GMT 904698027 LDAS-TEST31571783 operator_callback begin processing LDAS-TEST31571783
09/05/08-18:00:19 PDT 09/06/08-01:00:19 GMT 904698033 LDAS-TEST31571771 mgr::rmJobFiles response has 11400 bytes
09/05/08-18:00:19 PDT 09/06/08-01:00:19 GMT 904698033 LDAS-TEST31571783 reattach done processing LDAS-TEST31571783, closing sock5
09/05/08-18:06:35 PDT 09/06/08-01:06:35 GMT 904698409 IDLE ::getPid appending pids 784 obtained from ps
09/05/08-18:06:35 PDT 09/06/08-01:06:35 GMT 904698409 IDLE ::getPid appending pids 810 obtained from ps
09/05/08-18:06:35 PDT 09/06/08-01:06:35 GMT 904698409 IDLE ::getPid appending pids 848 obtained from ps
09/05/08-18:06:35 PDT 09/06/08-01:06:35 GMT 904698409 IDLE cntlmon::cleanupLDASoutput next cleanup at 1220666795 secs
09/05/08-18:08:43 PDT 09/06/08-01:08:43 GMT 904698537 IDLE ::getPid appending pids 23569 obtained from ps
09/05/08-18:43:46 PDT 09/06/08-01:43:46 GMT 904700640 IDLE emergency_callback In the last hour, 0 emails were sent out, 0 emails blocked; so far 26393 emails were sent, 1090 unsent (throttle exceeded), 0 emails were blocked, 271 emails generated by ldas since 09/02/08-04:10:16 PDT
09/05/08-18:43:46 PDT 09/06/08-01:43:46 GMT 904700640 IDLE emergency_callback In the last hour, 0 emails were sent out, 0 emails blocked; so far 26393 emails were sent, 1090 unsent (throttle exceeded), 0 emails were blocked, 271 emails generated by ldas since 09/02/08-04:10:16 PDT
09/05/08-19:06:35 PDT 09/06/08-02:06:35 GMT 904702009 IDLE ::getPid appending pids 784 obtained from ps
09/05/08-19:06:35 PDT 09/06/08-02:06:35 GMT 904702009 IDLE ::getPid appending pids 810 obtained from ps
09/05/08-19:06:35 PDT 09/06/08-02:06:35 GMT 904702009 IDLE ::getPid appending pids 848 obtained from ps
09/05/08-19:06:35 PDT 09/06/08-02:06:35 GMT 904702009 IDLE cntlmon::cleanupLDASoutput next cleanup at 1220670395 secs
09/05/08-19:08:44 PDT 09/06/08-02:08:44 GMT 904702138 IDLE ::getPid appending pids 23569 obtained from ps
09/05/08-19:38:32 PDT 09/06/08-02:38:32 GMT 904703926 LDAS-TEST31572496 operator_callback begin processing LDAS-TEST31572496
09/05/08-19:38:35 PDT 09/06/08-02:38:35 GMT 904703929 LDAS-TEST31572491 mgr::rmJobFiles response has 6282 bytes
09/05/08-19:38:35 PDT 09/06/08-02:38:35 GMT 904703929 LDAS-TEST31572496 reattach done processing LDAS-TEST31572496, closing sock5
09/05/08-19:38:53 PDT 09/06/08-02:38:53 GMT 904703947 LDAS-TEST31572502 operator_callback begin processing LDAS-TEST31572502
09/05/08-19:38:54 PDT 09/06/08-02:38:54 GMT 904703948 LDAS-TEST31572500 mgr::rmJobFiles response has 6080 bytes
09/05/08-19:38:54 PDT 09/06/08-02:38:54 GMT 904703948 LDAS-TEST31572502 reattach done processing LDAS-TEST31572502, closing sock5
09/05/08-19:43:46 PDT 09/06/08-02:43:46 GMT 904704240 IDLE emergency_callback In the last hour, 0 emails were sent out, 0 emails blocked; so far 26393 emails were sent, 1090 unsent (throttle exceeded), 0 emails were blocked, 271 emails generated by ldas since 09/02/08-04:10:16 PDT
09/05/08-19:43:46 PDT 09/06/08-02:43:46 GMT 904704240 IDLE emergency_callback In the last hour, 0 emails were sent out, 0 emails blocked; so far 26393 emails were sent, 1090 unsent (throttle exceeded), 0 emails were blocked, 271 emails generated by ldas since 09/02/08-04:10:16 PDT
09/05/08-19:54:19 PDT 09/06/08-02:54:19 GMT 904704873 LDAS-TEST31572620 operator_callback begin processing LDAS-TEST31572620
09/05/08-19:54:23 PDT 09/06/08-02:54:23 GMT 904704877 LDAS-TEST31572610 mgr::rmJobFiles response has 55873 bytes
09/05/08-19:54:23 PDT 09/06/08-02:54:23 GMT 904704877 LDAS-TEST31572620 reattach done processing LDAS-TEST31572620, closing sock5
09/05/08-20:06:35 PDT 09/06/08-03:06:35 GMT 904705609 IDLE ::getPid appending pids 784 obtained from ps
09/05/08-20:06:35 PDT 09/06/08-03:06:35 GMT 904705609 IDLE ::getPid appending pids 810 obtained from ps
09/05/08-20:06:35 PDT 09/06/08-03:06:35 GMT 904705609 IDLE ::getPid appending pids 848 obtained from ps
09/05/08-20:06:35 PDT 09/06/08-03:06:35 GMT 904705609 IDLE cntlmon::cleanupLDASoutput next cleanup at 1220673995 secs
09/05/08-20:08:45 PDT 09/06/08-03:08:45 GMT 904705739 IDLE ::getPid appending pids 23569 obtained from ps
09/05/08-20:22:44 PDT 09/06/08-03:22:44 GMT 904706578 LDAS-TEST31572809 operator_callback begin processing LDAS-TEST31572809
09/05/08-20:22:47 PDT 09/06/08-03:22:47 GMT 904706581 LDAS-TEST31572803 mgr::rmJobFiles response has 6282 bytes
09/05/08-20:22:47 PDT 09/06/08-03:22:47 GMT 904706581 LDAS-TEST31572809 reattach done processing LDAS-TEST31572809, closing sock5
09/05/08-20:37:50 PDT 09/06/08-03:37:50 GMT 904707484 LDAS-TEST31572922 operator_callback begin processing LDAS-TEST31572922
09/05/08-20:37:52 PDT 09/06/08-03:37:52 GMT 904707486 LDAS-TEST31572917 mgr::rmJobFiles response has 5272 bytes
09/05/08-20:37:52 PDT 09/06/08-03:37:52 GMT 904707486 LDAS-TEST31572922 reattach done processing LDAS-TEST31572922, closing sock5
09/05/08-20:43:46 PDT 09/06/08-03:43:46 GMT 904707840 IDLE emergency_callback In the last hour, 0 emails were sent out, 0 emails blocked; so far 26393 emails were sent, 1090 unsent (throttle exceeded), 0 emails were blocked, 271 emails generated by ldas since 09/02/08-04:10:16 PDT
09/05/08-20:43:46 PDT 09/06/08-03:43:46 GMT 904707840 IDLE emergency_callback In the last hour, 0 emails were sent out, 0 emails blocked; so far 26393 emails were sent, 1090 unsent (throttle exceeded), 0 emails were blocked, 271 emails generated by ldas since 09/02/08-04:10:16 PDT
09/05/08-21:06:35 PDT 09/06/08-04:06:35 GMT 904709209 IDLE ::getPid appending pids 784 obtained from ps
09/05/08-21:06:35 PDT 09/06/08-04:06:35 GMT 904709209 IDLE ::getPid appending pids 810 obtained from ps
09/05/08-21:06:35 PDT 09/06/08-04:06:35 GMT 904709209 IDLE ::getPid appending pids 848 obtained from ps
09/05/08-21:06:35 PDT 09/06/08-04:06:35 GMT 904709209 IDLE cntlmon::cleanupLDASoutput next cleanup at 1220677595 secs
09/05/08-21:08:46 PDT 09/06/08-04:08:46 GMT 904709340 IDLE ::getPid appending pids 23569 obtained from ps
09/05/08-21:11:18 PDT 09/06/08-04:11:18 GMT 904709492 LDAS-TEST31573176 operator_callback begin processing LDAS-TEST31573176
09/05/08-21:11:20 PDT 09/06/08-04:11:20 GMT 904709494 LDAS-TEST31573163 mgr::rmJobFiles response has 7696 bytes
09/05/08-21:11:20 PDT 09/06/08-04:11:20 GMT 904709494 LDAS-TEST31573176 reattach done processing LDAS-TEST31573176, closing sock5
09/05/08-21:42:07 PDT 09/06/08-04:42:07 GMT 904711341 LDAS-TEST31573414 operator_callback begin processing LDAS-TEST31573414
09/05/08-21:42:08 PDT 09/06/08-04:42:08 GMT 904711342 LDAS-TEST31573413 mgr::rmJobFiles response has 6080 bytes
09/05/08-21:42:08 PDT 09/06/08-04:42:08 GMT 904711342 LDAS-TEST31573414 reattach done processing LDAS-TEST31573414, closing sock5
09/05/08-21:43:46 PDT 09/06/08-04:43:46 GMT 904711440 IDLE emergency_callback In the last hour, 0 emails were sent out, 0 emails blocked; so far 26393 emails were sent, 1090 unsent (throttle exceeded), 0 emails were blocked, 271 emails generated by ldas since 09/02/08-04:10:16 PDT
09/05/08-21:43:46 PDT 09/06/08-04:43:46 GMT 904711440 IDLE emergency_callback In the last hour, 0 emails were sent out, 0 emails blocked; so far 26393 emails were sent, 1090 unsent (throttle exceeded), 0 emails were blocked, 271 emails generated by ldas since 09/02/08-04:10:16 PDT
09/05/08-22:06:35 PDT 09/06/08-05:06:35 GMT 904712809 IDLE ::getPid appending pids 784 obtained from ps
09/05/08-22:06:35 PDT 09/06/08-05:06:35 GMT 904712809 IDLE ::getPid appending pids 810 obtained from ps
09/05/08-22:06:35 PDT 09/06/08-05:06:35 GMT 904712809 IDLE ::getPid appending pids 848 obtained from ps
09/05/08-22:06:35 PDT 09/06/08-05:06:35 GMT 904712809 IDLE cntlmon::cleanupLDASoutput next cleanup at 1220681195 secs
09/05/08-22:08:47 PDT 09/06/08-05:08:47 GMT 904712941 IDLE ::getPid appending pids 23569 obtained from ps
09/05/08-22:18:37 PDT 09/06/08-05:18:37 GMT 904713531 LDAS-TEST31573713 operator_callback begin processing LDAS-TEST31573713
09/05/08-22:18:41 PDT 09/06/08-05:18:41 GMT 904713535 LDAS-TEST31573687 mgr::rmJobFiles response has 8394 bytes
09/05/08-22:18:41 PDT 09/06/08-05:18:41 GMT 904713535 LDAS-TEST31573713 reattach done processing LDAS-TEST31573713, closing sock5
09/05/08-22:43:46 PDT 09/06/08-05:43:46 GMT 904715040 IDLE emergency_callback In the last hour, 0 emails were sent out, 0 emails blocked; so far 26393 emails were sent, 1090 unsent (throttle exceeded), 0 emails were blocked, 271 emails generated by ldas since 09/02/08-04:10:16 PDT
09/05/08-22:43:46 PDT 09/06/08-05:43:46 GMT 904715040 IDLE emergency_callback In the last hour, 0 emails were sent out, 0 emails blocked; so far 26393 emails were sent, 1090 unsent (throttle exceeded), 0 emails were blocked, 271 emails generated by ldas since 09/02/08-04:10:16 PDT
09/05/08-22:52:11 PDT 09/06/08-05:52:11 GMT 904715545 LDAS-TEST31573943 operator_callback begin processing LDAS-TEST31573943
09/05/08-22:52:14 PDT 09/06/08-05:52:14 GMT 904715548 LDAS-TEST31573917 mgr::rmJobFiles response has 8376 bytes
09/05/08-22:52:14 PDT 09/06/08-05:52:14 GMT 904715548 LDAS-TEST31573943 reattach done processing LDAS-TEST31573943, closing sock5
09/05/08-22:53:17 PDT 09/06/08-05:53:17 GMT 904715611 LDAS-TEST31573962 operator_callback begin processing LDAS-TEST31573962
09/05/08-22:53:19 PDT 09/06/08-05:53:19 GMT 904715613 LDAS-TEST31573959 mgr::rmJobFiles response has 5272 bytes
09/05/08-22:53:19 PDT 09/06/08-05:53:19 GMT 904715613 LDAS-TEST31573962 reattach done processing LDAS-TEST31573962, closing sock5
09/05/08-23:01:38 PDT 09/06/08-06:01:38 GMT 904716112 IDLE cntlmon::GlobusChannelAccept newsock gtxiosock14 addr quark.ligo.caltech.edu port 59216
09/05/08-23:01:38 PDT 09/06/08-06:01:38 GMT 904716112 IDLE createClient Namespace "client23" created for user Mary_O_Lei
09/05/08-23:01:38 PDT 09/06/08-06:01:38 GMT 904716112 IDLE cntlmon::GlobusChannelRead registered user Mary_O_Lei as priviledged client23
09/05/08-23:01:44 PDT 09/06/08-06:01:44 GMT 904716118 IDLE ::client23::GlobusChannelHandler read 129 bytes globus channel '~~122 cmonStatistics::showReply job-Stats 0 0 $client:new cntlmon::jobStatsGraph .+ 904694517-904716117 {pass fail reject} .+ .+ '
09/05/08-23:01:44 PDT 09/06/08-06:01:44 GMT 904716118 IDLE client23::evalCmd received cmd: cntlmon::jobStatsGraph .+ 904694517-904716117 {pass fail reject} .+ .+
09/05/08-23:01:50 PDT 09/06/08-06:01:50 GMT 904716124 IDLE client23::bgCmd replied to cmd cntlmon::jobStatsGraph .+ 904694517-904716117 {pass fail reject} .+ .+ 110179 bytes
09/05/08-23:02:01 PDT 09/06/08-06:02:01 GMT 904716135 IDLE ::client23::GlobusChannelHandler read 0 bytes globus channel ''
09/05/08-23:02:01 PDT 09/06/08-06:02:01 GMT 904716135 IDLE client23::delete closed socket gtxiosock14
09/05/08-23:02:01 PDT 09/06/08-06:02:01 GMT 904716135 IDLE client23::delete client handler "client23" destroyed.
09/05/08-23:04:25 PDT 09/06/08-06:04:25 GMT 904716279 LDAS-TEST31574013 operator_callback begin processing LDAS-TEST31574013
09/05/08-23:04:27 PDT 09/06/08-06:04:27 GMT 904716281 LDAS-TEST31573965 mgr::rmJobFiles response has 5878 bytes
09/05/08-23:04:27 PDT 09/06/08-06:04:27 GMT 904716281 LDAS-TEST31574013 reattach done processing LDAS-TEST31574013, closing sock5
09/05/08-23:06:35 PDT 09/06/08-06:06:35 GMT 904716409 IDLE ::getPid appending pids 784 obtained from ps
09/05/08-23:06:35 PDT 09/06/08-06:06:35 GMT 904716409 IDLE ::getPid appending pids 810 obtained from ps
09/05/08-23:06:35 PDT 09/06/08-06:06:35 GMT 904716409 IDLE ::getPid appending pids 848 obtained from ps
09/05/08-23:06:36 PDT 09/06/08-06:06:36 GMT 904716410 IDLE cntlmon::cleanupLDASoutput next cleanup at 1220684796 secs
09/05/08-23:08:48 PDT 09/06/08-06:08:48 GMT 904716542 IDLE ::getPid appending pids 23569 obtained from ps
09/05/08-23:37:35 PDT 09/06/08-06:37:35 GMT 904718269 LDAS-TEST31574277 operator_callback begin processing LDAS-TEST31574277
09/05/08-23:37:38 PDT 09/06/08-06:37:38 GMT 904718272 LDAS-TEST31574252 mgr::rmJobFiles response has 8403 bytes
09/05/08-23:37:38 PDT 09/06/08-06:37:38 GMT 904718272 LDAS-TEST31574277 reattach done processing LDAS-TEST31574277, closing sock5
09/05/08-23:37:42 PDT 09/06/08-06:37:42 GMT 904718276 LDAS-TEST31574279 operator_callback begin processing LDAS-TEST31574279
09/05/08-23:37:46 PDT 09/06/08-06:37:46 GMT 904718280 LDAS-TEST31574276 mgr::rmJobFiles response has 11231 bytes
09/05/08-23:37:46 PDT 09/06/08-06:37:46 GMT 904718280 LDAS-TEST31574279 reattach done processing LDAS-TEST31574279, closing sock5
09/05/08-23:40:56 PDT 09/06/08-06:40:56 GMT 904718470 LDAS-TEST31574328 operator_callback begin processing LDAS-TEST31574328
09/05/08-23:40:59 PDT 09/06/08-06:40:59 GMT 904718473 LDAS-TEST31574305 mgr::rmJobFiles response has 8358 bytes
09/05/08-23:40:59 PDT 09/06/08-06:40:59 GMT 904718473 LDAS-TEST31574328 reattach done processing LDAS-TEST31574328, closing sock5
09/05/08-23:43:46 PDT 09/06/08-06:43:46 GMT 904718640 IDLE emergency_callback In the last hour, 0 emails were sent out, 0 emails blocked; so far 26393 emails were sent, 1090 unsent (throttle exceeded), 0 emails were blocked, 271 emails generated by ldas since 09/02/08-04:10:16 PDT
09/05/08-23:43:46 PDT 09/06/08-06:43:46 GMT 904718640 IDLE emergency_callback In the last hour, 0 emails were sent out, 0 emails blocked; so far 26393 emails were sent, 1090 unsent (throttle exceeded), 0 emails were blocked, 271 emails generated by ldas since 09/02/08-04:10:16 PDT
09/05/08-23:50:46 PDT 09/06/08-06:50:46 GMT 904719060 LDAS-TEST31574360 operator_callback begin processing LDAS-TEST31574360
09/05/08-23:50:47 PDT 09/06/08-06:50:47 GMT 904719061 LDAS-TEST31574347 mgr::rmJobFiles response has 6080 bytes
09/05/08-23:50:47 PDT 09/06/08-06:50:47 GMT 904719061 LDAS-TEST31574360 reattach done processing LDAS-TEST31574360, closing sock5
09/06/08-00:06:35 PDT 09/06/08-07:06:35 GMT 904720009 IDLE ::getPid appending pids 784 obtained from ps
09/06/08-00:06:35 PDT 09/06/08-07:06:35 GMT 904720009 IDLE ::getPid appending pids 810 obtained from ps
09/06/08-00:06:36 PDT 09/06/08-07:06:36 GMT 904720010 IDLE ::getPid appending pids 848 obtained from ps
09/06/08-00:06:36 PDT 09/06/08-07:06:36 GMT 904720010 IDLE cntlmon::cleanupLDASoutput next cleanup at 1220688396 secs
09/06/08-00:06:41 PDT 09/06/08-07:06:41 GMT 904720015 LDAS-TEST31574482 operator_callback begin processing LDAS-TEST31574482
09/06/08-00:06:45 PDT 09/06/08-07:06:45 GMT 904720019 LDAS-TEST31574467 mgr::rmJobFiles response has 7393 bytes
09/06/08-00:06:45 PDT 09/06/08-07:06:45 GMT 904720019 LDAS-TEST31574482 reattach done processing LDAS-TEST31574482, closing sock16
09/06/08-00:08:49 PDT 09/06/08-07:08:49 GMT 904720143 IDLE ::getPid appending pids 23569 obtained from ps
09/06/08-00:43:46 PDT 09/06/08-07:43:46 GMT 904722240 IDLE emergency_callback In the last hour, 0 emails were sent out, 0 emails blocked; so far 26393 emails were sent, 1090 unsent (throttle exceeded), 0 emails were blocked, 271 emails generated by ldas since 09/02/08-04:10:16 PDT
09/06/08-00:43:46 PDT 09/06/08-07:43:46 GMT 904722240 IDLE emergency_callback In the last hour, 0 emails were sent out, 0 emails blocked; so far 26393 emails were sent, 1090 unsent (throttle exceeded), 0 emails were blocked, 271 emails generated by ldas since 09/02/08-04:10:16 PDT
09/06/08-00:47:42 PDT 09/06/08-07:47:42 GMT 904722476 LDAS-TEST31574821 operator_callback begin processing LDAS-TEST31574821
09/06/08-00:47:45 PDT 09/06/08-07:47:45 GMT 904722479 LDAS-TEST31574798 mgr::rmJobFiles response has 8403 bytes
09/06/08-00:47:45 PDT 09/06/08-07:47:45 GMT 904722479 LDAS-TEST31574821 reattach done processing LDAS-TEST31574821, closing sock5
09/06/08-01:06:35 PDT 09/06/08-08:06:35 GMT 904723609 IDLE ::getPid appending pids 784 obtained from ps
09/06/08-01:06:36 PDT 09/06/08-08:06:36 GMT 904723610 IDLE ::getPid appending pids 810 obtained from ps
09/06/08-01:06:36 PDT 09/06/08-08:06:36 GMT 904723610 IDLE ::getPid appending pids 848 obtained from ps
09/06/08-01:06:36 PDT 09/06/08-08:06:36 GMT 904723610 IDLE cntlmon::cleanupLDASoutput next cleanup at 1220691996 secs
09/06/08-01:08:50 PDT 09/06/08-08:08:50 GMT 904723744 IDLE ::getPid appending pids 23569 obtained from ps
09/06/08-01:43:46 PDT 09/06/08-08:43:46 GMT 904725840 IDLE emergency_callback In the last hour, 0 emails were sent out, 0 emails blocked; so far 26393 emails were sent, 1090 unsent (throttle exceeded), 0 emails were blocked, 271 emails generated by ldas since 09/02/08-04:10:16 PDT
09/06/08-01:43:46 PDT 09/06/08-08:43:46 GMT 904725840 IDLE emergency_callback In the last hour, 0 emails were sent out, 0 emails blocked; so far 26393 emails were sent, 1090 unsent (throttle exceeded), 0 emails were blocked, 271 emails generated by ldas since 09/02/08-04:10:16 PDT
09/06/08-01:47:35 PDT 09/06/08-08:47:35 GMT 904726069 LDAS-TEST31575237 operator_callback begin processing LDAS-TEST31575237
09/06/08-01:47:37 PDT 09/06/08-08:47:37 GMT 904726071 LDAS-TEST31575228 mgr::rmJobFiles response has 6282 bytes
09/06/08-01:47:37 PDT 09/06/08-08:47:37 GMT 904726071 LDAS-TEST31575237 reattach done processing LDAS-TEST31575237, closing sock5
09/06/08-02:01:13 PDT 09/06/08-09:01:13 GMT 904726887 LDAS-TEST31575333 operator_callback begin processing LDAS-TEST31575333
09/06/08-02:01:15 PDT 09/06/08-09:01:15 GMT 904726889 LDAS-TEST31575326 mgr::rmJobFiles response has 6080 bytes
09/06/08-02:01:15 PDT 09/06/08-09:01:15 GMT 904726889 LDAS-TEST31575333 reattach done processing LDAS-TEST31575333, closing sock5
09/06/08-02:06:22 PDT 09/06/08-09:06:22 GMT 904727196 LDAS-TEST31575408 operator_callback begin processing LDAS-TEST31575408
09/06/08-02:06:26 PDT 09/06/08-09:06:26 GMT 904727200 LDAS-TEST31575403 mgr::rmJobFiles response has 55873 bytes
09/06/08-02:06:26 PDT 09/06/08-09:06:26 GMT 904727200 LDAS-TEST31575408 reattach done processing LDAS-TEST31575408, closing sock5
09/06/08-02:06:36 PDT 09/06/08-09:06:36 GMT 904727210 IDLE ::getPid appending pids 784 obtained from ps
09/06/08-02:06:36 PDT 09/06/08-09:06:36 GMT 904727210 IDLE ::getPid appending pids 810 obtained from ps
09/06/08-02:06:36 PDT 09/06/08-09:06:36 GMT 904727210 IDLE ::getPid appending pids 848 obtained from ps
09/06/08-02:06:36 PDT 09/06/08-09:06:36 GMT 904727210 IDLE cntlmon::cleanupLDASoutput next cleanup at 1220695596 secs
09/06/08-02:08:51 PDT 09/06/08-09:08:51 GMT 904727345 IDLE ::getPid appending pids 23569 obtained from ps
09/06/08-02:39:28 PDT 09/06/08-09:39:28 GMT 904729182 LDAS-TEST31575636 operator_callback begin processing LDAS-TEST31575636
09/06/08-02:39:30 PDT 09/06/08-09:39:30 GMT 904729184 LDAS-TEST31575631 mgr::rmJobFiles response has 6282 bytes
09/06/08-02:39:30 PDT 09/06/08-09:39:30 GMT 904729184 LDAS-TEST31575636 reattach done processing LDAS-TEST31575636, closing sock5
09/06/08-02:43:46 PDT 09/06/08-09:43:46 GMT 904729440 IDLE emergency_callback In the last hour, 0 emails were sent out, 0 emails blocked; so far 26393 emails were sent, 1090 unsent (throttle exceeded), 0 emails were blocked, 271 emails generated by ldas since 09/02/08-04:10:16 PDT
09/06/08-02:43:46 PDT 09/06/08-09:43:46 GMT 904729440 IDLE emergency_callback In the last hour, 0 emails were sent out, 0 emails blocked; so far 26393 emails were sent, 1090 unsent (throttle exceeded), 0 emails were blocked, 271 emails generated by ldas since 09/02/08-04:10:16 PDT
09/06/08-02:51:43 PDT 09/06/08-09:51:43 GMT 904729917 LDAS-TEST31575706 operator_callback begin processing LDAS-TEST31575706
09/06/08-02:51:47 PDT 09/06/08-09:51:47 GMT 904729921 LDAS-TEST31575663 mgr::rmJobFiles response has 9312 bytes
09/06/08-02:51:47 PDT 09/06/08-09:51:47 GMT 904729921 LDAS-TEST31575706 reattach done processing LDAS-TEST31575706, closing sock5
09/06/08-02:52:31 PDT 09/06/08-09:52:31 GMT 904729965 LDAS-TEST31575721 operator_callback begin processing LDAS-TEST31575721
09/06/08-02:52:32 PDT 09/06/08-09:52:32 GMT 904729966 LDAS-TEST31575719 mgr::rmJobFiles response has 5272 bytes
09/06/08-02:52:32 PDT 09/06/08-09:52:32 GMT 904729966 LDAS-TEST31575721 reattach done processing LDAS-TEST31575721, closing sock5
09/06/08-03:06:36 PDT 09/06/08-10:06:36 GMT 904730810 IDLE ::getPid appending pids 784 obtained from ps
09/06/08-03:06:36 PDT 09/06/08-10:06:36 GMT 904730810 IDLE ::getPid appending pids 810 obtained from ps
09/06/08-03:06:36 PDT 09/06/08-10:06:36 GMT 904730810 IDLE ::getPid appending pids 848 obtained from ps
09/06/08-03:06:36 PDT 09/06/08-10:06:36 GMT 904730810 IDLE cntlmon::cleanupLDASoutput next cleanup at 1220699196 secs
09/06/08-03:08:52 PDT 09/06/08-10:08:52 GMT 904730946 IDLE ::getPid appending pids 23569 obtained from ps
09/06/08-03:22:16 PDT 09/06/08-10:22:16 GMT 904731750 LDAS-TEST31575923 operator_callback begin processing LDAS-TEST31575923
09/06/08-03:22:19 PDT 09/06/08-10:22:19 GMT 904731753 LDAS-TEST31575912 mgr::rmJobFiles response has 7696 bytes
09/06/08-03:22:19 PDT 09/06/08-10:22:19 GMT 904731753 LDAS-TEST31575923 reattach done processing LDAS-TEST31575923, closing sock5
09/06/08-03:24:07 PDT 09/06/08-10:24:07 GMT 904731861 LDAS-TEST31575945 operator_callback begin processing LDAS-TEST31575945
09/06/08-03:24:11 PDT 09/06/08-10:24:11 GMT 904731865 LDAS-TEST31575894 mgr::rmJobFiles response has 9312 bytes
09/06/08-03:24:11 PDT 09/06/08-10:24:11 GMT 904731865 LDAS-TEST31575945 reattach done processing LDAS-TEST31575945, closing sock5
09/06/08-03:43:46 PDT 09/06/08-10:43:46 GMT 904733040 IDLE emergency_callback In the last hour, 0 emails were sent out, 0 emails blocked; so far 26393 emails were sent, 1090 unsent (throttle exceeded), 0 emails were blocked, 271 emails generated by ldas since 09/02/08-04:10:16 PDT
09/06/08-03:43:46 PDT 09/06/08-10:43:46 GMT 904733040 IDLE emergency_callback In the last hour, 0 emails were sent out, 0 emails blocked; so far 26393 emails were sent, 1090 unsent (throttle exceeded), 0 emails were blocked, 271 emails generated by ldas since 09/02/08-04:10:16 PDT
09/06/08-03:59:31 PDT 09/06/08-10:59:31 GMT 904733985 LDAS-TEST31576230 operator_callback begin processing LDAS-TEST31576230
09/06/08-03:59:32 PDT 09/06/08-10:59:32 GMT 904733986 LDAS-TEST31576228 mgr::rmJobFiles response has 6080 bytes
09/06/08-03:59:32 PDT 09/06/08-10:59:32 GMT 904733986 LDAS-TEST31576230 reattach done processing LDAS-TEST31576230, closing sock5
09/06/08-04:06:36 PDT 09/06/08-11:06:36 GMT 904734410 IDLE ::getPid appending pids 784 obtained from ps
09/06/08-04:06:36 PDT 09/06/08-11:06:36 GMT 904734410 IDLE ::getPid appending pids 810 obtained from ps
09/06/08-04:06:36 PDT 09/06/08-11:06:36 GMT 904734410 IDLE ::getPid appending pids 848 obtained from ps
09/06/08-04:06:36 PDT 09/06/08-11:06:36 GMT 904734410 IDLE cntlmon::cleanupLDASoutput next cleanup at 1220702796 secs
09/06/08-04:08:53 PDT 09/06/08-11:08:53 GMT 904734547 IDLE ::getPid appending pids 23569 obtained from ps
09/06/08-04:43:46 PDT 09/06/08-11:43:46 GMT 904736640 IDLE emergency_callback In the last hour, 0 emails were sent out, 0 emails blocked; so far 26393 emails were sent, 1090 unsent (throttle exceeded), 0 emails were blocked, 271 emails generated by ldas since 09/02/08-04:10:16 PDT
09/06/08-04:43:46 PDT 09/06/08-11:43:46 GMT 904736640 IDLE emergency_callback In the last hour, 0 emails were sent out, 0 emails blocked; so far 26393 emails were sent, 1090 unsent (throttle exceeded), 0 emails were blocked, 271 emails generated by ldas since 09/02/08-04:10:16 PDT
09/06/08-04:45:17 PDT 09/06/08-11:45:17 GMT 904736731 LDAS-TEST31576538 operator_callback begin processing LDAS-TEST31576538
09/06/08-04:45:21 PDT 09/06/08-11:45:21 GMT 904736735 LDAS-TEST31576511 mgr::rmJobFiles response has 11433 bytes
09/06/08-04:45:21 PDT 09/06/08-11:45:21 GMT 904736735 LDAS-TEST31576538 reattach done processing LDAS-TEST31576538, closing sock5
09/06/08-05:00:59 PDT 09/06/08-12:00:59 GMT 904737673 LDAS-TEST31576654 operator_callback begin processing LDAS-TEST31576654
09/06/08-05:01:01 PDT 09/06/08-12:01:01 GMT 904737675 LDAS-TEST31576650 mgr::rmJobFiles response has 5272 bytes
09/06/08-05:01:01 PDT 09/06/08-12:01:01 GMT 904737675 LDAS-TEST31576654 reattach done processing LDAS-TEST31576654, closing sock5
09/06/08-05:06:36 PDT 09/06/08-12:06:36 GMT 904738010 IDLE ::getPid appending pids 784 obtained from ps
09/06/08-05:06:36 PDT 09/06/08-12:06:36 GMT 904738010 IDLE ::getPid appending pids 810 obtained from ps
09/06/08-05:06:36 PDT 09/06/08-12:06:36 GMT 904738010 IDLE ::getPid appending pids 848 obtained from ps
09/06/08-05:06:36 PDT 09/06/08-12:06:36 GMT 904738010 IDLE cntlmon::cleanupLDASoutput next cleanup at 1220706396 secs
09/06/08-05:08:53 PDT 09/06/08-12:08:53 GMT 904738147 IDLE ::getPid appending pids 23569 obtained from ps
09/06/08-05:35:36 PDT 09/06/08-12:35:36 GMT 904739750 LDAS-TEST31576939 operator_callback begin processing LDAS-TEST31576939
09/06/08-05:35:38 PDT 09/06/08-12:35:38 GMT 904739752 LDAS-TEST31576928 mgr::rmJobFiles response has 5878 bytes
09/06/08-05:35:38 PDT 09/06/08-12:35:38 GMT 904739752 LDAS-TEST31576939 reattach done processing LDAS-TEST31576939, closing sock5
09/06/08-05:43:46 PDT 09/06/08-12:43:46 GMT 904740240 IDLE emergency_callback In the last hour, 0 emails were sent out, 0 emails blocked; so far 26393 emails were sent, 1090 unsent (throttle exceeded), 0 emails were blocked, 271 emails generated by ldas since 09/02/08-04:10:16 PDT
09/06/08-05:43:46 PDT 09/06/08-12:43:46 GMT 904740240 IDLE emergency_callback In the last hour, 0 emails were sent out, 0 emails blocked; so far 26393 emails were sent, 1090 unsent (throttle exceeded), 0 emails were blocked, 271 emails generated by ldas since 09/02/08-04:10:16 PDT
09/06/08-05:50:33 PDT 09/06/08-12:50:33 GMT 904740647 LDAS-TEST31577042 operator_callback begin processing LDAS-TEST31577042
09/06/08-05:50:37 PDT 09/06/08-12:50:37 GMT 904740651 LDAS-TEST31577036 mgr::rmJobFiles response has 11231 bytes
09/06/08-05:50:37 PDT 09/06/08-12:50:37 GMT 904740651 LDAS-TEST31577042 reattach done processing LDAS-TEST31577042, closing sock5
09/06/08-06:05:45 PDT 09/06/08-13:05:45 GMT 904741559 LDAS-TEST31577121 operator_callback begin processing LDAS-TEST31577121
09/06/08-06:05:46 PDT 09/06/08-13:05:46 GMT 904741560 LDAS-TEST31577118 mgr::rmJobFiles response has 6080 bytes
09/06/08-06:05:46 PDT 09/06/08-13:05:46 GMT 904741560 LDAS-TEST31577121 reattach done processing LDAS-TEST31577121, closing sock5
09/06/08-06:06:36 PDT 09/06/08-13:06:36 GMT 904741610 IDLE ::getPid appending pids 784 obtained from ps
09/06/08-06:06:36 PDT 09/06/08-13:06:36 GMT 904741610 IDLE ::getPid appending pids 810 obtained from ps
09/06/08-06:06:36 PDT 09/06/08-13:06:36 GMT 904741610 IDLE ::getPid appending pids 848 obtained from ps
09/06/08-06:06:36 PDT 09/06/08-13:06:36 GMT 904741610 IDLE cntlmon::cleanupLDASoutput next cleanup at 1220709996 secs
09/06/08-06:08:54 PDT 09/06/08-13:08:54 GMT 904741748 IDLE ::getPid appending pids 23569 obtained from ps
09/06/08-06:20:51 PDT 09/06/08-13:20:51 GMT 904742465 LDAS-TEST31577237 operator_callback begin processing LDAS-TEST31577237
09/06/08-06:20:53 PDT 09/06/08-13:20:53 GMT 904742467 LDAS-TEST31577232 mgr::rmJobFiles response has 7393 bytes
09/06/08-06:20:53 PDT 09/06/08-13:20:53 GMT 904742467 LDAS-TEST31577237 reattach done processing LDAS-TEST31577237, closing sock5
09/06/08-06:43:46 PDT 09/06/08-13:43:46 GMT 904743840 IDLE emergency_callback In the last hour, 0 emails were sent out, 0 emails blocked; so far 26393 emails were sent, 1090 unsent (throttle exceeded), 0 emails were blocked, 271 emails generated by ldas since 09/02/08-04:10:16 PDT
09/06/08-06:43:46 PDT 09/06/08-13:43:46 GMT 904743840 IDLE emergency_callback In the last hour, 0 emails were sent out, 0 emails blocked; so far 26393 emails were sent, 1090 unsent (throttle exceeded), 0 emails were blocked, 271 emails generated by ldas since 09/02/08-04:10:16 PDT
09/06/08-07:06:36 PDT 09/06/08-14:06:36 GMT 904745210 IDLE ::getPid appending pids 784 obtained from ps
09/06/08-07:06:36 PDT 09/06/08-14:06:36 GMT 904745210 IDLE ::getPid appending pids 810 obtained from ps
09/06/08-07:06:36 PDT 09/06/08-14:06:36 GMT 904745210 IDLE ::getPid appending pids 848 obtained from ps
09/06/08-07:06:36 PDT 09/06/08-14:06:36 GMT 904745210 IDLE cntlmon::cleanupLDASoutput next cleanup at 1220713596 secs
09/06/08-07:08:55 PDT 09/06/08-14:08:55 GMT 904745349 IDLE ::getPid appending pids 23569 obtained from ps
09/06/08-07:43:46 PDT 09/06/08-14:43:46 GMT 904747440 IDLE emergency_callback In the last hour, 0 emails were sent out, 0 emails blocked; so far 26393 emails were sent, 1090 unsent (throttle exceeded), 0 emails were blocked, 271 emails generated by ldas since 09/02/08-04:10:16 PDT
09/06/08-07:43:46 PDT 09/06/08-14:43:46 GMT 904747440 IDLE emergency_callback In the last hour, 0 emails were sent out, 0 emails blocked; so far 26393 emails were sent, 1090 unsent (throttle exceeded), 0 emails were blocked, 271 emails generated by ldas since 09/02/08-04:10:16 PDT
09/06/08-08:06:36 PDT 09/06/08-15:06:36 GMT 904748810 IDLE ::getPid appending pids 784 obtained from ps
09/06/08-08:06:36 PDT 09/06/08-15:06:36 GMT 904748810 IDLE ::getPid appending pids 810 obtained from ps
09/06/08-08:06:36 PDT 09/06/08-15:06:36 GMT 904748810 IDLE ::getPid appending pids 848 obtained from ps
09/06/08-08:06:36 PDT 09/06/08-15:06:36 GMT 904748810 IDLE cntlmon::cleanupLDASoutput next cleanup at 1220717196 secs
09/06/08-08:08:56 PDT 09/06/08-15:08:56 GMT 904748950 IDLE ::getPid appending pids 23569 obtained from ps
09/06/08-08:10:05 PDT 09/06/08-15:10:05 GMT 904749019 LDAS-TEST31578017 operator_callback begin processing LDAS-TEST31578017
09/06/08-08:10:07 PDT 09/06/08-15:10:07 GMT 904749021 LDAS-TEST31578009 mgr::rmJobFiles response has 6282 bytes
09/06/08-08:10:07 PDT 09/06/08-15:10:07 GMT 904749021 LDAS-TEST31578017 reattach done processing LDAS-TEST31578017, closing sock5
09/06/08-08:11:59 PDT 09/06/08-15:11:59 GMT 904749133 LDAS-TEST31578057 operator_callback begin processing LDAS-TEST31578057
09/06/08-08:12:00 PDT 09/06/08-15:12:00 GMT 904749134 LDAS-TEST31578054 mgr::rmJobFiles response has 6080 bytes
09/06/08-08:12:00 PDT 09/06/08-15:12:00 GMT 904749134 LDAS-TEST31578057 reattach done processing LDAS-TEST31578057, closing sock5
09/06/08-08:39:35 PDT 09/06/08-15:39:35 GMT 904750789 LDAS-TEST31578241 operator_callback begin processing LDAS-TEST31578241
09/06/08-08:39:39 PDT 09/06/08-15:39:39 GMT 904750793 LDAS-TEST31578228 mgr::rmJobFiles response has 55873 bytes
09/06/08-08:39:39 PDT 09/06/08-15:39:39 GMT 904750793 LDAS-TEST31578241 reattach done processing LDAS-TEST31578241, closing sock5
09/06/08-08:43:46 PDT 09/06/08-15:43:46 GMT 904751040 IDLE emergency_callback In the last hour, 0 emails were sent out, 0 emails blocked; so far 26393 emails were sent, 1090 unsent (throttle exceeded), 0 emails were blocked, 271 emails generated by ldas since 09/02/08-04:10:16 PDT
09/06/08-08:43:46 PDT 09/06/08-15:43:46 GMT 904751040 IDLE emergency_callback In the last hour, 0 emails were sent out, 0 emails blocked; so far 26393 emails were sent, 1090 unsent (throttle exceeded), 0 emails were blocked, 271 emails generated by ldas since 09/02/08-04:10:16 PDT
09/06/08-08:57:46 PDT 09/06/08-15:57:46 GMT 904751880 LDAS-TEST31578384 operator_callback begin processing LDAS-TEST31578384
09/06/08-08:57:48 PDT 09/06/08-15:57:48 GMT 904751882 LDAS-TEST31578379 mgr::rmJobFiles response has 5272 bytes
09/06/08-08:57:48 PDT 09/06/08-15:57:48 GMT 904751882 LDAS-TEST31578384 reattach done processing LDAS-TEST31578384, closing sock5
09/06/08-08:58:43 PDT 09/06/08-15:58:43 GMT 904751937 LDAS-TEST31578402 operator_callback begin processing LDAS-TEST31578402
09/06/08-08:58:47 PDT 09/06/08-15:58:47 GMT 904751941 LDAS-TEST31578386 mgr::rmJobFiles response has 8403 bytes
09/06/08-08:58:47 PDT 09/06/08-15:58:47 GMT 904751941 LDAS-TEST31578402 reattach done processing LDAS-TEST31578402, closing sock5
09/06/08-08:59:18 PDT 09/06/08-15:59:18 GMT 904751972 LDAS-TEST31578413 operator_callback begin processing LDAS-TEST31578413
09/06/08-08:59:20 PDT 09/06/08-15:59:20 GMT 904751974 LDAS-TEST31578409 mgr::rmJobFiles response has 6282 bytes
09/06/08-08:59:20 PDT 09/06/08-15:59:20 GMT 904751974 LDAS-TEST31578413 reattach done processing LDAS-TEST31578413, closing sock5
09/06/08-09:06:36 PDT 09/06/08-16:06:36 GMT 904752410 IDLE ::getPid appending pids 784 obtained from ps
09/06/08-09:06:36 PDT 09/06/08-16:06:36 GMT 904752410 IDLE ::getPid appending pids 810 obtained from ps
09/06/08-09:06:37 PDT 09/06/08-16:06:37 GMT 904752411 IDLE ::getPid appending pids 848 obtained from ps
09/06/08-09:06:37 PDT 09/06/08-16:06:37 GMT 904752411 IDLE cntlmon::cleanupLDASoutput next cleanup at 1220720797 secs
09/06/08-09:08:57 PDT 09/06/08-16:08:57 GMT 904752551 IDLE ::getPid appending pids 23569 obtained from ps