{"id":30,"date":"2017-02-01T07:52:18","date_gmt":"2017-02-01T06:52:18","guid":{"rendered":"https:\/\/blog.kurschies.de\/?p=30"},"modified":"2017-02-23T07:21:08","modified_gmt":"2017-02-23T06:21:08","slug":"mystery-performance-issue-with-background-process","status":"publish","type":"post","link":"https:\/\/blog.kurschies.de\/index.php\/2017\/02\/01\/mystery-performance-issue-with-background-process\/","title":{"rendered":"mystery performance issue with background process"},"content":{"rendered":"

recently I’ve got some\u00a0serious Performance issues on an 11.2 database. The mystery: Neither the AWR nor the ADDM report showed any obviously problems, but the linux system showed a very high load. So i tried to get some Kind of live view… Cloud Control was not availeble yet, so I used the great and free tool Mumbai<\/a><\/p>\n

\"\"<\/a><\/p>\n

And it seems quite usual… but wait – what are These resources at the bottom of the line? Resource Group “other”… great, now i know exactly what it is \ud83d\ude41 But in the right lower corner is the list of top consuming processes, and there are some “E00x” processes there.<\/p>\n

Just to be sure, i’ve checked with native SQL if the chart is correct:<\/p>\n

\r\nselect\r\n   count(*) count\r\n  ,trunc(SAMPLE_TIME, 'HH24') hour\r\n  ,session_id\r\n  ,session_serial#\r\n  ,program\r\n-- and whatever might be interesting, like EVENT, P1 & P1TEXT, etc.\r\nfrom DBA_HIST_ACTIVE_SESS_HISTORY\r\nwhere session_type='BACKGROUND'      -- just get background processes\r\n  and wait_class != 'Idle'           -- ignor all idle processes\r\n  and program not like 'rman%'       -- rman is approved backgrouond process to be not idle for a longer time\r\n  and sample_time > sysdate-1        -- only last 24 hours\r\ngroup by trunc(SAMPLE_TIME, 'HH24'), session_id  ,session_serial#  ,program --dont Forget to add the columns you might added\r\nhaving count(*) > 180                -- only show if process was active half the time\r\norder by 2 desc\r\n;\r\n<\/pre>\n
the Output was like this:<\/p>\n
\r\nCOUNT   HOUR                    SESSION_ID      SESSION_SERIAL#   PROGRAM\r\n360     201x-xx-xx 02:00:00     1282            21723             oracle@orcl (E002)\r\n359     201x-xx-xx 01:00:00     1282            21723             oracle@orcl (E002)\r\n360     201x-xx-xx 00:00:00     1282            21723             oracle@orcl (E002)\r\n<\/pre>\n
wow – what’s this? just to explain the column “count”: we are querying an ASH table. ASH is captured every second, every 10th capture is flushed to disk – so we have 6 snapshots per minute or 360 per hour… meaning in above query the background process was running all time.<\/p>\n
So next step: figure out what the hell the E00x process is doing. But thanks to Oracle, the documentation<\/a> is (as usual) quite good. It’s the EMON Slave Process, which “Performs database event management and notifications”. Ok, but why are they – or at lease some of these processes – running all time? After some research i found oracle bug 9735536 and a corresponding bugfix… and as Workaround the Suggestion to kill the Ennn processes together with the EMNC (EMON Coordinator Process) and voila – the System runs smoothly again… at least for some hours, after the next EMON process was running mad.<\/p>\n
after installing the patch the issue was solved, so we can continue planning the 12c upgrade…<\/p>\n
Benjamin<\/p>\n","protected":false},"excerpt":{"rendered":"
recently I’ve got some\u00a0serious Performance issues on an 11.2 database. The mystery: Neither the AWR nor the ADDM report showed any obviously problems, but the linux system showed a very high load. So i tried to get some Kind of … Continue reading →<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":true,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[1],"tags":[7,9],"class_list":["post-30","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-oracle","tag-performance"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p8m6YC-u","jetpack-related-posts":[],"_links":{"self":[{"href":"https:\/\/blog.kurschies.de\/index.php\/wp-json\/wp\/v2\/posts\/30","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.kurschies.de\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.kurschies.de\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.kurschies.de\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.kurschies.de\/index.php\/wp-json\/wp\/v2\/comments?post=30"}],"version-history":[{"count":19,"href":"https:\/\/blog.kurschies.de\/index.php\/wp-json\/wp\/v2\/posts\/30\/revisions"}],"predecessor-version":[{"id":56,"href":"https:\/\/blog.kurschies.de\/index.php\/wp-json\/wp\/v2\/posts\/30\/revisions\/56"}],"wp:attachment":[{"href":"https:\/\/blog.kurschies.de\/index.php\/wp-json\/wp\/v2\/media?parent=30"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.kurschies.de\/index.php\/wp-json\/wp\/v2\/categories?post=30"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.kurschies.de\/index.php\/wp-json\/wp\/v2\/tags?post=30"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}