Top 10 Reasons for poor Oracle performance

Top 10 Reasons for poor Oracle performance

What are the most common root-causes of poor Oracle performance? Every expert will give you a different opinion, but I’ve prepared my list based on our hundreds of Oracle tuning engagements:

Bad Design – The number one offender to poor performance is over-normalization of Oracle tables, excessive (unused indexes) and 15-way table joins for what should be a simple fetch.

Poor server optimization – Setting the server kernel parameters and I/O configuration (e.g. direct I/O) has a profound impact on Oracle performance

Bad disk I/O configuration – Inappropriate use of RAID5, disk channel bottlenecks and poor disk striping.

Poor Optimizer Statistics – Prior to Oracle 10g (automatic statistics), a common cause of poor SQL performance was missing/stale CBO statistics and missing histograms.

Object contention – Failing to set ASSM, freelists or freelist_groups for DML-active tables and indexes can cause very slow DML performance.

Under-allocated RAM regions – Not allocating enough RAM for shared_pool_size, pga_aggregate_target and db_cache_size can cause an I/O-bound database.

Non-reentrant SQL – All SQL should use host variables/cursor_sharing=force to make SQL reusable within the library cache.

Un-set initialization parameters – Many of the initialization parameters are made to be set by the DBA (db_file_multiblock_read_count, optimizer_index_caching) and failing to set these parameters properly results in poorly optimized execution plans.

Excessive nested loop joins – In 64-bit Oracle systems we have gigabytes available for RAM sorts and hash joins. Failing to set pga_aggregate_target to allow the CBO to choose hash joins can result in very slow SQL performance.

Human Misfeasance – The DBA’s failure to monitor their database (STATSPACK/AWR), set-up exception reporting alerts (OEM) and adjusting their instance to match changing workloads is a major cause of poor performance.

The Oracle Documentation lists these ten reasons for poor performance.

1. Bad Connection Management
The application connects and disconnects for each database interaction. This problem is common with stateless middleware in application servers. It has over two orders of magnitude impact on performance, and it is totally unscalable.

2. Bad Use of Cursors and the Shared Pool
Not using cursors results in repeated parses. If bind variables are not used, then there is hard parsing of all SQL statements. This has an order of magnitude impact in performance, and it is totally unscalable. Use cursors with bind variables that open the cursor and execute it many times. Be suspicious of applications generating dynamic SQL.

3. Getting Database I/O Wrong
Many sites lay out their databases poorly over the available disks. Other sites specify the number of disks incorrectly, because they configure disks by disk space and not I/O bandwidth.

4. Redo Log Setup Problems
Many sites run with too few redo logs that are too small. Small redo logs cause system checkpoints to continuously put a high load on the buffer cache and I/O system. If there are too few redo logs, then the archive cannot keep up, and the database will wait for the archive process to catch up.

5. Serialization of data blocks in the buffer cache due to lack of free lists, free list groups, transaction slots (INITRANS), or shortage of rollback segments.
This is particularly common on INSERT-heavy applications, in applications that have raised the block size to 8K or 16K, or in applications with large numbers of active users and few rollback segments.

6. Long Full Table Scans
Long full table scans for high-volume or interactive online operations could indicate poor transaction design, missing indexes, or poor SQL optimization. Long table scans, by nature, are I/O intensive and unscalable.

7. In Disk Sorting
In disk sorts for online operations could indicate poor transaction design, missing indexes, or poor SQL optimization. Disk sorts, by nature, are I/O-intensive and unscalable.

8. High Amounts of Recursive (SYS) SQL
Large amounts of recursive SQL executed by SYS could indicate space management activities, such as extent allocations, taking place. This is unscalable and impacts user response time. Recursive SQL executed under another user ID is probably SQL and PL/SQL, and this is not a problem.

9. Schema Errors and Optimizer Problems
In many cases, an application uses too many resources because the schema owning the tables has not been successfully migrated from the development environment or from an older implementation. Examples of this are missing indexes or incorrect statistics. These errors can lead to sub-optimal execution plans and poor interactive user performance. When migrating applications of known performance, export the schema statistics to maintain plan stability using the DBMS_STATS package.

Likewise, optimizer parameters set in the initialization parameter file can override proven optimal execution plans. For these reasons, schemas, schema statistics, and optimizer settings should be managed together as a group to ensure consistency of performance.

10. Use of Nonstandard Initialization Parameters
These might have been implemented based on poor advice or incorrect assumptions. In particular, parameters associated with SPIN_COUNT on latches and undocumented optimizer features can cause a great deal of problems that can require considerable investigation.

Steps to Recover DB via RMAN

Steps to Recover DB via RMAN::

Lets suppose my Original DB is PROD: and I want to duplicate this db to other system as TEST
I assume that oracle software is installed on following location. You can change according to situation.
c:\oracle\10g

1) backup target database PROD using RMAN.

2) Create password file for new db i.e. TEST

orapwd file= 'c:\oracle\10g\database\orapwTEST password=tuesday

3) Create directory structures for the clone database:

BDUMP (c:\oracle\admin\TEST\bdump)
UDUMP (c:\oracle\admin\TEST\udump)
CDUMP (c:\oracle\admin\TEST\cdump)
ADUMP (c:\oracle\admin\TEST\adump)
DATAFILES (c:\oracle\admin\TEST\PROD\datafile) (remember we have included PROD dir here)
ONLINELOG (c:\oracle\admin\TEST\PROD\onlinelog)
ARCHIVES (c:\oracle\admin\TEST\archives)
CONTROLFILE (c:\oracle\admin\TEST\paktest)

4) Copy pfile and edit it for the duplicate db.
*.db_file_name_convert=('c:\oracle\oradata\PROD',' c:\oracle\oradata\TEST ')
*.log_file_name_convert=('c:\oracle\oradata\PROD',' c:\oracle\oradata\TEST ')
*.control_files=' c:\oracle\oradata\TEST\control01.ctl',' c:\oracle\oradata\TEST\control02.ctl'
*.audit_file_dest=' c:\oracle\oradata\TEST\adump'
*.background_dump_dest=' c:\oracle\oradata\TEST\bdump'
*.core_dump_dest=' c:\oracle\oradata\TEST\cdump'
*.user_dump_dest=' c:\oracle\oradata\TEST\udump'
*.db_name='paktest'
*.instance_name='paktest'

5) Create oracle service and Startup the duplicate database in nomount stage

oradim -new -sid TEST -intpwd tuesday -startmode auto -pfile 'c:\oracle\10g\database\initTEST.ora'

set ORACLE_SID=paktest
sqlplus / as sysdba
startup nomount
exit;

6) Configure the listener and tnsnames.ora for the duplicate database. Try to connect to original database with sys user.
Also try to connect to auxiliary database with sys user.

7) We you are ok with connectivity test then connect to target and auxiliary database from command prompt as:


rman target sys/your_sys_password@PROD auxiliary sys/tuesday@TEST


8) list backupset and check all backups are available.
RMAN> list backupset of database;

9) Run the following command to duplicate the entire database. Use skip tablespace clause to exclude tablespaces.
run {
set until time "to_date('2007-08-27 18:00:00','YYYY-MM-DD HH24:MI:SS')";
allocate channel ch1 type disk;
allocate auxiliary channel ch2 type disk;
duplicate target database to paktest;
}

Most Commonly used DBA Scripts:1

Script To Find The Active_Session_Waits

SET LINESIZE 200
SET PAGESIZE 1100
COLUMN username FORMAT A15
COLUMN osuser FORMAT A15
COLUMN sid FORMAT 99999
COLUMN serial# FORMAT 9999999
COLUMN wait_class FORMAT A15
COLUMN state FORMAT A19
COLUMN logon_time FORMAT A20
SELECT NVL(a.username, '(oracle)') AS username, a.osuser, a.sid, a.serial#, d.spid AS process_id, a.wait_class, a.seconds_in_wait, a.state, a.blocking_session, a.blocking_session_status, a.module, TO_CHAR(a.logon_Time,'DD-MON-YYYY HH24:MI:SS') AS logon_timeFROM v$session a, v$process dWHERE a.paddr = d.addrAND a.status = 'ACTIVE'ORDER BY 1,2;

Script To Find Database Usage High Watermark :

COLUMN name FORMAT A40
COLUMN highwater FORMAT 999999999999
COLUMN last_value FORMAT 999999999999
SET PAGESIZE 24SELECT hwm1.name, hwm1.highwater, hwm1.last_valueFROM dba_high_water_mark_statistics hwm1WHERE hwm1.version = (SELECT MAX(hwm2.version) FROM dba_high_water_mark_statistics hwm2 WHERE hwm2.name = hwm1.name)ORDER BY hwm1.name;

COLUMN FORMAT DEFAULT

Display The Value of Dynamically Memory Pools :

COLUMN name FORMAT A40
COLUMN value FORMAT A40
SELECT name, value FROM v$parameter WHERE SUBSTR(name, 1, 1) = '_'ORDER BY name;

COLUMN FORMAT DEFAULT


Display Feature Usage Statistics :

COLUMN name FORMAT A50
COLUMN detected_usages FORMAT 999999999999
SELECT u1.name, u1.detected_usagesFROM dba_feature_usage_statistics u1WHERE u1.version = (SELECT MAX(u2.version) FROM dba_feature_usage_statistics u2 WHERE u2.name = u1.name)ORDER BY u1.name;

COLUMN FORMAT DEFAULT

Displays Scheduler Information About Job Classes :

SET LINESIZE 200
COLUMN service FORMAT A20
COLUMN comments FORMAT A40
SELECT job_class_name, resource_consumer_group, service, logging_level, log_history, commentsFROM dba_scheduler_job_classesORDER BY job_class_name;

Displays scheduler information about job programs :
SET LINESIZE 250
COLUMN owner FORMAT A20
COLUMN program_name FORMAT A30
COLUMN program_action FORMAT A50
COLUMN comments FORMAT A40
SELECT owner, program_name, program_type, program_action, number_of_arguments, enabled, commentsFROM dba_scheduler_programsORDER BY owner, program_name;

Displays scheduler information about job schedules :

SET LINESIZE 250
COLUMN owner FORMAT A20
COLUMN schedule_name FORMAT A30
COLUMN start_date FORMAT A35
COLUMN repeat_interval FORMAT A50
COLUMN end_date FORMAT A35
COLUMN comments FORMAT A40
SELECT owner, schedule_name, start_date, repeat_interval, end_date, commentsFROM dba_scheduler_schedulesORDER BY owner, schedule_name;

Displays scheduler job information :

SET LINESIZE 200
COLUMN owner FORMAT A20
COLUMN next_run_date FORMAT A35
SELECT owner, job_name, enabled, job_class, next_run_dateFROM dba_scheduler_jobsORDER BY owner, job_name;

Displays scheduler information for running jobs :

SET LINESIZE 200
COLUMN owner FORMAT A20
SELECT owner, job_name, running_instance, elapsed_timeFROM dba_scheduler_running_jobsORDER BY owner, job_name;

Displays information on all database sessions with the username column displayed as a heirarchy if locks are present :

SET LINESIZE 500
SET PAGESIZE 1000
COLUMN username FORMAT A15
COLUMN machine FORMAT A25
COLUMN logon_time FORMAT A20
SELECT LPAD(' ', (level-1)*2, ' ') NVL(s.username, '(oracle)') AS username, s.osuser, s.sid, s.serial#, s.lockwait, s.status, s.module, s.machine, s.program, TO_CHAR(s.logon_Time,'DD-MON-YYYY HH24:MI:SS') AS logon_timeFROM v$session s CONNECT BY PRIOR s.sid = s.blocking_session START WITH s.blocking_session IS NULL;

SET PAGESIZE 14

Displays segment advice for the specified segment :

SET SERVEROUTPUT ON SIZE 1000000
SET LINESIZE 200
SET VERIFY OFF
DECLARE l_object_id NUMBER;
l_task_name VARCHAR2(32767) := 'SEGMENT_ADVISOR_TASK';
l_object_type VARCHAR2(32767) := UPPER('&1');
l_attr1 VARCHAR2(32767) := UPPER('&2');
l_attr2 VARCHAR2(32767) := UPPER('&3');
BEGIN IF l_attr2 = 'NULL' THEN l_attr2 := NULL;
END IF;

DBMS_ADVISOR.create_task ( advisor_name => 'Segment Advisor', task_name => l_task_name);

DBMS_ADVISOR.create_object ( task_name => l_task_name, object_type => l_object_type, attr1 => l_attr1, attr2 => l_attr2, attr3 => NULL, attr4 => 'null', attr5 => NULL, object_id => l_object_id);
DBMS_ADVISOR.set_task_parameter ( task_name => l_task_name, parameter => 'RECOMMEND_ALL', value => 'TRUE');

DBMS_ADVISOR.execute_task(task_name => l_task_name);FOR cur_rec IN (SELECT f.impact, o.type, o.attr1, o.attr2, f.message, f.more_info FROM dba_advisor_findings f JOIN dba_advisor_objects o ON f.object_id = o.object_id AND f.task_name = o.task_name WHERE f.task_name = l_task_name ORDER BY f.impact DESC) LOOP DBMS_OUTPUT.put_line('..');

DBMS_OUTPUT.put_line('Type : ' cur_rec.type); DBMS_OUTPUT.put_line('Attr1 : ' cur_rec.attr1);
DBMS_OUTPUT.put_line('Attr2 : ' cur_rec.attr2); DBMS_OUTPUT.put_line('Message : ' cur_rec.message);
DBMS_OUTPUT.put_line('More info : ' cur_rec.more_info);
END LOOP;
DBMS_ADVISOR.delete_task(task_name => l_task_name);
EXCEPTION WHEN OTHERS THEN DBMS_OUTPUT.put_line('Error : ' DBMS_UTILITY.format_error_backtrace);
DBMS_ADVISOR.delete_task(task_name => l_task_name);
END;

Displays information about database services :
SET LINESIZE 200
COLUMN name FORMAT A30
COLUMN network_name FORMAT A50
SELECT name, network_nameFROM dba_servicesORDER BY name;

Displays information on all database session waits :

SET LINESIZE 200
SET PAGESIZE 1000
COLUMN username FORMAT A20
COLUMN event FORMAT A30
COLUMN wait_class FORMAT A15
SELECT NVL(s.username, '(oracle)') AS username, s.sid, s.serial#, sw.event, sw.wait_class, sw.wait_time, sw.seconds_in_wait, sw.state FROM v$session_wait sw, v$session sWHERE s.sid = sw.sid ORDER BY sw.seconds_in_wait DESC;

Displays scheduler information about window groups :

SET LINESIZE 250
COLUMN comments FORMAT A40
SELECT window_group_name, enabled, number_of_windows, commentsFROM dba_scheduler_window_groupsORDER BY window_group_name;SELECT window_group_name, window_nameFROM dba_scheduler_wingroup_membersORDER BY window_group_name, window_name;

Displays scheduler information about windows :

SET LINESIZE 250
COLUMN comments FORMAT A40
SELECT window_name, resource_plan, enabled, active, commentsFROM dba_scheduler_windowsORDER BY window_name;