Wikipedia

Search results

Sharing Memory—Automatically

TECHNOLOGY: Talking Tuning


Sharing Memory—Automatically (By Kimberly Floss)                                                                                         Published In 

Oracle Magazine

Put away your scripts and let Oracle automatically resize your memory pools.
It used to be a challenge to size the various memory pools that comprise the Oracle system global area (SGA) for optimal performance. But that was before Oracle Database 10g.
For example, the buffer cache had to be large enough to keep frequently used blocks readily available for fast retrieval—but not so large that the database couldn't allocate memory in one of the other SGA pools when needed. Since various types of applications and usage patterns tax the assorted pools differently, and since the workload can vary minute by minute, manuall y resizing the SGA components could feel like a never-ending task.
If there wasn't enough free SGA to allocate memory to a specific pool when needed, the database would raise an out-of-memory error, such as 
ORA-04031: unable to allocate ...


Also, before Oracle9i Database Release 2, adjusting the pool sizes required bouncing the server—hardly practical in a production environment.
That's why Oracle Database 10g's Automatic Shared Memory Management (ASMM) is such a welcome improvement. First introduced in Oracle Database 10g and further enhanced in Oracle Database 10g Release 2, ASMM automatically sizes many of the memory pools while the database is running, allocating and de-allocating memory as needed. As the workload composition changes, Oracle Database 10g enlarges the appropriate pools and reduces the sizes of other automatically sized pools accordingly. In short, ASMM can save you a lot of trouble—and improve overall performance as well.
Let's take a closer look at how it works and how to use it.

SGA Memory Components

The linchpin of Oracle Database 10g's memory management scheme is the new SGA_TARGETinitialization parameter. The value of this setting determines the total amount of SGA memory that can be allocated across both manually and automatically sized pools. (See Table 1, in the online version of this article at oracle.com/technology/oramag/oracle/ 05-sep/o55tuning.html.) TheSGA_TARGET value imposes a ceiling on the amount of RAM devoted to the Oracle SGA.
Oracle Database 10g Release 2 can automatically tune the size of the shared pool, buffer cache, Java pool, large pool, and streams pool. However, you must still manually tune several SGA components, including the log buffer (at startup time only), the keep buffer cache, the recycle buffer cache, and all of the nonstandard block-size buffer caches (for example, the 32K and 16K buffer caches set by the db_32K_cache_size and db_16K_cache_size parameters). Manually sized components consume SGA from the target value first, and then the remainder of SGA memory is spread across the various autosized pools. In other words, if you set parameter values for any of these manually tuned pools, Oracle Database 10g Release 2 subtracts their sizes from SGA_TARGET first, before allocating memory to the automatically allocated pools.
The SGA_TARGET setting value also includes a small amount of fixed-size SGA. Oracle Database 10g Release 2 sets the fixed size based on the operating system and other criteria. You can see the amount (in bytes) of the fixed-size SGA and totals of other major elements that occupy the SGA by querying the V$SGA view, as follows: 
SQL> select * from v$sga;
NAME                      VALUE
------------------        ----------
Fixed Size                  1247780
Variable Size             124319196
Database Buffers           41943040
Redo Buffers                 262144


You can query this same view to determine an initial size for SGA_TARGET when you switch from manual to autotuning, by summing all the components as follows: 
SQL> select sum(value)/1024/1024 "Megabytes" from v$sga;

Megabytes
------------------
 160


Listing 1 shows an example of total real memory allocation for the current SGA from theV$SGA_DYNAMIC_COMPONENTS view (introduced in Oracle9i Database), which contains both manual and autotuned SGA components.
Code Listing 1: Query of V$SGA_DYNAMIC_COMPONENTS 
SQL> select component, current_size from v$sga_dynamic_components;

COMPONENT                          CURRENT_SIZE
-------------------------------     ------------
shared pool                         92274688
large pool                           8388608
java pool                            8388608
streams pool                        12582912
DEFAULT buffer cache                33554432
KEEP buffer cache                    4194304
RECYCLE buffer cache                 4194304
DEFAULT 2K buffer cache                    0
DEFAULT 4K buffer cache                    0
DEFAULT 8K buffer cache                    0
DEFAULT 16K buffer cache                   0
DEFAULT 32K buffer cache                   0
ASM Buffer Cache                           0

13 rows selected.


Using ASMM

As with the other manageability features of Oracle Database 10g, ASMM requires you to set theSTATISTICS_LEVEL parameter to at least TYPICAL (the default), so make sure this setting is correct before you try to enable ASMM. You can enable it in the following ways: 
  • Set SGA_TARGET to a nonzero value in the initialization parameter file (pfile).
  • Use Oracle Enterprise Manager (or Oracle Grid Control) or ALTER SYSTEM to dynamically set a nonzero value for SGA_TARGET in the server parameter file (spfile).
To use the command line ( ALTER SYSTEM ), set a value for SGA_TARGET and then set the parameters for all of the autotuned pools to 0, as follows: 
SQL> alter system set sga_target=160M scope=both;
System altered.

SQL> alter system set db_cache_size=0;
System altered.

SQL> alter system set shared_pool_size=0;
System altered.

<
                               repeat for each autotuned pool>
                            


If you don't set the values for each of the autotuned pools to zero after switching to ASMM, whatever value you had set for a parameter will function as a lower threshold value for that pool—ASMM won't go below a nonzero value, even if it needs the memory for another autotuned pool.
Rather than entering each of these ALTER SYSTEM commands for each of the autotuned pools as shown above, you can use Oracle Enterprise Manager to accomplish the same thing in one step, as part of the switch from Manual to Automatic tuning, by clicking the Enable button on the Memory Parameters page. This also shows you at a glance the various memory allocations at any time. To open the Memory Parameters page, from Database Control's Administration page, click the Advisor Central link (under the Related Links heading near the bottom of the page), and then click the Memory Advisor link.
The Memory Parameters page shows the memory allocations across all major components over time, since the last restart of the database, as shown in Figure 1. Lower down on the page you'll see a pie chart and a table displaying the Current Allocations. The chart shows the allocations as percentages of the total SGA that can be allocated, while the table shows the component and a value, in megabytes, of the allocation.
Figure 1: Memory Parameters page

Also on the Memory Parameter page, adjacent to the Total SGA Size field, is an Advice button—new in Oracle Database 10g Release 2. This lets you assess the impact of increasing (or decreasing) the system's target SGA. The v$sga_target_advice view also provides this information, as shown in Listing 2.
The v$sga_target_advice view gives you the information you need to modify theSGA_TARGET parameter (up to the size of the SGA_MAX_SIZE , if necessary). For the example in Listing 2, we can increase the SGA_TARGET to 200 and thereby reduce physical reads by about 10%.
Code Listing 2: Query of V$SGA_TARGET_ADVICE 
                               
SQL> select sga_size, sga_size_factor, estd_db_time, estd_db_time_factor, estd_physical_reads
 from v$sga_target_advice order by sga_size_factor;
 SGA_SIZE     SGA_SIZE_FACTOR     ESTD_DB_TIME      ESTD_DB_TIME_FACTOR    ESTD_PHYSICAL_READS
---------    -----------------   --------------    -------------------    -------------------
120           .75                421               1                      26042
160          1                   421               1                       8223
240          1.5                 421               1                       7340
280          1.75                421               1                       7340       
200          1.25                421               1                       7340
320          2                   421               1                       7340

6 rows selected.
                       

SHARED_POOL_SIZE Value in Oracle Database 10g 


In Oracle Database 10g, the shared pool size is the actual value of the parameter only—yet the overhead must still be accounted for in the shared pool. This means that if you're migrating to Oracle Database 10g from a prior release and you plan to manually size the various memory pools, you must make the shared_pool_size parameter value slightly larger in Oracle Database 10g than in your prior release to account for the startup overhead. In Oracle Database 10g, you can see precisely how much this startup overhead is by querying the v$sgainfodynamic view—you'll find a value for "Startup overhead in shared pool."If you're going to manually tune Oracle Database 10g SGA pools, note one key difference in how it interprets the shared pool value. In releases prior to Oracle Database 10g, the shared pool size was set equal to the shared_pool_sizeinitialization parameter value, plus the amount of internal SGA overhead—memory that the database allocated during startup to maintain state for various server components of the SGA. So if you set theshared_pool_sizeparameter to 48M in Oracle9i Database, for example, the actual shared pool size allocated by the database might have been 60MB, assuming a startup overhead amount of 12MB. 

Conclusion

It doesn't matter how much memory the system has if it's not properly allocated across the appropriate SGA pools at runtime. Yet since the system requirements can change from one minute to the next, these pools must be constantly adjusted. Oracle Database 10g's ASMM is a huge time-saver, and it also reduces the chances of ORA-04031 errors. The result? Your system will perform better—and you'll save time and trouble.


Oracle Automatic Shared Memory Management

Here’s a problem with ASSM that used to exist in older versions of Oracle had disappeared by 11.2.0.4 and then re-appeared in 12.1.0.2 – disappearing again by 12.2.0.1. It showed up on MoS a few days ago under the heading: “Insert is running long with more waits on db file sequential read”.
The obvious response to this heading is to question the number of indexes on the table – because big tables with lots of indexes tend to give you lots of random I/O as Oracle maintains the indexes – but this table had no indexes. The owner of the problem supplied several of bits of information in the initial post, with further material in response to follow-up questions, including the tkprof summary of the 10046/level 12 trace of the insert and two extracts from the trace file to show us some of the “db file sequential read” waits – the first extract made me wonder if there might be some issue involving 16KB blocks but the second one dispelled that illusion.
There are several buggy things that can appear with ASSM and large-scale DML operations, and sometimes the problems can appear long after the original had done the dirty deed, so I thought I’d create a simple model based on the information supplied to date – and discovered what the problem (probably) was. Here’s how it starts – I’ve created a tablespace using ASSM, and in this tablespace I’ve created a table which has 48 columns with a row length of 290 bytes (roughly matching the OP’s table), and I’ve hacked out a simple PL/SQL block that loops around inserting arrays of 100 rows at a time into the table for a total of 1M rows before committing.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
rem
rem     Script:         assm_cleanout.sql
rem     Author:         Jonathan Lewis
rem     Dated:          Jan 2018
rem     Purpose:
rem
rem     Last tested
rem             12.2.0.1
rem             12.1.0.2        Lots of blocks left "not full"
rem             11.2.0.4
rem
 
rem
rem     using OMF, so no file-name needed
rem     Ran this bit as SYS, and altered test user to have unlimited quota
rem
 
/*
create
        tablespace test_8k_assm
        datafile size 1G
        extent management local
        autoallocate
        segment space management auto
;
*/
 
rem
rem     Create the table, pre-allocate some space.
rem     This means we should get consistent 8M extents and not initial little ones
rem
 
create table t1 (
        v001 varchar2(5), v002 varchar2(5), v003 varchar2(5), v004 varchar2(5), v005 varchar2(5),
        v006 varchar2(5), v007 varchar2(5), v008 varchar2(5), v009 varchar2(5), v010 varchar2(5),
        v011 varchar2(5), v012 varchar2(5), v013 varchar2(5), v014 varchar2(5), v015 varchar2(5),
        v016 varchar2(5), v017 varchar2(5), v018 varchar2(5), v019 varchar2(5), v020 varchar2(5),
        v021 varchar2(5), v022 varchar2(5), v023 varchar2(5), v024 varchar2(5), v025 varchar2(5),
        v026 varchar2(5), v027 varchar2(5), v028 varchar2(5), v029 varchar2(5), v030 varchar2(5),
        v031 varchar2(5), v032 varchar2(5), v033 varchar2(5), v034 varchar2(5), v035 varchar2(5),
        v036 varchar2(5), v037 varchar2(5), v038 varchar2(5), v039 varchar2(5), v040 varchar2(5),
        v041 varchar2(5), v042 varchar2(5), v043 varchar2(5), v044 varchar2(5), v045 varchar2(5),
        v046 varchar2(5), v047 varchar2(5), v048 varchar2(5)
)
segment creation immediate
tablespace test_8k_assm
storage(initial 8M)
;
 
alter table t1 allocate extent (size 8M);
alter table t1 allocate extent (size 8M);
 
rem
rem     Simple anonymous pl/sql block
rem     Large insert, handled with array inserts
rem     Can modify loop count and array size very easily
rem
 
declare
        type tab_array is table of t1%rowtype;
        junk_array tab_array;
begin
 
        select
                'xxxxx', 'xxxxx', 'xxxxx', 'xxxxx', 'xxxxx', 'xxxxx', 'xxxxx', 'xxxxx', 'xxxxx', 'xxxxx',
                'xxxxx', 'xxxxx', 'xxxxx', 'xxxxx', 'xxxxx', 'xxxxx', 'xxxxx', 'xxxxx', 'xxxxx', 'xxxxx',
                'xxxxx', 'xxxxx', 'xxxxx', 'xxxxx', 'xxxxx', 'xxxxx', 'xxxxx', 'xxxxx', 'xxxxx', 'xxxxx',
                'xxxxx', 'xxxxx', 'xxxxx', 'xxxxx', 'xxxxx', 'xxxxx', 'xxxxx', 'xxxxx', 'xxxxx', 'xxxxx',
                'xxxxx', 'xxxxx', 'xxxxx', 'xxxxx', 'xxxxx', 'xxxxx', 'xxxxx', 'xxxxx'
        bulk collect into
                junk_array
        from
                all_objects
        where
                rownum  <= 100 -- > comment to avoid WordPress format issue
        ;
 
        for i in 1..10000 loop
                forall j in 1..junk_array.count
                        insert into t1 values junk_array(j) ;
        end loop;
 
end;
 
commit;
The number of rows per block after this insert is 24, with 1038 bytes free space left (808 due to the pctfree = 10, then the bit that was too small to take a 25th row before breaching the pctfree barrier). This means we should report 1M/24 = 41,666 full blocks and one block with some free space. So we query the table using the dbms_space package:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
declare
        m_unformatted_blocks    number;
        m_unformatted_bytes     number;
        m_fs1_blocks            number;
        m_fs1_bytes             number;
        m_fs2_blocks            number;
        m_fs2_bytes             number;
 
        m_fs3_blocks            number;
        m_fs3_bytes             number;
        m_fs4_blocks            number;
        m_fs4_bytes             number;
        m_full_blocks           number;
        m_full_bytes            number;
 
begin
        dbms_space.SPACE_USAGE(
                segment_owner           => 'TEST_USER',
                segment_name            => 'T1',
                segment_type            => 'TABLE',
                unformatted_blocks      => m_unformatted_blocks,
                unformatted_bytes       => m_unformatted_bytes,
                fs1_blocks              => m_fs1_blocks ,
                fs1_bytes               => m_fs1_bytes,
                fs2_blocks              => m_fs2_blocks,
                fs2_bytes               => m_fs2_bytes,
                fs3_blocks              => m_fs3_blocks,
                fs3_bytes               => m_fs3_bytes,
                fs4_blocks              => m_fs4_blocks,
                fs4_bytes               => m_fs4_bytes,
                full_blocks             => m_full_blocks,
                full_bytes              => m_full_bytes
        );
 
        dbms_output.new_line;
        dbms_output.put_line('Unformatted                   : ' || to_char(m_unformatted_blocks,'999,999,990') || ' / ' || to_char(m_unformatted_bytes,'999,999,999,990'));
        dbms_output.put_line('Freespace 1 (  0 -  25% free) : ' || to_char(m_fs1_blocks,'999,999,990') || ' / ' || to_char(m_fs1_bytes,'999,999,999,990'));
        dbms_output.put_line('Freespace 2 ( 25 -  50% free) : ' || to_char(m_fs2_blocks,'999,999,990') || ' / ' || to_char(m_fs2_bytes,'999,999,999,990'));
        dbms_output.put_line('Freespace 3 ( 50 -  75% free) : ' || to_char(m_fs3_blocks,'999,999,990') || ' / ' || to_char(m_fs3_bytes,'999,999,999,990'));
        dbms_output.put_line('Freespace 4 ( 75 - 100% free) : ' || to_char(m_fs4_blocks,'999,999,990') || ' / ' || to_char(m_fs4_bytes,'999,999,999,990'));
        dbms_output.put_line('Full                          : ' || to_char(m_full_blocks,'999,999,990') || ' / ' || to_char(m_full_bytes,'999,999,999,990'));
 
end;
/
The results aren’t what we expect:
1
2
3
4
5
6
Unformatted                   :            0 /                0
Freespace 1 (  0 -  25% free) :       35,001 /      286,728,192
Freespace 2 ( 25 -  50% free) :            1 /            8,192
Freespace 3 ( 50 -  75% free) :            0 /                0
Freespace 4 ( 75 - 100% free) :           67 /          548,864
Full                          :        6,665 /       54,599,680
We have one block marked as 25 – 50% free (that’s the one block with 16 rows in it, which means about 40% space currently free) but our 41,666 full blocks are actually reported as 6,665 full blocks and 35,001 blocks with some space available. That’s going to hurt eventually if some process wants to insert more rows and finds that it has to fail its way through 35,001 blocks before finding a block which has enough free space.
So what happens when I repeat the PL/SQL block (and commit)? Here are the results from calls to dbms_space after the next two cycles:
1
2
3
4
5
6
7
8
9
10
11
12
13
Unformatted                   :            0 /                0
Freespace 1 (  0 -  25% free) :       70,002 /      573,456,384
Freespace 2 ( 25 -  50% free) :            2 /           16,384
Freespace 3 ( 50 -  75% free) :            0 /                0
Freespace 4 ( 75 - 100% free) :          220 /        1,802,240
Full                          :       13,330 /      109,199,360
 
Unformatted                   :          256 /        2,097,152
Freespace 1 (  0 -  25% free) :      105,003 /      860,184,576
Freespace 2 ( 25 -  50% free) :            3 /           24,576
Freespace 3 ( 50 -  75% free) :            0 /                0
Freespace 4 ( 75 - 100% free) :          117 /          958,464
Full                          :       19,995 /      163,799,040
Every time we execute the PL/SQL block we leave a trail of 35,001 more blocks which are flagged as “not quite full”.
Looking at the session stats while running the insert loop I can tell that Oracle isn’t checking to see whether or not it should be using those blocks. (A quick way of proving this is to flush the buffer cache before each execution of the PL/SQL and note that Oracle doesn’t read back the 105,000 blocks before inserting any data). So somehow, sometime, someone might get a nasty surprise – and here’s one way that it might happen:
Since I know I my data fits 24 rows per block I’m going to modify my PL/SQL block to select one row into the array then loop round the insert 25 times – so I know I’m inserting a little bit more than one block’s worth of data. Starting from the state with 105,003 blocks marked as “Freespace 1” this is what I saw – first, the free space report after inserting 25 rows:
1
2
3
4
5
6
Unformatted                   :          240 /        1,966,080
Freespace 1 (  0 -  25% free) :        1,074 /        8,798,208
Freespace 2 ( 25 -  50% free) :            0 /                0
Freespace 3 ( 50 -  75% free) :            0 /                0
Freespace 4 ( 75 - 100% free) :          133 /        1,089,536
Full                          :      123,927 /    1,015,209,984
Then a few wait events and session statistics for the insert:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
---------------------------------------------------------
SID:    39:TEST_USER - jonathan
Session Events - 09-Jan 16:57:18
Interval:-      6 seconds
---------------------------------------------------------
Event                                             Waits   Time_outs        Csec    Avg Csec    Max Csec
-----                                             -----   ---------        ----    --------    --------
db file sequential read                          15,308           0         128        .008           3
db file scattered read                           20,086           0         271        .014           4
 
---------------------------------
Session stats - 09-Jan 16:57:18
Interval:-  6 seconds
---------------------------------
Name                                                                     Value
----                                                                     -----
session logical reads                                                  269,537
physical read total IO requests                                         35,401
db block gets                                                          229,522
consistent gets                                                         40,015
physical reads                                                         124,687
physical reads cache                                                   124,687
db block changes                                                       208,489
physical reads cache prefetch                                           89,293
redo entries                                                           207,892
redo size                                                           16,262,724
undo change vector size                                                  1,720
deferred (CURRENT) block cleanout applications                         103,932
table scan blocks gotten                                                20,797
HSC Heap Segment Block Changes                                              25
The session has read and updated almost all of the level 1 bitmap blocks. I don’t know exactly what triggered this re-read, but seems to be related to the number of rows inserted (or, perhaps, the amount of space used rather than the row count) as an insert crosses the pctfree boundary and fails over to the next block. I’ve only done a couple of little tests to try and get a better idea of why an insert sometimes sweeps through the bitmap blocks – so I know that inserting 2 or 3 rows at a time will also trigger the cleanout – but there are probably several little details involved that need to be identified.
You might note a couple of details in the stats:
  • Because I had flushed the buffer cache before the insert Oracle did its “cache warmup” tablescanning trick – if this had not happened I would probably have done a single block read for every single bitmap block I touched.
  • There are 103,932 block cleanout applications – but 208,000 db block changes and redo entries. Roughly half the latter are for data block cleanouts (OP code 4.1) and half are the state changes on the level 1 bitmap blocks (OP code 13.22). You’ll notice that neither change produces any undo.
  • I’ve also included the HSC Heap Segment Block Changes statistics to show you that not all changes to Heap Segment Blocks show up where you might expect them.

Finally:

If you re-run the tests on 11.2.0.4 and 12.2.0.1 you get the following results after the intial script run – the problem doesn’t exist:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
11.2.0.4
========
Unformatted                   :            0 /                0
Freespace 1 (  0 -  25% free) :            0 /                0
Freespace 2 ( 25 -  50% free) :            1 /            8,192
Freespace 3 ( 50 -  75% free) :            0 /                0
Freespace 4 ( 75 - 100% free) :           67 /          548,864
Full                          :       41,666 /      341,327,872
 
12.2.0.1
========
Unformatted                   :            0 /                0
Freespace 1 (  0 -  25% free) :            0 /                0
Freespace 2 ( 25 -  50% free) :            1 /            8,192
Freespace 3 ( 50 -  75% free) :            0 /                0
Freespace 4 ( 75 - 100% free) :           67 /          548,864
Full                          :       41,666 /      341,327,872

So how does this help the OP.

  • First, there may be a huge mess still waiting to be cleaned in the table – but at 34M blocks I’m not too keen to suggest running the dbms_space routine to find out what it looks like – but maybe that’s necessary.
  • Secondly – an upgrade to 12.2 will probably avoid the problem in future.
  • Thirdly – if the number of rows per block is very close to uniform, write a little code to do a loop that inserts (say) 2 * expected number of rows per block as single row inserts and rolls back; the inserts will probably trigger a massive though perhaps not complete cleanout, so rinse and repeat until the cleanout is complete. Try to find a time when you don’t mind the extra load to get this job done.
  • Finally – on the big job that does the bulk insert – repeat the dummy insert/rollback at the end of the job to clean up the mess made by the job.

Add on:

Prompted by comment #2 below, I should add that if the problem has been fixed in 12.2 then possibly there’s a bug report and patch for it already. If there isn’t then the OP could raise an SR (referencing this blog note), and request a bug fix or back-port from 12.2.   
                                                                                                         Thanks , Jonathan Lewis