Troubleshooting ORA-3137

Oracle Customer Support has observed a large volume of issues reported by customers experiencing ORA-3137 [12333] and occasionally ORA-3137 [1010] errors when connecting to an Oracle RDBMS using the Oracle JDBC driver.  This document enumerates troubleshooting approaches that have proven successful.  

Note that while this problem may be encountered when making a JDBC connection to the database, it is rare that the problem actually comes from the JDBC driver itself.  Instead, the problem usually is caused from one of a small number of database bugs.

TROUBLESHOOTING STEPS

Known Database Issues

Unpublished Bug 8625762 - ORA-3137 [12333] due to bind data not read from wire

This bug affects RDBMS version lower than version 11.2.  It is fixed in version 11.2 of the database.

It can occur intermittently, due to bind data being left unread database connection.  This is a bind peeking bug.

This bug is also known for causing
ORA-3137: TTC protocol internal error : [1010] [] [] [] [] [] [] []

Unpublished Bug 9703463 - ORA-3137 [12333] or ORA-600 [kpobav-1] When Using Bind Peeking

This bug affects versions 11.1.0.6, 11.1.0.7, and 11.2.0.1 of the RDBMS.  It is fixed in version 11.2.0.2 of the database.

It can also occur intermittently; similarly to unpublished Bug:8625762, this is a bind peeking bug.

Unpublished Bug:9243912 - Additional diagnostics for ORA-3137 [12333] / OERI:12333

This bug affects versions of the RDBMS lower than version 11.2.0.2.  It is fixed in version 11.2.0.2 of the database.

This bug addresses scenarios in which the ORA-3137 [12333] error occur bind issues during the client/server communication process.  When this particular bug is fixed, the result may be the uncovering of an additional, more descriptive error:  ORA-600 [kpobav-1]

Bug 7390077  ORA-03137: TTC PROTO INTERNAL ERROR: [12333] [8] [80] [65]

This bug affects the RDBMS 11.1.0.6 and is resolved with RDBMS 11.1.0.7.

typical errors:
ORA-03137: TTC protocol internal error : [12333] [10] [83] [74] [] [] [] []
ORA-03137: TTC protocol internal error : [12333] [7] [120] [108] [] [] [] []

example stack:

ORA-03137: TTC protocol internal error : [12333] [7] [120] [108] [] [] [] []
----- Current SQL Statement for this session (sql_id=c0czhbm4hcz98) -----
DELETE FROM PORTAL_CONCURRENT_USER_SCRATCH
WHERE LAST_LOGIN < :intervalStartDate


----- Call Stack Trace -----
calling call entry argument values in hex
location type point (? means dubious value)
-------------------- -------- -------------------- ----------------------------
skdstdst()+002c bl 105e2771c
ksedst1()+0064 bl 101fad134
ksedst()+0028 bl ksedst1() 60000B8104C4A30 ?
600000002050032 ?
dbkedDefDump()+07fc bl 101fae954
ksedmp()+0048 bl 101fad130
opiierr()+0168 bl ksedmp() 000000000 ?
opitsk()+1344 bl 103ec9a5c
opiino()+09a0 bl opitsk() 110098B40 ? 000000000 ?
opiodr()+0b48 bl 103ec7a30
opidrv()+0440 bl opiodr() 3C61672F72 ? 41032E668 ?

This problem does not happen if the JDBC 10.2.0.3 driver is used but is nevertheless considered a database bug.

The following bugs have been marked as potential duplicates of this case:
Bug 7573690 Bug 7607470 Bug 8490909

Unpublished Bug 9373370  - Data Base Returns Wrong CURSORID When There Is an ORA-1013

This bug affects the 10.2, 11.1, 11.2.0.1, and 11.2.0.2 databases.  It is discussed in the following notes:

Note:9373370.8 Bug 9373370 - The wrong cursor may be executed by JDBC thin following a query timeout / ORA-3137 [12333]
Note:9967872.8 Bug 9967872 - The wrong cursor may be executed by JDBC thin following a query timeout

While the bug primarily manifests in ORA-1006 or ORA-1008 errors, the problem may also result in ORA-600 [12333] or ORA-3137 [12333] errors appearing on the server side.

Unpublished Bug 18841764 - ORA-12592 TNS:BAD PACKET OR ORA-3137 OR ORA-1 PDIT_BAS_SCPBI

This bug affects the 12.1.0.2 database. It is discussed in the following notes:

Note:18841764.8 Bug 18841764 - Network related error like ORA-12592 or ORA-3137 or ORA-3106 may be signaled
Note:1905674.1 Occasional ORA-12592 (TNS:BAD PACKET) and ORA-03137 [12333] When Running SQL*Loader Jobs

This bug can occur when using SQL Loader.  The problem may result in ORA-12592 or ORA-3137 [12333] errors.

Known JDBC Issues

Bug 9445675  NO MORE DATA TO READ FROM SOCKET WHEN USING END-TO-END METRICS

This bug does affect the JDBC driver.  This bug may be the cause when all of the following conditions are met:

1. You are using the 10.1.x.x or the 11.2.0.1 JDBC driver; the bug does not affect 10.2.x.x, or 11.1.x.x versions of the driver, nor versions 11.2.0.2 or above
2. You are using end-to-end metrics in your Java code
3. The server side ORA-3137 [12333] error is accompanied by the client side Java exception "No more data to read from socket"

This bug is fixed in the 11.2.0.2 version of the JDBC driver and above.  It is discussed in the following notes:

Note 9445675.8 Bug 9445675 - "No more data" / ORA-3137 using end to end metrics with JDBC Thin

Note 1081275.1 "java.sql.SQLRecoverableException: No more data to read from socket" is Thrown When End-to-end Metrics is Used

SQLException: Protocol Violation and ORA-3137 [12333] when using JDBC 12.1.0.1

Application side:

SQLException: Protocol Violation: [14, 118]

SQLException: Protocol Violation: [111, ]

SQLException: Protocol Violation: [110, ]

Database side:
*** MODULE NAME:(JDBC Thin Client) 2017-07-27 09:51:55.150
*** CLIENT DRIVER:(jdbcthin) 2017-07-27 09:51:55.150
*** ACTION NAME:() 2017-07-27 09:51:55.150

--- PROTOCOL VIOLATION DETECTED ---
----- Dump Cursor sql_id=gxx1qcjkkaw20 xsc=0x7fe882624238 cur=0x7fe882d82108 -----

ORA-03137: TTC protocol internal error : [12333] [7] [3] [94] [] [] [] []

This is resolved by upgrading the JDBC from 12.1.0.1 to 12.1.0.2.

Troubleshooting Approaches

1.  Disable Bind Peeking

A common thread through the database bugs listed above is that the problem does not occur if bind peeking is shut off.  Temporarily disabling bind peeking allows you to confirm whether this is the case in your environment as well.

This can be done dynamically:

SQL> alter system set "_optim_peek_user_binds"=false;

If the ORA-3137 [12333] error no longer reproduces after disabling bind peeking, then in most cases, the problem is due to one of the database bugs listed above, or due to some variant of one of those bugs.

Note:  disabling bind peeking can have an impact on the database.  Specifically, it can alter the explain plan for some queries that use bind variables.  This should be taken into account, particularly before considering the use of this option as a long-term workaround.

2.  Apply Relevant Database Patches

Whenever possible, the recommended solution when running a database version 11.1.0.7 is to apply the most recent PSU patch.
PSU 11.1.0.7.8 Patch 12419384 includes Patch:9703463.

For 11.1.0.7, Patch 9243912 can be applied on top of 11.1.0.7.8 ( no conflict).

Patch:9703463  can also be applied individually but requires PSU 11.1.0.7.6

Patch:8625762  may also be applicable to databases version 11.1.0.7

If you have disabled bind peeking as a test, and the error no longer reproduces, then applying one of these patches would be the next step.

Alternatively, if you are unable to disable bind peeking for some reason, you may consider applying one of these patches as a diagnostic test in itself, to confirm whether the patch fixes the problem for you.

Patch:18841764 fixes the SQL Loader issue that may affect database version 12.1.0.2.

3.  Upgrade the Database to Version 11.2.0.3

Many of the above bugs are fixed in version 11.2.0.3 of the database.

4.  Contact Oracle Support if the problem still persists

For JDBC connections that have resulted in the ORA-3137 error, the above database-centric approaches have resolved the issue almost all of the time.  If disabling bind peeking has no effect and the ORA-3137 [12333] still occurs, please contact Oracle Support, as this is a relatively rare scenario in which further investigation is needed.

Reference  -Troubleshooting ORA-3137 [12333] Errors Encountered When Using Oracle JDBC Driver (Doc ID 1361107.1)