Extending AbstractLoadTransactionsRunMode Extending PlaybackRunMode

FAQTroubleshooting 23-5

23.1.4.5 Develop Possible Causes

Problem solving involves developing possible causes.

23.1.4.6 Test Each Candidate Cause Against the Specification

Test each candidate cause against the specification: ■ Each possible cause must fit all the items in the specification ■ If you end up with no causes then go back and refine the process ■ Causes must explain both the IS and the IS not but COULD be ■ Determine the most probable cause ■ Do not discount any causes that fit

23.1.4.7 Confirm the Cause

Confirm the cause so that you can devise an action plan. You can: ■ Devise ways to test the possible causes ■ Observe ■ Test assumptions Provide comparison A test case often is the key to establishing something to compare the problem with. If it reproduces the issue then it does not help the problem analysis as such, but it is extremely useful when passing the problem to the next team to work on the fix. It also allows quicker testing of potential fixes and solutions workarounds, not to mention you would be gaining experience. If there is no comparison, create a test case If it does not reproduce then it provides something to compare the problem system with and perhaps even a possible work around. Development Description Knowledge and experience You can use your knowledge and experience to recognize possible causes ■ Seen before ■ Seen it in the documentation ■ Support note or through search engine Distinctions and changes You can make a list of distinctions and changes to narrow down causes: ■ Only at this site or on one platform ■ Just after upgrade ■ When load increased ■ Only on Thursdays Examine each of the symptoms and comparisons Consider each of the facts and ensure that they are relevant and that they are not conflicting Step Description 23-6 Oracle Fusion Middleware Developers Guide for Oracle Adaptive Access Manager ■ Experiment ■ Test solution and monitor The main point here is to devise action plans to prove or disprove the theories. It is important to communicate the reason for each action plan. Especially when asking for a negative test, i.e. a test that is to prove something is not true. People might assume all action plans are attempts to solve the problem and resist any thing they think is not directed in the direction.

23.1.4.8 Failures

When one solution fails, just start back at the beginning and apply the approach once again, updated with the new results. Really complex problems will often take several iterations. The process is not infallible. Main causes of failure are: ■ Poor or incorrect problem statement ■ Inaccurate or vague information ■ Missing the key distinctions in IS vs. IS NOT ■ Allowing assumptions to distort judgment ■ Not involving a broader set of skills

23.2 Troubleshooting Tools

This section contains information about tools and processes you can use to investigate and troubleshoot issues with your system. Table 23–1 lists the general and OAAM-specific tools you can use for troubleshooting problems. Table 23–2 provides items to check for when troubleshooting the system. Table 23–1 Troubleshooting Tools Category Description General Tools ■ Middleware Enterprise Manager ■ Database Enterprise Manager ■ Monitor Data in DMS ■ Audit Data ■ PingNetwork Check Tools OAAM Specific Tools ■ Dashboard ■ Monitor Data ■ Log files FAQTroubleshooting 23-7 Table 23–3 summarizes problems and the checks you can perform to troubleshoot and solve the problem. Table 23–2 Troubleshooting Tips Tips Reason Check the operating system Some issues may be platform specific. For example, Java keystores created on non-IBM platforms will not work on IBM platforms Check WebLogic Server version Make sure OAAM is installed on a WebLogic server certified for 11g Check the JDK Sun or JRockit Make sure the JDK is certified for the Identity Management 11g Suite Change logging configuration through Enterprise Manager Make sure the log level is changed appropriately before tracing and debugging Search for log messages through Enterprise Manager Log messages record information you deem useful or important to know about how a script executes. Use the Execution Context ID to search for log messages The ECID is a unique identifier that can be used to correlate individual events as being part of the same request execution flow. Use the WebLogic Console to monitor database connection pool Check the health of the connection pool through the WebLogic Console. 23-8 Oracle Fusion Middleware Developers Guide for Oracle Adaptive Access Manager Table 23–3 Problems and Tips Problem Checks You Can Perform Common Troubleshooting Use Cases ■ Most of the operations are slow ■ Server is throwing out of memory exceptions ■ Server is throwing encryption related exceptions ■ Connection pool related errors occur when starting the server ■ Errors while starting managed servers after upgrade from 11.1.1.4 to 11.1.1.5 ■ OAAM CLI script issues ■ SOAP call issues ■ Native integration issues Most of the Operations are Slow ■ Check performance of OAAM policies – Use the dashboard to see the performance of the rules – Tune rules or their parameterd if necessary ■ Check the database using Enterprise Manager and see if there are any queries that are slow. Follow Enterprise Manager recommendation to add suggested indexes ■ Check if the application server CPU is high Take a thread dump if possible ■ Check the connectivity and network speed between application server and database ■ Use the IP of the database machine in data source settings Server is Throwing Out of Memory Exceptions ■ Check the configuration of the OAAM WebLogic Domain ■ See if all the OAAM web applications are deployed on the same managed servers ■ Increase the heap size of the managed server Connection Pool Errors ■ Make sure the database listener is running ■ Use IP address rather than name in JDBC URL ■ Make sure the database service name is correct ■ Make sure the connection pool is not too large Check if there are too many managed servers accessing the same database Errors While Starting the Managed Server After Upgrade ■ Make sure encryption keys are properly copied ■ Make sure all manual steps are followed that are in the upgrade documentation ■ Check the WebLogic Console and make sure all web applications are targeted properly to their managed servers