Oracle apps dba Interview Q&A's
Oracle apps dba Interview Q&A's:
===========================
What is the significance of utl_file_dir parameter in init.ora file?
=====================================================================
Answer The value of this parameter is the group of directories to which u r database can write,
means u r database packages have permission to write to flat files in these directories.
What is APPLPTMP environment variable?
=======================================
Answer This is the temporary file location for the pl/sql temp files.
If this variable was not set then the concurrent programs may error-ed out.
DIAG_PARAM ="Y" in adcmctl.sh. for CM issues
oracle apps concurrent managers interview question and common information's
Question:Error:
================
After submitting the concurrent program, i am getting the status as inactive and no manager and when i clicked on diagnostics button its showing a message like
This request will not be processed because there are currently no managers running that can process it.
This is an error condition. There are one or more managers defined that can process this request that should be running at this time, but do not appear to be running.
Contact your system administrator and/or check the manager log files for errors.
Solution:
=========
to resolve that, just u simple startup concurrent manager by executing the ./adcmctl.sh script
Question:
=========
want to create a new “MANAGER”
Ans:
====
sysadmin guide for that
http://download-uk.oracle.com/docs/cd/B25516_11/current/html/docset.html
To figure out the issue.
1) Check your CRM is showing any jobs in conflicts.
2) Check for locks.
3)280295.1 REQUESTS.sql Script for Parent/Child Request IDs and Trace File IDs
ANALYZEPENDING.SQL - Analyze all Pending Requests Doc ID: Note:134033.1
Question :
===========
problem is My technical consultant has submitted two concurrent requests. But it is long time running. the status is running. There is nothing to find in the log file. Where can i know what is exact problem
Solution
========
are the concurrent requests are custom or default concurrent programs....did you check the concurrent managers status.....
(a) check the concurrent managers status
(b) check the concurrent manager logfiles
if the concurrent requests are custom...i mean....if it is the report based...check the report query and tune the query wether custom tables are having indexes or not.......check the schema statistics of apps schema
2:
Try enabling trace on the particular concurrent request,may be that should help to diagnose the problem?
Question:
=========
What is environment variable APPLCSF in Apps environment?
Ans:
==
If environment variable APPLCSF is set then u can
get the logfiles of concurrent managers under $APPLCSF/$APPLLOG.
If APPLCSF is not set, Request Log files will go to
$<MODULE>_TOP/$APPLLOG
and output files will go to
$<MODULE>_TOP/$APPLOUT
Here <MODULE_TOP> is GL_TOP, AP_TOP, AR_TOP etc etc.
QUESTION:
=========
How and why to configure PCP(Parallel Concurrent Processing)?
Answer:
========
PCP is nothing but a load balancing methodology for concurrent processing server. To load balance on concurrent processing server, this is required. In a multinode architecture, it is being used.
Check the following metalink notes
How to Set Up Parallel Concurrent Processing (PCP) in Apps 11i?
Doc ID: Note:388495.1
Oracle E-Business Suite Release 11i with 9i RAC: Installation and Configuration using AutoConfig
Doc ID: Note:279956.1
Question:
=========
CM is not working after cloning?
Answer
======
1.Bounce CM
2.check FND_NODES correctlly updated for concurent manager
3.if is there custom tops ..if so means...add those to any env file..after that set env
then Bounce the concurrnet managers..
Question:
=========
Concurrent manager "Inactive, No Manager"
1. We are facing this problem even when no other job is running
2. We dont face this problem on daily basis- it can happen sometimes thrice in a day and sometimes once in a month.
3. This program generally runs atleast 10 times in a day.
4. Configuration of the concurrent manager is (this is not standard manager):
No. of processes is: 8, Cache Size: 10, Sleep time: 10
Answer:
=========
sol_1
====
This problem seems to be inactive ICM.
When ICM is not up u will get error inactive-no manager.
Just check with manager and make it up.
ps -ef|grep LIBR
if it shows the manager is down then start the CM using script adcmctl.sh.
And if the manager is up but it is showing the inactive-nomanager then check for the log file and do the needful.
As my idea y dont u try with cmclean.sql.(not supported by oracle),Check in UAT before trying in prod.
1.Bring down CM.
2.run cmclean.sql
3.Bring up CM.
Sol_2:
======
Most importantly, see try reassigning the request to a different manager instead of the Standard Manager (maybe a custom one which only it belongs to) and see if you get the same behavior. Also check to see there are no unnecessary incompatibilities listed for the Program under the Program definitions.
sol_3:
========
I'd like to add that we checked the work schedule which is set to standard (24X7) and specialised rules against the standard Concurrent manager in the define manager screen. In adition we recreated the Conc mgr views, also created our own manager which gave the same message Phase Inactive No Manager.
Sol_4:
=======
What is the value of your GSM Enabled profile setting?
We had it set at Y but since changed it to N. We have also changed the appldcp context value to NO. Nothing seems to make any difference
Sol_5:
=======
after cloning you are getting inactive manager
go to applmgr user
$TNS_ADMIN / path
change the tns file as
edit
(SID=FNDSM) to (SID=FNDSM_VIS)
Sol_6:
=====
check the disc space of this instance, and increase
QUESTION:
=========
What are the differences between the functionality of purging concurrent request, FNDCPPUR and CMCLEAN.
Answer:
=======
FNDCPPUR is executable name of Purge Concurrent Request and/or Manager Data Program.
When you run this request, it will delete all the requests from Tables and log, req and manager log files from OS leve.
cmclean.sql is used to clean the concurrent manager tables, when there is any problem with managers starting up..
for more check the following note
CMCLEAN.SQL - Non Destructive Script to Clean Concurrent Manager Tables
Doc ID: Note:134007.1
Question:
=======
If ICM stop the othar manager ST which are already
started will continue but the new manager will not start
At the same time confict resolution manager will pickup all the request
How to know the ICM is down
Answer:
========
FNDSM is ICMs agent which will be doing the job for ICM(starting stoppin managers etc..).
if that goes down everything else related to concurrent processing shuts off.
So you could grep for this FNDSM process continously at OS level to check ICMs status using
a script
LOCKING:
========
Check V$session and V$locked_object. You can formulate a query by joining these two views to get the required info..Do a describe on them and try to create a query.
>How to resolve locking
using the two views mentioned above you can find out which session is blockin which sessions.If the culprit session is INACTIVE, then you may kill it using alter system kill session command. That would release the blockage
Question:
=========
what is PMON cyle,QUEUE size and sleep time:
ANS:
===
While the basic ICM definition should not be changed, you can if required modify the sleep time
(number of seconds the ICM waits between checking for new concurrent requests),
PMON (process monitor) cycle time (number of sleep cycles the ICM waits between checking for
failed workers), and queue size (duration between checks for number of active workers,
measured in PMON cycles). If Parallel Concurrent Processing (described below) is being used,
you can also set some options for this.
what does these failed "workers" refer to?? What are these workers?
A/These workers are nothing but the Operating System processes associated with the concurrent
managers.
Question:
==========
Reasons when concurrent manager fail?
In concurrent manager -> administer
Actual /target values different? Wat do you mean by it?
Answer:
=========
Reasons when concurrent manager fail?
There can be N number of reason for CM failure , check CM log file for exact message and cause
Few of them are --
Database listener down,
wrong password while starting CM,
corruption in FND_NODES ...
In concurrent manager -> administer Actual /target values different?
Wat do you mean by it?
Actual is actual number of manager running at that time and target is the number of Manager should be
running at that particular time.
Qestion:
========
How to check the CM is up?
Answer:
========
ps -ef|grep FNDLIBR
Question:
========
which concurrent manager will run, which concurrent programs?
ANS_!:
========
Basically there are four types of Conc Mgrs
1 Internal Concurrent Manager
2 Conflict resoluction manager
3 Standard Manager
4 Transactional Managers
ICM - monitors other managers
SM picks up all the requests which or not there in it's exclusion list
Each transaction manager can process only the programs contained in its program library.
In the Concurrent Program Definition there is a column called REQUEST_TYPE by which we can assign the program to a particular manager.
You can use include and exclude options for the managers depending upon the your requirement if you want the program to be run by only one user.
You can navigate to Administer Concurrent Managers from System Adminsrator reqponsiblity, select maanger and view it's requests to see what all requests it is running.
Document which gives you more details is "System Administrator’s Guide"
Ans_2
=====
you can know this by navigate to concurrent>>manager>>define and there just query for that manager what you search for. There , the manager has been defined that it will run which requests or programs.
ANS_3:
========
From concurrent manager define and administrator screen we can find out what are all the concurrent programs that manager is running.
We can find out currently running request and which manager queue this request is running from the script $FND_TOP/sql/afrqrun.sql.
But my question is, I want general script or query to find from back end,
Which request (completed, errored, schedule, running etc) goes to which manager?
ANS_4:
=======
Use this query. Provide short name as concurrent_program_name.
SELECT A.INCLUDE_FLAG, A.QUEUE_APPLICATION_ID, C.USER_CONCURRENT_QUEUE_NAME,a
B.CONCURRENT_PROGRAM_NAME
FROM APPLSYS.FND_CONCURRENT_QUEUE_CONTENT A, APPLSYS.FND_CONCURRENT_PROGRAMS B, APPS.FND_CONCURRENT_QUEUES_VL C
WHERE type_id = b.concurrent_program_id and b.concurrent_program_name = '&SHORT_NAME' and c.concurrent_queue_id = a.concurrent_queue_id
/
Output will show you Which program is excluded from standard manager and included in other manager.
Question:
=========
What to do when Concurrent request is queing up and not being completed.
Did u started the concurrent manager...
ANSWER:
=========
A/Start the conc manger .. $COMMON_TOP/admin/scripts/adcmctl.sh start apps/apps
But still it not complited
B/Are there any errors in the Standard Manager log?
$APPLCSF/log/<Context Name>/FNDSM9999.mgr
Question
==========
What cmclean.sql will do?
Answer:
=========
cmclean.sql is non destructive script to clean CM tables. To run this script, CM has to be down since it does some updates on the tables. The script resets the flags of the requests to 'COMPLETED' and clean the CM tables so allowing the concurrent managers to come up. You need to run this script whenever you have an issue with the CM startup.
EXEC FND_CONC_CLONE.SETUP_CLEAN;commit ==> to clean fnd_tables
SQL> select NODE_NAME,SERVER_ADDRESS from fnd_nodes;
DESCRIPTION
Concurrent managers cleanup script.
To be called by rapidinstall ONLY.
- Delete all site specific information (nodes, services, printers, etc).
- Delete requests that are in Completed and Running states.
- Delete/reset all concurrent manager state information.
- Cancel all Pending Oracle Reports based requests.
Question:
========
What is CONCSUB
Ans:
=====
CONCSUB is a utility run at the operating system level to submit a concurrent program to the concurrent manager.
For more check the metalink note...Using The CONCSUB Utility
Doc ID: Note:9611.1
Question:
=========
How to purge concurrent request espcially those who has .req and .log extension name on OS side, is it safe to purge concurrent request?
Answer:
========
A/To avoid running out of space on your disk drives, you should periodically
delete Oracle Applications log files, output files and purge these tables
with the FNDCPPUR program (Purge Concurrent Request and/or Manager Data).
The FNDCPUR purge program maintains the number of log and output files the operating system retains, and manage tables that store information about concurrent requests and concurrent manager processes.
You can run the program FNDCPPUR once and automatically resubmit the program for your specific time intervals.
Depending on the usage you might have a different strategy for purging the concurrent requests log/out files eg.
- every 30 days for normal usage
- every two weeks (14 days) for heavy usage
- if you are using the AGE mode, set the Mode Value to 7 or higher to retain the most recent days of concurrent request data, log files, and report output files.
WARNING : When you purge concurrent request information, you lose audit details which are used by the Signon Audit Concurrent Requests report.
If your business requires to see the requests run back in time then you need to keep archive on disk or tape.
B/The log and out files go to a common location for all modules.
The concurrent manager first looks for the environment variable
$APPLCSF. If this is set, it creates a path using two other
environment variables: $APPLLOG and $APPLOUT
It places log files in $APPLCSF/$APPLLOG, output files go in
$APPLCSF/$APPLOUT
So for example, if you have this environment set:
$APPLCSF = /u01/appl/common
$APPLLOG = log
$APPLOUT = out
The concurrent manager will place log files in /u01/appl/common/log,
and output files in /u01/appl/common/out
Note that $APPLCSF must be a full, absolute path, and the other two
are directory names.
If $APPLCSF is not set, it places the files under the product top of
the application associated with the request. For example, a PO report
would go under $PO_TOP/$APPLLOG and $PO_TOP/$APPLOUT
Logfiles go to: /u01/appl/po/9.0/log
Output files to: /u01/appl/po/9.0/out
All these directories must exist and have the correct permissions.
Note that all concurrent requests produce a log file, but not necessarily
an output file.
Concurrent manager logfiles follow the same convention, and will be
found in the $APPLLOG directory.
Use the find command to move files which are more than N days old to another directory and zip them up or archive to tape. You can also use utilities like TSM (Trivoli Storage Manager) for backing up the filesystem.
eg. find $APPLCSF/$APPLLOG -type f -mtime +45|xargs -i mv -f {} /tmp/45more_log
Question:
=========
15 custom CM's , in which one is down . so what is the way to troubleshoot it to bring it up.
Answer:
=====
you can go with anyone:
1. 1st check the log file of the cm. if your error message is not enough to find out the problem, then change DIAG_PARAM ="" to DIAG_PARAM ="Y" in adcmctl.sh.
2.you can use cmclean.sql and then check the status of that cm.
3. check the values of target and actual column for that cm in frontend.
Question:
===========
CM is down Status Inactive and No Manager?
Ans:
===
1Check in icm log file to see issue
$APPLCSF/$APPLLOG/<SID>_<timestamp>.log Check in icm log file to see issue
$APPLCSF/$APPLLOG/<SID>_<timestamp>.log
Ans_2:
======
This is way too big of a question to be answered in one post but in general:
1. Ensure you have enough Standard Manager processes to handle all of the requests. The default number is 3, this can be increased in the Manager Definitions (Workshift).
2. Ensure that the number of rows in FND_CONCURRENT_PROCESSES and FND_CONCURRENT_REQUESTS don't exceed 50K or there will be performance issues.
3. Ensure that you don't have any unnecessary manager processes running in the manager queues.
4. Set a low priority if possible on the jobs which generally run the longest. If you are consistently getting slow performance on jobs, check to see if they also execute slowly from the SQL PLUS. If they do, the issue is with the code, not the managers.
DBA_EXTENTS or DBA_OBJECTS Table..there you can get the SIZE of the Objects
Question:
=========
submitted a request and it completed with the status as completed normal
But not able to see log file?
Answer:
========
A, RRA profile agaent & apps listener adalnctl.sh
B, whatever you have operating system , you need to restart the listener again even if it's working status and if the users are connected to the Apps. then you have to restart the Apatche first then the listener of the application 8.0.6.
Question:
========
DB tier and Concurrent Manager on one node
Apps Tier(Forms, Apache) on another node.
How to move the Concurrent Manager to a new node
Ans:
====
Theoretically, you'll need to add a node to the Apps Tier using the Rapid Clone addnode.pl
This should give you an overall idea.You'll need to tweak this to your requirement.
So, 1) you run preclone on DB node, CM node, Web/Forms node
2) Deregister existing CM node
3) Disable security option, run autoconfig on DB node, restart listener
4) Copy files to the new node
5) Run post clone on DB tier
6) Run post clone on Web/Forms Tier
7) Run post clone on new node to configure for CM node
8 ) Run addclone.pl on Apps tier.
How do I deregister an application tier from the Net Services Topology Data Model?
Answer:
To deregister the current application tier from the Net Services Topology Data Model, invoke the following command:
perl <AD_TOP>/bin/adgentns.pl appspass=<APPSpwd> contextfile=<CONTEXT> -removeserver
Refer Note: Note:218089.1 Autoconfig FAQ
Regarding step ( 8 ) : This would need to be run from Forms/Web node I believe. Not familiar with it's prompts, but it should be straightforward as we have the NEW CM node already ready by the time we execute this. Please ensure that once this is run, the entries in the context file on this node reflect your new node. I would check entries in the following section in context XML
<oa_admin_server>
<oa_cp_server>
For addnode.pl refer to note: Note:230672.1 Cloning Oracle Applications Release 11i
After everything is done, I would also check FND tables:
FND_NODES
FND_APP_SERVERS
The entries here should match with what we needed to start with.
1) Entry for DB node
2) Entry for Forms/Web Node
3) Entry for *NEW* CM node
4) No entries for *OLD* CM node
I hope you are doing this on test first? Let me know how it goes.
Question:
========
Want to configur PCP?
Ans:
===
241370.1 Concurrent Manager Setup and Configuration Requirements in an 11i RAC Environment
312731.1 Configuring Oracle Applications Release 11i with
10g Release1 Real Application Clusters and Automatic Storage Management
Section 3.11 Configuring Parallel Concurrent Processing
Question:
=========
Why only for Concurrent Manager, we specify apps/apps password & not for other?
Ans:
======
Concurrent Manager is used for processing requests..it means...for example...I want to run one report based on Current Month GL Balance...So we have to run one report to show balance...this balance will be stored in the Database tables..right.....
Concurrent Manager runs this request by connecting to the database using apps user password.....
Got it..hope it is clear.....'
Question:
========
which situations we need to bounce the concurrent manager ? and application services ?
Ans:
====
doc id: 105133.1(given in teachme oracle-Metalink Notes Section).
When you modify the Printer Driver you have to restart the Manager which runs the request which is attached to that Printer Driver, however,if you do not know which manager then you have to restart the Internal manager because the printer driver can be used by multiple managers and multiple requests. If only a concurrent program definition is modified, running a verify on the Internal Manager will pick up the changes without the need for bouncing the manager.
QUESTION:
==========
I have defined a new manager and submitted request in it.
But the job is always running and it never stops.
ANS:
====
issue initially the process specified for the new ASAP manager is below 10.
But as per the thumb rule for that specified application and the manager
the process should be more than 20.Thumb rule for the manager which we created was
thumbrule= 4 + (2 * profile value).
According to the above thumbrule the intial pocess of 8 was increased to 25 then the manager runs normally.
q/a
Below is the relevant content from adcmctl.sh:
printf "Shutting down concurrent managers for $TWO_TASK ...\n" >> $LOGFILE
$FND_TOP/bin/CONCSUB "$unpw" SYSADMIN $WAIT_PARAM 'System Administrator' SYSADMIN CONCURRENT FND SHUTDOWN
exit_code=$?
QUESTION
==========
CM is not shuting down.
Ans?
===
As u defined that CM is hanged due to some pending request,
the one option that i think would be feasible is to make cancel that request from backend
keeping status_code='E' and phase_code='C' ,
So , once the request got completed the CM will automatically come down with no problem
run Cmclean.sql and start the Cm
precisley knowing your FNDLIBR process you can try this in command line:
ps -aex|grep FNDLIBR|grep ORCL
The above command will list out FNDLIBR process related to instance ORCL only.
Question :
=======
CM is up but Oracle Applications Manager. it shows down?
Ans:
======
- Dashborad collection manager is down
- GSM might be disabled
- Monitoring/SErvice Update in OAM might be diabled
References:
Basic information about Concurrent Managers (Doc ID 69336.1)
Concurrent Processing - Product Information Center (PIC) (Doc ID 1304305.1)
Note:134007.1 CMCLEAN.SQL
Note:388495.1 Parallel Concurrent Processing (PCP) in Apps 11i?
Note:279956.1 Oracle E-Business Suite Release 11i with 9i RAC: Installation and Configuration using AutoConfig
Note:9611.1 CONCSUB Utility
Note:241370.1 Concurrent Manager Setup and Configuration Requirements in an 11i RAC Environment
Note:312731.1 Configuring Oracle Applications Release 11i with 10g Release1 Real Application Clusters and Automatic Storage Management Section 3.11 Configuring Parallel Concurrent Processing
Note:105133.1 when to bounce Cm
Note:392246.1 Requests Stuck with Inactive No Manager
Note:2181143
Note:233428.1 merge the APPL_TOP on the DB tier with the APPL_TOP on the apps tier
280295.1 REQUESTS.sql Script for Parent/Child Request IDs
Note:134033.1 Analyze all Pending Requests
Note 189708.1 DBA to configure two files i.e. uiprint.txt & default.ppd
Note 91151.1 Concurrent Managers Rel. 10.7 Tuning and Installation Issues
NOTE:74717.1 - TROUBLESHOOTING CONCURRENT MANAGER STARTUP PROBLEMS - Applications 11.0
NOTE:97798.1 - ORA-904 starting Concurrent Managers
NOTE:1057802.1 - Concurrent Processing - Best Practices for Performance for Concurrent Managers in E-Business Suite
NOTE:200360.1 - 11i : Oracle Application Object Library Concurrent Manager Setup Test
NOTE:168071.1 - ICMLOG.sql - Script for Retrieving Only the Internal Manager Logfile
NOTE:171855.1 - Concurrent Processing - CCM.sql Diagnostic Script to Diagnose Common Concurrent Manager Issues
NOTE:179661.1 - E-Business Suite Diagnostics 11i Test Catalog
NOTE:185036.1 - Unable to Startup or Shutdown the Concurrent Managers in 11.5.5 by Using the Syntax adcmctl.sh apps/<password> Start or Stop
NOTE:1059201.6 - Status Pending With 'Inactive No Manager' for Concurrent Requests
NOTE:113135.1 - Conflict Resolution Manager FNDCRM Does Not Start; No Error Message
NOTE:123607.1 - CONCSUB SYSADMIN 'System Administrator' Gives Invalid Responsibility Error
NOTE:1304305.1 - Concurrent Processing - Product Information Center (PIC)
NOTE:2055375.6 - Concurrent Managers and PMON Method
NOTE:2120154.6 - Uable to Start the CCM after Installation - ICM Core Dumps with Status 139
NOTE:213021.1 - Concurrent Processing (CP) / APPS Reporting Scripts
NOTE:230121.1 - Routine AFPEIM Encountered an Error While Starting Concurrent Manager on Cloned Instance
NOTE:421245.1 - E-Business Suite Diagnostics References for R12
Tips:
===
To trace CM request:
select v.request_id,
s.sid,
s.serial#,
s.event,
p.spid
from apps.fnd_conc_requests_form_v v,
v$process p,
v$session s
where v.request_id = ‘5029302'
and v.oracle_process_id = p.spid(+)
and p.addr = s.paddr(+)
/
Form the session ID find the PID
SQL> col sid for 999999
SQL> col serial# for 999999
SQL> col username for a20
SQL> col osuser for a20
SQL> select s.sid,s.serial#,s.username,s.osuser
from v$session s,v$process p
where s.paddr=p.addr;
Set the trace as below what is happening/executing we can comes to know
SQL> -- set the OS PID
SQL> oradebug setospid 864
Windows thread id: 864, image: ORACLE.EXE
SQL> -- or set the Oracle pid
SQL> oradebug setorapid 13
Windows thread id: 864, image: ORACLE.EXE
SQL> -- set the trace file size to unlimitd
SQL> oradebug unlimit
Statement processed.
SQL> -- now turn on trace for SCOTT
SQL> oradebug event 10046 trace name context forever, level 12
Statement processed.
SQL> -- run some queries in another session and then turn trace off
SQL> oradebug event 10046 trace name context off
Statement processed.
Find the trace in UDUMP
http://www.petefinnigan.com/ramblings/how_to_set_trace.htm==> to set trace using oradebug
453527.1
===================================================================
Query to find the Request id complete date
select request_id,
to_char(REQUESTED_START_DATE, 'mm/dd HH24:MI:ss') req_dt,
to_char(actual_START_DATE, 'mm/dd HH24:MI:ss') start_dt,
phase_code,
status_code,
to_char(ACTUAL_COMPLETION_DATE , 'mm/dd HH24:MI:ss') end_dt
from FND_CONCURRENT_REQUESTS
where (REQUESTED_START_DATE
between to_date('06/18 05:04:59','mm/dd HH24:MI:ss')
and to_date('06/18 05:10:00','mm/dd HH24:MI:ss'))
order by REQUESTED_START_DATE
/
Question 1: What are different types of concurrent manager?
Answer There are several types of concurrent manager. Important ones:
Internal Manager
Standard Manager.
Conflict Resolution Manager
Internal monitors
Service Manager
Transaction Manager
Custom Manager Defined
Read more on below link
Oracle Concurrent Manager
Question 2: What is Internal Concurrent Manager?
Answer : It is responsible for controlling all other concurrent managers. Its main task is to make sure that all other concurrent managers are up and running. It controls other manager through the requests made to the service manager. It also starts, stops and restarts the service manager for all nodes.
Question 3: What is Conflict Resolution Manager (CRM)?
Answer: It takes care of resolving the program incompatibilities and checks if a request in queue can be run in parallel with the running request. If a program is identified as run alone, then it prevents concurrent managers from starting other programs in the same conflict domain.
Question 4: What is Standard Manager?
Answer : Standard Manager is the master concurrent manager. It is always running and can take care of processing any concurrent request. If at all, no other manager is assigned to a program, that program will be picked by standard manager.
Question 5:
What happened behind the scene when a concurrent request is submitted?
Answer
1) Once a concurrent request is submitted by the user, the table FND_CONCURRENT_REQUESTS is automatically updated with the details of the request. The table is also updated with the information about the schedule of the concurrent request whether it’s immediately scheduled or scheduled at a fixed time.
2)a) If the request is incompatible/constraints defined ,Once the request time to run is arrived,it status is set to pending/Standby.Now the conflict resolution manager takes care of the request and finds out what are the incompatibilities and set the status pending normal when the incompatibilities are cleared.
b) If there are no incompatibilities then Once the request time to run is arrived,it status is set to pending/Normal
c) ALL the standard concurrent Manager and special manager continuously poll the FND_CONCURRENT_REQUESTS table .The job of a concurrent manager is to execute concurrent requests that are in Pending / Normal phase / status and that it is qualified to run according to its specialization rules.
Concurrent Manager Processes
– Act independently
– Select only requests that: (a) match the manager specialization rules, (b) are Pending/Normal, (c) have a requested start time <= sysdate
d) Once the request is processed, the FND_CONCURRENT_REQUESTS table is updated with the status.
Question 6: Business user create the task that Concurrent request is taking a lot of time to complete. What will be your approach for debugging it?
Answer
1) First find the status of the concurrent request. It may be scheduled later on or it could be pending/standby mode or all the concurrent manager are occupied running other request. If it is pending/standy, we need to find the incompatible program running and inform user.Many times the users schedule the request to run at a later time.
2) Find out the database sid of the concurrent request and check it is waiting on any locks. We will kill the blocking session to order to complete the job
3) We can run a trace on the request id to find the sql running and then generate the explain plan for it. You can see if the sid is stuck on particular sql . If it is particular sql, then it is good to check the statistics of the table involved . We can look for tuning option for that query
4) We can check the parameters with which the request is run. (For example, once a user came saying the request is not printing the output. On Checking the possible things, it was realized that he scheduled the request with print copies = 0.)
Question 7: What happens when the internal concurrent manager dies abruptly? Are all the managers also killed immediately after it?
Answer
No ALL the standard manager keep running and executing the request. if the internal manager dies, below queue control request are not performed
a) Starts all other processes.
b) Executes “control requests” submitted by the administrator.
c) Activate/Deactivate/Abort Concurrent Manager
d) Terminate Concurrent Request
e) Monitors processes, restarting any that failed.
f) Sets the target number of processes for each service based on the current work shift.
Question 8: Does the internal manager run or schedule any request for itself?
Answer
No, the internal manager does not run or schedule any requests. It has nothing to do with scheduling requests, or deciding which manager will run a particular request. Its function is only to run ‘queue control’ requests
a) Starts all other processes.
b) Executes “control requests” submitted by the administrator.
c) Activate/Deactivate/Abort Concurrent Manager
d) Terminate Concurrent Request
e) Monitors processes, restarting any that failed.
f) Sets the target number of processes for each service based on the current work shift.
Question 9: How do I process more concurrent requests in parallel?
Answer
We can increase the target processes of the concurrent manager in order to increase the parallelism. This can be done using define concurrent manager form or through direct update from sqlplus
Question 10 : If the internal manager goes down, do I need to kill all the managers before restarting the internal manager?
Answer
No, if the internal manager goes down you need not kill all the managers. You can simply start the internal manager using startmgr.
Concurrent Manager troubleshooting
Question 11 : What are the problems u have faced while shutting down applications?
Answer
While shutting down application generally concurrent manager won’t go down because some or the other request may be running. We will see what are the concurrent requests running by querying fnd_concurrent_requests, fnd_concurrent_program_vl, v$session, v$process and v$sqltext.
If that request is only doing some select statement then we will kill those requests, otherwise we will check what time it will take to complete by querying the previous runs of that request and then we will decide what to do.
Oracle Concurrent Manager Secrets
Question 12: What are Internal Monitors ?
Answer: Internal Monitors are used specifically in PCP to allow for ICM failover to other available middle tier nodes.
a) Place an Internal Monitor on any node where the ICM can start in case of a failure.
b) Internal Monitors are seeded on every registered node by default.
c) If the ICM goes down, the Internal Monitor will attempt to start a new ICM on the local node.
d) If multiple ICMs are started, only the first will stay active. The others will gracefully exit.
Question 13: Can I delete concurrent manager?
Answer:
Yes, you can delete any concurrent manager. For deleting, query for the manager in the defined concurrent manager form and then delete the row.
Deleting the predefined concurrent managers is not recommended and it should never be done. Deletion may cause instability in the system.
Question 14: How can you know which trace file is created for the particular request?
Answer
You can find out the same using the script given below. The trace will be located in the udump location of the database server.
prompt
accept request prompt ‘Please enter the concurrent request id for the appropriate concurrent program:’
prompt
column traceid format a8
column tracename format a80
column user_concurrent_program_name format a40
column execname format a15
column enable_trace format a12
set lines 80
set pages 22
set head off
SELECT ‘Request id: ‘||request_id, ‘Trace id: ‘||oracle_Process_id, ‘Trace flag: ‘||req.enable_trace, ‘Trace Name: ‘||dest.value||’ ‘||lower(dbnm.value)||’ora’||oracle_process_id||’.trc’, ‘Prog. Name: ‘||prog.user_concurrent_program_name, ‘File name: ‘||execname.execution_file_name||execname.subroutine_name , ‘Status :’||decode(phase_code, ‘R’, ‘Running’)||’ ‘||’-‘||decode(status_code, ‘R’, ‘Normal’), “SID Serial: “||ses.sid||” , “||ses.serial#, “Module : “||ses.module
from fnd_concurrent_requests req,
v$session ses, v$process proc,
v$parameter dest, v$parameter dbnm,
fnd_concurrent_programs_v1 prog,
fnd_executables execname
where req.request_id = &request
and req.oracle_process_id=proc.spid(+)
and proc.addr = ses.paddr(+)
and dest.name=’user_dump_dest’
and dbnm.name=’db_name’
and req.concurrent_program_id =
prog.concurrent_program_id
and req.program_application_id =
prog.application_id
and prog.application_id =
execname.application_id
and
prog.executable_id=execname.executable_id;
Top 30 Most Useful Concurrent Manager Queries
Question 15: Explain how parallel concurrent processing(PCP) works?
Answer
In case of parallel concurrent processing, all the managers are assigned a primary and a secondary node. The managers are started in their primary node by default. In case of node failure or Oracle instance failure, all the concurrent managers on that node are switched to their secondary nodes. Once the primary node is available again the concurrent managers on the secondary nodes are migrated back to the primary node. During the migration process, a manager may be spread across both primary and secondary nodes.
In case of parallel concurrent processing, it may happen that in a node where parallel concurrent processing is configured, the Oracle instance may or may not be running. The node which is not running Oracle, the concurrent managers connects via Net8 to a node which is running Oracle.
The internal concurrent manager can run on any node, and can activate and deactivate concurrent managers on all nodes. Since the internal concurrent manager must be active at all times, it needs high fault tolerance. To provide this fault tolerance, parallel concurrent processing uses internal monitor processes. The job of the internal monitor process is to constantly monitor the internal manager and start it when it fails. Only one internal monitor process can be active on a single node. You decide which nodes have an internal monitor process when you configure your system. You can also assign each internal monitor process a primary and a secondary node to ensure fail over protection. Internal monitor processes, like concurrent managers, can be assigned work shifts, and are activated and deactivated by the internal concurrent manager.
Parallel Concurrent Processing
Question 16: What are the circumstances in which you need to bounce the concurrent manager?
Answer: There can be many situation where you need to bounce the concurrent manager
a) When you modify the definition of the printers
b) When you modify the environment variables. Suppose you have changed the APPLTMP and APPLPTMP variable.
c) When all the requests are pending and hanging and no processing happening
d) patch application requires the bounce of the CM
e) We have many global hangs in the system due to locks by several concurrent manager and other processes
Question 17: What are the reasons a concurrent manager hangs?
Answer:
The concurrent manager hangs due to many reasons. A few of them are:
– Long running jobs
– The internal manager was activated by someone other then owner of the application system
– The operating system files system is full
– It’s not able to create the log file
– You’ve shut down the internal manager, but actual has a number in it
– The database is hanging may be because the archive log files have filled
– Pending/standby requests are too many
Question 18: How can we enable/disable Conflict Resolution Manager?
Answer: It can be done using profile options “Concurrent: Use ICM”. Set it to “Y” to enable Conflict Resolution manager. To disable it, set the profile option to “N”.
Question 19: What are Transaction Managers?
Answer : Transaction managers provide synchronous job processing by continually monitoring a DBMS pipe for requests to come through from a client-side application. The job of a transaction manager is to process this job immediately and send information back to the client using the pipe.
a) Transaction Managers Provide Synchronous Job Processing
b) A client makes a request for a specific transaction manager to run a program, and waits for the results of that program
c) Product teams’ programs are linked directly into the transaction manager executables
d) PO, CRP, INV, AR, and OE all ship transaction managers
Question 20: How the mechanism of viewing log and output files works from browser?
Answer
The sequence of events is as follows:
1. A user within an Applications session requests to view a log or out file.
2. The browser receives the request and spawns the cgi program FNDWRR.exe
3. FNDWRR.exe logs into the database and queries FND_CONCURRENT_REQUESTS to discover on which node the files for this request are stored.
4. FNDWRR.exe constructs the service name for the file server on that node. And makes the tns call to contact the listener for this service name.
5. The listener responds by spawning the local FNDFS executable, as defined in it’s listener.ora file. Now, FNDFS and FNDWRR.exe are able to communicate directly now, using RPC calls.
6. FNDWRR.exe requests FNDFS to transfer the file that was selected by the user.
7. FNDFS transfers the file contents to a temporary file directory on the web server’s node.
8. The web server displays the file contents to the user.
Question 21: Why does the Concurrent manager put a concurrent program into a queue? Why doesn’t the manager simply let the program run?
Answer: Because at any given point in time a concurrent manager can run no more than say 10 programs concurrently. This figure of 10 is configurable of course. First the manager puts a submitted program into a queue, next the manager checks if there is a slot available (i.e. Less than 10 programs are currently running). If a slot is found available, the concurrent manager then runs the program, or else it keeps the concurrent program in a queue with status Pending.
Question 22: What to do if a request is Inactive/No Manager
Answer: This is quite tricky questions
a) Make certain that there is at least one active manager with specialization rules that allow the program.
b) If you have confirmed the previous point, then the problem may be a stale Worker Request View
– The view is used internally to map requests to managers
– The view is regenerated when managers are created, or specialization rules are altered
c) You can manually regenerate the view
FNDLIBR FND FNDCPBWV apps/apps SYSADMIN ‘System Administrator’ SYSADMIN
Question 23: What are Service Managers
Answer: Service Managers are spawned on the middle-tier nodes of a GSM enabled system in order to act as an agent of the ICM. When the ICM sees that it needs An Service Manager to perform some function, such as start a concurrent manager process, on a middle-tier node, it will make remote procedure control (RPC) calls to the Apps listener on that node to start the Service manager. Once the Service manager has been started and initialized, the ICM communicates directly to the SM through RPC, giving it information to manage the services on that node. The SM is spawned from the APPS TNS Listener.The APPS TNS Listener must be started on every middle-tier node in the system, and started by the user that starts ICM (e.g. applmgr) .TNS Listener spawns Service Manager to run as agent of ICM for the local node
The Service Manager is started by ICM on demand when needed. If no management actions are needed on a node a Service Manager will not be started by ICM until necessary. When ICM exits its Service Managers exit as well.
Question 24: What all is performed by ICM Process Monitor?
Answer: The ICM itself and each process spawned by the ICM has an entry in FND_CONCURRENT_PROCESSES and holds a uniquely named DBMS lock.
The unique lock of the ICM has a format of FNDCPLK_ICM. This database session lock is the method that the ICM ensures each PMON cycle that manager and service processes are still alive. If the ICM can get the DBMS session lock of a process, the ICM will start a new process for that manager or service.
This is the reason you will often see entries like dead process found ,starting new process in ICM log files.
===========================
What is the significance of utl_file_dir parameter in init.ora file?
=====================================================================
Answer The value of this parameter is the group of directories to which u r database can write,
means u r database packages have permission to write to flat files in these directories.
What is APPLPTMP environment variable?
=======================================
Answer This is the temporary file location for the pl/sql temp files.
If this variable was not set then the concurrent programs may error-ed out.
DIAG_PARAM ="Y" in adcmctl.sh. for CM issues
oracle apps concurrent managers interview question and common information's
Question:Error:
================
After submitting the concurrent program, i am getting the status as inactive and no manager and when i clicked on diagnostics button its showing a message like
This request will not be processed because there are currently no managers running that can process it.
This is an error condition. There are one or more managers defined that can process this request that should be running at this time, but do not appear to be running.
Contact your system administrator and/or check the manager log files for errors.
Solution:
=========
to resolve that, just u simple startup concurrent manager by executing the ./adcmctl.sh script
Question:
=========
want to create a new “MANAGER”
Ans:
====
sysadmin guide for that
http://download-uk.oracle.com/docs/cd/B25516_11/current/html/docset.html
To figure out the issue.
1) Check your CRM is showing any jobs in conflicts.
2) Check for locks.
3)280295.1 REQUESTS.sql Script for Parent/Child Request IDs and Trace File IDs
ANALYZEPENDING.SQL - Analyze all Pending Requests Doc ID: Note:134033.1
Question :
===========
problem is My technical consultant has submitted two concurrent requests. But it is long time running. the status is running. There is nothing to find in the log file. Where can i know what is exact problem
Solution
========
are the concurrent requests are custom or default concurrent programs....did you check the concurrent managers status.....
(a) check the concurrent managers status
(b) check the concurrent manager logfiles
if the concurrent requests are custom...i mean....if it is the report based...check the report query and tune the query wether custom tables are having indexes or not.......check the schema statistics of apps schema
2:
Try enabling trace on the particular concurrent request,may be that should help to diagnose the problem?
Question:
=========
What is environment variable APPLCSF in Apps environment?
Ans:
==
If environment variable APPLCSF is set then u can
get the logfiles of concurrent managers under $APPLCSF/$APPLLOG.
If APPLCSF is not set, Request Log files will go to
$<MODULE>_TOP/$APPLLOG
and output files will go to
$<MODULE>_TOP/$APPLOUT
Here <MODULE_TOP> is GL_TOP, AP_TOP, AR_TOP etc etc.
QUESTION:
=========
How and why to configure PCP(Parallel Concurrent Processing)?
Answer:
========
PCP is nothing but a load balancing methodology for concurrent processing server. To load balance on concurrent processing server, this is required. In a multinode architecture, it is being used.
Check the following metalink notes
How to Set Up Parallel Concurrent Processing (PCP) in Apps 11i?
Doc ID: Note:388495.1
Oracle E-Business Suite Release 11i with 9i RAC: Installation and Configuration using AutoConfig
Doc ID: Note:279956.1
Question:
=========
CM is not working after cloning?
Answer
======
1.Bounce CM
2.check FND_NODES correctlly updated for concurent manager
3.if is there custom tops ..if so means...add those to any env file..after that set env
then Bounce the concurrnet managers..
Question:
=========
Concurrent manager "Inactive, No Manager"
1. We are facing this problem even when no other job is running
2. We dont face this problem on daily basis- it can happen sometimes thrice in a day and sometimes once in a month.
3. This program generally runs atleast 10 times in a day.
4. Configuration of the concurrent manager is (this is not standard manager):
No. of processes is: 8, Cache Size: 10, Sleep time: 10
Answer:
=========
sol_1
====
This problem seems to be inactive ICM.
When ICM is not up u will get error inactive-no manager.
Just check with manager and make it up.
ps -ef|grep LIBR
if it shows the manager is down then start the CM using script adcmctl.sh.
And if the manager is up but it is showing the inactive-nomanager then check for the log file and do the needful.
As my idea y dont u try with cmclean.sql.(not supported by oracle),Check in UAT before trying in prod.
1.Bring down CM.
2.run cmclean.sql
3.Bring up CM.
Sol_2:
======
Most importantly, see try reassigning the request to a different manager instead of the Standard Manager (maybe a custom one which only it belongs to) and see if you get the same behavior. Also check to see there are no unnecessary incompatibilities listed for the Program under the Program definitions.
sol_3:
========
I'd like to add that we checked the work schedule which is set to standard (24X7) and specialised rules against the standard Concurrent manager in the define manager screen. In adition we recreated the Conc mgr views, also created our own manager which gave the same message Phase Inactive No Manager.
Sol_4:
=======
What is the value of your GSM Enabled profile setting?
We had it set at Y but since changed it to N. We have also changed the appldcp context value to NO. Nothing seems to make any difference
Sol_5:
=======
after cloning you are getting inactive manager
go to applmgr user
$TNS_ADMIN / path
change the tns file as
edit
(SID=FNDSM) to (SID=FNDSM_VIS)
Sol_6:
=====
check the disc space of this instance, and increase
QUESTION:
=========
What are the differences between the functionality of purging concurrent request, FNDCPPUR and CMCLEAN.
Answer:
=======
FNDCPPUR is executable name of Purge Concurrent Request and/or Manager Data Program.
When you run this request, it will delete all the requests from Tables and log, req and manager log files from OS leve.
cmclean.sql is used to clean the concurrent manager tables, when there is any problem with managers starting up..
for more check the following note
CMCLEAN.SQL - Non Destructive Script to Clean Concurrent Manager Tables
Doc ID: Note:134007.1
Question:
=======
If ICM stop the othar manager ST which are already
started will continue but the new manager will not start
At the same time confict resolution manager will pickup all the request
How to know the ICM is down
Answer:
========
FNDSM is ICMs agent which will be doing the job for ICM(starting stoppin managers etc..).
if that goes down everything else related to concurrent processing shuts off.
So you could grep for this FNDSM process continously at OS level to check ICMs status using
a script
LOCKING:
========
Check V$session and V$locked_object. You can formulate a query by joining these two views to get the required info..Do a describe on them and try to create a query.
>How to resolve locking
using the two views mentioned above you can find out which session is blockin which sessions.If the culprit session is INACTIVE, then you may kill it using alter system kill session command. That would release the blockage
Question:
=========
what is PMON cyle,QUEUE size and sleep time:
ANS:
===
While the basic ICM definition should not be changed, you can if required modify the sleep time
(number of seconds the ICM waits between checking for new concurrent requests),
PMON (process monitor) cycle time (number of sleep cycles the ICM waits between checking for
failed workers), and queue size (duration between checks for number of active workers,
measured in PMON cycles). If Parallel Concurrent Processing (described below) is being used,
you can also set some options for this.
what does these failed "workers" refer to?? What are these workers?
A/These workers are nothing but the Operating System processes associated with the concurrent
managers.
Question:
==========
Reasons when concurrent manager fail?
In concurrent manager -> administer
Actual /target values different? Wat do you mean by it?
Answer:
=========
Reasons when concurrent manager fail?
There can be N number of reason for CM failure , check CM log file for exact message and cause
Few of them are --
Database listener down,
wrong password while starting CM,
corruption in FND_NODES ...
In concurrent manager -> administer Actual /target values different?
Wat do you mean by it?
Actual is actual number of manager running at that time and target is the number of Manager should be
running at that particular time.
Qestion:
========
How to check the CM is up?
Answer:
========
ps -ef|grep FNDLIBR
Question:
========
which concurrent manager will run, which concurrent programs?
ANS_!:
========
Basically there are four types of Conc Mgrs
1 Internal Concurrent Manager
2 Conflict resoluction manager
3 Standard Manager
4 Transactional Managers
ICM - monitors other managers
SM picks up all the requests which or not there in it's exclusion list
Each transaction manager can process only the programs contained in its program library.
In the Concurrent Program Definition there is a column called REQUEST_TYPE by which we can assign the program to a particular manager.
You can use include and exclude options for the managers depending upon the your requirement if you want the program to be run by only one user.
You can navigate to Administer Concurrent Managers from System Adminsrator reqponsiblity, select maanger and view it's requests to see what all requests it is running.
Document which gives you more details is "System Administrator’s Guide"
Ans_2
=====
you can know this by navigate to concurrent>>manager>>define and there just query for that manager what you search for. There , the manager has been defined that it will run which requests or programs.
ANS_3:
========
From concurrent manager define and administrator screen we can find out what are all the concurrent programs that manager is running.
We can find out currently running request and which manager queue this request is running from the script $FND_TOP/sql/afrqrun.sql.
But my question is, I want general script or query to find from back end,
Which request (completed, errored, schedule, running etc) goes to which manager?
ANS_4:
=======
Use this query. Provide short name as concurrent_program_name.
SELECT A.INCLUDE_FLAG, A.QUEUE_APPLICATION_ID, C.USER_CONCURRENT_QUEUE_NAME,a
B.CONCURRENT_PROGRAM_NAME
FROM APPLSYS.FND_CONCURRENT_QUEUE_CONTENT A, APPLSYS.FND_CONCURRENT_PROGRAMS B, APPS.FND_CONCURRENT_QUEUES_VL C
WHERE type_id = b.concurrent_program_id and b.concurrent_program_name = '&SHORT_NAME' and c.concurrent_queue_id = a.concurrent_queue_id
/
Output will show you Which program is excluded from standard manager and included in other manager.
Question:
=========
What to do when Concurrent request is queing up and not being completed.
Did u started the concurrent manager...
ANSWER:
=========
A/Start the conc manger .. $COMMON_TOP/admin/scripts/adcmctl.sh start apps/apps
But still it not complited
B/Are there any errors in the Standard Manager log?
$APPLCSF/log/<Context Name>/FNDSM9999.mgr
Question
==========
What cmclean.sql will do?
Answer:
=========
cmclean.sql is non destructive script to clean CM tables. To run this script, CM has to be down since it does some updates on the tables. The script resets the flags of the requests to 'COMPLETED' and clean the CM tables so allowing the concurrent managers to come up. You need to run this script whenever you have an issue with the CM startup.
EXEC FND_CONC_CLONE.SETUP_CLEAN;commit ==> to clean fnd_tables
SQL> select NODE_NAME,SERVER_ADDRESS from fnd_nodes;
DESCRIPTION
Concurrent managers cleanup script.
To be called by rapidinstall ONLY.
- Delete all site specific information (nodes, services, printers, etc).
- Delete requests that are in Completed and Running states.
- Delete/reset all concurrent manager state information.
- Cancel all Pending Oracle Reports based requests.
Question:
========
What is CONCSUB
Ans:
=====
CONCSUB is a utility run at the operating system level to submit a concurrent program to the concurrent manager.
For more check the metalink note...Using The CONCSUB Utility
Doc ID: Note:9611.1
Question:
=========
How to purge concurrent request espcially those who has .req and .log extension name on OS side, is it safe to purge concurrent request?
Answer:
========
A/To avoid running out of space on your disk drives, you should periodically
delete Oracle Applications log files, output files and purge these tables
with the FNDCPPUR program (Purge Concurrent Request and/or Manager Data).
The FNDCPUR purge program maintains the number of log and output files the operating system retains, and manage tables that store information about concurrent requests and concurrent manager processes.
You can run the program FNDCPPUR once and automatically resubmit the program for your specific time intervals.
Depending on the usage you might have a different strategy for purging the concurrent requests log/out files eg.
- every 30 days for normal usage
- every two weeks (14 days) for heavy usage
- if you are using the AGE mode, set the Mode Value to 7 or higher to retain the most recent days of concurrent request data, log files, and report output files.
WARNING : When you purge concurrent request information, you lose audit details which are used by the Signon Audit Concurrent Requests report.
If your business requires to see the requests run back in time then you need to keep archive on disk or tape.
B/The log and out files go to a common location for all modules.
The concurrent manager first looks for the environment variable
$APPLCSF. If this is set, it creates a path using two other
environment variables: $APPLLOG and $APPLOUT
It places log files in $APPLCSF/$APPLLOG, output files go in
$APPLCSF/$APPLOUT
So for example, if you have this environment set:
$APPLCSF = /u01/appl/common
$APPLLOG = log
$APPLOUT = out
The concurrent manager will place log files in /u01/appl/common/log,
and output files in /u01/appl/common/out
Note that $APPLCSF must be a full, absolute path, and the other two
are directory names.
If $APPLCSF is not set, it places the files under the product top of
the application associated with the request. For example, a PO report
would go under $PO_TOP/$APPLLOG and $PO_TOP/$APPLOUT
Logfiles go to: /u01/appl/po/9.0/log
Output files to: /u01/appl/po/9.0/out
All these directories must exist and have the correct permissions.
Note that all concurrent requests produce a log file, but not necessarily
an output file.
Concurrent manager logfiles follow the same convention, and will be
found in the $APPLLOG directory.
Use the find command to move files which are more than N days old to another directory and zip them up or archive to tape. You can also use utilities like TSM (Trivoli Storage Manager) for backing up the filesystem.
eg. find $APPLCSF/$APPLLOG -type f -mtime +45|xargs -i mv -f {} /tmp/45more_log
Question:
=========
15 custom CM's , in which one is down . so what is the way to troubleshoot it to bring it up.
Answer:
=====
you can go with anyone:
1. 1st check the log file of the cm. if your error message is not enough to find out the problem, then change DIAG_PARAM ="" to DIAG_PARAM ="Y" in adcmctl.sh.
2.you can use cmclean.sql and then check the status of that cm.
3. check the values of target and actual column for that cm in frontend.
Question:
===========
CM is down Status Inactive and No Manager?
Ans:
===
1Check in icm log file to see issue
$APPLCSF/$APPLLOG/<SID>_<timestamp>.log Check in icm log file to see issue
$APPLCSF/$APPLLOG/<SID>_<timestamp>.log
Ans_2:
======
This is way too big of a question to be answered in one post but in general:
1. Ensure you have enough Standard Manager processes to handle all of the requests. The default number is 3, this can be increased in the Manager Definitions (Workshift).
2. Ensure that the number of rows in FND_CONCURRENT_PROCESSES and FND_CONCURRENT_REQUESTS don't exceed 50K or there will be performance issues.
3. Ensure that you don't have any unnecessary manager processes running in the manager queues.
4. Set a low priority if possible on the jobs which generally run the longest. If you are consistently getting slow performance on jobs, check to see if they also execute slowly from the SQL PLUS. If they do, the issue is with the code, not the managers.
DBA_EXTENTS or DBA_OBJECTS Table..there you can get the SIZE of the Objects
Question:
=========
submitted a request and it completed with the status as completed normal
But not able to see log file?
Answer:
========
A, RRA profile agaent & apps listener adalnctl.sh
B, whatever you have operating system , you need to restart the listener again even if it's working status and if the users are connected to the Apps. then you have to restart the Apatche first then the listener of the application 8.0.6.
Question:
========
DB tier and Concurrent Manager on one node
Apps Tier(Forms, Apache) on another node.
How to move the Concurrent Manager to a new node
Ans:
====
Theoretically, you'll need to add a node to the Apps Tier using the Rapid Clone addnode.pl
This should give you an overall idea.You'll need to tweak this to your requirement.
So, 1) you run preclone on DB node, CM node, Web/Forms node
2) Deregister existing CM node
3) Disable security option, run autoconfig on DB node, restart listener
4) Copy files to the new node
5) Run post clone on DB tier
6) Run post clone on Web/Forms Tier
7) Run post clone on new node to configure for CM node
8 ) Run addclone.pl on Apps tier.
How do I deregister an application tier from the Net Services Topology Data Model?
Answer:
To deregister the current application tier from the Net Services Topology Data Model, invoke the following command:
perl <AD_TOP>/bin/adgentns.pl appspass=<APPSpwd> contextfile=<CONTEXT> -removeserver
Refer Note: Note:218089.1 Autoconfig FAQ
Regarding step ( 8 ) : This would need to be run from Forms/Web node I believe. Not familiar with it's prompts, but it should be straightforward as we have the NEW CM node already ready by the time we execute this. Please ensure that once this is run, the entries in the context file on this node reflect your new node. I would check entries in the following section in context XML
<oa_admin_server>
<oa_cp_server>
For addnode.pl refer to note: Note:230672.1 Cloning Oracle Applications Release 11i
After everything is done, I would also check FND tables:
FND_NODES
FND_APP_SERVERS
The entries here should match with what we needed to start with.
1) Entry for DB node
2) Entry for Forms/Web Node
3) Entry for *NEW* CM node
4) No entries for *OLD* CM node
I hope you are doing this on test first? Let me know how it goes.
Question:
========
Want to configur PCP?
Ans:
===
241370.1 Concurrent Manager Setup and Configuration Requirements in an 11i RAC Environment
312731.1 Configuring Oracle Applications Release 11i with
10g Release1 Real Application Clusters and Automatic Storage Management
Section 3.11 Configuring Parallel Concurrent Processing
Question:
=========
Why only for Concurrent Manager, we specify apps/apps password & not for other?
Ans:
======
Concurrent Manager is used for processing requests..it means...for example...I want to run one report based on Current Month GL Balance...So we have to run one report to show balance...this balance will be stored in the Database tables..right.....
Concurrent Manager runs this request by connecting to the database using apps user password.....
Got it..hope it is clear.....'
Question:
========
which situations we need to bounce the concurrent manager ? and application services ?
Ans:
====
doc id: 105133.1(given in teachme oracle-Metalink Notes Section).
When you modify the Printer Driver you have to restart the Manager which runs the request which is attached to that Printer Driver, however,if you do not know which manager then you have to restart the Internal manager because the printer driver can be used by multiple managers and multiple requests. If only a concurrent program definition is modified, running a verify on the Internal Manager will pick up the changes without the need for bouncing the manager.
QUESTION:
==========
I have defined a new manager and submitted request in it.
But the job is always running and it never stops.
ANS:
====
issue initially the process specified for the new ASAP manager is below 10.
But as per the thumb rule for that specified application and the manager
the process should be more than 20.Thumb rule for the manager which we created was
thumbrule= 4 + (2 * profile value).
According to the above thumbrule the intial pocess of 8 was increased to 25 then the manager runs normally.
q/a
Below is the relevant content from adcmctl.sh:
printf "Shutting down concurrent managers for $TWO_TASK ...\n" >> $LOGFILE
$FND_TOP/bin/CONCSUB "$unpw" SYSADMIN $WAIT_PARAM 'System Administrator' SYSADMIN CONCURRENT FND SHUTDOWN
exit_code=$?
QUESTION
==========
CM is not shuting down.
Ans?
===
As u defined that CM is hanged due to some pending request,
the one option that i think would be feasible is to make cancel that request from backend
keeping status_code='E' and phase_code='C' ,
So , once the request got completed the CM will automatically come down with no problem
run Cmclean.sql and start the Cm
precisley knowing your FNDLIBR process you can try this in command line:
ps -aex|grep FNDLIBR|grep ORCL
The above command will list out FNDLIBR process related to instance ORCL only.
Question :
=======
CM is up but Oracle Applications Manager. it shows down?
Ans:
======
- Dashborad collection manager is down
- GSM might be disabled
- Monitoring/SErvice Update in OAM might be diabled
References:
Basic information about Concurrent Managers (Doc ID 69336.1)
Concurrent Processing - Product Information Center (PIC) (Doc ID 1304305.1)
Note:134007.1 CMCLEAN.SQL
Note:388495.1 Parallel Concurrent Processing (PCP) in Apps 11i?
Note:279956.1 Oracle E-Business Suite Release 11i with 9i RAC: Installation and Configuration using AutoConfig
Note:9611.1 CONCSUB Utility
Note:241370.1 Concurrent Manager Setup and Configuration Requirements in an 11i RAC Environment
Note:312731.1 Configuring Oracle Applications Release 11i with 10g Release1 Real Application Clusters and Automatic Storage Management Section 3.11 Configuring Parallel Concurrent Processing
Note:105133.1 when to bounce Cm
Note:392246.1 Requests Stuck with Inactive No Manager
Note:2181143
Note:233428.1 merge the APPL_TOP on the DB tier with the APPL_TOP on the apps tier
280295.1 REQUESTS.sql Script for Parent/Child Request IDs
Note:134033.1 Analyze all Pending Requests
Note 189708.1 DBA to configure two files i.e. uiprint.txt & default.ppd
Note 91151.1 Concurrent Managers Rel. 10.7 Tuning and Installation Issues
NOTE:74717.1 - TROUBLESHOOTING CONCURRENT MANAGER STARTUP PROBLEMS - Applications 11.0
NOTE:97798.1 - ORA-904 starting Concurrent Managers
NOTE:1057802.1 - Concurrent Processing - Best Practices for Performance for Concurrent Managers in E-Business Suite
NOTE:200360.1 - 11i : Oracle Application Object Library Concurrent Manager Setup Test
NOTE:168071.1 - ICMLOG.sql - Script for Retrieving Only the Internal Manager Logfile
NOTE:171855.1 - Concurrent Processing - CCM.sql Diagnostic Script to Diagnose Common Concurrent Manager Issues
NOTE:179661.1 - E-Business Suite Diagnostics 11i Test Catalog
NOTE:185036.1 - Unable to Startup or Shutdown the Concurrent Managers in 11.5.5 by Using the Syntax adcmctl.sh apps/<password> Start or Stop
NOTE:1059201.6 - Status Pending With 'Inactive No Manager' for Concurrent Requests
NOTE:113135.1 - Conflict Resolution Manager FNDCRM Does Not Start; No Error Message
NOTE:123607.1 - CONCSUB SYSADMIN 'System Administrator' Gives Invalid Responsibility Error
NOTE:1304305.1 - Concurrent Processing - Product Information Center (PIC)
NOTE:2055375.6 - Concurrent Managers and PMON Method
NOTE:2120154.6 - Uable to Start the CCM after Installation - ICM Core Dumps with Status 139
NOTE:213021.1 - Concurrent Processing (CP) / APPS Reporting Scripts
NOTE:230121.1 - Routine AFPEIM Encountered an Error While Starting Concurrent Manager on Cloned Instance
NOTE:421245.1 - E-Business Suite Diagnostics References for R12
Tips:
===
To trace CM request:
select v.request_id,
s.sid,
s.serial#,
s.event,
p.spid
from apps.fnd_conc_requests_form_v v,
v$process p,
v$session s
where v.request_id = ‘5029302'
and v.oracle_process_id = p.spid(+)
and p.addr = s.paddr(+)
/
Form the session ID find the PID
SQL> col sid for 999999
SQL> col serial# for 999999
SQL> col username for a20
SQL> col osuser for a20
SQL> select s.sid,s.serial#,s.username,s.osuser
from v$session s,v$process p
where s.paddr=p.addr;
Set the trace as below what is happening/executing we can comes to know
SQL> -- set the OS PID
SQL> oradebug setospid 864
Windows thread id: 864, image: ORACLE.EXE
SQL> -- or set the Oracle pid
SQL> oradebug setorapid 13
Windows thread id: 864, image: ORACLE.EXE
SQL> -- set the trace file size to unlimitd
SQL> oradebug unlimit
Statement processed.
SQL> -- now turn on trace for SCOTT
SQL> oradebug event 10046 trace name context forever, level 12
Statement processed.
SQL> -- run some queries in another session and then turn trace off
SQL> oradebug event 10046 trace name context off
Statement processed.
Find the trace in UDUMP
http://www.petefinnigan.com/ramblings/how_to_set_trace.htm==> to set trace using oradebug
453527.1
===================================================================
Query to find the Request id complete date
select request_id,
to_char(REQUESTED_START_DATE, 'mm/dd HH24:MI:ss') req_dt,
to_char(actual_START_DATE, 'mm/dd HH24:MI:ss') start_dt,
phase_code,
status_code,
to_char(ACTUAL_COMPLETION_DATE , 'mm/dd HH24:MI:ss') end_dt
from FND_CONCURRENT_REQUESTS
where (REQUESTED_START_DATE
between to_date('06/18 05:04:59','mm/dd HH24:MI:ss')
and to_date('06/18 05:10:00','mm/dd HH24:MI:ss'))
order by REQUESTED_START_DATE
/
Question 1: What are different types of concurrent manager?
Answer There are several types of concurrent manager. Important ones:
Internal Manager
Standard Manager.
Conflict Resolution Manager
Internal monitors
Service Manager
Transaction Manager
Custom Manager Defined
Read more on below link
Oracle Concurrent Manager
Question 2: What is Internal Concurrent Manager?
Answer : It is responsible for controlling all other concurrent managers. Its main task is to make sure that all other concurrent managers are up and running. It controls other manager through the requests made to the service manager. It also starts, stops and restarts the service manager for all nodes.
Question 3: What is Conflict Resolution Manager (CRM)?
Answer: It takes care of resolving the program incompatibilities and checks if a request in queue can be run in parallel with the running request. If a program is identified as run alone, then it prevents concurrent managers from starting other programs in the same conflict domain.
Question 4: What is Standard Manager?
Answer : Standard Manager is the master concurrent manager. It is always running and can take care of processing any concurrent request. If at all, no other manager is assigned to a program, that program will be picked by standard manager.
Question 5:
What happened behind the scene when a concurrent request is submitted?
Answer
1) Once a concurrent request is submitted by the user, the table FND_CONCURRENT_REQUESTS is automatically updated with the details of the request. The table is also updated with the information about the schedule of the concurrent request whether it’s immediately scheduled or scheduled at a fixed time.
2)a) If the request is incompatible/constraints defined ,Once the request time to run is arrived,it status is set to pending/Standby.Now the conflict resolution manager takes care of the request and finds out what are the incompatibilities and set the status pending normal when the incompatibilities are cleared.
b) If there are no incompatibilities then Once the request time to run is arrived,it status is set to pending/Normal
c) ALL the standard concurrent Manager and special manager continuously poll the FND_CONCURRENT_REQUESTS table .The job of a concurrent manager is to execute concurrent requests that are in Pending / Normal phase / status and that it is qualified to run according to its specialization rules.
Concurrent Manager Processes
– Act independently
– Select only requests that: (a) match the manager specialization rules, (b) are Pending/Normal, (c) have a requested start time <= sysdate
d) Once the request is processed, the FND_CONCURRENT_REQUESTS table is updated with the status.
Question 6: Business user create the task that Concurrent request is taking a lot of time to complete. What will be your approach for debugging it?
Answer
1) First find the status of the concurrent request. It may be scheduled later on or it could be pending/standby mode or all the concurrent manager are occupied running other request. If it is pending/standy, we need to find the incompatible program running and inform user.Many times the users schedule the request to run at a later time.
2) Find out the database sid of the concurrent request and check it is waiting on any locks. We will kill the blocking session to order to complete the job
3) We can run a trace on the request id to find the sql running and then generate the explain plan for it. You can see if the sid is stuck on particular sql . If it is particular sql, then it is good to check the statistics of the table involved . We can look for tuning option for that query
4) We can check the parameters with which the request is run. (For example, once a user came saying the request is not printing the output. On Checking the possible things, it was realized that he scheduled the request with print copies = 0.)
Question 7: What happens when the internal concurrent manager dies abruptly? Are all the managers also killed immediately after it?
Answer
No ALL the standard manager keep running and executing the request. if the internal manager dies, below queue control request are not performed
a) Starts all other processes.
b) Executes “control requests” submitted by the administrator.
c) Activate/Deactivate/Abort Concurrent Manager
d) Terminate Concurrent Request
e) Monitors processes, restarting any that failed.
f) Sets the target number of processes for each service based on the current work shift.
Question 8: Does the internal manager run or schedule any request for itself?
Answer
No, the internal manager does not run or schedule any requests. It has nothing to do with scheduling requests, or deciding which manager will run a particular request. Its function is only to run ‘queue control’ requests
a) Starts all other processes.
b) Executes “control requests” submitted by the administrator.
c) Activate/Deactivate/Abort Concurrent Manager
d) Terminate Concurrent Request
e) Monitors processes, restarting any that failed.
f) Sets the target number of processes for each service based on the current work shift.
Question 9: How do I process more concurrent requests in parallel?
Answer
We can increase the target processes of the concurrent manager in order to increase the parallelism. This can be done using define concurrent manager form or through direct update from sqlplus
Question 10 : If the internal manager goes down, do I need to kill all the managers before restarting the internal manager?
Answer
No, if the internal manager goes down you need not kill all the managers. You can simply start the internal manager using startmgr.
Concurrent Manager troubleshooting
Question 11 : What are the problems u have faced while shutting down applications?
Answer
While shutting down application generally concurrent manager won’t go down because some or the other request may be running. We will see what are the concurrent requests running by querying fnd_concurrent_requests, fnd_concurrent_program_vl, v$session, v$process and v$sqltext.
If that request is only doing some select statement then we will kill those requests, otherwise we will check what time it will take to complete by querying the previous runs of that request and then we will decide what to do.
Oracle Concurrent Manager Secrets
Question 12: What are Internal Monitors ?
Answer: Internal Monitors are used specifically in PCP to allow for ICM failover to other available middle tier nodes.
a) Place an Internal Monitor on any node where the ICM can start in case of a failure.
b) Internal Monitors are seeded on every registered node by default.
c) If the ICM goes down, the Internal Monitor will attempt to start a new ICM on the local node.
d) If multiple ICMs are started, only the first will stay active. The others will gracefully exit.
Question 13: Can I delete concurrent manager?
Answer:
Yes, you can delete any concurrent manager. For deleting, query for the manager in the defined concurrent manager form and then delete the row.
Deleting the predefined concurrent managers is not recommended and it should never be done. Deletion may cause instability in the system.
Question 14: How can you know which trace file is created for the particular request?
Answer
You can find out the same using the script given below. The trace will be located in the udump location of the database server.
prompt
accept request prompt ‘Please enter the concurrent request id for the appropriate concurrent program:’
prompt
column traceid format a8
column tracename format a80
column user_concurrent_program_name format a40
column execname format a15
column enable_trace format a12
set lines 80
set pages 22
set head off
SELECT ‘Request id: ‘||request_id, ‘Trace id: ‘||oracle_Process_id, ‘Trace flag: ‘||req.enable_trace, ‘Trace Name: ‘||dest.value||’ ‘||lower(dbnm.value)||’ora’||oracle_process_id||’.trc’, ‘Prog. Name: ‘||prog.user_concurrent_program_name, ‘File name: ‘||execname.execution_file_name||execname.subroutine_name , ‘Status :’||decode(phase_code, ‘R’, ‘Running’)||’ ‘||’-‘||decode(status_code, ‘R’, ‘Normal’), “SID Serial: “||ses.sid||” , “||ses.serial#, “Module : “||ses.module
from fnd_concurrent_requests req,
v$session ses, v$process proc,
v$parameter dest, v$parameter dbnm,
fnd_concurrent_programs_v1 prog,
fnd_executables execname
where req.request_id = &request
and req.oracle_process_id=proc.spid(+)
and proc.addr = ses.paddr(+)
and dest.name=’user_dump_dest’
and dbnm.name=’db_name’
and req.concurrent_program_id =
prog.concurrent_program_id
and req.program_application_id =
prog.application_id
and prog.application_id =
execname.application_id
and
prog.executable_id=execname.executable_id;
Top 30 Most Useful Concurrent Manager Queries
Question 15: Explain how parallel concurrent processing(PCP) works?
Answer
In case of parallel concurrent processing, all the managers are assigned a primary and a secondary node. The managers are started in their primary node by default. In case of node failure or Oracle instance failure, all the concurrent managers on that node are switched to their secondary nodes. Once the primary node is available again the concurrent managers on the secondary nodes are migrated back to the primary node. During the migration process, a manager may be spread across both primary and secondary nodes.
In case of parallel concurrent processing, it may happen that in a node where parallel concurrent processing is configured, the Oracle instance may or may not be running. The node which is not running Oracle, the concurrent managers connects via Net8 to a node which is running Oracle.
The internal concurrent manager can run on any node, and can activate and deactivate concurrent managers on all nodes. Since the internal concurrent manager must be active at all times, it needs high fault tolerance. To provide this fault tolerance, parallel concurrent processing uses internal monitor processes. The job of the internal monitor process is to constantly monitor the internal manager and start it when it fails. Only one internal monitor process can be active on a single node. You decide which nodes have an internal monitor process when you configure your system. You can also assign each internal monitor process a primary and a secondary node to ensure fail over protection. Internal monitor processes, like concurrent managers, can be assigned work shifts, and are activated and deactivated by the internal concurrent manager.
Parallel Concurrent Processing
Question 16: What are the circumstances in which you need to bounce the concurrent manager?
Answer: There can be many situation where you need to bounce the concurrent manager
a) When you modify the definition of the printers
b) When you modify the environment variables. Suppose you have changed the APPLTMP and APPLPTMP variable.
c) When all the requests are pending and hanging and no processing happening
d) patch application requires the bounce of the CM
e) We have many global hangs in the system due to locks by several concurrent manager and other processes
Question 17: What are the reasons a concurrent manager hangs?
Answer:
The concurrent manager hangs due to many reasons. A few of them are:
– Long running jobs
– The internal manager was activated by someone other then owner of the application system
– The operating system files system is full
– It’s not able to create the log file
– You’ve shut down the internal manager, but actual has a number in it
– The database is hanging may be because the archive log files have filled
– Pending/standby requests are too many
Question 18: How can we enable/disable Conflict Resolution Manager?
Answer: It can be done using profile options “Concurrent: Use ICM”. Set it to “Y” to enable Conflict Resolution manager. To disable it, set the profile option to “N”.
Question 19: What are Transaction Managers?
Answer : Transaction managers provide synchronous job processing by continually monitoring a DBMS pipe for requests to come through from a client-side application. The job of a transaction manager is to process this job immediately and send information back to the client using the pipe.
a) Transaction Managers Provide Synchronous Job Processing
b) A client makes a request for a specific transaction manager to run a program, and waits for the results of that program
c) Product teams’ programs are linked directly into the transaction manager executables
d) PO, CRP, INV, AR, and OE all ship transaction managers
Question 20: How the mechanism of viewing log and output files works from browser?
Answer
The sequence of events is as follows:
1. A user within an Applications session requests to view a log or out file.
2. The browser receives the request and spawns the cgi program FNDWRR.exe
3. FNDWRR.exe logs into the database and queries FND_CONCURRENT_REQUESTS to discover on which node the files for this request are stored.
4. FNDWRR.exe constructs the service name for the file server on that node. And makes the tns call to contact the listener for this service name.
5. The listener responds by spawning the local FNDFS executable, as defined in it’s listener.ora file. Now, FNDFS and FNDWRR.exe are able to communicate directly now, using RPC calls.
6. FNDWRR.exe requests FNDFS to transfer the file that was selected by the user.
7. FNDFS transfers the file contents to a temporary file directory on the web server’s node.
8. The web server displays the file contents to the user.
Question 21: Why does the Concurrent manager put a concurrent program into a queue? Why doesn’t the manager simply let the program run?
Answer: Because at any given point in time a concurrent manager can run no more than say 10 programs concurrently. This figure of 10 is configurable of course. First the manager puts a submitted program into a queue, next the manager checks if there is a slot available (i.e. Less than 10 programs are currently running). If a slot is found available, the concurrent manager then runs the program, or else it keeps the concurrent program in a queue with status Pending.
Question 22: What to do if a request is Inactive/No Manager
Answer: This is quite tricky questions
a) Make certain that there is at least one active manager with specialization rules that allow the program.
b) If you have confirmed the previous point, then the problem may be a stale Worker Request View
– The view is used internally to map requests to managers
– The view is regenerated when managers are created, or specialization rules are altered
c) You can manually regenerate the view
FNDLIBR FND FNDCPBWV apps/apps SYSADMIN ‘System Administrator’ SYSADMIN
Question 23: What are Service Managers
Answer: Service Managers are spawned on the middle-tier nodes of a GSM enabled system in order to act as an agent of the ICM. When the ICM sees that it needs An Service Manager to perform some function, such as start a concurrent manager process, on a middle-tier node, it will make remote procedure control (RPC) calls to the Apps listener on that node to start the Service manager. Once the Service manager has been started and initialized, the ICM communicates directly to the SM through RPC, giving it information to manage the services on that node. The SM is spawned from the APPS TNS Listener.The APPS TNS Listener must be started on every middle-tier node in the system, and started by the user that starts ICM (e.g. applmgr) .TNS Listener spawns Service Manager to run as agent of ICM for the local node
The Service Manager is started by ICM on demand when needed. If no management actions are needed on a node a Service Manager will not be started by ICM until necessary. When ICM exits its Service Managers exit as well.
Question 24: What all is performed by ICM Process Monitor?
Answer: The ICM itself and each process spawned by the ICM has an entry in FND_CONCURRENT_PROCESSES and holds a uniquely named DBMS lock.
The unique lock of the ICM has a format of FNDCPLK_ICM. This database session lock is the method that the ICM ensures each PMON cycle that manager and service processes are still alive. If the ICM can get the DBMS session lock of a process, the ICM will start a new process for that manager or service.
This is the reason you will often see entries like dead process found ,starting new process in ICM log files.
Comments
Post a Comment