OEM 13c Configure agent to monitor WebLogic Servers secured with custom certificate

When adding "Oracle Fusion Middleware/WebLogic Domain" to EM 13c with t3s (instead of t3) JMX Protocol, you may get following errors

Failed to discover WebLogic: java.lang.RuntimeException:-java.lang.RuntimeException:-javax.naming.CommunicationException-t3s://wls.dbaplus.ca:7002: -Destination-192.15.49.108,-7002-unreachable;-nested-exception-is: ---javax.net.ssl.SSLHandshakeException: -Received-fatal-alert: -protocol_version; -No-available-router-to-destination. No targets discovered. Check host, port, credentials, and protocol and be sure the server is running.

Check agent trace file <AGENT_INST>/sysman/log/emagent_perl.trc

oracle_wls.pl: 2020-04-26 11:23:42,899: WARN:  Start_discovery_output---------------
oracle_wls.pl: 2020-04-26 11:23:42,899: WARN:  No logging has been configured and default agent logging support is unavailable.
 Handshake failed: TLSv1.3, error = No appropriate protocol (protocol is disabled or cipher suites are inappropriate)
 Handshake failed: TLSv1.2, error = PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
 Handshake failed: TLSv1.1, error = PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
 Handshake failed: TLSv1, error = Received fatal alert: protocol_version
 Handshake failed: TLSv1.3, error = No appropriate protocol (protocol is disabled or cipher suites are inappropriate)
 Handshake failed: TLSv1.2, error = PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
 Handshake failed: TLSv1.1, error = PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
 Handshake failed: TLSv1, error = Received fatal alert: protocol_version
 FATAL: Error in discovery : Exception=java.lang.RuntimeException: java.lang.RuntimeException: javax.naming.CommunicationException t3s://wls.dbaplus.ca:7002: Destination 192.15.49.108, 7002 unreachable; nested exception is:
        javax.net.ssl.SSLHandshakeException: Received fatal alert: protocol_version; No available router to destination

If the WebLogic Server (WLS) is configured with custom certificate and you can find message 'unable to find valid certification path to requested target' as previous sample trace file, the issue can be fixed by importing the certificate of each CA involved in issuing the custom certificate into agent local keystore with following command,

  $AGENT_HOME/bin/emctl secure add_trust_cert_to_jks -trust_certs_loc <ca_certificate_file> -alias <certificate_alias> [-password <keystore_pwd>]

Here, <certificate_alias> is used to identify the certificates saved in the keystore, they must be unique for each certificate, <keystore_pwd> is the password of the keystore, the default value is welcome.

For example

emctl stop agent
emctl secure add_trust_cert_to_jks -password welcome -alias dbaplus-root -trust_certs_loc /home/oracle/Root_CA_Certificate.txt
emctl secure add_trust_cert_to_jks -password welcome -alias dbaplus-intermediate -trust_certs_loc /home/oracle/Intermediate_CA_Certificate.txt
emctl start agent

To list the certificates imported into agent monitor keystore,

 $AGENT_HOME/jdk/bin/keytool -list -alias <certificate_alias> -keystore   $AGENT_INSTANCE_HOME/sysman/config/montrust/AgentTrust.jks -storepass welcome -v

If needed, the certificates can be removed from keystore as following

 $AGENT_HOME/jdk/bin/keytool -delete -alias <certificate_alias> -keystore   $AGENT_INSTANCE_HOME/sysman/config/montrust/AgentTrust.jks -storepass welcome -v

Instead of checking trace file "emagent_perl.trc", more details of target discovery errors can be retrieved as following,

In the "Advanced" section of EM Discovery wizard, configure "External Parameters" or "Discovery Debug File Name" to assign a debug file name. Agent will write discovery message into the file with given file name under directory <AGENT_INST>/sysman/log.

The format of assigning debug file name in "External Parameters" is "-DFMWDiscoveryAgLogFile=<Log file name>", it can be used together with other parameters here. These parameters will be passed to the java process which makes a connection to the Administration Server. All the parameters must begin with -D.

Value for "Discovery Debug File Name" is pretty straight, just enter a valid OS file name. The agent side discovery messages for this session will be logged into this file. If this file already exists, it will be updated.