Ramesh Kodali
2013-07-26 18:28:32 UTC
Ramesh Kodali [https://community.jboss.org/people/kodali.rameshbabu] created the discussion
"JBoss Connection pool not recovered after a database outage with back ground validation enabled"
To view the discussion, visit: https://community.jboss.org/message/830034#830034
--------------------------------------------------------------
Hi,
To give a littler backround of the problem,
Recently we had a data base outage in production due to some locks applied on tables(due to abnormal termination of DB update jobs) and it causes all the connections in JDBC connection pool waiting at data base and waiting to get hold of tables. After some time, it leads all the connections in the connection pool got exhausted and no more conenctions in the pool. Aplication server started throrwing connection errors for new coming request as no more conenctions available in pool.
DBA killed those waiting threads and restarted data base instance. Though after DB servers restarts, still App servers can not recover the connections in connection pool and it forces us to restart App servers too.
After App servers restarts, problem solved.
*Now question is:* Why JBoss App servers required to restart after DB servers are back up? Will JBoss connection pool does not have mechanism to recover connections automatically?
P.S: Other team in our company not enabled the back ground validation in their ds.xml, their App servers does not require restart after DB back up. Their App servers picked up connections automatically.
What makes the back ground validation different here?
Your valueble inputs are much apprecitated. Also if it a problem, can you please suggest the solution to fix this.
We are using JBoss AS 5.0
Here below our JBoss ds.xml configuration:
We are using the Informix data base.
<local-tx-datasource>
       <jndi-name>JNDIName</jndi-name>
       <connection-url>Connection_URL</connection-url>
       <driver-class>com.informix.jdbc.IfxDriver</driver-class>
       <!-- Use the security domain defined in conf/login-config.xml -->
       <security-domain>Sample-POLICY</security-domain>
       <depends>jboss.security:service=JaasSecurityDomain,domain=ServerMasterPassword</depends>
       <new-connection-sql>SET LOCK MODE TO WAIT </new-connection-sql>
       <check-valid-connection-sql>select * from sysmaster:sysdual</check-valid-connection-sql>
       <min-pool-size>5</min-pool-size>
       <max-pool-size>200</max-pool-size>
  <idle-timeout-minutes>3</idle-timeout-minutes>
       <background-validation-millis>20000</background-validation-millis>
       <validate-on-match>false</validate-on-match>
       <query-timeout>15</query-timeout>
       <set-tx-query-timeout>true</set-tx-query-timeout>
       <prepared-statement-cache-size>20</prepared-statement-cache-size>
       <exception-sorter-class-name>org.jboss.resource.adapter.jdbc.vendor.InformixExceptionSorter
       </exception-sorter-class-name>
       <metadata>
           <type-mapping>InformixDB</type-mapping>
       </metadata>
</local-tx-datasource>
--------------------------------------------------------------
Reply to this message by going to Community
[https://community.jboss.org/message/830034#830034]
Start a new discussion in Datasource Configuration at Community
[https://community.jboss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2077]
"JBoss Connection pool not recovered after a database outage with back ground validation enabled"
To view the discussion, visit: https://community.jboss.org/message/830034#830034
--------------------------------------------------------------
Hi,
To give a littler backround of the problem,
Recently we had a data base outage in production due to some locks applied on tables(due to abnormal termination of DB update jobs) and it causes all the connections in JDBC connection pool waiting at data base and waiting to get hold of tables. After some time, it leads all the connections in the connection pool got exhausted and no more conenctions in the pool. Aplication server started throrwing connection errors for new coming request as no more conenctions available in pool.
DBA killed those waiting threads and restarted data base instance. Though after DB servers restarts, still App servers can not recover the connections in connection pool and it forces us to restart App servers too.
After App servers restarts, problem solved.
*Now question is:* Why JBoss App servers required to restart after DB servers are back up? Will JBoss connection pool does not have mechanism to recover connections automatically?
P.S: Other team in our company not enabled the back ground validation in their ds.xml, their App servers does not require restart after DB back up. Their App servers picked up connections automatically.
What makes the back ground validation different here?
Your valueble inputs are much apprecitated. Also if it a problem, can you please suggest the solution to fix this.
We are using JBoss AS 5.0
Here below our JBoss ds.xml configuration:
We are using the Informix data base.
<local-tx-datasource>
       <jndi-name>JNDIName</jndi-name>
       <connection-url>Connection_URL</connection-url>
       <driver-class>com.informix.jdbc.IfxDriver</driver-class>
       <!-- Use the security domain defined in conf/login-config.xml -->
       <security-domain>Sample-POLICY</security-domain>
       <depends>jboss.security:service=JaasSecurityDomain,domain=ServerMasterPassword</depends>
       <new-connection-sql>SET LOCK MODE TO WAIT </new-connection-sql>
       <check-valid-connection-sql>select * from sysmaster:sysdual</check-valid-connection-sql>
       <min-pool-size>5</min-pool-size>
       <max-pool-size>200</max-pool-size>
  <idle-timeout-minutes>3</idle-timeout-minutes>
       <background-validation-millis>20000</background-validation-millis>
       <validate-on-match>false</validate-on-match>
       <query-timeout>15</query-timeout>
       <set-tx-query-timeout>true</set-tx-query-timeout>
       <prepared-statement-cache-size>20</prepared-statement-cache-size>
       <exception-sorter-class-name>org.jboss.resource.adapter.jdbc.vendor.InformixExceptionSorter
       </exception-sorter-class-name>
       <metadata>
           <type-mapping>InformixDB</type-mapping>
       </metadata>
</local-tx-datasource>
--------------------------------------------------------------
Reply to this message by going to Community
[https://community.jboss.org/message/830034#830034]
Start a new discussion in Datasource Configuration at Community
[https://community.jboss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2077]