[jboss-user] [Performance Tuning] - JBOSS Hanging Problem

Discussion:

Mihir Sanghvi

2013-03-15 06:49:59 UTC

Mihir Sanghvi [https://community.jboss.org/people/sanghvimihir] created the discussion

"JBOSS Hanging Problem"

To view the discussion, visit: https://community.jboss.org/message/802847#802847

--------------------------------------------------------------
Hi

We have a JBOSS hanging problem at a cliant location, due to which we are having to restart the server every few hours. I have a quich problem description from my team member below. Can someone please help us in finding the solution. Can provide more details on request.

Jboss is hanging when someone clicks on certain links/some functinalities of the Application .

Radoslav Husar

2013-03-15 09:04:21 UTC

Permalink

Radoslav Husar [https://community.jboss.org/people/rhusar] created the discussion

"Re: JBOSS Hanging Problem"

To view the discussion, visit: https://community.jboss.org/message/802869#802869

--------------------------------------------------------------
Maybe your application has a infinite loop somewhere? What objects are being added to PermGen? Obtain a report using jstack and see where its stuck:

http://docs.oracle.com/javase/1.5.0/docs/tooldocs/share/jstack.html http://docs.oracle.com/javase/1.5.0/docs/tooldocs/share/jstack.html

Also, your AS version is pretty old, upgrade to newer.
--------------------------------------------------------------

Reply to this message by going to Community
[https://community.jboss.org/message/802869#802869]

Start a new discussion in Performance Tuning at Community
[https://community.jboss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2078]

Mihir Sanghvi

2013-03-18 10:32:54 UTC

Permalink

Mihir Sanghvi [https://community.jboss.org/people/sanghvimihir] created the discussion

"Re: JBOSS Hanging Problem"

To view the discussion, visit: https://community.jboss.org/message/803216#803216

--------------------------------------------------------------
Thanks Redoslav and Peter. My technical team memeber is looking into this. Will revert with observations.
--------------------------------------------------------------

Reply to this message by going to Community
[https://community.jboss.org/message/803216#803216]

Start a new discussion in Performance Tuning at Community
[https://community.jboss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2078]

Mihir Sanghvi

2013-03-20 14:36:21 UTC

Permalink

Mihir Sanghvi [https://community.jboss.org/people/sanghvimihir] created the discussion

"Re: JBOSS Hanging Problem"

To view the discussion, visit: https://community.jboss.org/message/803837#803837

--------------------------------------------------------------
We dont seem to be getting any of the usual problems like OOM, errors in log files etc.

It may quite be possible that the databse/data is eating up memory due to bad queries. Any way to validate or monitor memory usage from database using some tool?
--------------------------------------------------------------

Reply to this message by going to Community
[https://community.jboss.org/message/803837#803837]

Start a new discussion in Performance Tuning at Community
[https://community.jboss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2078]

Peter Johnson

2013-03-20 14:43:47 UTC

Permalink

Mihir Sanghvi

2013-03-20 14:57:50 UTC

Permalink

Mihir Sanghvi [https://community.jboss.org/people/sanghvimihir] created the discussion

"Re: JBOSS Hanging Problem"

To view the discussion, visit: https://community.jboss.org/message/803846#803846

--------------------------------------------------------------
So what we understand is that the app server is not really getting hanged. But as time passes that application becomes painstakingly slow and it "looks" like as if the server hanged, so the users are just restarting the server.

Ill try and get you the results of the thread dump from my tech team. Meanwhile, do you still think it is a java problem and not a databse problem?
--------------------------------------------------------------

Reply to this message by going to Community
[https://community.jboss.org/message/803846#803846]

Start a new discussion in Performance Tuning at Community
[https://community.jboss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2078]

Peter Johnson

2013-03-20 15:15:00 UTC

Permalink

Peter Johnson [https://community.jboss.org/people/peterj] created the discussion

"Re: JBOSS Hanging Problem"

To view the discussion, visit: https://community.jboss.org/message/803855#803855

--------------------------------------------------------------
That could be caused by one of two problems:

a) You have one or more requests that are "hanging". When they hang, the user might close the browser and try again. For example, there might be an infinite loop in the code which from that point on consumes processing power (and memory, but that usually is not too significant, but that all depends on the app). Or there might be some other haning issue such as a database deadlock. A thread dump will find this. (I have had people swear that there was no such issue in their code, yet while the system was idle - no requests coming in - the CPU was running at some high percentage. A thread dump showed exactly where the loop was.) Eventually, all of the threads in the http request thread pool get used up any no-one can get on the system any more. (Just yesterday I helped some colleagues find such an issue in their portal code using a thread dump.)

b) You have a memory leak. Eventually the heap fills up to the point that the JVM has to constantly take full garbage collections. You can verify this by turning on one of the GC logging options (-verbose:gc should be enough to catch a heap problem, also us -Xloggc:xxx to log to a file). One way to find a memory leak is to attach JVisualVM to the process and use it to take heap snapshots about 10 or 15 minutes apart while the system is busy. Then have it compare the heap snapshots and show you the differences. That should tell you which object(s) have increased in numbers.
--------------------------------------------------------------

Reply to this message by going to Community
[https://community.jboss.org/message/803855#803855]

Start a new discussion in Performance Tuning at Community
[https://community.jboss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2078]

Mihir Sanghvi

2013-03-21 13:12:46 UTC

Permalink

Bhalchandra Madhekar

2013-06-11 19:33:56 UTC

Permalink

Bhalchandra Madhekar [https://community.jboss.org/people/bmadhekar] created the discussion

"Re: JBOSS Hanging Problem"

To view the discussion, visit: https://community.jboss.org/message/822645#822645

--------------------------------------------------------------
try to check memory allocation on jConsole, additionally analyze the application characteristics to determine what type of objects are created based on targeted/ specific test cases.
--------------------------------------------------------------

Reply to this message by going to Community
[https://community.jboss.org/message/822645#822645]

Start a new discussion in Performance Tuning at Community
[https://community.jboss.org/choose-container!input.jspa?contentType=1&containerType=14&container=2078]

Peter Johnson

2013-03-15 14:44:58 UTC

Permalink