I have always shown people 'verbose' that is the way the heap is utilized when threads are acting on application. You have good tools to read verbose file generated by servers and plot it for better UI(I prefer IBM PMAT, pattern modelling and analysis tool).
A more intelligent set of people (esp in java) will not be all happy with this only. They would ask .."Which objects are there in heap?" ..Verbose has no answer. A Heap-dump would be having the answer.
The heap dump refers to a file (hprof or txt or phd file) that CAN be collected during the application-run. This is 'stop-the-world' and lengthy process and it will force JVM for compaction also, that means its not a thing to trigger every 10 minutes...although it can be! (For some servers, this has to be enabled before starting the server)
A heap dump obtained is a big file with cryptographic data present in it if opened in simple text editors. This can be read here but beyond my scope at this point. This has to be opened by specialized tool for reading heapDumps (like MDD4J, IBM-HeapDumpAnalyzer, Eclipse MAT --The best so far I found).
Once opened, you will find a lot of object names present at heap when the snapshot was taken. The UI will show number of each objects, size of each object. Thats ok. The best thing it has is the chain. Not all objects are created directly by the programmer. And once the class unloads, all its objects are cleared..great! No!.. Some objects due to bad coding remains forever.
Example: Cache Objects(implemented without soft referencing)
Other important object that stays in heap are due to big architectural dependencies, these are the objects like Connection, which gets created from a pool (usually) and is binded to various objects, something like Driver/Factory binding. so basically some object (say A) is created by B (directly) and registered by C (indirectly like binding). As B unloads, A remains in the heap as it has reference(C) living, that will kill automatic cleaning up feature of JVM.
There can be various other things available for keeping the object alive. The worst part is, in a heap you cant find the age of the object (I couldn't). Each big object (size wise and number wise) must be traversed to its dominator (parent object) to catch the leak....leak???... yes all the time B loads, it creates A' and A'' A''' A'''' and we have a lot of A's in heap (since C is something that HAS to be on heap and it holds ref. to A's) and one day our Java code will throw "OutOfMemoryException" ...yes thats like loop forever and create new Object in a loop!
God gifted quality of bad coding practices can lead to disaster.. this is not functional defect... This is quality defect!.. The code is perfect if code just means giving desired output. Yes it gives desired output but may fail to do so forever and absolutely not when concurrently accessed by million users forever.
To find these errors, we cant go through code and find defect..Not possible, We might find practice to be perfect but results are disastrous ...so ONLY one hope...."HEAP" dear coders.... please start reading heap!
I am doing these nowadays..Once I get more info... I ll share! thanks!
ZeeK
Wednesday, February 17, 2010
Tuesday, January 5, 2010
playing JAVA - PDF - Image Addition
I found this very interesting...
iText.jar
use this
and enjoy writing on pdf file (even images) from java.
Queries on classpath.. lol try all bin lib jre jdk -classpath %PATH% google... If it doesnt help please ask me.
Tutorial for same in roseIndia is available but perhaps the import statement is wrong with them. change the import :)
iText.jar
use this
and enjoy writing on pdf file (even images) from java.
Queries on classpath.. lol try all bin lib jre jdk -classpath %PATH% google... If it doesnt help please ask me.
Tutorial for same in roseIndia is available but perhaps the import statement is wrong with them. change the import :)
Sunday, August 30, 2009
The JVM-Garbage To Begin with
Recently I was asked to asked to analyze the behavior of heap in java application (with lots of short lived objects and highly transactional processes).
Here is what I came to know :-
Java is a programming language, where object creation and destruction is handled by JVM itself and this indeed is handled by one of its process called GC (garbage collector process). Four algorithm is available (not specific to Java vendor), For the defined case, gencon is considered to be best.
For most of the cases, gencon policy will work fine
Heap (memory) is divided into two physical part: Nursery (NEW) and Tenured (OLD) Space, Nursery space is logically further divided into Allocated (Eden) and Survivor Space.
New (young) objects gets a place in Eden.
Once the eden is full (or partly filled depends on the setting (JVM option) ), the process called SCAVENGE is called (in gencon policy of garbage collection).
This process will clean up the eden space i.e, deallocates the memory space of the objects no longer in use, and copies the objects to 'survivor' space (or tenured space depending upon the age of object).
As soon as the scavenger is called, the initial eden space is now referred as survivor space (yes empty) and objects in previous survivor space (yes now eden/allocated space) gets their age increased by 1.
As stated previously, the objects can get copied logically to survivor or physically to Tenured area. The destination depends upon the "ThreshholdTenuredAge" parameter (in JVM setting or dynamically set by JVM in default case)
Scavenge process is light weight (wrt CPU utilization), frequent scavenge is called (as the application supposedly had lots of short lived objects)
The major overhead while cleaning garbage is clearing up Tenured Area, where the objects with high age has been allotted space. This is done by GLOBAL garbage collection, which is triggered when the space in Tenured in not enough to accommodate new object from nursery to tenured area (through scavenge; this is called Allocation Failure). Global GC clears up Tenured Area and frees the memory!
Global GC(and scavenge also) is "Stop the World" Phase where the application thread is paused and ONLY gc thread works.
Garbage collection process mainly has three phases (and all the phases work as single step (stop the world, hehe) )
1. Mark - That is marking all LIVE objects
2. Sweep - Cleaning/ freeing and updating free list.
3. Compact - (optional) reduce fragmentation (time taking)
Bad coding is natural gift to all the expert programmers (no offense) still this can get worse if the JavaHeap is not perfectly set, that is BAD JVM OPTION.
For instance, The JVM OPTION -Xmx512m means set maximum heap size to 512MB and the java application regularly throws OOM Exception(OutOfMemory Exception), tuning is required.
Need not be happy, the above objective case is not always the situation.
The case can be really complicated as improving performance of an application based on Memory leaks or availability is a tough and subjective task.
Google for Java Vendor Specific JVM OPTIONs (depends on version too) and do dome permutation to tune in a good application. However, the theoretical aspect can by analyzed to reduce the set of eligible jvm settings.
In order to analyze the garbage collection, enable the verbose in JVM setting (as the name suggests, verbose (who speaks a lot) is a log which tells the whole story of heap in elaborated xml format. This complicated log can be analyzed on various freely available tool and hence the memory behavior can be studied. Later parameters like fixing nursery size, tenured size, total heap size, size ratio, Garbage collection policy, max tenured age etc can be modified.
Make sure while tuning, you degrade one feature to get another, settle as soon as end user says OK.
Happy tuning
(please google for various JVM options)
Here is what I came to know :-
Java is a programming language, where object creation and destruction is handled by JVM itself and this indeed is handled by one of its process called GC (garbage collector process). Four algorithm is available (not specific to Java vendor), For the defined case, gencon is considered to be best.
For most of the cases, gencon policy will work fine
Heap (memory) is divided into two physical part: Nursery (NEW) and Tenured (OLD) Space, Nursery space is logically further divided into Allocated (Eden) and Survivor Space.
New (young) objects gets a place in Eden.
Once the eden is full (or partly filled depends on the setting (JVM option) ), the process called SCAVENGE is called (in gencon policy of garbage collection).
This process will clean up the eden space i.e, deallocates the memory space of the objects no longer in use, and copies the objects to 'survivor' space (or tenured space depending upon the age of object).
As soon as the scavenger is called, the initial eden space is now referred as survivor space (yes empty) and objects in previous survivor space (yes now eden/allocated space) gets their age increased by 1.
As stated previously, the objects can get copied logically to survivor or physically to Tenured area. The destination depends upon the "ThreshholdTenuredAge" parameter (in JVM setting or dynamically set by JVM in default case)
Scavenge process is light weight (wrt CPU utilization), frequent scavenge is called (as the application supposedly had lots of short lived objects)
The major overhead while cleaning garbage is clearing up Tenured Area, where the objects with high age has been allotted space. This is done by GLOBAL garbage collection, which is triggered when the space in Tenured in not enough to accommodate new object from nursery to tenured area (through scavenge; this is called Allocation Failure). Global GC clears up Tenured Area and frees the memory!
Global GC(and scavenge also) is "Stop the World" Phase where the application thread is paused and ONLY gc thread works.
Garbage collection process mainly has three phases (and all the phases work as single step (stop the world, hehe) )
1. Mark - That is marking all LIVE objects
2. Sweep - Cleaning/ freeing and updating free list.
3. Compact - (optional) reduce fragmentation (time taking)
Bad coding is natural gift to all the expert programmers (no offense) still this can get worse if the JavaHeap is not perfectly set, that is BAD JVM OPTION.
For instance, The JVM OPTION -Xmx512m means set maximum heap size to 512MB and the java application regularly throws OOM Exception(OutOfMemory Exception), tuning is required.
Need not be happy, the above objective case is not always the situation.
The case can be really complicated as improving performance of an application based on Memory leaks or availability is a tough and subjective task.
Google for Java Vendor Specific JVM OPTIONs (depends on version too) and do dome permutation to tune in a good application. However, the theoretical aspect can by analyzed to reduce the set of eligible jvm settings.
In order to analyze the garbage collection, enable the verbose in JVM setting (as the name suggests, verbose (who speaks a lot) is a log which tells the whole story of heap in elaborated xml format. This complicated log can be analyzed on various freely available tool and hence the memory behavior can be studied. Later parameters like fixing nursery size, tenured size, total heap size, size ratio, Garbage collection policy, max tenured age etc can be modified.
Make sure while tuning, you degrade one feature to get another, settle as soon as end user says OK.
Happy tuning
(please google for various JVM options)
Subscribe to:
Posts (Atom)