Sunday, July 13, 2008
Design Pattern Tutorial (Design Pattern 101)
So, make up your mind to read further.
It has helped so many other developers and hence it should also help you. And one more interesting fact is that a lot of developers use design patterns without being aware that they are using one. For example, if you have used wait(), notify(), notifyAll() is an observer pattern. So you have been using it without knowing. Then why should you learn it. Because all the design patterns are solution to specific problem and focused towards object oriented concepts, reusability, maintainability. And also, it is not just one, there are 23 core design patterns and each one of them have an intent and the specific way of application. It is a reusable concept/solution.
Apart from the solution, the design pattern let the developers to speak a common and a quick language. For example, if you ask someone "how to go to a place where lot of people are standing on a platform to go to a distant place. The people will be carried by a series of wagons that are connected and pulled by an electric or diesel engine. Also, the entire vehicle goes on two tracks that are separated by 1.5 metres". What would be the answer? Aren't you elaborating and complicating too much. Instead of asking the way to railway station, you simply beat around the bush and simply irritating the stranger. The very same (shame) thing happens if you do not know design patterns. Design patterns improves your productivity by making you solve problems in a right way and also to have your discussion short and to the point.
These are some of the reasons why one should know design patterns. After all, it is you, who is going to benefit by learning it. You ll end up giving a better and nicer software to your customer.
Watch out for a series of blogs on patterns.
Upcoming Write-up: Types of Design Patterns.
Saturday, July 12, 2008
JVM it too Hot with HotSwap
Apart from being specifications and methodologies to write tools, these specification can help you a lot to understand JVM deeply. Recently, I was reading through HotSwap and suddenly wanted to write a simple tool using which I can reload classes in a live remote JVM. I was successful in loading the class. The only disadvantage is that you cannot reload the class if you have added any new members to it. However, this is the limitations of JVM which may be addressed in upcoming Java major release. The rest of this write up gives you, the specification and algorithm (simple steps) used.
Specification
Write a simple tool which let you to reload classes in a running JVM without restarting your application. Also provide a simple user interface (probably using SWING).
Steps
1. Use Java Debug Interface API to connect to remote JVM. In order to connect to remote JVM, you the hostname/ip address of the remote box and port number on which your JVM runs. In order to hotswap the classes, you need start JVM in debug mode and if you start JVM using "socket" debugger.
2. Also subscribe for few important events such as JVM Exit, Disconnect Event. (for more info refer Java Debug Interface API Spec).
3. If the connection is successful, you get an instance of the remote JVM and you can once again use JDI to replace classes.
4. You have to write the logic of reading the class file (flat file) from the file system and give to JDI. JDI pumps the class file to remote JVM which reloads the class after making lot of checks.
Friday, September 14, 2007
Precious book on Java – Effective Java by Joshua Bloch
By Java, I mean Core Java. This book made to understand the elegance of Java and its strong APIs. While reading I often refer Java Libraries written by the author of this book. Each of his words has a meaning to it. The one two three four chapters I like most are Threads, Exceptions, Object Creation and Deletion and of course Classes and Interfaces. Though, the other chapters are equally good, I particularly like these four chapters because they are cornerstone to Java. The author has given a lot of Best Practices and if you apply those best practices, your code is much maintainable, readable and comprehensible.
Threads offer you a greater flexibility but writing thread safe application is harder. The current day applications have seriously bugs and if they are running properly, it is nothing more than mere coincidence. The book also explains the results of over synchronization and wait/notify. The chapter on exceptions is more fulfilling and it gives two great thoughts – Exception Chaining and Exception Translation which is handy when your application has many layers. The book gives thoughts on object creation, object deletion, classes and interfaces.
This is a right book for you to have a copy of this book if you are really interested in writing code effectively. This book helps you to think Java in an art from.
Sunday, September 9, 2007
Favor Composition (“has a”) over Inheritance (“is a”)
!!! Composition and Inheritance should complement each other !!!
You need to implement different types of sorting algorithm. But there are many sorting available and you should implement bunch of them. Based on the client requirement, you need use any one of the sorting algorithm (when there is low memory, you need to go for insertion sort but if the memory is high you can go for quick or merge sort). The bottom line is the client knows which algorithm is needed and your framework has to do the job. If the memory is low, the clients decide to go with insertion sort and the framework needs to use the sorting algorithm. Also, if the client is interested, the client should be able to fit in their own algorithm “weird sorting” into your framework. How will you go about with this problem? How will you design classes?
There are two ways of solving this problem. The first way is very crudest way where you have all the sorting algorithms implemented in a same class. The single class will have methods – binarySort, insertionSort, heapSort and so on. This straight away blow up the design principle – open close principle. For adding up new sorting “weird sorting”, you need add a new “weird” method. It produces a maintenance nightmare. The second way is slightly smarter way where in the sorting algorithms are implemented as class for each class inheriting from a class “Sorting” which is abstract. But the clients have to use them based on their requirement and most importantly they cannot change the sorting algorithm dynamically.
The third approach to this problem is implement an abstract class or an interface “Sorting” that has a method “sort”. Each sorting algorithm implements this “sort” method and as the result you have many sorting algorithm. When you want add “weird sorting”, it is as simple as to add new class implementing the “sort” method. Your code follows open close principle and this avoids a lot of testing. You can for sure say that your new code does not introduce a bug in the old code. So far, we talked about inheritance. This is usual stuff.
How will you allow others to invoke the sorting method? You need to extend each of the sorting class and so that others invoke the “sort method”. But this method leads to class explosion. When a new sorting is implemented, you need to change/add code. But instead of doing this, you can have the sorting algorithm as a component with a “has a relationship”. For example,
public class Client {
private Sorting sortingAlgorithm;
public void setSortingAlgorithm(Sorting sortingAlgorithm) {
this.sortingAlgorithm = sortingAlgorithm
}
public void someOperation() {
sortingAlgorithm.sort(); //first
/// some operation
sortingAlgorithm.sort(); //second
}
Consider the method someOperation() of Client class. Also assume that Client is a shared object and so many people decide on the particular sorting algorithm. Now the sorting algorithm can be changed dynamically based on various factors. If your application has memory management module, it can play its part to decide on the particular sorting algorithm. In the above example, the first method could be a different sorting and the second sorting could be a different sorting. This, what we mean by flexibility.
In order to engage people in using composition, most developers argue the words “Favor Composition over Inheritance”. These words are simply phrased to give you the power of composition. These words should not be taken literally and no composition works greatly without employing inheritance. So both “Composition”and “Inheritance” should complement each other in a true object oriented perspective. It is time to etch
!!! Composition and Inheritance should complement each other !!!
Open for Extension and Closed for Modification
This is one of the fundamentally design thought that every designers should deeply analysis. When we say open, it does not necessarily carry the real meaning of “open” and close does not imply the real meaning of “close”. In the context of designing, these two words are related to level of flexibility given to programmers/developers. This fundamentally principle suggests that an application design should be flexible enough to accommodate features to the application.
But a sudden spike of thought that comes to our mind – for every product release, we add a lot of features. Before proceeding further, let us ask some questions.
- Does your product have added few features for each release?
- Do you feel that the base architecture remain fairly stable over many releases?
- Do the developers know the consequences to their features when the underlying framework changes?
- How easy or tough adding of features are? (Easy or tough is very relative J )
- How this fundamental design principle helps the framework to be flexible?
There will be two scenarios where the developers will modify the code. When the software is buggy, they do not have any other way but to modify the code. The developers typically identify the root cause of the bug and fix them. But some of the developers open the code to add new features. The modification of the existing code for adding new features conveys that there is something wrong. It is a bad design for a framework if it forces it clients to modify the code when adding new features. Apart from forcing the modifications, the design should not force itself to undergo a massive code change to support new functionalities. When it comes to framework, the designers should also see the framework from another perspective apart from providing basic functionalities. The perspective is that the framework is just a contract or guidelines given to the clients. The bottom line is the framework is just contract that governs and facilitates proper functioning of the applications. When it comes to an application, the application should foresee some changes in requirements and burn flexibilities in the design to accommodate those requirements.
The design principle, “Open for Extension; Closed for Modification” is a great tool and thought for building software. This is just a thought and for achieving this one needs a lot of things to be done and known. In few of my next blogs, I ll be writing on the design principles and these design principles could be a one-liner, a design pattern that is commonly needed or concepts taken from Java library.
Wednesday, September 5, 2007
Improving Test Coverage using Code Coverage
- Are we doing it like a black box testing without looking at the source code?
- Do we look at the source code during unit testing?
- How do we make sure that the unit testing cover maximum source code?
The answer to the second question is that every developer has to look at the source code while writing a unit test plan. This is very crucial and the test cases has to be based on the source code. The developer of the source code is the best person to know about the source code and hence it is advisable and desirable that the author of the software carry out the unit testing.
Before answering the third question, let us discuss on testing effectiveness. The effectiveness of testing is the covering the entire source code with minimal test cases. When the effectiveness is high, the quality and productivity of the software will also be high. As human are prove to make mistakes, it is often essential to use tools to improve effectiveness. Code coverage is one such technique to do unit testing effectively. Code coverage pin points the area of the source code that is not tested. Most of the code coverage tools work from package level to source line number level. Once the unit testing session is over, the developer can immediately see the results and improve the test plan to make it more effective.
For example, during iteration #1, the developer will execute the test cases and run code coverage in parallel. During the unit testing or after the unit testing, the developer can take a look at the code coverage reports. Since the code coverage tool reports the coverage at line level, it is easier for the developer to add test cases then and there. Most of the tools have features to generate a comprehensive reports at desired level (application, package, class, method and line level).
There are few code coverage tools (open source and commercial) available for Java. Emma and Cobertura are the most popular code coverage tools in open source arena. Both of them are much matured and used by many developers across globe. There are lot of tutorials available for both the tools. Kindly refer respective websites for more information.
Tuesday, September 4, 2007
Software Engineering Practices and Tools – Now or Never
Recently, I was browsing through Google Video. There was an interesting presentation on Static Analysis. The speaker was a researcher and talked on the importance of Static Analysis. I have been using few open source tools like Findbugs, PMD, Checkstyle and Cobertura. I should confess honestly that I have not been using them by heart but as a process. These are more a sort of personal process. I should also admit that I did not get a real understanding of the tools usage. I am not going to blame myself because we did not live in the world of true parallelism. A couple of years back, a true parallelism is above layman’s reach. Only the high end users and top enterprises use true parallelism.
There were two parallelisms possible in the past. They were super scalability and instruction pipelining. I lost tracking the advancements in hardware industry as the growth was tremendous (I too do not have competency to track the developments). Until recently, we were living in the age of true fastness. In 1990 a C program might have run in 10 nanoseconds but in the year 2000 the same C program might have run in 5 nanoseconds. No multiprocessing or multithreading. The reason is the hardware manufacturers were able to achieve fastness in clock frequency. Simply, there were able to execute more instructions per second sequentially. Nowadays, in each physical processor we have many logical processors. Each logical processor runs in parallel. The process of sequential execution is fading away. We are forced to learn multithreading to tap the advantage of multi-cores. As human we are not so much used to concurrency and that’s our limitation.
In older days, the computers are meant for geeks. But Java, Web 2.0 and Web technologies gave a lead to computers. With these infrastructures, now a layman can explore the power of Internet. We cannot imagine a day in this planet without the Internet – Mails, Blogging, Community Software and, Messengers. The world has become a virtual family. A computer Engineer will handle Software in a different way than a layman. The software that is being developed should be easy to use and reliable. How can you achieve reliability? How will you study functionality in detail without tools?
With wide deployment of Software, Software security is gaining momentum. It was “ok” to leave vulnerabilities in the past but today within 15 minutes of your software release, the applications are being hacked. A couple of years back, a Honeypot was deployed in the Internet. Within 15 minutes, an attacker took over the honeypot. But the Honeypot was protected enough so that the attacker was locked inside. So writing secured software is going to get harder and harder.
In order address the issues from all sides, the fundamental characters have to strong. Software Engineering practices and following the practices by heart is need for the hour. Use of tools helps us to uncover most of the low hanging issues that may go undetected in the final product. Matured Software Engineering practices together with tools can improve the quality and productivity. You will release software with fewer bugs in lesser duration. In the future, big companies are going to survive. But the companies which follow the practices by heart are going to become big companies.
Your managers are not responsible if you do not follow processes or use tools. Now it is time for a paradigm shift. Now or Never.
Saturday, September 1, 2007
Profiling Tools for Java Applications
Tools are primarily used for two reasons – Quality and Productivity. It helps us to drill down the problem. When it comes to Java, one has a lot of tools both open source and commercial. There are lots of development tools available for Java and you can find consolidated information at http://java-source.net/. This blog discusses profiling Java applications and gives guidelines on when and where to profile.
Believe me, profiling has to be considered as last resort. It is just like debugging a bug in your application. While coding, the developers should concentrate on addressing the requirements rather than concentration on the performance. The development team chooses the technology, protocols and algorithms that perform the job effectively. Profiling should be done selectively and on need basis. Never profile the application to improve the overall performance of the application. If you want to improve the performance of the whole application, start with design document. Evaluate the algorithms, data structures and infrastructure.
Profiling has to be done only on critical paths. Only 20% of the code is used by the users 80% of the time and the rest of 80% of code is used only 20% of time. It is enough to profile that 20% of code that is used 80% of time. Once you have decided to profile the application (even for fun), you need to have a tool that gives you reliable data. Though you can always settle for printing the time in each method entry and exit, most of the times you will have access the source code and you might be able to add print statements because of various reasons. Profiling tools become handy and once again for Java, you have many open source profiling tools. Netbeans profiler is a great tool to profile the application for fastness, memory consumption. It can profile entire application or part of the application. You can even profile a single statement and get to know their performance cost. You can find the Netbeans IDE and Profiler at http://www.netbeans.org
I have been using Netbeans and Netbeans profiler for the past one year and I am quite satisfied with its feature and results. It integrates nicely will any application and that is the crux. Before using Netbeans, I tried to use few open source profilers but I spent a lot time to find how to use the tool. But with Netbeans, I bet, you will take off within 10 minutes. After installing Netbeans profiler, it is enough to spend few minutes in going through the “Profiler” menus and you can happily explore its functionality in your free time.
Monday, July 2, 2007
Experiences With Netbeans
For experimental sake, I installed Netbeans and download Netbeans happened quite quickly and within 30 minutes, I was able to install Netbeans and set up the project. Since I needed to set a debugging session, my task was simple. Just compile the sources and start the Tomcat in debug mode. I was happy to connect to remote tomcat (in the same host) and completed the debugging. Since I used to debug applications using Eclipse, the debugging with Netbeans became easy. I would give full marks to Netbeans for giving a small but efficient IDE.
Over the next few days, I was using Netbeans almost daily and that made to investigate more on the tool. Suddenly, I needed to work on a project which is a web application running on Tomcat and powered by Struts. Netbeans, The Crusader, came to my rescue. Netbeans has built-in Tomcat and feature that supports Struts. Soon, I have all configuration files ready and without any source, I built and ran the project. My test web application was launched in my default browser, Firefox. Once again, I am pushed to give full marks for integrated Tomcat. Two more stars for the feature to configure integrated Tomcat. It was a "WoW" feeling.
I don’t find any differences between Netbeans and Eclipse in other features like Refactoring, help, Views/Perspective etc. Both are equally good and I don’t have any points to support one in the basics aspects of IDE.
As of now, the main disadvantage with Netbeans is that there are only few plug-ins but for Eclipse already hundreds of plug-ins are available. Netbeans community should improve a lot and fast to make life easier for the developers. Unless this is addressed, the developers will be in dilemma whether to migrate to Netbeans completely. At least, the software development tools like Code Coverage, Metrics Calculation should be the first priority.
Hope the Netbeans Community will address this soon.
Thursday, June 21, 2007
Garbage Collection - Moving Closer
We had an overview of Garbage Collection from a programmers' perspective in the previous write-up. It was just an introduction and we did not discuss much from the JVM's perspective. Some of you would started ask a question, "Why should I know all these? I am just a developer". Thanks for your thoughtful question and I appreciate it. You might have high-end desktops for coding a "Hello, World" program and your application might work as expected. But an enterprise level Java application face a lot of challenges in terms of functionalities and performance. If you know the internal working of a system, you get a broader perspective and possibly lead to judicious usage of the system. Your knowledge on JVM will become handy and it will certainly pay-off in near future.
As we discussed in the previous write-ups, there are two factors that are key for successful garbage collection. The first work is to identify the garbage objects (objects that are no longer used) and the second work is to actually clean them up. Apart for cleaning up, the garbage collection algorithm also does memory organization by relocating objects in heap.
Garbage Collection - A Closer LookIn the case of Sun Hotspot JVM, the garbage collection takes place using generational collector. Basically, there are two postulates based on which the generational collector works and they are
- Most of the objects (more than 90% of the objects) are short lived. They die as soon as they are created.
- A smaller percentage of the objects are long lived. They live almost till the JVM is live.
Wednesday, June 20, 2007
Garbage Collection – Part 1
Java is platform independent and Java Virtual Machine, the platform dependent component of Java gives this platform independency. JVM is exceptional user space process and it runs from mobile to high end servers. JVM has to bring in platform independency in Class Loading, Threading, Input/Output and Garbage Collection. Looking from the operating system's perspective Java Virtual Machine is just another process. But things are not that easy as they seem. Starting from the loading of classes to the recycling of objects, JVM behaves like an operating system by itself. In this series of blogs, we will discuss how the objects are re-cycled, various garbage collection algorithms and how to tune your JVM for better performance. Our discussions will be based on Sun JDK 1.5 (Tiger) and will be use open source tools as it is freely available in order everyone can use them.
Overview of GC
JVM is just a mighty big process (or it could be a tiny process running in a mobile phone) – that’s the operating system’s perspective. JVM internally does a lot of magic but nothing is visible to the operating system. The operating system just services the JVM. Garbage Collection and Threads are integral part of JVM. There are at least few threads runs when the JVM alive and GC is one among them. But GC is a low priority thread and JVM runs it only on need basis. JVM will try to run GC only when there is memory scarcity. But when it runs, it stalls the other threads in the JVM. The user program creates the objects and uses them as long as they want. The user program de-references it when they no longer need it and JVM takes care of deleting the unused objects executing the finalizer.
There is question that comes to our mind immediately. Who decides the eligibility for the garbage collection? Or how the objects are picked up for deletion? Though JVM automatically deletes the unwanted object but it is developers’ responsibility to say that he/she does not want the objects anymore. The developers need not tell this explicitly but JVM understands it implicitly. Among the objects in the JVM, JVM chooses few objects as special ones and name them as “Root Objects”. Usually the local references of all the stack frames (local variables of all the methods that haven’t exited), string objects in constant pool of the class and the class variables or static variables will be termed as “Root Objects”. When JVM wants to do the garbage collection, it removes the unused object from the Heap. All the objects that are reachable from any of the root objects directly or any objects that are chained with the reachable objects are the objects that are currently being used. All other objects that are not reachable from root objects either directly or through object that are linked with root objects are termed as “Unused objects”. During garbage collection, the JVM marks the unused objects and deletes them freeing up memory. But the algorithm of finding the unused objects and deleting them greatly varies from implementation to implementation. Even a single JVM implementation might have many GC algorithms which can be used based on the application and situation. Before deleting, JVM checks whether the object has finalize. If it has one, JVM postpones the freeing up of the object until finalizer is run. So the objects which have finalizer, GC is has one additional step, that is, invoking finalizer. Until then JVM does not deletes the object.
So far we discussed theoretical aspect of the garbage collection process, the rest of this section explains with an example.
Looking at the figure, the objects that are yellow are the root objects. All the objects that are chained with root objects are currently being used (that are represented in pink color). The objects that are not reachable from any root object directly or indirectly are unused objects which are represented as dotted circle. There are chances for unused object being linked with each other but still they are unused objects and eligible for the garbage collection. The point is, the objects should be reachable from the any of the existing stack frame (Each method when invoked, JVM pushes a stack frame that contains local variable, operand stack).
Eligibility for GC
In the last section, we discussed GC from 10,000 feet. In this section we are going to see Java program and identify the objects that are eligible for garbage collection at various stages of the program. By “Eligible for GC”, we should understand that the objects are only eligible for GC and we are telling JVM that we no longer need the objects. It is up to JVM to delete those objects and recycle the memory. JVM will make every possible attempt to recycle memory.
Consider the above code, at line 18, 19 and 20 we are creating objects. Assume that the control is at line 21 after executing 20. At this point of time, we have references to four objects referred by “args”, “string”, “i” and “j”. So there are four root objects. All the root objects and the objects that are reachable from the root objects are not eligible for GC. Hence at line 20, there are no objects eligible for GC.
Consider the above code, at line 18, 19 and 20 we are creating objects. Assume that the control is at line 21 after executing 20. At this point of time, we have references to four objects referred by “args”, “string”, “i” and “j”. So there are four root objects. All the root objects and the objects that are reachable from the root objects are not eligible for GC. Hence at line 20, there are no objects eligible for GC.
When JVM executes the method “display”, the control goes to the method where it has references to the objects that are passed as arguments. Apart from the arguments, the method “display” has one more local variable “string” which also becomes a root object. After executing line 32, the total root object becomes 5. At line 34 and 37, even though the object referred by “i” and “j” are de-referenced, the objects pointed by “i” and “j” cannot be garbage collected as the method “main” already has a reference. At line 40, the object referred by “string” is eligible for garbage collection. Once again at line 43, the object referred by “str” is not eligible has the method “main” has a reference to it.
When the control gets back to the method “main”, after executing line 23, the object referred by the variable “i” becomes eligible for GC. Subsequently, the objects referred by variable “j” and “string” become eligible for GC when the line 25 and 27 are executed respectively.
- The root object is decided not only passed on the local variable of the method being executed. JVM goes through the entire stack to find the root objects.
- Apart from the stack, JVM also looks into “static” variable or class variables and keep them as root variables. So the static variables will be eligible for garbage collection when the class is unloaded from JVM
- Few JVM will implement the method area in Java Heap. That is the JVM allocated memory to hold code in the heap. Those objects are also becomes eligible for GC when the classes are unloaded.
Questions for Understanding
1. What is Garbage Collection?
2. How the objects are recycled?
3. What is the standard garbage collection algorithm recommended by Java Virtual Machine Specification?
4. Elaborate the object lifecycle.
5. What are the candidates of root objects? How they affect the garbage collection?
Question for Thinking
In your JVM, you have few objects that are eligible for the garbage collection. JVM also runs garbage collector. Now the question is, whether all the objects that are eligible for GC will be garbage collected?
Answer in single word "yes" or "no".
If you are not sure on what to answer, the next blog will open some of the concepts and eventually you will answer the question.
Keep Watching and Have a Great Day
Garbage Collection - Understanding the Death
By default, it works in a specific way but it can be changed based on the application requirements during starting up of Java Virtual Machine. We will be discussing Garbage Collection techniques and GC tuning techniques with simple example. In the entire exercise, we will rely on generating GC statistics from the JVM and internalize the Garbage collection.
Monday, June 18, 2007
java -Xms
Java is an interpreted language and lacks performance as it runs on Java Virtual Machine. This is an age old statement and probably true during its early stages. It is quite natural for any Software or Hardware to scale up or perform well. Java is an exception as it came fast exceptionally. Today with a lot of improvements and flexibilities Java Technologies gives an edge. Java Virtual Machine plays a crucial role for these performance improvements. A newbie can get started with Java and write a decent application with greater ease. In this blog, I would like to share the information about JVM, JDK, APIs, Tools and ideas related to Java Technology. As of now, I am planning to write regularly but of course I do not know the interval. Anyways, I am positive to blog and share some of the information I find interesting. Keep watching and give back your comments.
