Open for Extension and Closed for Modification

This is one of the fundamentally design thought that every designers should deeply analysis. When we say open, it does not necessarily carry the real meaning of “open” and close does not imply the real meaning of “close”. In the context of designing, these two words are related to level of flexibility given to programmers/developers. This fundamentally principle suggests that an application design should be flexible enough to accommodate features to the application.

But a sudden spike of thought that comes to our mind – for every product release, we add a lot of features. Before proceeding further, let us ask some questions.

1. Does your product have added few features for each release?
2. Do you feel that the base architecture remain fairly stable over many releases?
3. Do the developers know the consequences to their features when the underlying framework changes?
4. How easy or tough adding of features are? (Easy or tough is very relative J )
5. How this fundamental design principle helps the framework to be flexible?
   

There will be two scenarios where the developers will modify the code. When the software is buggy, they do not have any other way but to modify the code. The developers typically identify the root cause of the bug and fix them. But some of the developers open the code to add new features. The modification of the existing code for adding new features conveys that there is something wrong. It is a bad design for a framework if it forces it clients to modify the code when adding new features. Apart from forcing the modifications, the design should not force itself to undergo a massive code change to support new functionalities. When it comes to framework, the designers should also see the framework from another perspective apart from providing basic functionalities. The perspective is that the framework is just a contract or guidelines given to the clients. The bottom line is the framework is just contract that governs and facilitates proper functioning of the applications. When it comes to an application, the application should foresee some changes in requirements and burn flexibilities in the design to accommodate those requirements.

The design principle, “Open for Extension; Closed for Modification” is a great tool and thought for building software. This is just a thought and for achieving this one needs a lot of things to be done and known. In few of my next blogs, I ll be writing on the design principles and these design principles could be a one-liner, a design pattern that is commonly needed or concepts taken from Java library.

Improving Test Coverage using Code Coverage

For the past few years, the industry is undergoing a lot of advancements. Particularly, new software development model have been followed. The industry moved from waterfall model towards Agile and Iternation (incremental) model. Also, a lot of progress have been made in improving productivity by employing tools to aid development. One of the most important activity is Unit testing. Unit testing is essential to ensure that the developed code works as expected and without the unit testing the development is not complete. Unit testing has to be carried on the entire code not leaving a single line of code. It is during this phase, the developers make sure that the code will work properly as expected. There are few questions that we need to ask ourselves during unit testing

1. Are we doing it like a black box testing without looking at the source code?
2. Do we look at the source code during unit testing?
3. How do we make sure that the unit testing cover maximum source code?

Let us straight away get into the answers to the above questions. To answer the first question, the unit testing is not a black box testing. Doing it like a black box testing is not effective way of testing the software at unit level. For example, in the actual source code there could be lots of conditional and branching statements and loop statements. The black box testing does not test these language specific constructs properly but it focuses only on functionality of the software.

The answer to the second question is that every developer has to look at the source code while writing a unit test plan. This is very crucial and the test cases has to be based on the source code. The developer of the source code is the best person to know about the source code and hence it is advisable and desirable that the author of the software carry out the unit testing.

Before answering the third question, let us discuss on testing effectiveness. The effectiveness of testing is the covering the entire source code with minimal test cases. When the effectiveness is high, the quality and productivity of the software will also be high. As human are prove to make mistakes, it is often essential to use tools to improve effectiveness. Code coverage is one such technique to do unit testing effectively. Code coverage pin points the area of the source code that is not tested. Most of the code coverage tools work from package level to source line number level. Once the unit testing session is over, the developer can immediately see the results and improve the test plan to make it more effective.

For example, during iteration #1, the developer will execute the test cases and run code coverage in parallel. During the unit testing or after the unit testing, the developer can take a look at the code coverage reports. Since the code coverage tool reports the coverage at line level, it is easier for the developer to add test cases then and there. Most of the tools have features to generate a comprehensive reports at desired level (application, package, class, method and line level).

There are few code coverage tools (open source and commercial) available for Java. Emma and Cobertura are the most popular code coverage tools in open source arena. Both of them are much matured and used by many developers across globe. There are lot of tutorials available for both the tools. Kindly refer respective websites for more information.

Software Engineering Practices and Tools – Now or Never

Recently, I was browsing through Google Video. There was an interesting presentation on Static Analysis. The speaker was a researcher and talked on the importance of Static Analysis. I have been using few open source tools like Findbugs, PMD, Checkstyle and Cobertura. I should confess honestly that I have not been using them by heart but as a process. These are more a sort of personal process. I should also admit that I did not get a real understanding of the tools usage. I am not going to blame myself because we did not live in the world of true parallelism. A couple of years back, a true parallelism is above layman’s reach. Only the high end users and top enterprises use true parallelism.

There were two parallelisms possible in the past. They were super scalability and instruction pipelining. I lost tracking the advancements in hardware industry as the growth was tremendous (I too do not have competency to track the developments). Until recently, we were living in the age of true fastness. In 1990 a C program might have run in 10 nanoseconds but in the year 2000 the same C program might have run in 5 nanoseconds. No multiprocessing or multithreading. The reason is the hardware manufacturers were able to achieve fastness in clock frequency. Simply, there were able to execute more instructions per second sequentially. Nowadays, in each physical processor we have many logical processors. Each logical processor runs in parallel. The process of sequential execution is fading away. We are forced to learn multithreading to tap the advantage of multi-cores. As human we are not so much used to concurrency and that’s our limitation.

In older days, the computers are meant for geeks. But Java, Web 2.0 and Web technologies gave a lead to computers. With these infrastructures, now a layman can explore the power of Internet. We cannot imagine a day in this planet without the Internet – Mails, Blogging, Community Software and, Messengers. The world has become a virtual family. A computer Engineer will handle Software in a different way than a layman. The software that is being developed should be easy to use and reliable. How can you achieve reliability? How will you study functionality in detail without tools?

With wide deployment of Software, Software security is gaining momentum. It was “ok” to leave vulnerabilities in the past but today within 15 minutes of your software release, the applications are being hacked. A couple of years back, a Honeypot was deployed in the Internet. Within 15 minutes, an attacker took over the honeypot. But the Honeypot was protected enough so that the attacker was locked inside. So writing secured software is going to get harder and harder.

In order address the issues from all sides, the fundamental characters have to strong. Software Engineering practices and following the practices by heart is need for the hour. Use of tools helps us to uncover most of the low hanging issues that may go undetected in the final product. Matured Software Engineering practices together with tools can improve the quality and productivity. You will release software with fewer bugs in lesser duration. In the future, big companies are going to survive. But the companies which follow the practices by heart are going to become big companies.

Your managers are not responsible if you do not follow processes or use tools. Now it is time for a paradigm shift. Now or Never.

Profiling Tools for Java Applications

“An apprentice carpenter may want only a hammer and saw, but a master craftsman employs many precision tools”

Tools are primarily used for two reasons – Quality and Productivity. It helps us to drill down the problem. When it comes to Java, one has a lot of tools both open source and commercial. There are lots of development tools available for Java and you can find consolidated information at http://java-source.net/. This blog discusses profiling Java applications and gives guidelines on when and where to profile.

Believe me, profiling has to be considered as last resort. It is just like debugging a bug in your application. While coding, the developers should concentrate on addressing the requirements rather than concentration on the performance. The development team chooses the technology, protocols and algorithms that perform the job effectively. Profiling should be done selectively and on need basis. Never profile the application to improve the overall performance of the application. If you want to improve the performance of the whole application, start with design document. Evaluate the algorithms, data structures and infrastructure.

Profiling has to be done only on critical paths. Only 20% of the code is used by the users 80% of the time and the rest of 80% of code is used only 20% of time. It is enough to profile that 20% of code that is used 80% of time. Once you have decided to profile the application (even for fun), you need to have a tool that gives you reliable data. Though you can always settle for printing the time in each method entry and exit, most of the times you will have access the source code and you might be able to add print statements because of various reasons. Profiling tools become handy and once again for Java, you have many open source profiling tools. Netbeans profiler is a great tool to profile the application for fastness, memory consumption. It can profile entire application or part of the application. You can even profile a single statement and get to know their performance cost. You can find the Netbeans IDE and Profiler at http://www.netbeans.org

I have been using Netbeans and Netbeans profiler for the past one year and I am quite satisfied with its feature and results. It integrates nicely will any application and that is the crux. Before using Netbeans, I tried to use few open source profilers but I spent a lot time to find how to use the tool. But with Netbeans, I bet, you will take off within 10 minutes. After installing Netbeans profiler, it is enough to spend few minutes in going through the “Profiler” menus and you can happily explore its functionality in your free time.

CoW – Linux way of creating processes

Linux operating systems is one of THE most popular operating systems and continue to lead the embedded operating systems markets. In this blog, I would like to give an overview of a design decision in Linux and it is one of the reasons why I see Linux not only as Software but also as an Art. The kernel developers thought leadership is unquestionable. I am great fan of Linux Kernel DevelopersJ. Without bogging you down, let me come to the point directly.

UNIX operating systems create processes using the system call fork () and overlay (load) a binary/executable image using exec () system call. The system call fork () does not have a parameter and the system call exec () few parameters to load a program from a permanent storage (File System). Practically, the system calls fork () and exec () are twins; fork () is the elder brother and exec () is the younger one. Let us first see the functionality of “fork”. As you may know a process is program in execution that has a state such as data, heap, stack, pending signals, open files and environment variables. When you want to execute something, let us say running a command “ls” from the shell, the “shell” process typically calls fork (). When the fork () is successful, the kernel creates another process which is copy of the process. So each successful call to fork () returns twice – once in the called process (aka parent process) and second time in the newly created process (aka child process). As a standard, all the operating systems copy the address space of parent process and create another address space. There is a overhead involved while creating fork’ ing a process.

Shortly after the process is created, either parent or child process is loaded with some other program. Typically, one of the system calls in “exec” family of system calls is used. When “exec” is executed, the entire address space of the called process is recreated. So there is considerable amount time spent in this “double creation”. Surely, there is some kind of optimization can be done to gain substantial performance.

In Linux, fork () does not copy the address space but just simply creates the kernel data structures needed to the new process. Now, both the parent and child process uses the same address space and the entire address space is marked read-only. So, both the parent and child process can continue to read the process address. When any one process tries to write to a page (memory page), the kernel duplicates the address space and creates the address space. But this is unlikely to occur. Another scenario is the loading a new program into memory. When this occurs, the kernel any way creates a fresh address space and starts to execute the loaded program. By this approach, the “double creation” is avoided. The deferring the duplication of address space has given a performance and sometimes procrastination helpsJ.

The functionality of fork () is called CoW – Copy on Write. So from next time, when you see a Linux box and a running process think about

But you have two process after fork (), which process will be scheduled first? Yes, Linux is a masterpiece.