Make sure there’s no dirty picture – Have a good big picture!

 

Few years back I had worked with a Japanese customer for a health care imaging project at their site. I was relatively new to the domain. Most of the development team was there in India.

We had some solid requirements and prototyping on some new features. I had demonstrated one of the prototypes integrated to the product which essentially play back the images aligned to the ECG cycle acquired from the cardiac patient. The feature helps the sonographer to export the data as AVI. I simply tested the feature and quickly setup the demo as we’ve already crossed the deadline. Things were working just fine. This I had already seen in the prototype earlier. Fine. I demonstrated the feature successfully and was happy about it. I requested him to to try the new feature to get more feedback from him. He happily took over the system.

He simply registered a patient, then acquired the images along with ECG. He went on reviewing the acquired images, then he exported the cycles as AVI to the patient. He quit the application and opened the patient browser if appears under the current patient. Yea it’s there. I dint feel anything bad when he registered one more patient and start the application again. (Application won’t quit, it will be residing in memory always. They simply hide the window and release the resources and keeps on waiting for commands from parent application). He did the very same procedure again. It was working I was happy again. The progress bar has been displayed again and finished the export. Then he took the patient browser once more. Surprisingly the data wasn’t there under the current patient.

As I knew the code base well, I figured out within seconds what made it to disappear the data. I told him “I’m sorry, the new patient event not handled well. The code isn’t resetting/replacing the patient ID with new one. I can fix this. It’s just one line modification” and I was about to go back to my desk to modify the source.

He stopped me and smiled and explained, “Sarath-san, are you realizing that this is a serious security issue?” I could not fully understand what he really meant with security issue.

I did not reply. He continued “For programmers, it’s a one line code modification which takes to fix this issue. But think from a customer perspective. If you went a hospital with some small chest pain and the next patient is coming with some serious cardiac problem and what if his data being exported to your ID, then the doctor gives you the wrong treatment?”

The example was perfect. I could not think of a major heart surgery because of a system/sonographer’s mistake. That was kind of eye opener. Even after all these years I still remember the story.

Most of the time, programmers are either instructed to do some part of the code without having understanding of the impact. To make and maintain better software, it is also important to understand what is the big picture. What is the real impact. Always remember the issues being found during the development time is easy to fix, once it’s delivered to the customer site, the cost will be N times more, depends on the number of customers we’ve and the cost of deploying the patches. It starts from requirement capturing to implementation. It’s responsibility of the person who captures the requirements to test/inform the major uses cases, aspect and impact of the features. On the other hand, when we’re implementing something, it’s solely our responsibility to properly understand the requirement, big picture of this requirements and the impact area. It’s equally important to do regression testing to make sure that the existing functionalities aren’t broken.

When less competent people leads these activity there’s a high chance to have the information in pockets and do not explain the requirements properly to make sure that they’ve better control over the project (trust me, there are people). But that is actually making the product and themselves worse. Also it’s developers responsibility to bug the people enough to get maximum information and clarify the requirements.

 

Versioning Systems and 3 Way merge process

 

If you’re a developer you must know how important is a file comparison tool. Especially when working in a team with a source/versioning system. It’s quite often we compare several versions of source files. The usual comparison and merging with two different version of files are called 2-Way merge process.
To talk about the 3 way merging, first we must get a basic essence of versioning system and merge process. Because 3 way merge is part of it and became popular with it.
The conventional configuration management systems like Visual Source Safe provides the basic functionalities of configuration management. Things have changed much in these years. In the new age of software development, people rely on versioning system where individual developers can work independent with his own branch and merge the changes to main server, merge to other branches etc. The version of files controls everything. This helps us to work in collaborative work environment. If you know or not, even geographically distributed teams are working well with new age versioning systems.
Various versioning systems are available in the market today. Both free and commercial tools like IBM Rational Clearcase, CVS, Subversion, Git are the popular among the tools available.
Let’s see how versioning system works. When we check-in the files, obviously we’re creating a new version of the same file. In conventional source management systems, everyone works with the latest version of the files. We will be connecting to a the source management server and do the check-ins and checkouts but we’re getting limited with lot of other facts.

  • Other developers are unable to checkout the files and they keep waiting.
  • Everyone working in the latest version of the files available in the fileserver and making it’s hard to work with a different version of systems.
  • Increasing number of local copies and which is totally unsecure.
  • Less flexible to merge different versions of software.
  • Too much limitations of imposed by source controlling systems.

Ok Let’s think in a broader way. What if

  • We get a source control systems, where we can work independently on the replica of the latest software where every developer are independent and working in their own branch. Get the latest version of unmodified files easily?
  • Are you happy if you can work on a specific time based branch and work according to your convenience?
  • What if you many other developers modify the same file and easy way to merge everything back and forth with flexible tools?
  • What if you can easily get a tree representation of the versions of the files for flexible comparison and detailed information?
  • What if you’re free to work geographically distributed and work with same source and same team effectively?

If your answers are yes, the modern versioning systems are the answer for these queries. Versioning , branching and merging are the core features of a systems.
The following figure represents basic concept of a versioning system (Taken from Wikipedia.org)

image

In a versioning system, there will be a mainline/trunk which contains the latest version of the product. This will be maintained high in quality. One or more developers are working on the replica of mainline called branch during their development time. The modifications in the branch is independent and mainline in not affected. Once developer finish testing and merging, it’s the time to merge the code to mainline.

The red colored arrow to the trunk is called merging process which is explained below.

Trivial Merge

For each file in the branch, it’s checked out from specific version of the same file from parent branch (mainline). Once we checkout, other developers can also checkout the same file and merge to the mainline. But during this time, the file we checked out remains intact. If no other new versions of the same files are created in the mainline during the merge process, it’s called, trivial merge. This is something like a simple comparison. The different part of source code will be merged to the main.

Non Trivial Merge

There are different kind of situations for a trivial merge. Consider the following situations.

1. Developer working on a prior version of software and other developers updated the same files with a latest version in the mainline.

2. Get the modification of same files from another developer from his branch, which may created based on different base versions.

When you get two different versions of same file and ask to do the merge process, you’re just taking your comparison tools and make the best possible assumption to merge the files. This requires intensive human effort and also error prone. Some people do an alternative by asking to put tags while coding. Which is totally conventional.

Most of the versioning system provides a merge tool associated with it. This can smooth up the merge processes. Independent merge tools are also available in market, both free and commercial.

As I mentioned before, there will be a mainline/trunk where you can get the latest version of the product. Each files under the configuration management will have the version history of it’s own.Consider the following situation.

image

You can see Joe and Dave checkout same file, based on Version 1 in the mainline. Dave finished his work and checked in the file to mainline and version 2 created in mainline. During this time, Joe continue his development based on version 1. Finally before her merge back/during development time itself, he identifies there’s latest version of file is created. Now he required get the modifications of Dave to continue his development or merge back. If Joe blindly merge back, to mainline, Dave’s modification will be lost. Joe can make best assumptions on the conflicting points when he compare Dave’s modification and create the new version (2) which contains both his and latest modifications but this is error prone.

A three way merge can help us in this regard.

  • We’ve a base version Main(1)
  • We’ve a latest version Main(2)
  • Joe have his own modification Joe’s branch(1).

The merge tools can easily find the changed points based on the base version. It can easily create a modification with latest and latest local version without losing any modification. There would be some conflicting points where both developers merged same lines of source code. In this case, human interference is required. We’ve to choose the source code by seeing base, latest, latest local. User can easily identify which code he need to persist in new version (2).

The three way merge process is less error prone and requires only less human effort. The tools like KDiff3, Beyond Compare 3, Perforce provides merge tools for effectively do a three way merge process. All these tools can also attach to your existing source control system. However, most of the revision control software itself provides its own merge tool with their package.

 

An example merge process screenshot from KDiff3 is attached here(Click to Expand).

dirmergebig