IBM ClearCase MVFS Dynamic View Linux Patch
This is about getting ClearCase (a source configuration management system) dynamic views to work with a Unix client and a windows host. The documentation says you can't, maybe they should have looked into the issue a little first.

snapshots vs dynamic views
ClearCase supports snapshot views and dynamic views. The snapshot views are what every other source control system I've ever used does, it gives you the requested file in a local directory and it's a normal file. Dynamic views are a virtual file system that you mount that provide the same set of files that a snapshot view provides, only it is like it updates the file before every open, so you have the latest version. It also lets you add @@/main/2 to the end of a file to get the specified file version from any program that you can give a file name to.

There is a table 'Table 1. Rational ClearCase VOB access across platforms of different types' at the bottom of the following web page About interoperation between Windows computers and UNIX workstations, that can be summarized as follows. Every combination of ClearCase client or server running windows or Unix works with both dynamic views and snapshot views, except if the client is Unix and the server is windows, dynamic views don't work. It's right in the documentation and I was told that, so fine, we have windows servers, with a Linux client, I'll just use snapshot views.


Installing ClearCase, and getting it to work (skip if you have it working).
Work has been pushing IBM ClearCase for source control. After using CVS for years, including work projects, I got another taste of ClearCase. Some of us have Linux as our main desktop computer. First, they said ClearCase doesn't support Linux, then when another ClearCase admin said it does I got the next set of excuses, ClearCase won't work behind a firewall, for ClearCase to work the computers had to be on the Windows active directory. At the time the Linux computers were on a local private network with a firewall between then and the corporate network. Now, for reasons I kind of understand but aren't logical, the windows desktop computers are on the same private network behind the same firewall, and they work, so that excuse is out. As to the active directory, I thought I just needed kerberose to work, except ClearCase doesn't use kerberose, so I don't know what the active directory mention was about. Without any more excuses, or more likely figuring I would try and give up, they gave me the install files to ClearCase on Linux, along with a warning that it would only work on Red Hat Enterprise Linux and not the Fedora Core 3 I had installed. Whatever, the installer script was perl, and I'm a programmer, I got it to install.

After the learning curve on how to make a Unix VOB (where ClearCase stores its data) tag /vobs/IOS_SDT because the existing windows VOB tag \IOS_SDT wouldn't work because it doesn't start with a Unix slash, and I had to have the right group setup, I did get snapshot views to work. You can't register the same VOB multiple times in one region. You can register the same VOB in multiple regions, so we have one region for Windows and another for Unix with the appropriate path separators.

While the system is running Fedora Core 3, I've upgraded the kernel to 2.6.24.4, and ClearCase doesn't support so new of a kernel, so no MVFS or dynamic views. To not go back to the older kernel and loose suspend to disk I did the rest of the playing around under a supported Linux kernel in qemu. I did get the mvfs to compile under but it paniced when I went to use it. If anyone else has it working please let me know.


But why don't dynamic views work Unix to windows?
You can almost get dynamic views to work by the following. Mount the windows SMB or CIFS share in a local directory that matches the Unix VOB tag with the normal smbmount or mount command. Use the cleartool mount command to let the mvfs know it is available. Create a dynamic view, start the dynamic view, go to the right directory in that view and VOB, and ... ls, directory listing! All find a dandy until you open a file, which fails.

At some point in trying to get things to work was I using tcpdump to dump the communcation between the client and server, and I took another look. When a file is requested the response includes a file and a set of directories that start with c\1\2\3\4\4321.1234 I don't remember the exact sequence, but something like that, strange. Go to the CIFS mount point and if you substitute / for \ the file exists, and the contents is the file you requested. Which I thought was odd, because every other configuration management system I know if stores file differences not complete files. I guess when a dynamic view client requests a file it generates the file, tells the client the name, and lets the normal network based file system handle the details like seeking and reading the file as a program requests it.

Do you suppose a big company like IBM with their "enterprise" source control software, could possibly just need the path separators switched over and it would just work? It couldn't be that easy, otherwise they wouldn't spend so much time documenting and educating their users that it doesn't work.

So, a couple hours of digging in their Linux kernel module source code and I find the function that looks like it returns the file I saw in the tcpdump, add a printk, compile, load, read file, yup that's the string. Add a for loop to switch \ to /, compile, load, and it works! One loop and that was it. What I don't understand is why they didn't just return a forward slash from windows computers in the first place, most all the windows APIs will accept them, and it clearly accepts the forward slashes from Unix servers. It should have not been an issue. I'll give IBM this much, their code is GPL'ed, so here's the patch, enjoy. Now if I didn't have the source code, I couldn't fix it. If it wasn't Free Software, I couldn't put it up on my home page, who knows, maybe it will even make its way back to IBM.

mvfs_clnt.c.diff

With IBM not only Linux friendly, but pushing it, "IBM, Canonical/Ubuntu, Novell, Red Hat to Deliver Microsoft-Free Desktops Worldwide", will they accept external patches? Time will tell.


Submitted 08-08-2008, id 973, IBM Rational Request For Enhancement
http://www.ibm.com/developerworks/support/rational/rfe/execute?use_case=viewChangeRequest&CR_ID=973
Headline: [PATCH] Linux MVFS dynamic view, windows interoperability
Submitter's raning of priority: CP3 - Lack of the RFE functionality is a minor road block to deployment/adoption
Product: Rational ClearCase
Component: MVFS
Operating System: Linux

Description
While the documentation lists dynamic views on windows will work with VOBs hosted on Unix, the opposite is documented to not work, and in fact it doesn't. To reproduce, mount the windows VOB share with CIFS on the local Linux system, create a VOB tag with that mount point for the global path, use cleartool to mount the VOB tag, create and start a dynamic view with that VOB, and try to read files in the view. It gives an error. Listing directories works, checking out, checking in, and editing checked out files works, but files that aren't checked out don't.

The problem is very basic. When MVFS in the kernel requests a file from the windows based ClearCase host, the returned file string looks like the following with dos backslashes for directory separators.
c\cdft\11\19\042e52ae895c4e66ae12725f9e13a7d5
and with the back slashes it isn't found, it just needs to be changed to the following with forward slashes for Unix directory separators.
c/cdft/11/19/042e52ae895c4e66ae12725f9e13a7d5
Preferably this would happen on windows before the string was returned (after all dynamic views on windows using VOBs hosted on Unix work, and presumably Unix is returning forward slash path separators). Being done on windows would allow the fix on one platform rather than each Unix platform. As the MVFS on Linux has the GPL license and that was all the source code I had available I fixed it there and dynamic views now work.

The following patch fixes the bug that dynamic views on Linux can't access VOBs that are hosted on windows.
ClearCase version 7.0.1 (Wed May 30 17:04:58 EDT 2007)
7.0.1.1-RATL-RCC-IFIX02 (Tue May 13 14:43:13 EDT 2008)

Index: mvfs_clnt.c
===================================================================
RCS file: /var/adm/rational/clearcase/mvfs/cvs_root/mvfs_src/mvfs_clnt.c,v
retrieving revision 1.1
diff -u -p -r1.1 mvfs_clnt.c
--- mvfs_clnt.c	2 Jul 2008 18:09:25 -0000	1.1
+++ mvfs_clnt.c	2 Jul 2008 22:26:40 -0000
@@ -1980,7 +1980,15 @@ CRED_T *cred;
          * into a minimum size buffer, so we can free up the maxpathlen
          * buffer allocated at this level.
          */
+	int i;
 	mnp->mn_vob.cleartext.isvob = rrp->vob;
+	//printk("mfs_clnt_cltxt_locked: original rrp->text %s\n", rrp->text);
+	for(i=0; i<rrp->text_size && rrp->text[i]; ++i)
+	{
+		if(rrp->text[i]=='\\')
+			rrp->text[i]='/';
+	}
+	//printk("mfs_clnt_cltxt_locked: modified rrp->text %s\n", rrp->text);
 	error = mfs_set_cpname(mnp, rrp->text, rrp->text_size);
     } else {
 	MFS_CHK_STALE(error, vp);

Use case: This feature would be used when an organization has Unix or Linux clients and wish to access a dynamic view where the VOB is hosted on Windows.

Business Justification (private): We have an existing windows VOB server for a piece of software that has been windows only, but now is being ported to Linux. Switching to a Linux VOB server isn't an option at this time, and not having dynamic views on Linux is frustrating development without the above patch.


IBM response, November 23, 2008 "Uncommitted Candidate"
I got an e-mail today (June 19, 2009) asking about the patch, so I looked up the request for enhancement. The good news is it hasn't been rejected, but it's in a favorable limbo. At least it passed the initial and followup review.

Status: Uncommitted Candidate
August 20, 2008 This RFE is consistent with our strategy and product roadmap and IBM is continuing to evaluate.
Most recent IBM developer update: 23 Nov 2008 This RFE is consistent with our strategy and product roadmap.

Uncommitted Candidate: This request will not be delivered within the next year, but the theme is aligned with our two year strategy. IBM is soliciting feedback for this request, and within one year from submission, status will be updated to either Rejected or Planned for Future Release.

You don't need to wait on IBM to get dynamic views to work on a Linux client from a Windows VOB. The MVFS Linux kernel module source comes with clearcase, apply the patch, recompile, insert the module, and have at. A couple notes on how to use it. As outlined above the vob tag must start with a Unix style slash and must have a matching directory on the system (which should be empty). The remote vob must be mounted locally so that the VOB_NAME.vbs matches the vob global path entry, such as /mnt/remote_server/VOB_NAME.vbs, and it can't be in the vob tag directory. Then cleartool mount will then be able to activate the VOB and it can be used normally.


RFE Voted +1 by a non-IBM developer

Date: 21 Sep 2009 03:53 AM Eastern Time (ET)
Comments: Have rebuilt mvfs.ko from 7.0.1.6 on RHEL 5.3 and works fine.

Good, a vote of success on the IBM community page, maybe a little bit of activity will bring some more attention to this RFE and get it fixed.


Written by David Fries <david@fries.net>

Valid HTML 4.0! Valid HTML 4.0!
My pgp key is available and the fingerprint follows. See
http://www.gnupg.org/ for details.
pub 1024D/CB1EE8F0 2001-08-21 David D. Fries <david@fries.net>
Key fingerprint = 7079 F7EA D7EA 8E93 5B84 1900 008F 39D9 CB1E E8F0
uid David Fries <dfries@mail.win.org>
sub 1024g/D9B8B029 2001-08-21
Don't e-mail 266bbf64_4b96cec6 at spamcheck aerospace fries net