Subject: Multi branch / distributed development support - by: tekkie

Bhorsoft wrote:
We've been using SVN branching for a long time. Now we only work on release branches and never use the trunk - for us the trunk is like an appendix, it is there, but not needed. We only merge "upstream". Since we may be working on two major releases and a couple of patches for the same application, we rely on branches so we can work concurrently on different releases of the same code. We have had little problem merging from the earlier releases to the later release branches.

We've been using this system for over 5 years and have never lost or mangled a change in all that time.
I am glad that your business allows you to work in such a structured way, but really is that not every business has that luxury.

Subversion is a great tool if your branches and merges are predictable, unfortunately it can't keep up when branching/mergning becomes ad-hoc (yes I know this should be avoided but business is business).

View the original article here

Re: Measure development time with ClearCase. Is that possible?

This question is not answered.
Replies: 4 - Pages: 1 - Last Post: Nov 3, 2010 7:32 PM Last Post By: Patrick.Meyer Threads: [ Previous | Next ]
Posts: 315
Registered: Nov 28, 2006 05:43:16 AM Measure development time with ClearCase. Is that possible?
Posted: Aug 11, 2010 09:32:00 AM Hi folks,

We are working with ClearCase UCM and we got a request to provide a report that summarizes the invested time on each activity.
The dev. manager wants to know how much time was invested for each dev. activity, and then to summarize it all.

I'm trying to think if I can find that information in ClearCase data (we are not using CQ unfortunately).
I wouldn't summarize the time between creation of activity to delivery time since it could be that the activity will be delivered later to another stream.

I'm tending to list the activity content (change-set) and then calculate the time for each file-version, separately.
I'm not sure if it's possible , since ClearCase provides the creation time only. I can calculate the time between creation of version X to creation of version X+1 (or X+CHECKEDOUT), but does that time really indicate the invested time?

What do you think? Is that request reasonable? Is it possible to provide it?

Thanks!

Posts: 417
Registered: Sep 28, 2005 02:20:11 PM Re: Measure development time with ClearCase. Is that possible?
Posted: Aug 11, 2010 11:34:06 AM   in response to: cm2006's post I'm not sure that you could pull those kinds of metrics out of ANY SCM system. Why? Because the best you can do is determine when:
1) A person started working on an activity (by creating it in CC, or performing a "workon" action in CQ)
2) A person started editing a given file in conjunction with working on the issue
3) When a file was checked in as part of that activity.
4) When it was delivered in CC and -- if using CQ -- was moved to a "complete" state.

In a Pure-Clearcase environment, you're limited because there is no way to account for the time you spend:
- READING the code to gain an understanding of it.
- compiling/running/testing the code.
- thinking about how to solve the problem or test that it was solved.
- in modeling tools, meetings, code reviews, etc.

The best you can do is to get a "best guess" approximation. And that time may be wildly inaccurate. More importantly, it may be worse for developers who think problems through on whiteboards/paper or in modeling tools before starting coding.

Since the checkout time is lost when you check a file in, you would have to do something like:
1) Create a preop checkin trigger that uses ct lsco to get the checkin time and then attaches an attribute to the version being checked in containing the computed time spent in the checked out state and the original checkout time.
2) create a postop deliver -complete trigger that uses the checkout times to compute some sort of time spent in the activity based on:
- When the activity was created
- The checkout times and time spend checked out of the versions in the change set
- The time spent between starting the deliver (creation time of the deliverbl will work if you can't find it any other way) and completing the deliver.

This time would be sent wherever you want. You could attach it to the deliver activity, send it to an external database, email it, etc.

This, of course only collects time data moving forward. And like most metrics and statistics (Effective Lines Of Code, etc), it needs context to make any sense, and without that context can become the path "paved with good intentions..."

=================================================================
Brian Cowan
Advisory Software Engineer
ClearCase Customer Advocacy Group (CAG)
Rational Software
IBM Software Group
550 King St
Littleton, MA 01460

Phone: 1.978.899.9471
Web: http://www.ibm.com/software/rational/support/

Posts: 561
Registered: Nov 19, 2007 06:06:36 AM Re: Measure development time with ClearCase. Is that possible?
Posted: Aug 12, 2010 03:45:44 AM   in response to: brcowan's post That's interesting issue.

I can tell from my point of view, that sometimes we are asked if we can provide to one of our products (CompBL, a meaningful code comparison for UCM) a report that summarizes the time spent on development.

I do agree with all Brian's say. Take into consideration another two things:
1. Users may move file versions to another activity
2. It could be that user checks out 5 files simultaneously and then check them all in after one hour.
So is it considered as one working hour or 5 hours ...?

Regards,
Tamir Gefen, GoMidjets
http://www.gomidjets.com/

Posts: 207
Registered: Oct 31, 2006 04:24:09 PM Re: Measure development time with ClearCase. Is that possible?
Posted: Aug 12, 2010 10:44:10 PM   in response to: cm2006's post In my experience, gathering this type of metrics is a pretty useless undertaking...

Not only will developers find this endeavor highly suspect (they'll assume management will use the results as input for the next layoff round) they'll quickly find ways to game the system.

My 2c...

Jozef

Posts: 5
Registered: Sep 07, 2010 08:03:03 PM Re: Measure development time with ClearCase. Is that possible?
Posted: Nov 03, 2010 07:32:42 PM   in response to: cm2006's post Tgefen wrote:
That's interesting issue.

I can tell from my point of view, that sometimes we are asked if we can provide to one of our products (CompBL, a meaningful code comparison for UCM) a report that summarizes the time spent on development.

I do agree with all Brian's say. Take into consideration another two things:
1. Users may move file versions to another activity
2. It could be that user checks out 5 files simultaneously and then check them all in after one hour.
So is it considered as one working hour or 5 hours ...?

Regards,
Tamir Gefen, GoMidjets
http://www.gomidjets.com/

It is exactly what I need, The link you provided is very useful, Your effort is appreciated!

Use the search field to find all types of content in My developerWorks with that tag.

Use the slider bar to see more or fewer tags.

Popular tags shows the top tags for this particular type of content or application that you're viewing.

My tags shows your tags for this particular type of content or application that you're viewing.

Use the search field to find all types of content in My developerWorks with that tag. Use the slider bar to see more or fewer tags. Popular tags shows the top tags for this particular type of content or application that you're viewing. My tags shows your tags for this particular type of content or application that you're viewing.MoreLess 

View the original article here

Multi branch / distributed development support - by: Bhorsoft

Re:Multi branch / distributed development support 1 Day, 7 Hours ago Ah, yes.... and no.
Subversion sees the revision you create during the merge between branches as a new change (which strictly speaking it is as there's no way to know that you did not take action to resolve conflicts for example). So, although it can 'see' that change 123 has been integrated it cannot know that the merge change can be taken on faith, as it were.
This is where you might get conflicts when you 'drop' the source branches. Subversion sees, and ignores, 123 but 233 is a new change (just like the problem you have when bringing a --reintegrated branch back to life where the revision created on trunk is seen as a 'poison' revision in any subsequent integration from trunk to branch). You can always 'fix' the problem with the --record-only trick to tell Subversion to ignore some revisions during the --reintegrate. Merge with a --record only from the branch into the trunk will essentially tell Subversion that you are happy that the change is already dealt with.
Using the previous example, suppose the merge between branches resulted in revision 233 being created and I then drop the sourcebranch into the trunk (--reintegrate), I can forewarn Subversion that 233 should also be ignored in any future trunk merge using the --record-only merge technique. This should only be used if 233 was created without any incident (no additional action taken when merging between branches, otherwise I may overlook some legitimate integration issues).
No tool can reliably and automatically account for this sort of thing because during the merge between branches I may have to perform all sorts of integration actions both resolving direct conflicts and also potentially performing changes to different parts of the code base to accommodate the integrated change (123). The tool can only 'blindly' resolve the record that change 123 has been integrated with 233 (which Subversion does), it cannot understand any other actions taken in 233 and so will have to account for that in the drop to trunk. This requires a human to say 'well, actually 233 has no special actions in it, so go ahead and ignore it'. I guess a tool could look at 233 and infer that, if all the deltas in 123 and 233 are identical, then 233=123, but beyond that I think a person is needed to make a judgement call as to whether 233 can be ignored completely, ignored in part, or not ignored at all.
There are ways to minimise problems of this sort (by limiting the scope of changes, making changes as small as possible, integrating changes in parts; those that need no special actions and those that do, etc.) but that's all a bit beyond this thread or my capacity to explain in a single post
Hope that all makes sense
The administrator has disabled public write access. Re:Multi branch / distributed development support 9 Hours, 56 Minutes ago I guess the following closes this topic:
www.joelonsoftware.com/items/2010/03/17.html
I can’t tell you how many Subversion users have told me the following story: “We tried to branch our code, and that worked fine. But when it came time to merge back, it was a complete nightmare and we had to practically reapply every change by hand, and we swore never again and we developed a new way of developing software using if statements instead of branches.”
And more here

hginit.com/00.html
Almost every Subversion team I’ve spoken to has told me some variation on the very same story. This story is so common I should just name it “Subversion Story #1.” The story is this: at some point, they tried to branch their code, usually so that the shipping version which they gave their customers can be branched off separately from the version that the developers are playing with. And every team has told me that when they tried this, it worked fine, until they had to merge, and then it was a nightmare. What should have been a five minute process ended up with six programmers around a single computer working for two weeks trying to manually reapply every single bug fix from the stable build back into the development build.
And almost every Subversion team told me that they vowed “never again,” and they swore off branches.
THE END
The administrator has disabled public write access. Re:Multi branch / distributed development support 2 Hours, 41 Minutes ago We've been using SVN branching for a long time. Now we only work on release branches and never use the trunk - for us the trunk is like an appendix, it is there, but not needed. We only merge "upstream". Since we may be working on two major releases and a couple of patches for the same application, we rely on branches so we can work concurrently on different releases of the same code. We have had little problem merging from the earlier releases to the later release branches.
I think two of our keys is we merge daily. This ensures developers are working on the latest code. It also keeps conflicts to a minimum.
We also have centralized our merges. Only one person does the merges on a daily basis and runs a CI build immediately after the merge. That person can resolve most merge conflicts but will call on individual developers to work out any "tricky" conflicts.
We've been using this system for over 5 years and have never lost or mangled a change in all that time.
The administrator has disabled public write access. Vision without action is a daydream. Action without vision is a nightmare.
Japanese Proverb
T. V. Loy
Configuration Manager
View the original article here

Multi branch / distributed development support - by: mbools

Ah, yes.... and no.
Subversion sees the revision you create during the merge between branches as a new change (which strictly speaking it is as there's no way to know that you did not take action to resolve conflicts for example). So, although it can 'see' that change 123 has been integrated it cannot know that the merge change can be taken on faith, as it were.
This is where you might get conflicts when you 'drop' the source branches. Subversion sees, and ignores, 123 but 233 is a new change (just like the problem you have when bringing a --reintegrated branch back to life where the revision created on trunk is seen as a 'poison' revision in any subsequent integration from trunk to branch). You can always 'fix' the problem with the --record-only trick to tell Subversion to ignore some revisions during the --reintegrate. Merge with a --record only from the branch into the trunk will essentially tell Subversion that you are happy that the change is already dealt with.
Using the previous example, suppose the merge between branches resulted in revision 233 being created and I then drop the sourcebranch into the trunk (--reintegrate), I can forewarn Subversion that 233 should also be ignored in any future trunk merge using the --record-only merge technique. This should only be used if 233 was created without any incident (no additional action taken when merging between branches, otherwise I may overlook some legitimate integration issues).
No tool can reliably and automatically account for this sort of thing because during the merge between branches I may have to perform all sorts of integration actions both resolving direct conflicts and also potentially performing changes to different parts of the code base to accommodate the integrated change (123). The tool can only 'blindly' resolve the record that change 123 has been integrated with 233 (which Subversion does), it cannot understand any other actions taken in 233 and so will have to account for that in the drop to trunk. This requires a human to say 'well, actually 233 has no special actions in it, so go ahead and ignore it'. I guess a tool could look at 233 and infer that, if all the deltas in 123 and 233 are identical, then 233=123, but beyond that I think a person is needed to make a judgement call as to whether 233 can be ignored completely, ignored in part, or not ignored at all.
There are ways to minimise problems of this sort (by limiting the scope of changes, making changes as small as possible, integrating changes in parts; those that need no special actions and those that do, etc.) but that's all a bit beyond this thread or my capacity to explain in a single post
Hope that all makes sense
View the original article here

Subject: Multi branch / distributed development support - by: mbools

Tekkie:

Just so I keep my wires straight (it's not clear from the other posts on this thread), what do you mean by having problems with baseless merges?

View the original article here

Subject: Multi branch / distributed development support - by: mbools

Tekkie:

Ah, perhaps you were asking whether someone from another branch could --reintegrate between 5 and 6. Well, yes, but that's not a problem.

Suppose I 'drop' the branch as follows.

cd ~someone/trunk/wc
svn update
svn merge --reintegrate ^/branches/somebranch .
svn commit

Creating revision 123. This revision 123 is the 'poison' revision that will cause all sorts of trouble if we tried to merge from trunk into branch again.

Anyhow, I can sync (at pretty much any time between the drop and the next time I want to pull trunk changes up into somebranch) with:

cd ~someone/somebranch/wc
svn update
svn merge -c 123 --record-only ^/trunk
svn commit

Note, I am specifically merging only the poison revision in order to prevent problems in future. There is nothing in revision 123 that will impact the branch (it was, after all dropped from branch in the first place). Even if someone from another branch had, in the meantime, dropped revision 124 this would in no way be a problem.

Next time I merge trunk into somebranch I will see 124 as a new trunk change (no different to seeing any other trunk change) and revision 123 will be seen as 'already merged' because of my --record-only merge earlier, hence it will not cause trouble.

Even if someone adds to the somebranch before I do the bookkeeping --record-only merge it does not matter. Just so long as the bookkeeping is done before the next merge from trunk.

If you're disorganised and allow a change on trunk between the last integration into the branch and the 'drop' then you may have an issue, but that issue will simply be remerging work, Subversion will keep track of it just fine.

Suppose trunk was at 120 when you integrated up to somebranch (bringing somebranch up to date with the head of trunk). Then someone updated the trunk, adding 122. Then you 'drop' somebranch to create 123 on trunk (it would be better to pull 122 into somebranch, rebuild, retest and then drop---but let's suppose you need to mess trunk about for some reason). There may be some merge work in trunk between 122 and 123 changes. If you then merge --record-only 123 up to somebranch the next time you integrate trunk up to branch you will need to 'remerge' revision 122. The best way to avoid this is to stop working on trunk. Use trunk as a 'release' branch and use 'support' branches for defect resolution etc. That way you keep control over your integration patterns and can avoid this sort of complexity.

Maintaining trunk in this way also makes the 'drop' simpler because you can 'merge --reintegrate' and 'resolve --accept theirs-full' (not that this should be necessary if you're merging is organised properly). Assuming you make sure the source branch is always up-to-date with respect to branch before dropping (and why would you not? If you are merging during a release you are releasing untested code, which is pretty risky stuff---unless your system is properly segmented into sub-systems/modules in which case you are releasing these CI, and the providing the merging effort is isolated to one CI you should have no issues, but this is getting too complex for a forum post and somewhat off topic, so I'll stop now).

View the original article here

Subject: Multi branch / distributed development support - by: mbools

Tekkie:

Ah, perhaps you were asking whether someone from another branch could --reintegrate between 5 and 6. Well, yes, but that's not a problem.

Suppose I 'drop' the branch as follows.

cd ~someone/trunk/wc
svn update
svn merge --reintegrate ^/branches/somebranch .
svn commit

Creating revision 123. This revision 123 is the 'poison' revision that will cause all sorts of trouble if we tried to merge from trunk into branch again.

Anyhow, I can sync (at pretty much any time between the drop and the next time I want to pull trunk changes up into somebranch) with:

cd ~someone/somebranch/wc
svn update
svn merge -c 123 --record-only ^/trunk
svn commit

Note, I am specifically merging only the poison revision in order to prevent problems in future. There is nothing in revision 123 that will impact the branch (it was, after all dropped from branch in the first place). Even if someone from another branch had, in the meantime, dropped revision 124 this would in no way be a problem.

Next time I merge trunk into somebranch I will see 124 as a new trunk change (no different to seeing any other trunk change) and revision 123 will be seen as 'already merged' because of my --record-only merge earlier, hence it will not cause trouble.

Even if someone adds to the somebranch before I do the bookkeeping --record-only merge it does not matter. Just so long as the bookkeeping is done before the next merge from trunk.

If you're disorganised and allow a change on trunk between the last integration into the branch and the 'drop' then you may have an issue, but that issue will simply be remerging work, Subversion will keep track of it just fine.

Suppose trunk was at 120 when you integrated up to somebranch (bringing somebranch up to date with the head of trunk). Then someone updated the trunk, adding 122. Then you 'drop' somebranch to create 123 on trunk (it would be better to pull 122 into somebranch, rebuild, retest and then drop---but let's suppose you need to mess trunk about for some reason). There may be some merge work in trunk between 122 and 123 changes. If you then merge --record-only 123 up to somebranch the next time you integrate trunk up to branch you will need to 'remerge' revision 122. The best way to avoid this is to stop working on trunk. Use trunk as a 'release' branch and use 'support' branches for defect resolution etc. That way you keep control over your integration patterns and can avoid this sort of complexity.

Maintaining trunk in this way also makes the 'drop' simpler because you can 'merge --reintegrate' and 'resolve --accept theirs-full' (not that this should be necessary if you're merging is organised properly). Assuming you make sure the source branch is always up-to-date with respect to branch before dropping (and why would you not? If you are merging during a release you are releasing untested code, which is pretty risky stuff---unless your system is properly segmented into sub-systems/modules in which case you are releasing these CI, and the providing the merging effort is isolated to one CI you should have no issues, but this is getting too complex for a forum post and somewhat off topic, so I'll stop now).

View the original article here