For a long time people have suggested that we migrate from Subversion (our current version control system) to Git or Mercurial. This isn't a trivial undertaking, but if the CVS to Subversion migration in 2004 is any indication it would definitely be worthwhile. Unfortunately the path forward isn't obvious. Subversion, Git and Mercurial all work well. It's also unclear how much trouble switching away from Subversion would cause any downstream organizations who maintain their own versions of Wireshark. As a result I'm asking for input from the development community. Please vote for your preferred course of action below. Feel free to add additional answers as you see fit. If you downvote an answer, please explain your reason in a comment. As with any other question on this site, the accepted answer may not necessarily be the one with the most votes. :) asked 30 Aug '11, 16:37  Gerald Combs ♦♦ |
5 Answers:
Migrate to Git! Linus knows best. If you'd like to test Git a mirror of the SVN repository is available for cloning and browsing at http://code.wireshark.org/git/wireshark. (This isn't an endorsment of Git; we simply have a lot of vocal Git fans.) answered 30 Aug '11, 16:38 Gerald Combs ♦♦ edited 30 Aug '11, 16:43 2 (30 Aug '11, 19:05) helloworld 1 For developing the DVB-CI dissector, I import the svn repository into git. This allows me to keep local branches for new things I'm trying out. I can easily merge things between those branches and keep them in sync with updates made by others. If I used svn directly, I'd have no chance to commit things locally (and I'm not allowed to commit directly to the svn repository ;-( ). (31 Aug '11, 00:48) MartinKaiser I was also using git-svn. hopefully we get git as a main SCM. (31 Aug '11, 11:24) Andrei Emelt... 1 The git (or other dvcs) workflow outside of linux kernel work isn't clear to me. Does Gerald create a repo that core devs have push privs on, or does Gerald (or someone else) have to respond to pull requests and then deal with merge issues? How can access to the buildbots be delivered for repos other than the "main" one? (31 Aug '11, 12:42) grahamb ♦ I think buildbot shoue automatically build Core Devs' repositories and commits should enter main after having at least one successful build in any of the Core Developers repositories. (31 Aug '11, 13:43) rbalint @grahamb workflow can be defined as you want to. And as far as I know, it has not been defined for wireshark yet. Could be a different thread on the mailing list. For sure, Gerald (nor a 1 someone else) would not have to deal with merge issues. Each contributor would have to lead with merge issues. (01 Sep '11, 04:25) standel What I want most is to be able to checkpoint one or more intermediate states of my current patch before its ready to submit. E.g. - builds under linux - first working version - builds under windows - passes checkAPIs and be able to diff/revert back to them, before pushing a single, complete change to the main repo. I would want this branch to be local/private - I doubt I'd want to share my branch with anyone else. Would git enable this workflow for me? (04 Sep '11, 15:03) MartinM 1 @MartinM: Yes, with git you in fact build a sequence of commits (changeset) on top of the last public commit. The order of these commits can be changed, you can aggregate two or more commits in one, or split one in multiple commits or modify (amend) a commit. You can rebase at any time (i.e. download the latest public commits and move your changeset in front of them) - I personally rebase all my branches each morning. Commits can be copied between branches, bisection can be used to find what commit introduced a bug. When your changeset is ready, you can send it to a commiter. (04 Sep '11, 17:41) petithug @MartinM, I think you can do this already with git-svn as MartinK posted above. See also HERE. (05 Sep '11, 08:16) grahamb ♦ Remember how the site of linux kernel was attacked a few days ago (http://lxnews.org/2011/09/02/linux-kernel-website-hacked/)? I don't know if it is like what I imagined, but I'm sure the source codes (and the SHA-1 hash values) were secure enough since they are distributed among the core developers. Thus any eventual changes in the codes and/or the hash values by the attacker could be easily checked. This is one major advantage of git, but surely it depends on which git model is used. (07 Sep '11, 07:54) francesc @petithug, "When your changeset is ready, you can send it to a commiter." Which committer? I don't think that would work well in practice as committer availability/knowledge/time varies. I think a workflow that still involves Bugzilla is more practical. (07 Sep '11, 16:22) cmaynard ♦♦ @cmaynard, I haven't yet used Gerrit myself, but its workflow (taken from Gerrit docs) shows that once a patch is submitted, Gerrit automatically notifies the project owners/verifiers while the submitter notifies reviewers through Gerrit. These steps closely match up to the current patch submittal process with Bugzilla. (07 Sep '11, 16:58) helloworld @cmaynard, ...and Gerrit supports side-by-side or unified diff for patches (something Bugzilla does not). See Gerrit screenshots (probably slightly outdated). (07 Sep '11, 17:04) helloworld @helloworld, it's nice that gerrit allows you to compare side-by-side or unified, but my comment about viewcvs was geared toward comparing changes already committed. (07 Sep '11, 18:09) cmaynard ♦♦ @cmaynard: That was just an example. You can send it to a mailing list (there is even a git command to send it directly), store it in Bugzilla, or push it in something like Gerrit. See my comments below about a workflow that can be implemented with Gerrit. (07 Sep '11, 23:15) petithug @cmaynard, I agree with your thoughts on viewgit, and I wasn't arguing that (otherwise, I would've commented on that post instead of this one). I was only pointing out that Gerrit has better diff support for patches than Bugzilla does. (08 Sep '11, 15:34) helloworld showing 5 of 16 show 11 more comments |
Stay with Subversion! It does what I need, and I don't need any more. answered 30 Aug '11, 16:38 Gerald Combs ♦♦ 2 Change for the sake of change is silly. What are the current issues which Subversion cannot handle? (30 Aug '11, 18:48) MavEtJu Our inhouse version and merge procedures is setup to use SVN (30 Aug '11, 21:39) Anders ♦ My preferred option too, but see my comment to SYNbit's answer about making use of feature branches. (31 Aug '11, 00:03) grahamb ♦ 1 With svn feature branches are not available for everyone. Only for core developers! With git, feature branches are available for everyone. (31 Aug '11, 06:44) standel Slightly different aim with feature branches and local branches. I would think that most if not all dvcs would support local branches. (31 Aug '11, 09:30) grahamb ♦ Please do NOT post your own answers to the question if you're just going to vote for git/Subversion/Mercurial; this is kind of a "funny" question, in that the votes for the answers will be treated as votes for the choices given by Gerald's answers. I've converted "Change for the sake of change is silly. What are the current issues which Subversion cannot handle?" to a comment on "Stay with Subversion", and added its two votes to "Stay with Subversion". (31 Aug '11, 17:18) Guy Harris ♦♦ Personally I think viewcvs is far superior to what I've seen from viewgit. I really like having the ability to view changes side by side. Maybe that is possible from viewgit? Also, I've read discussions about versioning, and maybe it's just me, but I just don't get it. With subversion, it's very easy to know that r38904 is older than svn r38928. With git, hashes don't tell me much. Then there's the workflow. I'm sure it's because of my lack of git knowledge, but I don't really understand the git workflow. If a switch to git is made, then I guess I've got some reading to do ... (07 Sep '11, 16:30) cmaynard ♦♦ showing 5 of 7 show 2 more comments |
Migrate to Mercurial! Guido van Rossum and Joel Spolsky have the right idea. answered 30 Aug '11, 16:40 Gerald Combs ♦♦ Joel Spolsky has an excellent Mercurial tutorial: Hg Init (31 Aug '11, 08:45) Gerald Combs ♦♦ The last time I checked (was two years ago or so...) Mercurial had a better Windows support than Git with a community actively supporting Microsoft OS. (31 Aug '11, 23:36) Pascal Quantin @Pascal: How do you define "better"? Performance? Memory? OS integration (with File Explorer, etc.)? In any case, a lot of improvements can happen in 2 years. I wouldn't vote based on outdated info. (01 Sep '11, 07:28) bstn @bstn: I meant that core developers of Mercurial are Windows users and ensure that it works out of the box, fixing specific Windows issues. I also find the Mercurial user interface more intuitive (just a matter of taste) and globally it is really easy to use and perfectly suits my needs. But I will have another look at the current state of the art for Git as you suggested. (07 Sep '11, 23:54) Pascal Quantin |
I already use the Subversion under git (using git svn), so staying with Subversion is OK for me. On the other hand I voted for git because the work-flow that git naturally favors makes for better commits - if one ever try to do some bisecting on the git tree, one would discover that the current work flow is more like "commit until it builds", than the "commit a well-tested and peer reviewed changeset" that is the git way. Moving to git would, hopefully, naturally improve this. It seems that it is also possible to use mercurial with git (http://hg-git.github.com/) but that seems rather difficult in the other direction. But the sticky point will probably be that AFAIK it is not possible to use git (or mercurial) from svn and as you have to unlearn svn to learn git, that would leave a lot of developers unhappy. So if there is consensus on git then by all means switch to that, but I predict that we'll stay on subversion because people can use either svn, git or mercurial in this case. I just wish that the committers start using git (or mercurial) themselves, so we do not have as frequent breakage as today. answered 31 Aug '11, 07:56  petithug edited 31 Aug '11, 09:05 As mentioned elsewhere, feature branches are the preferred svn method to handle large "breaking" changes. (31 Aug '11, 09:31) grahamb ♦
(31 Aug '11, 11:49) Anders ♦ as mentioned in different places, "feature" branches with svn are not available to everyone ... only to core developers. (31 Aug '11, 12:24) standel 2 @Anders: One could do creative things with multiple central repositories. For instance, have an 'incoming' staging repository that the buildbots use and only push to the 'main' repository when the buildbots are green. (31 Aug '11, 13:23) leogah Did I mention this Git branching model before? I think it would be worthwhile working out the specifics for Wireshark development. (31 Aug '11, 13:57) Jaap ♦ 3 @Anders: One of the tools I put in place in my company is Gerrit, a peer review tool which is integrated with git. It in fact manages a central git repository, but still permits to all my developers (who are in 3 different continents) to develop offline and to need a connection only to upload a changeset, pull the latest version of the tree or do a review. This is the tool that Google is using for Android: https://review.source.android.com (31 Aug '11, 14:20) petithug another very interesting code review tool is Review Board (http://www.reviewboard.org) An example of a patch review : http://demo.reviewboard.org/r/48/diff/#index_header (31 Aug '11, 16:08) standel 2 It seems that Buildbot can be integrated with Gerrit, which could permit to build a nice workflow: 1 Any developer can upload their changeset to Gerrit. 2 Buildbots automatically pull the changeset from Gerrit and build for various platforms. If a build fails, the changeset is marked as failed by the Buildbot, and the developer has to update the changeset. 3 A committer do a review of the code, the developer does the suggested modifications until the reviewer and the Buildbots are happy. 4 When the committer marks the changeset as reviewed, the changeset is committed in the central repository. (05 Sep '11, 07:36) petithug @ Jaap, I thought git made life simple, that seems very complex to me. (05 Sep '11, 08:10) grahamb ♦ 1 @petithug: Your suggested workflow with integrating git with gerrit sounds really good. I guess that would be a real advantage over having SVN for the central repository and having people use git-svn to use git locally. (05 Sep '11, 11:15) SYN-bit ♦♦ @petithug & SYNbit All that is lacking is the manpower :-) I think we are stuggling to keep up with bugzilla as it is. (07 Sep '11, 09:10) Anders ♦ 2 @Anders: IMO using Gerrit decreases the load on commiters. First of all the burden of rebasing is on the developer. Then with git and Gerrit it is better to split a big commit in multiple commits, so reviews are easier. The fact that you can directly comment on the web a commit (line by line) should also help (and anybody can comment on a patch). I used svn + patches in bugzilla for years, and switching to git + gerrit definitively improved things. I used the difference in time to implement more thorough peer reviews, so with the same resources I was able to produce a far better code. (07 Sep '11, 11:23) petithug showing 5 of 12 show 7 more comments |
For me it does not really matter which SCM we use, as long as it does its job. Subversion does the job for me. However, I run into problems when I develop stuff when it is something substantial (which I can not do in one run, so sometimes things lie on the shelf for months). The way I do it now is to save my partial diff and then re-patch when I continue. No need to say that often (especially with the GUI stuff) I need to rewrite lots of stuff to make it even possible to reload my partial code. I have not explored the possibilities of subversion to make my life easier in this regard, so it could be that subversion has could already make my life easier in this regard. If not another SCM which does have functionality in this area might be beneficial. Can anyone with more experience on SCM's enlighten me on how I could make my life easier with any of the SCM's? answered 30 Aug '11, 22:24  SYN-bit ♦♦ For longer lived pieces of development work we should make use of feature branches in svn. A feature branch is created (usually in /reporoot/branches/nameofmyfeature) by copying from the trunk. The developer(s) switch their working copy to the branch and can commit away without upsetting anyone else. Other changes applied to the trunk are merged into the branch as required. When dev is complete, switch your working copy back to trunk, merge in your branch and test. When happy, commit back to trunk (using the --reintegrate option) and delete the branch. (31 Aug '11, 00:00) grahamb ♦ Pt2. Note that you are free to switch your working copy (or create another) between trunk and feature branches at will to work on either, although you should make sure your working copy is free from local changes before doing so as you don't want to carry changes destined from the branch direct into the trunk and vice-versa. To really make this useful it would be handy to direct the buildbots to do a run against a feature branch. (31 Aug '11, 00:01) grahamb ♦ 1 With svn, once you commit back to trunk, you have to delete your branch. Have you ever wonder why? With git (or mercurial), you don't need to! feature branch with git is LOCAL, you don't need to have any write access to the server. Imagine all the non-core developers who want to create a feature branch? They don't have any commit access to wireshark svn server. Have you already thought of them? (31 Aug '11, 06:43) standel @Graham: Thanks for the explanation on how to work on big changes. However, do I understand you correctly that I need to make a feature branch on the central repository to make this scheme work? I would rather keep my projects local until they are ready. Is this possible with svn (as you can see, I have not done any "homework" on svn). (31 Aug '11, 10:35) SYN-bit ♦♦ @standel, There is no absolute need to delete an svn feature branch but as work on it is complete after it has been merged to the trunk it''s usually deleted as a matter of hygiene. What you mustn't do is do further work on the feature branch and try to merge to trunk again. @SYNbit, As svn is a centralised vcs then yes the feature branch must be created in the main repo. What this does allow, though is other folks to have visibility of the work (and collaborate) on the branch and for buildbots to attempt to build the branch. (31 Aug '11, 12:38) grahamb ♦
There is no workflow possible with svn that are impossible with a dvcs (be it git or mercurial). They're just a lot more flexible than svn. (31 Aug '11, 16:14) standel From all the things I've read so far, git has more options to maintain local branches and merge them when needed, while SVN supports our current workflow better and people are more familiar with it at the moment. I also understand that it is possible to use git svn to import and export changes from SVN to git and back. Are there any downsides in keeping SVN for the central Wireshark repository and letting people use git for their local branches? (04 Sep '11, 03:27) SYN-bit ♦♦ @ standel And what do dvcs users do when finished with a feature or local branch? Deleting the svn feature branch just removes it from current view of "HEAD", it's still there in the history. Flexibility usually comes with the cost of complexity. @SYNbit I agree with you here, folks have already posted recipes for doing this. (05 Sep '11, 08:05) grahamb ♦ @grahamb When a user merges his branch with git/hg, he /can/ continue to develop on top of that branch without any further action if needed. Later, when he'll decide to merge again its new work, it will work. Simple as that. Of course, if the branch is not useful anymore because the feature is fully finished, he could choose to delete the branch (or keep it for historic reasons, whatever). My point is that a user won't delete the branch because working more on top of it after a merge could create a mess with the SCM on the next merge. (07 Sep '11, 05:26) standel about complexity coming with flexibility ... It's assuming there is no useless complexity in the product you're considering. For example: Do you think svn is more flexible than cvs? (I hope you think svn is more flexible :)) But ... would you say cvs is simpler than svn? (07 Sep '11, 06:21) standel showing 5 of 10 show 5 more comments |
...and don't add an answer if you're just supporting one of the existing answers - vote up the existing answer in question and, if you have something more to say, add a comment to it.