Git conversion - Revision history

TomYu at 18:40, 12 May 2012

2012-05-12T18:40:59Z

TomYu: speling

2012-04-30T17:40:28Z

speling

Ghudson: /* Content conversion */

2012-04-28T20:14:34Z

‎Content conversion

Ghudson: /* Hooks */

2012-04-15T15:30:29Z

‎Hooks

TomYu: /* Scripts using svn */

2012-04-13T20:07:26Z

‎Scripts using svn

Ghudson: /* Hooks */

2012-04-13T20:01:47Z

‎Hooks

Ghudson at 18:52, 13 April 2012

2012-04-13T18:52:55Z

Ghudson: /* Hooks */

2012-04-13T18:44:12Z

‎Hooks

Ghudson: /* Branch and tag cleanup */

2012-04-13T18:30:06Z

‎Branch and tag cleanup

Show changes

Ghudson: /* Content outside of standard Subversion layout */

2012-04-13T18:26:03Z

‎Content outside of standard Subversion layout

← Older revision		Revision as of 18:40, 12 May 2012
Line 1:		Line 1:
	=Notes on converting from Subversion to Git=		=Notes on converting from Subversion to Git=
		+
		+	These notes are primarily for historical reference. The krb5 repository migrated to Git during the weekend of 2012-05-11.

	==Content conversion==		==Content conversion==

@@ Line 13: / Line 13: @@
 Subversion commit authors are simply repository-local usernames.  git commit authors contain a real name and an email address.  git svn fetch supports a map file mapping Subversion authors to git authors.  We will prepare a map file containing the names of all historical Subversion authors, using @mit.edu addresses for all usernames.
-In git repositories, it is common for the author and date of a commit to reflect the original authorship even if the author doesn't have write access to the repository, with separate matadata for the commit date and committer.  This practice was not possible in Subversion, so our Subversion log metadata reflects only the actual committer.  It would probably be too much effort to remedy this mismatch at conversion time.
+In git repositories, it is common for the author and date of a commit to reflect the original authorship even if the author doesn't have write access to the repository, with separate metadata for the commit date and committer.  This practice was not possible in Subversion, so our Subversion log metadata reflects only the actual committer.  It would probably be too much effort to remedy this mismatch at conversion time.
 ====Summary lines====

@@ Line 3: / Line 3: @@
 ==Content conversion==
-Most likely, bulk conversion of the Subversion repository will be done with git svn clone, as it was done for our krb5-anonsvn repository on github.
+Most likely, bulk conversion of the Subversion repository will be done with git svn fetch, as it was done for our krb5-anonsvn repository on github.
 Because of the way git uses SHA-1 hashes, it is desirable to make the conversion as clean as possible now, as doing any cleanup later will be disruptive to downstream repositories.  Because of our project's long history and historical lack of consistent code documentation, our version control history is a valuable investigative tool, and effort applied to cleaning up our history during conversion will pay dividends in the future.
@@ Line 11: / Line 11: @@
 ====Commit authorship====
-Subversion commit authors are simply repository-local usernames.  git commit authors contain a real name and username.  git svn clone supports a map file mapping Subversion authors to git authors.  We should prepare a map file containing the names of all historical Subversion authors, using @mit.edu addresses for all usernames.
+Subversion commit authors are simply repository-local usernames.  git commit authors contain a real name and an email address.  git svn fetch supports a map file mapping Subversion authors to git authors.  We will prepare a map file containing the names of all historical Subversion authors, using @mit.edu addresses for all usernames.
-In git repositories, it is common for the author and date of a commit to reflect the original authorship even if the author doesn't have write access to the repository, and a Signed-off-by: line to indicate who pushed the change.  This practice was not possible in Subversion, so our Subversion log metadata reflects the actual committer.  It would probably be too much effort to remedy this mismatch at conversion time.
+In git repositories, it is common for the author and date of a commit to reflect the original authorship even if the author doesn't have write access to the repository, with separate matadata for the commit date and committer.  This practice was not possible in Subversion, so our Subversion log metadata reflects only the actual committer.  It would probably be too much effort to remedy this mismatch at conversion time.
 ====Summary lines====
-It is recommended practice in git to begin each commit message with a summary line of less than 50 characters, followed by a blank line.  Some tools, such as git log --oneline, assume this practice.  It is probably worth some amount of effort to make most of our converted log messages follow this practice.
+It is recommended practice in git to begin each commit message with a summary line of less than 50 characters, followed by a blank line.  Some tools, such as git log --oneline, assume this practice.  We will try to make many of our converted log messages follow this practice through automated transformations.
-Many of our commit messages begin with RT metadata headers.  These should be moved to the end of the message so they do not appear in summar lines.  If a message contains a Subject: header for the new ticket, the contents of that header can be copied to the front in order to create a summary.
+Many of our commit messages begin with RT metadata headers.  These will be moved to the end of the message so they do not appear in summary lines.  If a message contains a Subject: header for the new ticket, the contents of that header will be copied to the front in order to create a summary.
 ====RT ticket numbers====
-A Subversion commit containing a "ticket: new" line creates a new RT ticket.  As of recently, such log messages are rewritten to reference the newly created ticket number instead of "new".  It may be worthwhile to rewrite old log messages as part of the conversion.  (XXX but how to find the ticket numbers automatically?  There are 760 of these, plus 294 pullups, so doing it by hand would be prohibitive.)
+A Subversion commit containing a "ticket: new" line creates a new RT ticket.  As of recently, such log messages are rewritten to reference the newly created ticket number instead of "new".  It would be useful to rewrite old log messages as part of the conversion, but we will not do so as it would be too difficult to automatically find the ticket numbers (and there are about 750 of these revisions, which is too many to do by hand).
 ====git-svn metadata====
@@ Line 31: / Line 31: @@
 ====Mechanics====
-Log messages can be modified after conversion (but before the converted repository is made public) using git filter-branch, or the messages can be changed in the Subversion repository prior to conversion.  Converting log messages using git filter-branch is inherently somewhat slower due to the need to recompute SHA-1 hashes on all subsequent revisions.
+Log messages can be modified after conversion (but before the converted repository is made public) using git filter-branch, or the messages can be changed in the Subversion repository prior to conversion.  Converting log messages using git filter-branch is inherently somewhat slower due to the need to recompute SHA-1 hashes on all subsequent revisions.  We will use a Python script to edit messages in a copy of the Subversion repository prior to conversion.
 ===Content outside of standard Subversion layout===
-Our Subversion repository contains some content outside of the standard trunk/tags/branches layout.  By default git-svn will ignore this content.  Branches can be added to the git-svn conversion by specifying multiple -b options to git svn clone; alternatively, the svn repository could be preprocessed (via dump and load) to move particular subdirectories into the branches directory.  Although git-svn does not gracefully handle branch name collisions by default, we can work around this problem by editing .git/config before the first fetch, as documented in the CAVEATS section of the git-svn man page.  Branches can be easily renamed after conversion.
+Our Subversion repository contains some content outside of the standard trunk/tags/branches layout.  By default git-svn will ignore this content.  Branches can be added to the git-svn conversion by specifying multiple -b options to git svn init.  Although git-svn does not gracefully handle branch name collisions by default, we can work around this problem by editing .git/config before the first fetch, as documented in the CAVEATS section of the git-svn man page.  Branches can be easily renamed after conversion.
-The top-level tools directory contains local work on external testing tools (gssmonger and gsstest).  It is likely best to ignore these and create separate repositories for them.
+The top-level tools directory contains local work on external testing tools (gssmonger and gsstest).  It is likely best to ignore these and create separate repositories for them if necessary.
 The top-level ChangeLogs directory contains copies of the ChangeLog files which were removed from the tree in r17893 (2006-04-11).  It is likely best to ignore this as well.  The history of these deleted files will be present in the converted repository, and it is easier to access historical versions of deleted ChangeLog files in git than in svn.
@@ Line 75: / Line 75: @@
 Some of the branches in the repository are relics of the cvs-to-svn conversion and should be removed.  These branch names begin with "unlabeled".
-Tags in the converted repository will be expressed as branches with names beginning with "tags/".  These should be converted to git tags.
+Tags in the converted repository will be expressed as branches with names beginning with "tags/".  These will be converted to git tags.
 Some branches in /branches or the above list are not correctly copied from trunk:
@@ Line 83: / Line 83: @@
 * users/raeburn/branches/plugin (copied from trunk/src)
-The referrals branch can probably be fixed using git filter-branch.  git-svn does not behave well when it sees a branch copied from trunk/src (it fetches another copy of the history of src and places it on the branch), so the branches under users/raeburn will probably need to be omitted.
+The referrals branch will be fixed using git filter-branch.  git-svn does not behave well when it sees a branch copied from trunk/src (it fetches another copy of the history of src and places it on the branch), so the branches under users/raeburn will be omitted in the conversion.  If we develop a way to convert these branches, we can add them to the converted repository later.
-For better readability of "git branch" and "git tag -l" output, we probably want to remove branches and tags which aren't likely to be useful--very old branches and tags, tags intended for use as anchors for CVS merges, and KfM branches and tags (which are very numerous).  If it turns out they are needed, all branches and tags will still be available in the frozen Subversion repository.
+For better readability of "git branch" and "git tag -l" output, we will remove branches and tags which aren't likely to be useful--very old branches and tags, tags intended for use as anchors for CVS merges, and KfM branches and tags (which are very numerous).  If it turns out they are needed, all branches and tags will still be available in the frozen Subversion repository.
-Our release branches and tags have names like krb5-X-Y, because CVS did not allow periods in branch names.  We can rename these to krb5-X.Y during conversion.  We can also rename the old 1.0 branches and tags (which have names like V1_0_BRANCH) to match the naming scheme of later branches.
+Our release branches and tags have names like krb5-X-Y, because CVS did not allow periods in branch names.  We will rename these to krb5-X.Y during conversion.  We will also rename the 1.0 release branches and tags (which have names like V1_0_BRANCH) to match the naming scheme of later branches.
 ==Hooks==

@@ Line 99: / Line 99: @@
 * Send the commit log through rt-cvsgate on krbdev.mit.edu to process RT headers
-We will want to perform similar tasks for the git repository, although the "ticket: new" translation may be impractical (see below).  In addition, we may want to add checks to ensure that committers are recorded in commit logs when they are not the original authors of the commits (XXX investigate best practices for this).
+We will want to perform similar tasks for the git repository, although the "ticket: new" translation may be impractical (see below).  In addition, we may want to:
 rt-cvsgate expects RT headers to be at the beginning of the commit log.  We should modify it to accept headers at the end of the message as well.

@@ Line 116: / Line 116: @@
 * util/find-missing-eol-prop and util/fix-eol-prop use svn, but should be unnecessary after conversion to git.
 * The nightly Coverity build script uses svn to maintain a copy of the trunk
-* XXX cron jobs to generate snapshots?
+* cron jobs use svn to generate snapshots, but could be consolidated into the buildbot setup
-* XXX cron job to update github? (maybe a hook after conversion)
+* cron job to update github can be mostly superseded by a git hook, but there should still be a cron job in case there is a transient failure to push to github.
 ==Issues==

← Older revision		Revision as of 20:01, 13 April 2012
Line 100:		Line 100:

	We will want to perform similar tasks for the git repository, although the "ticket: new" translation may be impractical (see below). In addition, we may want to add checks to ensure that committers are recorded in commit logs when they are not the original authors of the commits (XXX investigate best practices for this).		We will want to perform similar tasks for the git repository, although the "ticket: new" translation may be impractical (see below). In addition, we may want to add checks to ensure that committers are recorded in commit logs when they are not the original authors of the commits (XXX investigate best practices for this).
		+
		+	rt-cvsgate expects RT headers to be at the beginning of the commit log. We should modify it to accept headers at the end of the message as well.

	===RT "ticket: new" support===		===RT "ticket: new" support===

← Older revision		Revision as of 18:52, 13 April 2012
Line 108:		Line 108:

	A simple option is to make a script for developers to run to allocate a ticket (out of the krbdev locker on Athena, or copied to the developer's homedir). The script would output a ticket number which the developer would put in the "ticket:" header of the commit log.		A simple option is to make a script for developers to run to allocate a ticket (out of the krbdev locker on Athena, or copied to the developer's homedir). The script would output a ticket number which the developer would put in the "ticket:" header of the commit log.
		+
		+	==Scripts using svn==
		+
		+	* util/mkrel uses svn to export a release tree.
		+	* util/find-missing-eol-prop and util/fix-eol-prop use svn, but should be unnecessary after conversion to git.
		+	* The nightly Coverity build script uses svn to maintain a copy of the trunk
		+	* XXX buildbot?
		+	* XXX cron jobs to generate snapshots?
		+	* XXX cron job to update github? (maybe a hook after conversion)

	==Issues==		==Issues==

@@ Line 91: / Line 91: @@
 ==Hooks==
 To support the "ticket: new" capability, we currently rewrite the log messages of the commits on the svn server. There is no easy way to have a server-side git hook rewrite the commits it receives without exposing some intermediate state. The post-receive hook could rewrite the commit messages, but it would have to move the pushed branch head to point at the rewritten commit stream, because the post-receive hook runs after the server updates the refs. "git push" doesn't automatically rebase from the server's state; the pushing client would have to have a hook to rebase its branches after this event. A second client that contacts the server between when the original pushed ref update occurs on the server and when the post-rewrite refs are exposed would also have to rebase, which could be inconvenient.
 For ticket numbers, git notes are an alternative that some people have suggested. They have the disadvantage of not being immutable, and might not clone so well across repositories. They're also moderately new, and it's plausible that some older OS distributions of git won't support them.
 ==Issues==

@@ Line 49: / Line 49: @@
 * users/coffman/keyring -- Linux keyring ccache support
 * users/coffman/pkinit -- PKINIT preauth module
-* users/hartmans/pkinit -- Apple PKINIT
+* users/hartmans/fast -- FAST support
 * users/hartmans/fast-negotiate -- FAST negotiation support
 * users/lhoward/aes-ccm -- AES-CCM cipher support (never merged due to issues with CCM and Kerberos)