Skip to content
This repository was archived by the owner on Sep 25, 2020. It is now read-only.

Convertet blog post to Markdown #8

Open
ooxi wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Convertet blog post to Markdown #8

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
314 changes: 308 additions & 6 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,312 @@
A small project with some helper scripts for moving a project of yours on
[Google Code project hosting](http://code.google.com/hosting/) to
[Github](https://github.com/).
Google Code → GitHub
====================

See this blog post giving example usage: <http://trentm.com/2012/03/google-code-to-github.html>
[This README was extracted from a blogpost on trentm.com](http://trentm.com/2012/03/google-code-to-github.html)

Herein a walk-through of moving a project from Google Code to GitHub. The
project is `python-markdown2`: [on Google Code](http://code.google.com/p/python-markdown2/) →
[on GitHub](https://github.com/trentm/python-markdown2). It was single-commiter
subversion project on Google Code with one tag and no branches. It had (has) a
number of issues in the issue tracker, a few wiki pages and a few downloads.

# wiki conversion
This includes a couple tools I wrote for converting between Google Code and
GitHub project wikis and issue trackers. _Note:_ I originally did this migration
almost a year ago, so I'm not positive whether recent changes in the GitHub
issues API might break this tool.

GitHub project
--------------

First create the new project on GitHub.

Source code repo
----------------

As per [GitHub help](http://help.github.com/svn-importing/), [svn2git](https://github.com/nirvdrum/svn2git)
is preferred. At first I hit an inscrutable error from `svn2git`:

error: pathspec 'master' did not match any file(s) known to git

The only reference I could find with a cure was:

http://mysz.tumblr.com/post/2515522812/git2svn-no-author-i-bledy-importu

which indicated that you need an entry in `authors.txt` for `(no author)`,
something like so:

(no author) = unknown author <[email protected]>;

My first import attempt _appeared_ to work, but failed to import all commits.
A subsequent attempt (this time with `--verbose` option) worked. Not sure if
`--verbose` makes a difference. I hope not. Here were my steps:

```bash
$ cat ~/tm/authors.txt
trentm = Trent Mick <[email protected]>
(no author) = unknown author <[email protected]>
$ mkdir ~/tm/python-markdown2
$ cd ~/tm/python-markdown2
$ svn2git https://python-markdown2.googlecode.com/svn --authors ~/tm/authors.txt
W: -empty_dir: trunk/test/tm-cases/blog.markdown
W: -empty_dir: trunk/test/tm-cases/extintro.markdown
W: -empty_dir: trunk/test/tm-cases/link_defn_alt_title_delims.tags
W: -empty_dir: trunk/test/tm-cases/safe_mode.html
W: -empty_dir: trunk/test/tm-cases/safe_mode.opts
W: -empty_dir: trunk/test/tm-cases/safe_mode.text
W: -empty_dir: trunk/markdown2.py
W: -empty_dir: trunk/bin/markdown2
W: -empty_dir: trunk/test/tm-cases/issue31_gt_escaping.html
W: -empty_dir: trunk/test/tm-cases/issue31_gt_escaping.tags
W: -empty_dir: trunk/test/tm-cases/issue31_gt_escaping.text
Found possible branch point: https://python-markdown2.googlecode.com/svn/trunk => https://python-markdown2.googlecode.com/svn/tags/v1.0.1.17, 240
Use of uninitialized value $u in substitution (s///) at /usr/local/git/libexec/git-core/git-svn line 1731.
Use of uninitialized value $u in concatenation (.) or string at /usr/local/git/libexec/git-core/git-svn line 1731.
refs/remotes/trunk: 'https://python-markdown2.googlecode.com/svn' not found in ''

Note: checking out 'trunk'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

git checkout -b new_branch_name

HEAD is now at acb4fca... TODOne a while back
error: branch 'master' not found.
Switched to a new branch 'master'
Counting objects: 1175, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (1069/1069), done.
Writing objects: 100% (1175/1175), done.
Total 1175 (delta 611), reused 0 (delta 0)
Removing duplicate objects: 100% (256/256), done.

HEAD is now at cb87cf8... Tweaks to 'header-ids' and 'toc' extras. Also add a ".postprocess(text)" hook for subclasses' convenience.
Running command: git branch -D master
Deleted branch master (was cb87cf8).
Running command: git checkout -f -b master
Switched to a new branch 'master'
Running command: git gc
Counting objects: 1290, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (1166/1166), done.
Writing objects: 100% (1290/1290), done.
Total 1290 (delta 681), reused 0 (delta 0)
Removing duplicate objects: 100% (256/256), done.
```

Then a sanity check diffing the HEAD state in an svn working copy and a git
clone:

```bash
$ diff -ru -x .svn -x .git python-markdown2.svn python-markdown2 | less
$ diff -ru -x .svn -x .git python-markdown2.svn/externals/which/which.py python-markdown2/externals/which/which.py
--- python-markdown2.svn/externals/which/which.py 2010-10-14 09:55:52.000000000 -0700
+++ python-markdown2/externals/which/which.py 2011-01-31 21:14:20.000000000 -0800
@@ -65,7 +65,7 @@
files without executable access.
"""

-__revision__ = "$Id: which.py 82 2007-11-06 05:44:22Z trentm $"
+__revision__ = "$Id$"
__version_info__ = (1, 1, 3)
__version__ = '.'.join(map(str, __version_info__))
__all__ = ["which", "whichall", "whichgen", "WhichError"]
```

That's an acceptable diff. I don't care about that `$Id$` expansion facility.

Now for ignore-patterns:

```bash
$ cd python-markdown2
$ svn propget svn:ignore ../python-markdown2.svn > .gitignore
$ git add .gitignore
$ git commit -m ".gitignore from svn:ignore" .gitignore
[master d31e29e] .gitignore from svn:ignore
1 files changed, 8 insertions(+), 0 deletions(-)
create mode 100644 .gitignore
```

In general you'd have to walk the full svn working copy for `svn:ignore`
properties on any directory. In my case I had a bunch to incorporate (manually)
into my `.gitignore` file:

```bash
$ find . -type d | grep -v .svn | xargs svn pl -v .
Properties on '.':
svn:ignore
*.pyc
tmp
externals
dist
MANIFEST
build
googlecode_upload.py

Properties on 'externals':
svn:ignore
pygments

Properties on 'externals/which':
svn:ignore
*.pyc

Properties on 'lib':
svn:ignore
*.pyc

Properties on 'perf':
svn:ignore
*.pyc
*.prof
tmp-*-cases

Properties on 'sandbox':
svn:ignore
*.html

Properties on 'test':
svn:ignore
*.pyc
```

Time to push this to github:

```bash
$ cd .../python-markdown2
$ git remote add origin [email protected]:trentm/python-markdown2.git
$ git push origin master
```

Google Code project page → README.md
------------------------------------

A Google Code project's front page is the wiki-formatted project page. A GitHub
project's front page (prose) is the README file. You might want to start with
your Google Code project [wiki page content](http://code.google.com/p/PROJNAME/admin)
and use that to create a `README.md` (or `README.txt` or whatever). I already
had a `README.txt` and I'm a Markdown guy (duh) so I converted it to `README.md`
to have it rendered nicely on my GitHub project page.

Wiki pages
----------

Google Code for a long time (from the get go?) allowed you to have a wiki for
your project whose content is in Subversion -- nice feature. GitHub more
recently added support for project wiki pages being in a git repo. This will
make moving wiki pages straightforward.

First, create your wiki on GitHub by clicking `Create Wiki Now` on your
[project's wiki page](https://github.com/USERNAME/PROJNAME/wiki). Then let's get
working copies of the old and the new:

```bash
$ cd ...
$ svn co https://python-markdown2.googlecode.com/svn/wiki python-markdown2-wiki.svn
$ git clone [email protected]:trentm/python-markdown2.wiki.git
```

At this point you could do a whole subversion history conversion to git, but I
don't care about the wiki editing history.

Let's just copy over and convert the files.

```bash
$ python googlecode2github/wikiconvert.py \
trentm/python-markdown2 \
python-markdown2-wiki.svn \
python-markdown2.wiki
```

`wikiconvert.py` is a tool I wrote to help with this conversion. It doesn't
cover the full Google Code wiki syntax, just enough for what was my typical
usage... so you may want to do some post tweaking. However, it should be a good
start. See (github/gollum)[https://github.com/github/gollum] for details on
other wiki tweaks you can do on GitHub. `wikiconvert` lives on [GitHub](https://github.com/trentm/googlecode2github).

Review, commit and push in `python-markdown2.wiki`:

```bash
$ cd python-markdown2.wiki
$ git add .
$ git diff --staged
$ git commit -m "convert wiki from code.google.com/python-markdown2 with 'googlecode2github/wikiconvert'"
```

Issues
------

Fully transplanting all issues (along with timing, status, user data, comments,
attachments, etc.) is a fool's errand. Instead we'll just create a shadow issue
for each google code issue so that (a) we have a pointer for all issues from the
new GitHub project and (b) there are no issue number collisions when new issues
are added on GitHub.

[`shadowissues.py`](https://github.com/trentm/googlecode2github/blob/master/shadowissues.py)
will do this for us. For each shadow issue it'll try to reproduce the title,
original description and the closed/open state. To make changes you GH issues,
the script will need you github username and API token. The latter is available
at [GitHub account admin](https://github.com/account#admin_bucket). The script
will ask interactively or you can add them to your `~/.gitconfig` via:

```bash
$ git config --add github.user [github_username]
$ git config --add github.token [github_api_token]
```

As a sanity check, *I suggest that you first run this against a test github
project* (e.g. I used `trentm/ghtest`). This is because `shadowissues.py` wants
the shadow issues to have the issue ids match those in the Google Code project.
You cannot delete issues on a github project (a good thing) so you only get one
chance:

```bash
$ python .../googlecode2github/shadowissues.py python-markdown2 trentm/ghtest
# Gathering code.google.com/p/python-markdown2 issues.
# Gathering any github.com/trentm/ghtest issues.
Migrating issue 1.
from: http://code.google.com/p/python-markdown2/issues/detail?id=1
to: https://github.com/trentm/ghtest/issues/1
...
```

Review those created GH-issues, then if things look good, run it for realz:

```bash
$ python .../googlecode2github/shadowissues.py python-markdown2 trentm/python-markdown2
# Gathering code.google.com/p/python-markdown2 issues.
# Gathering any github.com/trentm/python-markdown2 issues.
Migrating issue 1.
from: http://code.google.com/p/python-markdown2/issues/detail?id=1
to: https://github.com/trentm/python-markdown2/issues/1
...
```

One last change for issues: Now that you've moved to github, you want new issues
to be added there. You might want to change the Google Code new issue template
to something like this:

```
NOTICE: This project has moved to <https://github.com/trentm/python-markdown2>
Please report your issue here:

https://github.com/trentm/python-markdown2/issues
```

This can be done on the equivalent of http://code.google.com/p/python-markdown2/adminIssues
for your project.

Pointer from google code to github
----------------------------------

A few extra touches for your Google Code project to provide pointers to the new
GitHub location.

* Summary: append `(**MOVED TO GITHUB**)`
* Description: prepend `= Note: This project has been moved to [https://github.com/trentm/python-markdown2 trentm/python-markdown2 on GitHub]. =`
* Wiki pages: prepend `= Note: This project has been moved to [https://github.com/trentm/python-markdown2 trentm/python-markdown2 on GitHub]. =`

- <http://code.google.com/p/support/wiki/WikiSyntax>