Interfacing CGit and Gitolite

As many of you know, the KDE Project is transitioning to using Git with Gitolite and CGit. As such, I thought I’d update my aging Gitweb/posix-permissions installation of git to use CGit and Gitolite, and now my public git repository is kicking away. (If you’d like commit access any place or would like to host your own repo on my server, drop me a line.)

Since Gitolite manages git repositories, it has the option of generating the necessary information for Git’s shipped gitweb. This includes making a static list of repository names that should be included in gitweb as well as optionally adding the gitweb.owner entry inside .git/config and the description file at .git/description. The static list of repository names is boring and standard and easy. The owner and description specifications are standards set by the Git project for this kind of information. Hence, Gitolite supports interfacing with them.

Meanwhile, CGit uses its own configuration format for determining the owner and description and repository path. For interfacing with Gitolite, in the past I have created a hook that writes out a CGit-formated configuration file, which is then included in the main cgitrc with the include directive. Essentially I had to do this:

gitcode@starfox ~ $ cat web/cgit/generaterepos.sh 
#!/bin/sh
 
cd $(dirname "$0")
rm -f repos.tmp
 
cat ~/projects.list | while read gitname; do
        name=${gitname%.*}
        fullpath=/home/gitcode/repositories/$gitname
        owner=$(git --git-dir=$fullpath config --get gitweb.owner)
        desc=$(cat $fullpath/description)
        (
                echo repo.url=$name
                echo repo.name=$name
                echo repo.path=$fullpath
                echo repo.desc=$desc
                echo repo.owner=$owner
                echo repo.enable-log-filecount=1
                echo repo.enable-log-linecount=1
        ) >> repos.tmp
done
 
mv repos.tmp repos
 
gitcode@starfox ~ $ tail -n 1 web/cgit/cgitrc 
include=/home/gitcode/web/cgit/repos
 
gitcode@starfox ~ $ cat repositories/gitolite-admin.git/hooks/post-update.secondary 
#!/bin/sh
exec /home/gitcode/web/cgit/generaterepos.sh

This worked decently, but it was cumbersome and ugly, and was likely not to scale as features in both Gitolite and CGit are added and changed. Luckily, CGit supports the scan-path option, which builds an internal list of repositories automatically by scanning a directory for git folders. One such solution for integrating with Gitolite would be to simply point scan-path at Gitolite’s repository directory. This works fine, but it has three main shortcomings, which I’ve addressed this in a generic non-Gitolite-specific way in three patches. Let’s walk through them one by one.

project-list

We don’t want all Gitolite repositories showing up on CGit, and Gitolite provides a generic mechanism for controlling this: it writes a list of all the repositories selected for Gitweb to a file called projects.list. It’s just a flat file with each repository’s name written on a new line:

CheeseWhiz.git
Geoemail.git
MyCoolThangs.git

So, what about augmenting CGit’s scan-path feature with another setting called “project-list” that points to this file? That’s what this patch does. If project-list is set before scan-path is set, then scan-path only scans the git folders at project-list/${a line in the project-list file}. Problem solved, and this is a pretty generic way of doing it too.

remove-suffix

Most people store git repositories on disk at MyGitRepository.git. Notice the .git ending. However, most people prefer to see it listed as just “MyGitRepository” and they especially would like to clone it at gituser@domain.com:MyGitRepository, without needing the .git ending. Usually, CGit’s scan-path infers the repository name directly from the folder name. This patch adds a setting called “remove-suffix” that, if set to 1 (default is 0) before scan-path is set, will remove the .git suffix from the repository name and url while still pointing to the correct physical path. This as well is fairly generic and not specific to Gitolite or Gitweb, but rather Git’s usual conventions.

enable-gitweb-owner

CGit’s scan-path infers the owner of the repository from the posix owner’s UID name. But there is an additional Git standard for overriding this for any interface: the “gitweb.owner” configuration key in .git/config, which Gitolite understands and respects, as well as Gitweb. This patch simply calls Git’s internal C functions for fetching this information from the current repository’s config, and prefers this as the owner to the posix owner’s UID name. If gitweb.owner is not set in the configuration, it falls back to the posix owner’s UID name. This is a standard Git behavior. This occurs only for scan-path — cgitrc specified owners are preferred over these former two, obviously. Again, this configuration standard has been determined by the Git project, and both Gitolite and Gitweb respect it. So, this patch adds support inside CGit for it.

it works

Now instead of the include and the ugly set of scripts and hooks, I can just place this at the bottom of my cgitrc:

enable-gitweb-owner=1
remove-suffix=1
project-list=/home/gitcode/projects.list
scan-path=/home/gitcode/repositories 

and this integrates perfectly with Gitolite. All is harmonious in the Git universe.

On top of all this, I’ve cooked up a wicked good .htaccess file for CGit that allows me to have anonymous http pull at the same time as it rewrites the CGit urls to be pretty. Check it out:

Options FollowSymlinks ExecCGI
 
DirectoryIndex cgit.cgi
Allow from all
Order allow,deny
 
RewriteEngine on
 
SetEnv GIT_PROJECT_ROOT=/home/gitcode/repositories
 
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d 
RewriteRule "^(.*)/(.*)/(HEAD|info/refs|objects/(info/[^/]+|[0-9a-f]{2}/[0-9a-f]{38}|pack/pack-[0-9a-f]{40}\.(pack|idx))|git-(upload|receive)-pack)$" /git-http-backend.cgi/$1.git/$2 [NS,L,QSA]
 
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^.* /cgit.cgi/$0 [L,PT,NS]

A strange combination of stopping internal redirects and partial rewritings and odd stop conditions has made it so that the request gets forwarded and reformatted to git-http-backend if and only if it is first valid with cgit.cgi. Is this crackable? Can anyone figure out a backdoor to grab a repository that isn’t in projects.list?

I’ve also written a super generic script for uploading new repositories to my gitolite/cgit installation. From a git working directory, I run ~/Projects/uploadNewGit.sh "This is a description of my new git repo.", and wham-shabam, all the permissions get set and everything is uploaded just fine. Here is uploadNewGit, the latest version of which you can always find in my GitTools repository:

#!/bin/sh
 
GITOLITE_ADMIN="$HOME/Projects/gitolite-admin"
 
gitdir=$(readlink -f "$(pwd)")
name=`basename "$gitdir" | cut -d / -f 2 | cut -d ' ' -f 1`
description="$1"
 
if [ ! -d "$gitdir/.git" ]; then
        echo Not a git repo.
        exit 1
fi
if [ -z "$description" ]; then
        echo You need to specify a description argument.
        exit 1
fi
 
pushd "$GITOLITE_ADMIN/conf" > /dev/null
echo "Writing config.."
(echo
echo "  repo    $name"
echo "          RW+CD   =   $(whoami)"
echo "          R       =   @all"
echo "          $name \"$(git config --get user.name)\" = \"$description\"") >> gitolite.conf
git commit -a -m "Adding $name to repository."
git push
popd > /dev/null
 
url=`git --git-dir=$GITOLITE_ADMIN/.git remote -v | grep push | cut -f 2 | cut -d ' ' -f 1 | sed "s/$(basename $GITOLITE_ADMIN)/$name/"`
git remote rm origin 2> /dev/null
git remote add origin $url
git push -u --all
git push --tags

(As a side note, I’m not really sure the best way to quote commands inside of commands with variables that have spaces. something=$(command $(othercommand $argument)) has issues if argument has a space or if othercommand produces something with a space or if command produces something with a space (not totally certain about the latter two — I should check). But I can’t do this: something=”$(command “$(othercommand “$argument”)”)” because of obvious quoting problems. What’s the common solution to this? I’ve been using an awkward combination of the backtick operator `…` and the $(…) syntax but the backtick has some weird rules too. What’s the deal? Can somebody point me in a good place to read about this?)

Anyway, most of what I’ve written about in this post is new to me. Or at the very least, I’m a bit uneasy. So if you have any suggestions, by all means please tell me. I’m looking forward to seeing what the KDE sysadmins do in the end. Hopefully the CGit authors accept my patches.

Update: After some back and forth with Lars, the CGit maintainer, I’ve added a few more patches, including putting the gitweb.owner functionality behind configuration setting and also caching the scan, among various other improvements. You can check out all the commits I’ve made on this at the cgit for my cgit clone.

Update 2: I’ve gotten rid of my branch because my commits have been merged to cgit!