Desktop (deb) packages migration to GIT

As desktop team we manage a certain number of packages with various distro-patches, as discussed on the past cycle there is a good consensus in switching to the usage gbp pq (which is an-impossible-name-to-spell-and-remember which stands for git build-package patch queue) for managing patches in a way better way as debian salsa already does.

We had some quick discussion on trello, and while I’d personally prefer to use something that is based on gitlab for better git management, we can safely continue using the desktop-team launchpad git repo for this.

The workflow is quite simple, and described in the pq docs (while manual is here), and probably the hardest thing is to remember the actual command name :joy:.

# Example workflow
git clone lp:~ubuntu-desktop/my-package
cd my-package
# Generate and switch to a patch-queue/ubuntu branch where all the patches
# just git commits
gbp pq import
# Now work with actual code, use `git rebase` for changing order, or edit a patch,
# so do everything you like that git allows you to do.

# Now export this back to patches, and switch back to the ubuntu branch
gbp pq export --no-patch-numbers
debchange "Love this change!"
git add debian/patches/*
git commit -a -m "Fantastic work with this package, now it's really cool"

# Push the git branch to LP so that you can create the merge-proposal
git push lp:~username/my-package

Anyways, we need some organization in order to proceed. Personally I can help with the migration, while I guess we can automatize this with a script that basically does:

  • Import clone the bzr repo
  • Convert it to git
  • Pull upstream git code
  • Includes the ex-bzr branch to an ubuntu branch debian subfolder inside (keeping the history)
    • Check that pq import works correctly
    • Do the same for an ubuntu/bionic branch based on same upstream branch (or should we use tarballs?)
    • If patches don’t apply to the latest code just add something mentioning this I guess.
  • Set the ubuntu branch as default (or we want to use master and then revision-codename for others?)
  • Push it to LP (and maybe remove bzr branches too)

I can craft something like this if wanted, while I’ve not the permissions to then push to ~ubuntu-desktop (unless any three generous team mates won’t agree that I can join it :stuck_out_tongue:), but well that’s just non important as I can push under different umbrella and someone else will then push it there.

When? Let’s decide it, personally I think sooner is better than later as, once we get used to the system it really improves the speed of development.

We want to proceed in another way? Like each time that you work on something new you just use the script to migrate? Not sure what’s preferred, but personally I’d do it in a batch.

2 Likes

Hi Marco, thanks for starting this discussion

We’d talked about this a bit on IRC and IRL before. I think the consensus was that our git history should be based on Debian’s git repositories which have all already been converted. We also talked about whether to try to work to interleave our bzr history with their git (from SVN) history and decided that it was probably not worth the effort, although it is a shame to lose that history. Like you say we could keep the old bzr branches around for future archaeologists. I think I prefer this to having an ‘old’ branch in git that everybody ends up pulling all of the time.

So the procedure is something like

gbp clone https://salsa.debian.org/gnome-team/pkg.git
git remote add upstream git://gitlab.gnome.org/pkg.git
git remote add lp <URL under ~ubuntu-desktop>
git fetch upstream
git checkout -b ubuntu/master <version latest ubuntu version is based on>
vim debian/gbp.conf # change debian-branch to ubuntu/master and add debian-tag = ubuntu/%(version)s
vim debian/control.in # update Vcs-* to point to Launchpad
fakeroot debian/rules clean
git commit debian/gbp.conf debian/control*
gbp import-dsc ../path/to/ubuntu/upload.dsc # if this is a new release add --upstream-vcs-tag=<the release tag>
git push lp ubuntu/master upstream/whatever-tag upstream/whatever-branch pristine-tar

This looks cumbersome but a lot of it is first time setup stuff. For the stuff that isn’t, please see this bug, in particular the X team’s script which we might be able to make use of to make gbp clone set up all the necessary remotes.

Totally, if you’re up for scripting all the repos that would be cool. You’d need to know what the upstream URL is for each repository which might be hard to automate, although @jbicha recently did the git conversion in Debian including importing upstream history so maybe he has something that could help here.

I was going to poke people soon™ - a little while ago I started a wiki page where I was going to put some common recipes. I thought that’d be a bit more enduring than a thread here. Maybe some people could help to fill that out?

Maybe a lot of that could be done in https://wiki.debian.org/Gnome/Git and we’d only need to document the Ubuntu-specific parts.

I think that’s a useful enough feature it would make sense for someone to propose to gbp that a field in gbp.conf be added for that. Otherwise, we could add a debian/upstream/metadata file for every package and use its Repository field.

Most of the packages are accessible via https://git.gnome.org/browse/PACKAGENAME (which redirects to GitLab where appropriate, and the GitLab migration might finish by the end of the year) but there are several exceptions. So a shortcut hack would be to only add metadata in debian/ for the packages that don’t follow that convention.

I want to very quickly document common tasks in one place, so you can easily get started if you don’t know gbp yet. We should probably link to that page though.

The bug I linked is for that, although I think they’re leaning towards something in control, which is also fine. For our downstream case we would also like to be able to add a “debian” remote too, which I don’t think has been considered yet. I just sent a reply asking about that. (Although, given that we don’t change source package names, this could be hardcoded in a postclone script even without this as a feature.)

@3v1n0 @jbicha

How do I create a new merge requests for ubuntu-settings. The bzr branch is gone now. And git clone lp:~ubuntu-desktop/ubuntu-settings doesn’t work. :expressionless:

https://code.launchpad.net/~ubuntu-desktop/+junk/ubuntu-settings

But I think Launchpad won’t let you propose a merge for “junk” branches. Open a bug, attach your branch to the bug, and subscribe ubuntu-sponsors.

1 Like

I’m migrating packages when I touch them now. At least gdm3, gnome-control-center, gnome-terminal are done. Please feel free to do the same. If you don’t know how, contact me and I’ll help you out.

Vcs-Git fields should be correct, so you know where to look.

1 Like

Just an update on that… At this time all the repos we’ve ubuntu forks for have been migrated to git.

I’ve been using these scripts I wrote to automatize the whole process, based on the very detailed documentation that @didrocks wrote (thanks!) for this task (check also the main git page for more infos).

So @laney will probably use something like this to update the bzr branches so that they will notify the new location.

1 Like

@3v1n0

Let’s say I want to open a pr against file-roller. How should I proceed? In case of bzr it was like this bzr push lp:~user/source-package/my-fix

or

bzr push lp:~ubuntu-desktop/source-package/my-fix (for debian/patches)

How will this work in git? Should we use lp:~ubuntu-desktop/file-roller/+git/my-fix (here lp is actually git+ssh://, configured in .gitconfig) ?

Will unity branches also move to git?

How will this work in git? Should we use lp:~ubuntu-desktop/file-roller/+git/my-fix (here lp is actually git+ssh://, configured in .gitconfig) ?

Once you’ve configured your lp: alias on your .gitconfig, just do your edits, in any branch you wish, and then do:

git push lp:~<user>/ubuntu/+source/<source-package-name>

Then you’ll see the branch in OpenID transaction in progress and from there you can open a merge request against the upstream branch (ubuntu/master on lp:~ubuntu-desktop/ubuntu/+source/<source-package-name>).

If that source package doesn’t exist (yet), you can also just push to lp:~<user>/+git/<any-repo-name>. That would be like the old bzr +junk repos were.

Will unity branches also move to git?

Both bamf and nux have been moved, I will move unity itself soon too.

1 Like

We think we’ve more or less finished the migration now. In the background as I write this I’m running a script which is pushing commits to all of our converted bzr repositories to point users to the new location.

:slight_smile:

2 Likes