You are here

Evaluate move of LCFG configuration from svn to git

Project ID: 
Current stage: 
We want to know if it is possible and desirable to move the LCFG configuration data (headers) from subversion to git

Evaluate the possibility of switching from subversion to git for the management of the LCFG configuration data. Test a prototype based on a snapshot of the current data. Ensure that we can continue to have an efficient, low-overhead weekly testing/stable release cycle.


Current Situation

The LCFG configuration files (headers and package lists) are currently stored in a subversion revision-control repository which is run by the School of Informatics. This repository can be split down
into 3 separate strands:

Public data which will create sensible default configuration
policies for the general case
Public data which configures computers to suit the University of
Edinburgh computing environment
Private data which configures computers to suit the School of
Informatics computing environment

The separation of public and private data, which is all stored in the same single repository, is achieved using subversion ACLs. These ACLs ensure that only computing staff from the School of Informatics can read or modify files related to the dice environment. Further to these access restrictions, although the public data is readable by any user it is only modifiable by Informatics computing staff.

Each week a snapshot of the development "trunk" is taken and tagged as a new release. That release is first deployed to a number of testing machines. Once a release has been tested and any problems resolved it is tagged again as the stable release. The full stable release is rolled out to DICE machines, the public portion (the lcfg and ed sections) is bundled up and pushed to the LCFG public repository (http and rsync). Downstream users, such as the MDP team in IS, then test and apply the changes to their own systems.

Currently the only way in which external users can contribute new features or improve existing configurations is to create a patch file containing the required changes and then submit a bug report in the LCFG bug tracker. This is a rather unwieldy process which results in external contributors being treated as second-class citizens. As well as creating a situation where contributing to the LCFG project is more difficult than it should be it also creates more overhead for Informatics staff. Each patch has to be reviewed and then manually applied to the repository, there is no automation to simplify the process or assist in reducing the risk of patches being badly applied. Although not impossible to do with the subversion revision control system this is not a trivial task.

Broadening the Community

There has never been an explicit policy of excluding external users. Rather the current situation has occurred naturally because the project was created and managed by Informatics computing staff. There simply was not originally any specific requirement for extending write access beyond that team. In recent years the LCFG project has become much more widely adopted across the University of Edinburgh. Each succesful deployment has increased the number of people who know how to use and create LCFG components. If the project is to continue to grow and thrive then contributions must be welcomed and actively encouraged. To encourage more external contributions we should aim to make it possible for anyone to make contributions. Clearly we cannot provide direct write access into the main repository, a good strategy, which is used by many Open Source projects, would be to allow public contributions into a review system. Gatekeepers can then review submissions, respond to the contributors and commit or reject the changes in an efficient, transparent process.

We have previously considered restructuring the LCFG headers repository so that we can allow external contributors direct write access to the headers for components which they manage. The restructuring is probably a good idea to help improve comprehensibility and discoverability, however it was decided that allowing access like this is too much of a security risk for Informatics. In Informatics we have approximately 100 machines which follow the develop release in real time, any mistake to a widely used header could result in us having badly broken machines. This risk is considered acceptable when there is a local CO involved who can check the likely impact of any change and be relied upon to notify others in advance of any risky changes. They are also more easily contacted if a problem occurs. This problem pretty much rules out subversion as a choice for our revision control system if we wish to involve external contributors. A better alternative would be to use a revision control system with a distributed model (e.g. git). This would allow contributors to fork repositories, test their own changes locally and push back changes when ready. We can also then build a review system around that approach which should help raise the quality of all changes.

Good Practices

Beyond just lowering the barriers for external contributors a review-based system has a number of additional benefits. By using a review system for all changes to the configuration files we get "more eyes on the code". It would actively increase the number of people who have an understanding of the purpose of an LCFG component. There would also be a wider awareness of upcoming changes. It can also be a useful tool for helping to tutors new contributors on topics of good-practice such as expected style, syntax, config layout, etc, when it is not possible to work side-by-side in "real life". A review system also encourages users to use good, descriptive changelogs, there's no way to lazily submit changes with an overly terse, unhelpful or just plain wrong changelog entry.

  1. Import a snapshot of the current LCFG configuration data (the headers) into a git repository.
  2. Configure the gerrit review system for the git repository
  3. Get local COs to play with the system and give feedback
  4. Get external contributors to play with the system and give feedback
  5. Investigate how the dice headers could be split from the public lcfg and ed levels.
  6. Investigate how we would do the weekly testing/stable releases
Effort estimate: 
Two weeks