DICE development projects - roger

Unit:

What:

We have a considerable amount of user web content (homepages is one obvious example) mounted on our web servers via NFS and AMD. This project would provide a more secure alternative using a separate web server with AFS filespace for sensitive data.

Why:

Customer: School

Case statement: NFS mounted web space has many security disadvantages, as well as being only accessible from within the School. AFS, with proper safeguards, would allow both file access and web access in a secure and controlled manner.

When:

Status: proposal

Priority: medium

Time: 1 week

How:

Proposal:
To provide secure, on-request, user web space

Resources:
Sufficient AFS space to contain the content
Knowledge of AFS and apache/waklog

Plan/Issues:

Install basic test web server
Add support for authentication (CoSign & Kerberos) & waklog
Confirm ID naming scheme
Create AFS/Kerberos IDs for web server & users
Generate & store appropriate keytabs
Configure appropriate ACLs for filesystem access via command-line & browser
Generate volumes
Confirm filestructure &naming scheme and configure volume mount-points
Rebuild suEXEC
Configure CGI execution environment

(also see earlier AFS Web Server Plan/Issues Document)

Other:

Dependencies:
This project requires a working version of waklog for our web servers.
This project relates to, and will have an impact on, the "Move user and group web space to AFS" project.

Risks:

AFS performance could be affected - possibly by significant web traffic to/from the servers. Any such degradation could, perhaps, be reduced by separate storage/servers for web areas.
Using AFS makes web data potentially available from any location (via browswer or filesystem), thus increasing security concerns.
Future dependency on waklog. Note that the future of mod_waklog is uncertain - any reliance we place on it may mean that might have to become involved in supporting it - either locally or publically.

Milestones

Proposed date	Achieved date	Name	Description

Software Download Portal/Repository: Stage 1

boss — Fri, 25 Jan 2013 15:46:27 +0000

Project ID:

258

Current stage:

Manager:

Unit:

What:

Description: Requirements gathering for the implementation of a new Software Download Management Platform that would enable the distribution of research software and applications more effectively than the existing Informatics Software Download Database (ISDD).

Deliverables: A report examining the current state of ISDD, and a list of the perceived short-comings (real or imaginary) of ISDD (and whether they could be fixed). A description of the functionality required by a new Software Download Management Platform.

Why:

Customer: Edinburgh Research & Innovation, and all Informatics Research Staff (especially David Richardson, Dave Robertson, Bob Fisher)

Case statement: The UK Research Council’s focus on impact is changing the way we think about our research output. Impact features heavily in the upcoming Research Excellence Framework (REF) and we need to ensure we are capturing as much supporting evidence as possible.

The Informatics Software Download Database (ISDD) has been in existence for many years and is currently home to a number of legacy projects, but is perceived as being rather long in the tooth.

There is a confirmed expectation from the Scottish Government that the University takes active steps to simplify the process for companies to take advantage of freely available intellectual property, in parallel with commercial licensing activities. The proposed portal for open source technology will allow the School to lead the way in this initiative, working with Edinburgh Research and Innovation (ERI) and the collaborative University Technology project.

When:

Status: Final Report available on the DICE wiki.

Timescales: estimated half a week.

Priority:

Time:

How:

Proposal: We need to know what users actually want, and whether the existing ISDD could be reasonably modified to provide that functionality - and if not, what would be required of a new Software Download Management Platform.

Resources:

Plan:

Establish current and prospective user base.
Investigate IS data share, and assess possible use/relevance.
Examine the current state of ISDD, and list any perceived short-comings (real or imaginary) reported by the user base.
Investigate whether any deficiencies (if found) could be be corrected, and viability/feasibility of same.
Assess additional requirements not provided by ISDD
Enumerate functional requirements of a new Software Download Management Platform (to include existing and requested functionality).
Solicit user feedback and refine requirements list.
Prepare final requirements list.

Other:

Dependencies:

Risks:

Milestones

Proposed date	Achieved date	Name	Description

Streaming Media Service Requirements

boss — Fri, 25 Jan 2013 15:46:27 +0000

Project ID:

259

Current stage:

3s_Stalled

Manager:

Unit:

What:

Description: Investigate the needs of users and the requirements necessary to provide an operational Streaming Media service, as a precursor to implementation (Project 227).

Deliverables: A clarification of what is required by users of a Streaming Media service, an assessment of viability, and a list of functional requirements that would need to be implemented.

Why:

Customer:

Case statement: Though there is a central, IS, provided service, which would continue to be used for larger scale events. Something smaller and simpler for use in-house would be useful - and in order to implement this, a functional specification is required.

When:

Status:

Timescales: estimate 1 week.

Priority: Not enormously high - but it does, in theory, block project 227.

Time:

How:

Proposal: We need to know what were committing ourselves to in implementing a Streaming Media service, what's feasible, and how this sits with what users are actually asking for.

Resources:

Plan:

Establish current and prospective user base.
Consult with user base to establish current usage and performance.
Investigate current levels of service provided in-house and by IS.
Establish required functionality, based on user feedback. Clarify and refine as appropriate.
Draft initial requirements list and solicit user feedback.
Prepare final requirements list.

Other:

Dependencies:

Risks:

Milestones

Proposed date	Achieved date	Name	Description

Replace CVS with SVN on www.inf

boss — Fri, 25 Jan 2013 15:46:20 +0000

Project ID:

226

Current stage:

Manager:

Unit:

What:

Description:
The bulk of the content on www.inf is managed via CVS. This is rather old and not as feature rich as Subversion (SVN). In most cases a person familiar with 'cvs' commands from the command prompt can just replace them with 'svn'. If we do consider changing to SVN, we need to make sure the various existing CVS hooks (eg validation), authorisation and publishing via the web continue to work.

Deliverables:
The same service and functionality that we have at the moment with CVS but using SVN instead.

Improved authentication via SVN and Cosign, to replace the weak security of CVS pserver.

Preserved CVS history and revisions.

Why:

Customer: web publishers to www.inf, and some of these are automated database driven commits.

Case statement:
Most teaching material is published using CVS, a version control system that is getting a bit long in the tooth, and this should be ported to SVN. CVS can take considerable time (for instance, SVN is much quicker when changing directory structure).

Improved authentication is something we should have looked at ages ago.

When:

Status: Proposal

Timescales: estimate 3 weeks

Priority: Depends on who you ask - there's no urgency from the service point of view, unless doing away with pserver is seen as urgent, though an alternative would be to switch to cvs + ssh

A possible additional feature to do as part of this work would be to flag content that staff want to store in the repository, but not publish on the web
(see RT 60294).

Time:

How:

Proposal:

Resources:

A test service.
Computing staff time.

Plan:
As well as replacing CVS with SVN we need to inform users of the service how to switch, so resources are needed to provide (possibly) migration tools, and certainly we'd need to track down and replace all documentation detailing how to use CVS with SVN.

All users would have to do a final commit of any outstanding CVS edits, then the CVS service removed and replaced with SVN, before web publishing could resume. Users would have to checkout a new SVN working copy of their web pages.

The migration would need to preserve the CVS history and revisions so that historical checkouts and comparisons could still be done.

There are tools to assist with migration, such as cvs2svn, and other web resources.

Basic plan might be something like:

Install cvs2svn
Stop CVS uypdates ("freeze" repository)
Copy the CVS repository from the CVS server to SVN server
Use cvs2svn to create dumpfile of CVS repository
Create svn repository
Load the dumpfile into new SVN repository
Verify (test) the svn repository created
Check access svn from a remote machine
Test client
Check/add pre-commit hooks and post-commit hooks
Point users at SVN repository

Note that cvs2svn assumes a one-time use to convert a CVS repository to SVN, although various transformations can be applied during this process. It isn't suited to separate, partial migrations of a CVS repository to SVN. It doesn't create a persistent record of which CVS version relates to which SVN revision number, so if only parts of CVS are converted at a time, branching information will be lost. CVS modules can, however, be converted one at a time.

Other:

Dependencies:

subversion and component
cosign

Risks: Inability to publish to the www.inf while the migration was happening.

We miss some automated process or publishing mechanism that is no longer able to publish as it hasn't been switched to using SVN

Confusion caused by the change in technology.

What if, sometime after the migration, we discover some problem/flaw. Can we revert back to CVS again preserving the revisions/history that may have been made via SVN in the mean time?

Milestones

Proposed date	Achieved date	Name	Description

Web search front end to package repository

boss — Fri, 25 Jan 2013 15:46:18 +0000

Project ID:

214

Current stage:

Manager:

Unit:

https://wiki.inf.ed.ac.uk/DICE/FinalProjectReport214

What:

Description: Provide a web interface for users to search for packages. For example, produce a list of available text editors for a given platform.

Deliverables: A user-facing web page and back-end scripts, implementing a specified search mechanism.

The final version is now available.

Why:

Customer: Primarily users.

Case statement:

When:

Status: Currently awaiting sign-off. Has been advertised and made available to users.

Timescales:

Priority:

Time:

How:

Proposal:
It's not always easy for users to find out what packages we have. Searching our repositories could reveal this, but they are not in a user-accessible form. A tool that provided simple search facilities would be useful, providing a starting-point for software investigation or package installation requests.

Resources: 3 weeks. May be able to make use of existing systems.

Plan:

Identify repositories to be used, and constrain search methods.
Investigate implementation possibilities (evaluate repository querying tools).
Produce web page and scripts to demonstrate simple search.
Advertise for evaluation and feedback.
Refine web page and scripts, assess viability of any requested features.

Final report URL:

Other:

Dependencies:

Risks:
The tools should provide no risk to system operation (no attempt at automatic installation is intended), but some inconvenient user requests may be generated.

Use outside Informatics should require access control (as information about installed software versions could be gleaned from searches).

Milestones

Proposed date	Achieved date	Name	Description

SL6 RAT packages

boss — Fri, 25 Jan 2013 15:46:17 +0000

Project ID:

211

Current stage:

5_Completed

Manager:

https://wiki.inf.ed.ac.uk/DICE/FinalProjectReport211

Unit:

rat-unit

What:

Description: This project should port the current F13 rat package lists to SL6 whilst trying to identify old and unused packages and updating versions where appropriate.

Deliverables: a full set of software for supporting teaching and research within informatics.

Why:

Customer: The school

Case statement:

When:

Status:

Timescales:

Priority:

Time: A significant amount of effort was expended in gathering requirements, porting, and getting feedback. Hopefully this will be lessened if/when a more formal requirements-gathering framework is in place.

How:

Proposal:

Resources:

Plan: A significant amount of consultation with Course Organisers and Teaching Assistants will be required to determine what software is required and what is redundant. Consultation will be organised in two phases, for Semester 1 and Semester 2. Each phase will involve establishing current and required versions, checking for local (or other) SRPM, and using pkgforge to build each package.

Once built, packages will be added to packages list (lcfg/core/packages/dice/dice_sl6_rat.rpms) and tested - first on testsl6.inf (locatelli), and then rolled out to lab and desktop machines.

Final report URL:

Other:

Dependencies:

Risks: We now have a more accurate list of teaching software requirements (having weeded-out the dead wood), and the equivalent process next year should be quicker. However, if the Software Requirements framework isn't implemented, we will continue to have late requests which cause additional work.

Milestones

Proposed date	Achieved date	Name	Description

Teaching/Research Software Requirements Framework

boss — Fri, 25 Jan 2013 15:46:15 +0000

Project ID:

199

Current stage:

5_Completed

Manager:

https://wiki.inf.ed.ac.uk/DICE/FinalProjectReport199

Unit:

rat-unit

What:

Description: The project will develop a framework for obtaining, publishing and resourcing teaching and research software in a timely manner.

Deliverables: An agreed framework with the ISS/director of teaching about when teaching software requirements can be changed.

Why:

Customer: staff (with a little bit of COs)

Case statement: We do not currently have a definitive set of software requirements for each of the teaching courses and for that requested by researchers and no guidance to staff as to how much advanced notice is required to install requested software. This leads to a messy scramble at the start of some courses where staff are asking for software at the last minute, software is unavailable because of licensing issues, upgrades break coursework and software is removed for various reasons ranging from obsolescence to security issues. We are also probably expending resources maintaining software that is no longer used by anyone.

When:

Status:

Timescales: Ideally we should have something in place in the run up to next year and the SL6 deployment.

Priority:

Time:

How:

Proposal: Develop a framework that drives timely changes to software installed to meet teaching and research requirements.

Resources:

Plan: Produce initial proposal (discussion document) for timescales and
policies related to provision of software for teaching courses.

Consult with ISS/Director of teaching and determine suitable
milestones in a courses development where software requirements
should be identified, and also determine a suitable timetable for
allowing staff to make annual changes to required software.

Final report URL:

Other:

Dependencies:

Risks: none

Milestones

Proposed date	Achieved date	Name	Description

Review roles mechanism

boss — Fri, 25 Jan 2013 15:46:14 +0000

Project ID:

197

Current stage:

Manager:

Unit:

https://wiki.inf.ed.ac.uk/DICE/FinalProjectReport197

What:

Description: A more structured approach to the creation and allocation of roles is needed. The current number of roles has become unmanageable and therefore needs to be rationalised.

The original discussion document is here.

A list of possible role deletions is here.

Deliverables: A more simplified and structured approach to role/entitlement creation and allocation.

Why:

Customer: All DICE account users.

Case statement: The existing roles and capabilities namespace and management procedures lack a cohesive structure and policy. Some roles are obsolete, and some are unused. The namespace would benefit from a clearout, and revised management practices would prevent further confusion.

When:

Status:

Timescales:

Priority: Low/Medium - whilst desirable, there is nothing being blocked by this project.

Time:

How:

Proposal: There are a number of factors to bear in mind, these points are taken from various discussions:

It needs to be clear who makes the actual decisions concerning roles and entitlements
Units should be in charge of the entitlements for their servers. And we should discourage people from piggybacking on existing entitlements.
Issues arise where the ownership of entitlements is unclear or across management boundaries, e.g. who decides what goes in a generic role like people
It needs "strong ownership". The roles system joins a lot of different technical systems together, as well as being our main expression of policy. The lack of single decision maker for that system means that it ends up being pulled in many different directions

Resources:

Plan:

Investigate current namespace for roles and capabilities.
Identify any moribund instances.
Analyse current groupings and evaluate relevance.
Identify areas of responsibility.
Review and revise groupings and ownership.

Final report URL:

Other:

Dependencies: There is some involvement with the Prometheus life-cycle code, which may be used, or may be affected, by any proposed changes in roles management procedures.

Risks:

Milestones

Proposed date	Achieved date	Name	Description
	2012-07-18		Decide on nature of roles review, and extent of actions required.
	2012-07-04		Produce recommendations report, based on consensual feedback

Migration of Mailing Lists to Central Service

boss — Fri, 25 Jan 2013 15:46:14 +0000

Project ID:

193

Current stage:

4d_Dropped

Manager:

Unit:

https://wiki.inf.ed.ac.uk/DICE/FinalProjectReport193

What:

Description: This project will evaluate whether the new mailing list service provided by IS (http://www.ed.ac.uk/schools-departments/information-services/about/new/m...) is suitable for moving the School's mailing lists across to and if the evaluation is positive will carry out the move.

Deliverables: The transfer of the School's mailing lists to the central server

Why:

Customer: School.

Case statement: We have a requirement to move School services to Centrally provided equivalents where appropriate. This project will determine if the central mailing list server falls into this category and if it does will perform the move.

When:

Status: Shelved. Not currently required.

Timescales: Evaluating the central service should take less than a week. The time taken to migrate the lists will depend on any issues encountered. Three weeks seems like a reasonable estimate for this.

Priority: University policy requires us to do this if possible. Apart from that, there is no urgent need to move the data.

Time:

How:

Proposal:

Resources:

Plan:

Final report URL:

Other:

Dependencies: Knowledge of the School's Mailman service and the new central service based on Sympa

Risks:

Milestones

Proposed date	Achieved date	Name	Description

Introduction/Integration of new event management system

boss — Fri, 25 Jan 2013 15:46:14 +0000

Project ID:

195

Current stage:

4d_Dropped

Manager:

Unit: