DICE development projects - 1_Proposal

Research Data Audit

cms — Fri, 23 Aug 2013 15:31:57 +0000

Project ID:

290

Current stage:

1_Proposal

Assigned Priority:

Manager:

cms

Unit:

services-unit

Summary:

An audit of all research data within the School

What:

This project will attempt to all research data held on School storage and create appropriate metadata (owner, status etc) for each item of data identified. The project will also encompass decisions regarding how the audit should be conducted, how the metadata should be stored and how best to ensure that the audit data remains current after the end of the project

Why:

The requirements of many funding bodies regarding the creation of, access to and eventual deletion of data are becoming ever stricter. This audit will be a useful first step towards ensuring that the requirements of these funders are met. It will also be extremely useful for management of the School's storage.

How:

To be decided as part of the project. Much of the work of actually collecting the data can be CSOs or even by research staff

Effort estimate:

one week for decision making.. up to four weeks CSO/other time for actual collection of data

Other:

Dependencies:An audit of group space usage was conducted by Lindsey Brown in 2011. This project will build on that work

Risks: Not doing this risks failing to comply with the requirements of funding bodies with detrimental affects on the School's reputation and income

Projects Database replacement

boss — Fri, 25 Jan 2013 15:46:16 +0000

Project ID:

204

Current stage:

1_Proposal

Manager:

gdutton

Unit:

rat-unit

What:

Description: Replace the existing UG4/MSc project tracking database with a more integrated and maintainable service.

Deliverables:

Data to be mastered in (or with significant reference to) the School database.
Functionally identical replacement of the existing interfaces.
System to be free of existing data integrity issues.
Modifications to ensure all administration can be performed by project coordinators.
Single structure allowing data to be partitioned and functional behaviour to differ by both cohort and academic year as teaching requirements change.

Why:

Customer: School of Informatics. The service will be used by teaching staff, teaching support staff and students.

Case statement:

The projects database was based on code provided by an undergraduate student some time around 2004. It was likely designed as a quick stop-gap, and required complete reconstruction each year, duplicating not only all working data but also its codebase. The system was, as a result, profoundly unreliable, unmaintainable and unsafe.

The system was taken on by computing support in 2006 in order to achieve several urgent goals of reducing the linear maintenance expansion and adding some form of secure authentication and authorisation, as well as a degree of stability. Maintenance has been ongoing since and, though hugely improved, even minor functional changes (as required by teaching policy) often have knock-on effects which risk availability and integrity of data, so maintenance continues to be expensive.

Additionally, much of the data stored in the projects database is unnecessary as it is (often inaccurate) duplication of data mastered in the School database. Administrative load could be reduced by integrating this data store.

When:

Status:

The project deliverable will replace a working (albeit fragile and expensive) service.
Other portions of project administration are already becoming integrated with Theon, reducing data duplication elsewhere.
Maintenance of the system typically required every year, and is comparatively expensive as it almost always entails multiple changes throughout the system.

Timescales: Not entirely known. Basic functionality is very simple indeed, but the administration interface is surprisingly powerful and adapting the system to be an integral part of the school's wider data model is complex.

The transition should take into accoun

Priority: TBC. Points to consider in 'status' above.

Time:

Estimates:

1-2 weeks evaluation, (new) requirements capture, basic prototyping

3-4 weeks production code development and testing.

1-2 weeks QA, testing, extended requirements (if time). Extensive user documentation.

(FTE necessarily

How:

Proposal:

Evaluate possible technical solutions and, in parallel, discuss existing requirements with those concerned, eliciting official procedures which are not always clear in existing system. (Note that additional requirements will almost certainly be generated at this stage; these might influence development but should be addressed separately.)

Implement integrated data model within Theon

(re)implement interface according to evaluation.

Resources:

CO time
An apache web server (host need not be dedicated)
Access to school database
[desirable] Working knowledge of school database
[desirable] Knowledge of typical user and data interactions of the system

Plan:

Implementation plan to be developed as part of project: see proposal.

Rework data model to integrate with Theon and reduce re-keying. Develop scripts to safely transfer project data into new model.

Develop new framework, then replicate functionality of each existing form or report into new framework.

Other:

Dependencies:

The new project will depend upon, and interact (to a presently undefined degree) with, the "Theon" school database service.

Some "Webmark" forms depend upon the data provided by this service.

Various internal web pages and policy documents refer to project site URLs
representing views of the projects data. Consideration should be given to maintaining current URLs and notice given of any deprecation.

Risks:

Risk of overrun due to personal underestimate or conflicting demands - not disastrous, as slippage simply mandates extended use of existing system.

Risk of dangerous interactions with Theon - to be specifically guarded against.

URL: https://www.dice.inf.ed.ac.uk/units/rat/documentation/unit/projectsdb.html

Milestones

Proposed date	Achieved date	Name	Description

Webmark Reform

boss — Fri, 25 Jan 2013 15:46:16 +0000

Project ID:

205

Current stage:

1_Proposal

Manager:

gdutton

Unit:

rat-unit

What:

Description: Enhance the existing 'Webmark' service, partially in response to recent requests, to produce a more general-purpose form generation and submission system, to be known as 'Reform'.

Deliverables:

A refreshed version of the existing Webmark service which functions similarly to the existing one.
Additional core functionality such as field validation, multiple data sources, chained inputs and outputs.
Generic input, output, form source APIs.
Allow external sources to define form contents.
Configuration format reworked to act as complete form descriptor (where appropriate)
Draft form storage
Session change handling.
All forms to be converted to new configuration format, or full support for legacy configuration
Project choice agreement form, if still required
Partial delegation of form structure to authors

Why:

Customer:

School of Informatics; Computing Support, Undergraduate and Graduate offices.

Features typically requested by members of the teaching offices, with input from academic staff.

Case statement:

New forms and features of the additional forms are now being requested by members of academic and teaching staff, and the existing Webmark framework can no longer cope easily with these changes.

For the past four years, most new forms have demanded some new functionality of the system, and while most could be integrated neatly, others were bolted on hastily and have demonstrated a limit to the flexibility of the core of the system. Continued augmentation is now difficult to do in a safe and clear way, and in particular integration with established School Database systems, which would save considerable effort for administrative and academic staff in teaching duties, is difficult. Requests continue, in particular a new requirement for a "project choice agreement" form which requires more complex interaction with school data, and demand for additional features continues to be expressed by ISS and teaching staff.

Furthermore, form configuration is straightforward at present, but enforces limited functionality. Without redevelopment, existing form commitments cannot be fulfilled in their entirety and future additions and modifications will be comparatively expensive and error-prone.

When:

Status:

Webmark was originally developed in 2007 as a means of generating the multiple paper marking forms required by both College and School for final-year undergraduate and MSc students. The initial design specifically limited the scope of the project and the first revisions met the requirements well.

The existing Webmark service continues to function reliably and allows the straightforward addition of new forms where the demands of those forms do not differ from existing ones, however new features become increasingly difficult to add to the existing structure.

Requests both for additional forms and extended features of existing forms have been pending for some time.

Timescales:

New form configuration is always ongoing and will likely be performed in parallel to development of the new service. The sooner the service is redeveloped, the more customer time will be saved.

Focus on this work will be affected by high-prior

Priority:

Time: Estimates:

~1 week to plan architecture for core
~2 weeks to fully rework core including
~1 week to reimplement dropped core features in new API
~1 weeks to refactor existing I/O and to add new modules
~2 weeks documentation, testing, support training

How:

Proposal: To follow the plan according to the description to provide the deliverables within the specified time.

Resources:

CO time
One Apache-capable DICE server (already exists)
Access to the school database

Plan:

Produce architectural overview, recommending development options.
Reconfirm requirements and pending augmentations of existing forms.
Work out wider picture wrt database connectivity.
Rework / rewrite core to allow full extensibility. Augment core with new functionality as required. Remove functionality where it can be provided in a more generic manner.
Rework inputs / outputs to use new core API
Document APIs and configuration format.
Evaluate form drafting functionality, potentially as generic input/output.
Add new input/output modules. Ensure these are self-documenting at a level usable by form authors
Adapt existing forms to new format (possibly delegated)
QA against existing form output

Other:

Dependencies:

Forms depend upon the "Theon" School Database , the Projects Database (though see project sortable list"; also utilises the ISS Request Tracker system.

At completion, expected to continue to depend on both the School Database and the ISS RT. Non-submission data (for example, for form coordination) might reasonably be expected to rely on a dedicated database.

Risks:

Usual risk of slippage.
Semi-public service: security of input to receive extra attention.
Insufficiently flexible redesign will be no more adaptable to customer requests.
Overly flexible redesign will be too complex for authors / support to administer.

URL: https://www.dice.inf.ed.ac.uk/units/rat/documentation/unit/webmark/

Milestones

Proposed date	Achieved date	Name	Description

KDC Rekey

boss — Fri, 25 Jan 2013 15:46:11 +0000

Project ID:

180

Current stage:

1_Proposal

Manager:

toby

Unit:

inf-unit

What:

Description: Migrate our KDCs away from single-DES keys.

Deliverables: Re-keyed KDC.

Why:

Customer: All

Case statement: For security reasons, we need to move away from using single-DES keys
in our KDCs

When:

Status:

Timescales:

Priority: High

Time:

How:

Proposal:

Resources: 1-2 weeks CO time.

Plan:

Evaluation:

Investigate encryption types and decide which to use.
Look for other organisations who have undertaken a rekey and seek to
learn from their experiences.

Implementation:

Remove both default_tkt_enctypes and default_tgs_enctypes from
krb5.conf everywhere. DONE
On KDC only, add new encryption algorithms to both supported_enctypes
and kdc_supported_enctypes. There is a complete list of supported
encryption types in the Admin manual MIT supply with each Kerberos
release.
Take des out of supported_enctypes on the KDC. Don't remove des from
kdc_supported_enctypes - kerberos component should be checked so that
it doesn't use same resource for supported and kdc_supported
attributes.
Rekey the TGT, using the --keepold option, to add the new encryption
types. Services will be rekeyed as they upgrade, users as they change
their passwords.

Flag day...

TGT - remove the old TGT from the database. This is scary, as you
either have to hand edit the database, or you have to rekey and not
--keepold, which breaks all of the clients.

Most vital is that throughout this all, it must be possible for DES
keys to still be issued, as the afs service key must remain a single
DES key. This means making sure that whilst the KDC defaults to
created DES keys, it is still prepared to issue and handle them.

Other:

Dependencies:

Risks: Kerberos underpins much of our infrastructure, so major changes present significant risk if problems arise.

Milestones

Proposed date	Achieved date	Name	Description

Prometheus Enhancements

boss — Fri, 25 Jan 2013 15:46:11 +0000

Project ID:

179

Current stage:

1_Proposal

Manager:

toby

Unit:

inf-unit

What:

Description: This project gathers together prometheus development work which didn't
happen as part of the main prometheus project.

Deliverables: Enhanced prometheus service.

Why:

Customer: All

Case statement: This page outlines the current state of Prometheus:

When:

Status: Pending

Timescales:

Priority: High

Time:

How:

Proposal:

Resources: 2-4 weeks, depending on progress of main project

Plan: Detailed plan pending, but will be based on work identified here.

Other:

Dependencies:

Risks:

Milestones

Proposed date	Achieved date	Name	Description

Inventory Improvement

boss — Fri, 25 Jan 2013 15:46:03 +0000

Project ID:

146

Current stage:

1_Proposal

Manager:

ascobie

Unit:

us-unit

What:

Description: Ultimately provide an automated inventory solution that will monitor and track items accurately. In the first instance, we should evaluate the current system and identify areas where we need improvements e.g. tracking self-managed machines.

Deliverables:

Evaluation phase

A report on the existing system - covering both good and bad aspects.

Implementation phase

A design for a replacement system.
A replacement system.

Why:

Customer:

Case statement: The current inventory system does not meet all our requirements and there is concern that tracking machines has now become unreliable. This is particularly the case for self-managed machines. We are not able to easily answer fundamental inventory questions - this is a particular problem for planning expenditure and machine allocation.

The current system is overly complicated with the flow of information difficult to fathom.

When:

Status:

Timescales: Difficult to judge - depends on Alastair availability. Aiming to clarify and document requirements by mid T2 2012.

Priority:

Time:

How:

Proposal:

Resources:

Plan:

Gain an understanding of existing inventory system
Meet all interested CO/CSOs to :-
- Assess the current mechanism identifying areas where we consider there to be inconsistencies/failings.
- Identify the positive aspects of the existing system so that these are not lost in any future system
Clarify and document our inventory requirements.
Design a simplified system
Implement new system

Other:

Dependencies:

Risks:

Milestones

Proposed date	Achieved date	Name	Description
	2012-07-01	Requirements	Aiming to clarify and document requirements by mid T2 2012.
	2012-06-01	Capture	Discuss existing systems with interested CO/CSOs