DICE development projects - services-unit

Assigned Priority:

Manager:

Unit:

Summary:

Review Handling of Sensitive Data Within the School

What:

Examine the way sensitive data, both personal and research, is currently handled within the School and to suggest ways in which this could be improved.

Why:

It is important that the School can be confident that all the requirements (both legal and as imposed by funding bodies) for the correct securing, retention and eventual disposal of secure data are being met. Not doing so opens the School up to both the threat of legal proceeding and the potential loss of research funding. This is not an easy task and has traditionally been overlooked or ignored as a consequence but it is becoming increasingly clear that this is an issue the School must address in a timely fashion.

When:

Though there are no hard deadlines, this is something which should be done as soon as possible.

How:

An important first step is to determine what sensitive data the School possesses. This information should be generated by the Research Data Audit project and the sensitive data audit currently under way. The Archiving Project shoudl provide details of such things as mandated retention periods and access criteria. With this information to hand, decisions can be taken as to how the School can improve its handling of sensitive data, acting in consultation with the University units such as Registry and the library.

Effort estimate:

3 weeks

Other:

Dependencies:Devproj 290 (research data audit), devproj 194 (Meeting the School's Archiving Needs), Sensitive data audit
Risks: The Risks are all concerned with not doing this project. Failing to address this issue will leave the School open to legal action and loss of research funding

Provision of secure web service for user data

roger — Mon, 03 Mar 2014 14:04:29 +0000

Project ID:

301

Current stage:

Manager:

roger

Unit:

What:

We have a considerable amount of user web content (homepages is one obvious example) mounted on our web servers via NFS and AMD. This project would provide a more secure alternative using a separate web server with AFS filespace for sensitive data.

Why:

Customer: School

Case statement: NFS mounted web space has many security disadvantages, as well as being only accessible from within the School. AFS, with proper safeguards, would allow both file access and web access in a secure and controlled manner.

When:

Status: proposal

Priority: medium

Time: 1 week

How:

Proposal:
To provide secure, on-request, user web space

Resources:
Sufficient AFS space to contain the content
Knowledge of AFS and apache/waklog

Plan/Issues:

Install basic test web server
Add support for authentication (CoSign & Kerberos) & waklog
Confirm ID naming scheme
Create AFS/Kerberos IDs for web server & users
Generate & store appropriate keytabs
Configure appropriate ACLs for filesystem access via command-line & browser
Generate volumes
Confirm filestructure &naming scheme and configure volume mount-points
Rebuild suEXEC
Configure CGI execution environment

(also see earlier AFS Web Server Plan/Issues Document)

Other:

Dependencies:
This project requires a working version of waklog for our web servers.
This project relates to, and will have an impact on, the "Move user and group web space to AFS" project.

Risks:

AFS performance could be affected - possibly by significant web traffic to/from the servers. Any such degradation could, perhaps, be reduced by separate storage/servers for web areas.
Using AFS makes web data potentially available from any location (via browswer or filesystem), thus increasing security concerns.
Future dependency on waklog. Note that the future of mod_waklog is uncertain - any reliance we place on it may mean that might have to become involved in supporting it - either locally or publically.

Milestones

Proposed date	Achieved date	Name	Description

Research Data Audit

cms — Fri, 23 Aug 2013 15:31:57 +0000

Project ID:

290

Current stage:

1_Proposal

Assigned Priority:

Manager:

Unit:

Summary:

An audit of all research data within the School

What:

This project will attempt to all research data held on School storage and create appropriate metadata (owner, status etc) for each item of data identified. The project will also encompass decisions regarding how the audit should be conducted, how the metadata should be stored and how best to ensure that the audit data remains current after the end of the project

Why:

The requirements of many funding bodies regarding the creation of, access to and eventual deletion of data are becoming ever stricter. This audit will be a useful first step towards ensuring that the requirements of these funders are met. It will also be extremely useful for management of the School's storage.

How:

To be decided as part of the project. Much of the work of actually collecting the data can be CSOs or even by research staff

Effort estimate:

one week for decision making.. up to four weeks CSO/other time for actual collection of data

Other:

Dependencies:An audit of group space usage was conducted by Lindsey Brown in 2011. This project will build on that work

Risks: Not doing this risks failing to comply with the requirements of funding bodies with detrimental affects on the School's reputation and income

Securing web servers

neilb — Thu, 22 Aug 2013 10:22:31 +0000

Project ID:

287

Current stage:

3_Implementation

Manager:

neilb

Unit:

https://wiki.inf.ed.ac.uk/DICE/Project287SecuringWebServers

Summary:

See what we can do to limit the possibility of a web server being compromised, and if compromised, limit the damage that could be done.

What:

The MPU have taken steps to secure/harden the external facing ssh servers. Web servers are equally external facing and we should make sure they are as secure as they could be. And should they be compromised, limit what could be done from them.

Why:

A compromised web server could lead to:

reputational damage
theft of personal/private/restricted data
the possibility of hosting illegal material
our equipment being used in a crime, eg phishing/fraud.

When:

These possible threats are not new, but it's percived that there's a growing risk of damage being done. So sooner rather than later.

How:

Possible options

Making sure software is up-to-date with latest security fixes/patches
Separate web instances so if one is exploited, damage can be limited.
Tighter control over who can publish web content, CGIs or web apps
Remove superfluous software from servers
Limit what can be done from a web server, eg no SSH outbound. Local DNS only, block outbound DNS, close outbound firewall holes
Better monitoring, eg monitor network traffic

A longer list of possible options is listed on the "Report" wiki page.

In the first instance, we only need to be more secure than the next guy, so we'll probably look at the low hanging fruit (ie the easy to do) first.

It would be nice if we could come up with simple header files people could include to implement simple fixes. Unfortunately some fixes will depend greatly on the web service you are trying to harden.

Plan:

List possible steps to harden/secure sites - done
Prioritise list (probably do the easiest ones first) - draft order added to wiki page
Work through the list:
Either implement as a simple header and/or document what's required to
achieve an generic solution. Site owners will need to decide how to
implement/configure more specific solution for their site.
Inform site managers of new header/docs.
Move onto next in priority list.
Repeat until out of time.
Document project.

Effort estimate:

4 weeks

Final report URL:

Other:

Dependencies: network, security knowledge

Risks: see Why? section

The effort includes implementing any changes across the various managed web servers, there are 5 main ones: www.inf, groups.inf, wcms.inf, homepages/wiki.inf, aiai.ed. each may/will have different "superfluous" software.

What about other web (app) sites, eg rbs.inf, computing.help, ifile.inf, webprint.inf, pp.inf, lcfg.org etc

If we're going to limit the software on web servers hosting user generated content/CGIs, then how will users develop their content? Logon to the live service? (undesirable) Or will we need to maintain a second development version of the service which users can log into, but is not accessible outside of inf? (sounds more likely, at the expensive of maintaining more servers)

Services Unit internal documentation review

alisond — Thu, 22 Aug 2013 09:49:48 +0000

Project ID:

283

Current stage:

Unit:

Summary:

Review of Services Unit internal documentation

Other:

Dependencies: project 280

Risks:

Deploy Git and Gerrit as full services

cms — Tue, 05 Feb 2013 10:54:57 +0000

Project ID:

272

Current stage:

3_Implementation

Assigned Priority:

Manager:

Unit:

Summary:

The School has been running a Git and Gerrit not-a-service for several years now. This project will upgrade Git and Gerrit to full services

What:

Set up new virtual server for Git/Gerrit
produce header files to allow Git/Gerrit servers to be easily set up
ensure that service data is being backed up
update documentation and helper programs

Why:

Git and Gerrit have seen a reasonable take-up by the School since the introduction of the not-a-service. It is being used to manage the Prometheus project, and has also been used for coursework group practicals in the past. Given the need to update the server on which the service is running to SL6, It makes sense to promote the service to a full service now.

How:

build obtain necessary RPMS
create header file to install RPMS, generate certificates and configure gerrit web interface
Build new virtual server
Install latest versions of Git/Gerrit on new server
Move data from not-a-service to new server
Test compare data for discrepancies
port/update helper/support programs
Set up mirrors/database dumps
Determine best way of configuring gerrit automatically.
deploy new service

Effort estimate:

In total the project should take around 2 man-weeks

Other:

Dependencies:
postgresql, apacheconf, cosign infrastructure

Risks:

Investigate AFS on ECDF storage

boss — Fri, 25 Jan 2013 15:46:29 +0000

Project ID:

268

Current stage:

2_Evaluation

Manager:

Unit:

https://wiki.inf.ed.ac.uk/DICE/Project268AFSonECDF

Summary:

AFS access to the ECDF provided 500GB per research user.

What:

Description: The University is proposing to provide each research active user with 500GB disk space. Unfortunately, IS are currently proposing that this space will be presented by NFS, CIFS or SSHFS. This project would investigate whether we (working with IS) can add AFS as an access mechanism.

Deliverables: Report of the investigations, and a working AFS path to the space if possible.

Why:

Customer:
All research active users within School

Case statement:
It wouldn't make sense to not use the space being made available to us, however for better integeration with our file system, AFS access would seem a better option than the proposed filesystems.

When:

Status:
Proposal

Timescales:
It's been resourced this third (T1, 2013) for 3 weeks.

Priority:
Medium/low - It's not mission critical, and at the last CCPAG (Feb 2013) ECDF were still waiting for the go ahead to provide the space.

Time:
1 hour so far

How:

Resources: Total of 3 weeks.

Proposal:
Two weeks to create an .ed.ac.uk AFS cell on GPFS nodes. One week to benchmark end-user performance and effect on GPFS cluster.

Plan:

Get to know how ECDF GPFS works.
Research strategies for running AFS on GPFS.
Assuming it is possible, mount GPFS on machine in Forum, or get access to machine at ECDF that does have access and setup AFS. Probably a test cell first.
If it isn't possible, can we get access to raw disk space and mount as a normal /vicep
Test and bench mark, check performance and affect on cluster.
If all is good, then make what ever changes to run it as a full service - eg hardware, final cell name.

What if a researcher doesn't want "their" space as AFS. Some may want to opt out and use one of the IS proposed protocols. eg researchers with Windows machines?

Final report URL:

Other:

Dependencies:
Access to ECDF storage.

Risks:
This would be a new service, not impacting on current services, so risks would be low. Perhaps performance issues affecting network/ECDF.

Milestones

Proposed date	Name	Description
22/3/2013	access	Get access to ECDF and GPFS
	testcell	Setup test AFS cell
	perf	Performance testing
	results	Present results/findings/thoughts
	production	Turn into production service if all went well
	writeup	Final Report

Do we still do these?

Use Central Lists Service to host Informatics Database Generated Mailing Lists

boss — Fri, 25 Jan 2013 15:46:28 +0000

Project ID:

263

Current stage:

Manager:

neilb

Unit:

http://www.ed.ac.uk/schools-departments/information-services/services/computing/comms-and-collab/email/lists/sympa

What:

Description: The Informatics Database currently generates the list memberships for about 860 locally run mailman lists (out of a total of 1560 lists).
This project would implement hosting of those lists on the central IS, Sympa based lists service.

Deliverables: A service similar to what we currently have, but hosted on the IS service.

Why:

Customer: Primarily Student Services, as they have the most DB generated lists, but also the School in general. Also the members of those lists.

Case statement: We have a requirement to move School services to Centrally provided equivalents where appropriate. It should also save us resources.

When:

Status: pending

Timescales: I suspect most time would be spend familiarising ourselves with Sympa's synchronising facilities, and whatever mechanism IS implement to allow automated list creation. Also on figuring out the criteria to produce the required list memberships in Prometheu

Priority:

Time: I would estimate it might only take 1 to 2 weeks to sort out the technical aspects of driving list creation and synchronising of list memberships.

A bigger unknown would be how long it would take to migrate the current conduit logic into Prometheus - if we decided to do it that way. There would be the basic "how we do it in Prometheus" - say 1 week for those with experience of Prometheus conduits. And then another week to actually migrate the logic for the lists?

So in all it could be 2 to 4 weeks work.

How:

Proposal:

Resources: Access to Prometheus and/or Informatics DB.

Plan:

Other:

Dependencies:

Currently there is no way to automate the creation of mailing lists on the sympa service at our end. IS are currently looking at providing this facility, but it is not there yet. Other Schools are requesting this, and it is something they are planning on providing, but it isn't there yet. Whatever they provide, we will have to produce the XML file that Sympa requires to create a list from the command line.
Once a list is created, sympa can currently manage a list membership from an external MySQL DB, web page, LDAP.
Conduit knowledge may be required if we were to use the DB to generate web pages, or other files containing lists of users to be synced to the lists.
Prometheus knowledge, as Sympa can sync lists directly from LDAP, then this would seem an obvious place to source the data.

Risks:

The address for all Sympa mailing lists is a flat mail space @mlist.is.ed.ac.uk. They have no plans to support virtual domains, eg @lists.inf.ed.ac.uk. It's a first come, first served policy for list names. So our lists could actually be called something like informatics-staff@mlist.is.ed.ac.uk, or inf-it-t02@mlist.is.ed.ac.uk. We'd have to setup forwards so that staff@inf or it-t02@inf are forwarded appropriately.
We'd want to check that the Sympa lists are aware of these aliases, and will accept the forward, they should. But are we bothered about the "clunky" real list names?
As I mentioned the lists names are allocated on a first come, first served basis, so there's nothing stopping someone else creating a list called inf-students-jobs@mlist.is and potentially being nothing to do with Informatics. And any list creation mechanism we create would need to take into account that list names it wants to create, may already be taken.
There isn't apparently a way to automatically add new members as a one time event. eg as we do for the seminars@inf list. We'd need to investigate if this is indeed true, or decide if we can live without it and have some work around.
Does sympa provide the functionality users are used to, if not (would seem unlikely), is it a show stopper?
If this goes ahead, what do we do with the remaining manually managed lists?

Milestones

Proposed date	Achieved date	Name	Description

Software Download Portal/Repository: Stage 1

boss — Fri, 25 Jan 2013 15:46:27 +0000

Project ID:

258

Current stage:

3_Implementation

Manager:

roger

Unit:

What:

Description: Requirements gathering for the implementation of a new Software Download Management Platform that would enable the distribution of research software and applications more effectively than the existing Informatics Software Download Database (ISDD).

Deliverables: A report examining the current state of ISDD, and a list of the perceived short-comings (real or imaginary) of ISDD (and whether they could be fixed). A description of the functionality required by a new Software Download Management Platform.

Why:

Customer: Edinburgh Research & Innovation, and all Informatics Research Staff (especially David Richardson, Dave Robertson, Bob Fisher)

Case statement: The UK Research Council’s focus on impact is changing the way we think about our research output. Impact features heavily in the upcoming Research Excellence Framework (REF) and we need to ensure we are capturing as much supporting evidence as possible.

The Informatics Software Download Database (ISDD) has been in existence for many years and is currently home to a number of legacy projects, but is perceived as being rather long in the tooth.

There is a confirmed expectation from the Scottish Government that the University takes active steps to simplify the process for companies to take advantage of freely available intellectual property, in parallel with commercial licensing activities. The proposed portal for open source technology will allow the School to lead the way in this initiative, working with Edinburgh Research and Innovation (ERI) and the collaborative University Technology project.

When:

Status: Final Report available on the DICE wiki.

Timescales: estimated half a week.

Priority:

Time:

How:

Proposal: We need to know what users actually want, and whether the existing ISDD could be reasonably modified to provide that functionality - and if not, what would be required of a new Software Download Management Platform.

Resources:

Plan:

Establish current and prospective user base.
Investigate IS data share, and assess possible use/relevance.
Examine the current state of ISDD, and list any perceived short-comings (real or imaginary) reported by the user base.
Investigate whether any deficiencies (if found) could be be corrected, and viability/feasibility of same.
Assess additional requirements not provided by ISDD
Enumerate functional requirements of a new Software Download Management Platform (to include existing and requested functionality).
Solicit user feedback and refine requirements list.
Prepare final requirements list.

Other:

Dependencies:

Risks:

Milestones

Proposed date	Achieved date	Name	Description

Streaming Media Service Requirements

boss — Fri, 25 Jan 2013 15:46:27 +0000

Project ID:

259

Current stage:

3s_Stalled

Manager:

roger

Unit: