Note that detailed notes exist on private pages at DeveloperEvaluationNotes (project team only).

Evaluation of the technical development

In February 2006 the NeSC developers from the BRIDGES and DyVOSE projects answered the questions of the ESP-Grid project who evaluated their experiences, expectations and the problems they encountered during the development (see also ["NeSC_Shibbolized_Resources"]). The developers included:

• Oluwafemi Ajayi
• Micha Bayer
• Jipu Jiang
• Anthony Stell
• John Watt

Contact details confirmed at http://www.nesc.ac.uk/nesc/team.html with biographies.

The developers considered specifically the Shibbolizing of the Bridges web portal and DyVOSE work, and all the myriad of steps which had to be completed to make this work (including PERMIS - Privilege and Role Management Infrastructure Standards Validation), and identified for us:

1. What did you find difficult?
2. What makes Shibboleth a good solution for accessing a service like Bridges or the DyVOSE data?
3. What issues can you see in a real-world production of this with 100s of users, maybe a commercial data provider, issues for the future etc.?
4. What scalability issues can you identify?
5. How could the access control you've implemented be subverted by e.g. a bad person, or by an expert trying to get round the system for their own convenience, or by a careless user?

Scenario

To spark some real-world flavour a scenario was contrived: the developer was asked to imagine he has by chance met a manager of a faculty resource in the corridor, and he/she knows of the developer's experience and naively thinks he's the person who can just "Shibb their target..." - "This afternoon, if you've time?"

What did the developers find difficult?

In no particular order here is a summary of what the developers recorded as being difficulties during the development:

The operating system

One problem was to find a compatible platform or operating system that did not need everything built from source, so that it all ran correctly. The developers strongly recommend a flexible attitude to operating system, considering there are quite a few dependent packages that need to be installed.

Client operating system

Security and single sign-on (SSO) will be greatly improved if in future Shibboleth works with the operating system and not Internet browsers.

The portal container

Shibbolizing the portal container proved impossible. Servlet containers are used to host the BRIDGES portal on Tomcat, but Shibboleth protects resources hosted on Apache. It is very difficult to force the requests that come for the portal container through the Apache Shibboleth module first. The developers used the mod_jk (plug-in that handles the communication between Tomcat and Apache) so that Shibboleth can indirectly protect the Tomcat services. (Because mod_jk allows Apache to wrap Tomcat services, it will be difficult for a hacker to directly access a Tomcat service.) The developers are pleased that there does not seem to be a performance issue for using mod_jk as a wrapper/bridge. They strongly recommend that Tomcat is necessary to Shibbolize portals that require a Servlet container.

Boundaries between protocols

It was difficult (complicated) to understand where one protocol stopped and another started. Error messages would be returned - for example - referring to the PERMIS policy, or the PKI authentication methods or the Globus service implementation. Understanding where they were coming from could be very tricky. A lack of supporting documentation and difficulties navigating the source code meant the developers created their own documentation see http://www.nesc.ac.uk/hub/projects/etf/.

Find an established federation

Input from an established federation meant targets could be Shibbed in days rather than weeks. In BRIDGES' case, the SDSS Development Federation at EDINA ([http://sdss.ac.uk/ Shibboleth Development and Support Services (SDSS)]) normally responded to queries within 24 hours. Also the Identity Providers (IdPs) and Service Providers (SPs) could be configured and tested separately before joining them together.

Cookies

To provide authentication services for local users, the developers implemented Identity Provider - IdP (authentication site) using LDAP. The WAYF service (Where Are You From) stores a cookie on the user's Internet browser to identify their IdP for single-sign-on (SSO) purposes and for subsequent access to other service providers in the federation. But if a user turned off cookies in their browser then SSO cannot be achieved. Also if a user switches browser say from Microsoft Internet Explorer to Firefox, they will have to re-authenticate.

A non-cookie solution will make the current Shibboleth implementation more secure in the future. The SSO of Shibboleth makes it a good authentication solution for services like BLAST but it is not enough for authorisation. A service like BLAST would have to be designed to perform authorisation using attributes. At the moment, the developers are pushing attributes to service providers (SPs), but a pull model is better in some scenarios (i.e. requesting attributes only when needed).

Why Shibboleth?

In no particular order, here is a summary of why the developers think Shibboleth is a good solution for accessing a service like the BRIDGES portlet or the DyVOSE data:

Adding other projects to the infrastructure was not difficult

It is very easy to tailor attribute release/acceptance once Shibboleth is running. Adding other projects to the developers' infrastructure was not difficult. The developers interfaced Shibboleth with GridSphere successfully, and bypassed the GridSphere login to use the mod_auth_ldap authentication.

Single sign-on (SSO)

The single sign-on (SSO) capabilities of Shibboleth are desirable in the Grid context of BRIDGES.

AE need to add more here, important point?

A Scottish grid

Shibboleth looks quite promising especially for an academic environment where we would want to, say, give access to an application for anyone in Scotland as part of a Scottish grid. The developers would then not have to worry about managing their own user base but instead have arrangements with all other Scottish universities and institutions. If a user's distinguished name (DN) can be extracted programmatically from within the portal, offending users could be tracked and hopefully dealt with at their home institution. The developers are already able to trace user activity and user origin/details, (as an end resource the National Grid Service (NGS), for example, dictates this to the developers under the existing agreement).

The framework is easy to use

Although not easy to install, the main benefits of the framework are its flexibility, expandability, and ease of use. Shibboleth greatly reduces the administration load for authentication and authorisation. System administrators can easily and freely adjust user information on the Identity Provider (IdP) side without impacting about the (Service Provider) SP side. On the other hand, a SP can often adjust the access management policy without troubling the IdP. In this way, the administration boundary is very clear: the IdP is only responsible for the user information and information release policy, while the SP is only responsible for the authorisation policy of the resource. Also, adding or removing a particular organisation from the federation is relatively easy. There are no complex connections between an organisation and the whole framework. This feature gives the framework "good expandability".

Virtual Organisations (VOs)

The technology paradigm allows distributed access to resources and provides a nice, flexible way of transferring attributes between participants in a Virtual Organisation (VO). This is a good way of limiting and exposing access to resources as and when they are needed in a VO. However, this flexibility will be lost if the implementation uses only specific attributes.

AE: Final sentence has been re-worded as per MN's suggestion - does it capture the point well? (-- AlunEdwards [[DateTime(2006-04-30T13:58:26Z)]])
RS: "Not sure what you mean by last sentence in 1.3.5 Virtual Organisation Section.

All depends on SP asking for right attributes and doing right things with them and IdP knowing what to release and when and to whom etc. If we know all of these things in advance then all is well, if not then how do we know what to release etc. IdP can release everything (as we do right now), but really want to be able to say attribues X,Y,Z only should be release to SP1, but A,B,C to SP2. Right now we give A,B,C,X,Y,Z to SP1 and SP2." (-- AlunEdwards [[DateTime(2006-05-01T12:31:42Z)]])

Real-world production

Here are the issues (in no particular order) which the developers identified for a real-world production in the future:

PERMIS (Privilege and Role Management Infrastructure Standards Validation)

PERMIS is useful as a policy engine that interprets SAML (Security Assertion Markup Language), but its implementation requires a complex supporting infrastructure of specific technologies such as different LDAP (Lightweight Directory Access Protocol) versions, that makes it quite inflexible and time-consuming to make it work. This may mean that it could be an unattractive solution for some. Also the interface PERMIS provides is limited. If a service method protected by PERMIS fails to run, an Authorisation Exception occurs. There is no handle, such as a parameter saying "PERMIT/DENY" that the developer can use to control the flow. In addition, although technically PERMIS is authentication-agnostic it does require a local LDAP server back-end that matches the DNs (distinguished names) used in the authentication tokens (certificates). If PERMIS is implemented poorly, the developers warn that there is a danger here of the "illusion of security".

Scalability

It is hard to build a system where you have no real estimate of your potential user numbers e.g. when you open things to big institutions like other universities. You would ideally want to strike a balance and devise a system which handles roughly the load you expect, but with larger numbers of potential users the load becomes harder to estimate. Building extensibility into the system from the start would be the answer here. NeSC Glasgow is looking at performance aspects of portal technology which will feed into any investigation of high-load Shibboleth use now the grid's user base is increasing. Other specific scalability issues were identified:

• The developers have adopted a privilege management infrastructure (the PERMIS role-based access control system for authorisation) intending to avoid scalability problems associated with Access Control Lists.

MN adds (M/grid mapfiles) to end of final sentence, is this just a simple insert instruction, or something more complicated? (-- AlunEdwards [[DateTime(2006-04-30T13:58:26Z)]])

• DyVOSE is investigating dynamic delegation, which eventually will allow separate institutions to form a Virtual Organisation (VO) without re-writing their local security policies. DyVOSE demonstrates a simplified version of this between the universities of Glasgow and Edinburgh.
• A problem with expanding [*] is that user role which is specified by the Identity Provider (IdP) and required by the Service Provider (SP) may not be the same. So that, to access a specific grid service either the IdP must change user role to fit the SP specification or the SP must change its policy to fit the IdP. To solve this, the best way is to negotiate the relationship between roles, or unify different roles from different organisations.

xxxx AE need to clarify whole paragraph, espec [*] insert here: grid / user base / resource?  (-- AlunEdwards [[DateTime(2006-04-11T16:39:40Z)]])

• The developers would suggest that the [PERMIS/Globus] combination be stress-tested extensively for scalability. When combined together with more than 10 users, for example, the developers wonder whether performance might slow down chronically on any given system?

• In the current implementation, the LDAP server back-end that PERMIS uses is managed manually as the developers do not know of an automatic tool for this and because in any security process, human verification of credentials is essential at some point. The developers suggest that some kind of automated management tool such as this could be built, but then how would it fit in to the whole infrastructure?

• Scalability issues lie with the service providers (SPs) in the push model described in xxxx section [xxxx insert reference to the cookies in 1.5.3]. A pull model in which attributes are requested only when needed by the SP will have serious scalability issues.

MN: This point comes from Femi, which I don't think I agree with: it should be possible (in the BRIDGES-, DyVOSE-, VOTES- like situations to collect all the attributes you need at the start.  But that's *my* opinion! (markn 28Apr06)
RS: "Agree with you on the scalability issues with push/pull attribute model in section 1.4.2."
AE: need to check with Richard if this is an agreement with Femi or with Mark? (-- AlunEdwards [[DateTime(2006-05-01T12:31:42Z)]])

Security

The developers considered the following when asked to think how could their implementation be subverted (for example by an attacker, or by an expert trying to get round the system for their own convenience, or by a careless user?):

Firewall

Unless you tailor your firewall properly to block certain ports it is quite easy for someone who knows that the system is there just to reference the URI : port directly to gain access to the Shibbed portal, and bypass authentication. This is a weakness that attackers will get to know about and the firewalls must be configured properly.

Usability of PKI certificates

As the BRIDGES project progressed it became evident that a major user requirement would be an easy to use interface to the grid. Biologists were simply not going to use the applications if they had to do complicated things like get UK e-Science certificates for themselves. As PERMIS requires PKI certificates to get the username the developers used PERMIS as a look-up system rather than integrating it completely with the BLAST service.

Insert here the DCOCE findings and refs to Bruce's work for certificates to be usable or not used at all!

Public computer

If a user on a public computer does not log-out of a session he has requested to be remembered by the IdP, it is possible that someone can re-use his session to gain access to other federation sites, which would be the same problem as if his password got stolen.

Cookies

Cookies may pose security risks such as identity theft. A cookie is stored on the user's Internet browser to identify the user's Identity Provider (IdP) for single-sign-on (SSO) purposes. A SAML authentication assertion (identity handle), which is returned to the portal (service provider), is also stored as a cookie on the user's browser for subsequent access to other service providers in the federation. On shared computers, it is important that the cookies are deleted, but it is difficult, or impossible for IdP personnel or the SPs to technically mandate this. (It's a big drawback of browser/SSO technology).

Data in headers

Ideally, users' attribute certificates are returned as SAML responses to the service provider (SP). The portals could have the IdP release attributes using the LDAP server the developers set up. The identity handle is used to query the LDAP for user attributes. Because of the way GridSphere (Portal technology) handles data in headers, attributes are returned as short strings to the portal. This currently is not secured, but the developers are working on a better solution such as SAML assertion compression/hashing.

Authorisation using attributes

The BRIDGES portal uses attributes to interact with grid services e.g. BLAST. The developers think there are no performance issues or further security risks from this. They assert that the security questions really are: What do you do with attributes once exchanged at the service providers' side? What type of PMI (Privilege Management Infrastructure) does the service provider have? Can the PMI be driven with attributes such as Roles or distinguished name (DN)?

In conclusion

AE: suggest we ask Richard to draw conclusions from a strategic viewpoint? (-- AlunEdwards [[DateTime(2006-04-11T16:57:00Z)]])
RS: "For my conclusion. Main one is that this needs to be explored in much more detail. It is still way too difficult to set up Shibboleth, PERMIS, Grid and portal technology for the likes of ourselves - and we have been doing this for quite some time now. Showing others how to set up secure Shibboleth based access to Grid resources is non-trivial. We know how to do it one way, but if someone has a slightly different OS or different version of Globus, or wants to use OMII or UniCore or whatever, then we guarantee nothing.

It is also the case that the documentation needed for this needs to be improved greatly: This is what we have been trying to do but it is still non-trivial in that our documentation tends to be more like a complex installation and configuration recipe. Set classpath to this, configure XML file to that, make sure this .jar .gar is in the right place etc etc. Trying to let Joe Bloggs from university X know how to do these things is likely to be an issue , especially if they do not fully understand the complete picture.

I guess the more demonstrations and applications we build with Shib access then the more people can perceive the benefits of it. Tis a complex space and hence we have complex software solutions unfortunately. I am slightly concerned by this for the longer term though... if it is too complex then folk might simply say, forget about it." (-- AlunEdwards [[DateTime(2006-05-01T12:31:42Z)]])

Note that detailed notes exist on private pages at DeveloperEvaluationNotes (project team only).