ESPGRIDwiki: UsabilityWorkshopNeSC

Usability in e-Science Workshop NeSC January 26-27, 2006

• See http://www.nesc.ac.uk/esi/events/613/ and

  • http://mytea.org.uk/uws/ for details about the event.

  and http://www.nesc.ac.uk/action/esi/contribution.cfm?Title=613 for presenters' slides.

  Also http://mytea.org.uk - go there to see the blog

See also the paper that we submitted for the workshop: Security, usability and the new types of grid users.

DAY 1

• Monica Schraefel (mc schraefel), University of Southampton - opening address.

• Rob Proctor - National Centre for e-Social Science Opening Plenary

  • http://www.inf.ed.ac.uk/people/staff/Robert_Procter.html Specialist in research for usability. e-Science vision: "globally connected, scholarly community promoting the highest quality scientific research". We can no longer assume that "if we build it, they will come" Organisational usability:

    • Help build manageable infrastructures Provide dependable authZ, authN and acess control mechanisms Support research governance
    Community usability
    • Create incentives for sharing data
      • e.g. properly describing it so that it can be discovered and re-used by other people. Communities got to consider their policies for this.
      Develp sustainable models of technology supply
      • Some people don't trust OSS, some don't trust proprietary. Which does the community use?
      Community knowledge repositories
      • Standards, ontologies need to be agreed.
      Respond to research drivers
      • Demand for evidence-based research (policy-making).
    "Work: the 'Missing What' in e-Science"
    • There is a tendency to assume that there is only one kind of knowledge, only one kind of science and one scientific method. That's just not the real world, but we need to understand this in the design of tools. Knorr-Cetina's book in 1999 looked at 2 scientific cultures/disciplines (high energy physics and molecular biology). Different social organisations of this work. HEP: trans-national,

      expensive, lots of collaboration, collectivity of effort. MolBiol: much more individually-oriented lab research. The Human Genome Project was Mol Biol's first attempt at 'big

      science' >350 laboratories. Digital artefacts: e.g. astronomy's reconfiguration from an obesrvational field science to an image processing lab science. Is there a paradigm shift in Mol Biol to a 'theoretical' science that manipulates masses of sequence data, rather than biological samples etc.: in silico experiments, rather than wet science.

    Big Challenge: Global collaboration.
    • [Missed a bit!] Access grid: we need much more here. Virtual meetings, recovering information from meetings, synthesising artefacts. Record and review what happened in meetings. VRE developments.
    Trust (another big challenge)
    • e-Science raises big concerns about trust in technologies, trust in data and between collaborators. We need to research:
      • - what makes technologies trustab le and how to provide 'trust affordances' in e-infrastructure - how distributed communities impact formation of 'cultures of trust' + how to develop practrices to deal with this.
      Trust and mobile data
      • Visibility: easy for real, local physical artefacts, but when digitised, this changes visibility and accountablility of work practices. (Marina Jirotka, much work in this area).
    VOs - big challenge
    • Codifying of policies and auditing of interactions would provide and opportunity to 'visualise' VOs (for those people in the VO in particular). Mapping and displaying networks of people (research into this in terms of VOs). [MN: is this really useful?] Shows who the members are and how they are collaborating.
    Representing knowledge
    • Production of new forms of knowledge is a central feature of the e-Science vision. Investigating new forms of reasoning and their impact on requirements for tools). e.g. visualisation - how does this affect the way we work? What does it add to them? Ontologies: imperfect, may not add to knowledge in the long term. Proctor kind of blames the software developers for not seeing how domain scientists will value the ontologies. Scientists are interested in getting their work done *now*, not really building up a data bank for future generations.
    Methodologies
    • Different set of methodologies needed for designing and building e-infrastructure and tools. Need to build on the efforts of early adopters. Need new methods for requirements capture, including requirements for work practices that are only as yet imagined.
      • "we know of no scalable methods of req. analysis that document the needs of vastly different user populations, contine to document changing needs over decades..." (Zimmerman and Nardi, 2006).

    What is NCeSS doing>

    • Entangled data project (Essex). Oxford e-Social Science: Ethical, Legal and Institutional Dynamics of Grid Enabled Science. Lots more NCeSS have usability an explicit goal.

• Richard Sinnott - BRIDGES, Biomedical Research Informatics Delivered by Grid Enabled Services
  • Users needed a data grid and a compute grid. Background of scientists:
    • From "what's a browser" to "PERL script experts, Linux admin capabilities" But didn't have the time or inclination to use Globus or GSI, or the patience to get and use an X.509 certificate. Users were concerned about security, especially about confidentiality of their data: IPR

    Login to the portal, collections of tools (including MagnaVista from IBM: type in the name of a gene, go to central hub and find it in the various - grid based - databases). Also have a "GeneVista" solution that is based on OGSA-DAI. Gave them lots of personalisation options, and personal profiles etc. but the scientists weren't interested. A "don't have time to read the manual" attitude. They wanted a "google-like" interface. It works, but still suffers from changing schemas. Very off-putting for end users. BLAST interface: they don't need certificates, use the host certificate. Username and password to access the portal. "The customer is always right" If they don't use it, accept that they need something easier. Scientific pull for these technologies is their 'work': it has to change the way way they do their researech if they have the use for it.

• Jenny Ure: Grammars of Collaboration: Designing for e-Science
  • Lots of "local grammars" that would impact on the success of the e- Science vision. Vision and Reality: vision may be blind to certain realities (both technological and pragmatic due to the behaviour of humans). Gap between vision and reality: no easy way of negotiating expectations and access/agreements.

    e.g. NeuroGrid project - data were collected from different sources and from different purposes. She mentioned the "myth of shared protocols" (Example used: tracing inside the brain and outside the brain in different laboratories) Local grammars built into natural communities. Good to design around old or existing technologies: users recognise and know (almost intrinsically) how to make the 'system' work for them. Different visualisation techniques very useful for this. VOs heighten the need for synergy and alignment to common ends:

    • One size fits all (SAP vision) Global and local requirements (Napster vision)
    People may have different needs that cannot be shoe-horned into one (place). So may need local systems anyway. But if you add this to the 'system' imposed by the collaborating community, you can get costly solutions, both time-wise and by introducing double-entry etc. and errors.

• Trevor Dudds: Collaborative Interaction in Virtual Environments
  • CVEs = Collaborative Virtual Environments Case study CVEs he used were Urban Planning and Data Visualisation

• Jenny Fry and Ralph Schroeder - Oxford Internet Institute. "E- Science: the few from the social sciences"
  • To identify the ethical, social and legal barriers to e-Social Science. First step is to map out the different approaches:
    1. Practical/usability
    2. Attempted neutrality/value free (e.g. co-authorship and collaboration etc.)
    3. Advocacy/steering and aligning structures - fostering institutional and legal structures that enable distributed communication and collaboration).
    4. Critique and reflexive approach (new forms of labelling due to e-Science)
    Earlier approaches:
    • David and Spence's project-based typology (a steering approach). Critique (e.g. Wouters and Beaulieu computation-centric e-science
      • based on disiplinary analysis) e.g. criticising the whole overall aim of e-Science.
    Is technology development a driver? The OII looked at the existing e-Soc Sci nodes and looked at disciplines being facilitated (based on PI's parent discipline). Could talk about usability AND utility (are they really being used, are the really useful). Can look at issues at the macro- or policy level? (Has it's limits). Can also look down at systems and networks level. Can look at different sciences/domains, can look at different projects. If you can collect data/requirements from each of these levels, does it actually help in gathering it all up together? (Open question). Can you bring all of these into harmony? Would it be useful if you could? They are looking at the different levels of analysis and how useful they are.

• Panel discussion on "Infrastructure: should it be invisible?"
  • Panel: Frederik Orellana, CERN (University of Geneva); Bruce Beckles, Cambridge; Charles Forsyth, Vita Nuova (company in York); James Suter, UCL (computational chemistry). Charles Forsyth: yes it really should be invisible. Most users want a really simple (Google-like) interface, but some users want to develop against it. Also invisibility bad if the user can't understand James Suter - has produced two middleware schemes that makes it easier for end users (who have not engaged due to Globus being too hard). Steven Newhouse commented that maybe it should be obvious when the user is moving to use resources in another administrative domain. The infrastructure/middleware should be visible in that sense. Charles Forsyth had a customer who asked them explicitly not to put security into the grid because they have had bad experiences in other grids!

• Diane H. Sonnenwald, Goteborg University. Re-thinking usability: some thoughts
  • Project example: the nanoManipulator. Allows scientists to interact directly with physical samples, ranging from DNA to cells.
    • Did a field study - looked at researchers working alone and in groups. Over 6 months. Got design requirements, interface design study, usability testing etc. etc. etc. Very realistic (real studies) used. Did longitudinal field study. Some successes: (she gave a list of examples). Some failures: some scientists wanted the system, got it, liked it, but didn't use it that much. "Great front-end to the wrong scientific instrument". (Microscope that it's built against isn't up to the job, and they haven't been able to put their interface against different types of microscopes, because of commercial issues).
    Are our current approaches to design in e-science good enough? Can we do better?

• Panel session (security) with me, Bruce Beckles, Ivan Flechais and Brock Craft
  • See my notes at the end of this text document.

• Craig Ganoe- Supporting distributed scientific communities: Making

  CiteSeer collaborative

  • Talk I kind of missed

• Jeremy Leighton John: From pencil to digital: note taking in the field British Library
  • Manuscripts - characteristis and field notes (not talking about just writen/paper/vellum things, also digital). Really talking about tracts of information that is not intended to be published (e.g. drafts etc.). 3 aspects that are interesting: authenticity, confidentiality, styles and layout.
    • Confidentiality issues may be huge - e.g. if some of the information contained therein got out it would be catastrophic.
    Origin and history of field note books. Clip board science Freestyle information: thoughts that the writer didn't intend being published etc. Not sure about the relevance to usability and grids but really interesting talk! During the questions, it became clear that Jeremy was trying to get more interaction with today's scientists to capture their media before they are too old and degraded.

• Jeremy Frey (in for Simon Coles), EPSRC National Crystallography Service, Southampton - Enabling the reusability of scientific data: Experiences with designing an open access infrastructure for sharing datasets.
  • Data and the publication problem
    • e.g. about 25 million chemicals can be made, but only about 2 million have found their way into scientific databases. Of those 2 million, not all are easy to actually get at. "The use of the scientific record is being impaired." IPR in Journals etc. Also, someone works out the structure of something, finds that it wasn't the substance that s/he was after, the structure doesn't get published. Need to publish data AND put in context. Bypass peer review and publish to the world. Crystal Structure Report Archive - based on an ePrint system. eBank JISC funded project, uses Dublin Core. How do you get these things published? Toolbox written to take files etc. to upload it into the databases. Minimal number of manual metadata entries - many can be hardwired into the system. Peer review removed from self deposit publication. Simple checks for consistency. Unique identifiers (URL) for the files, the molecule and collection of the bits of the data.

DAY 2

• PANEL SESSION "Research Council Q&A session" with Tony Hey, Maia Dimitrova (VRE programme, JISC), Jim Fleming (EPSRC)

  • Microsoft now looking for collaborations. Other people opened by talking vaguely about usability. Tony Hey reckons that JISC will do FEC - it just needs a new contract. Decision has been taken in principle, but nothing much has happened. Maia agreed but it sounded, in general, that it wasn't going to happen soon. Possible for one or two projects to get it, [but then, surely, that would cause problems of its own]. A little inconclusive!

• Christine Hine, University of Surrey - The politics and practice of distributed information in systematics
  • Christine is a sociologist of science and technology (not usability expert or computing expert). Looking at systematics (classifying organisms in biology). This discipline is using shared databases etc. but is not strictly e-Science. No high end computing but much distribution, collaboration etc. Idea that if you build it, they will come (it has happened in this community). Conclusion that usability has politics. Wherever you are in the world, the collected materials that will underpin your science will be somewhere else (e.g. Kew etc.) Convention on Biological Diversity, 1992 mandated that signatories must share their collections. Always interpreted to mean sharing their digital resources. Strong tradition of collaboration distribution in this field. So high pressure to develop digital solutions (ways of solving the problems of the discipline AND of the image of the discipline). Making tentative steps towards electronic publishing, don't totally trust it (as they have to work to very long time-scales: hundreds of years). They always want to create physical records of everything they do. Christine talked about the mixture of moving people around, digitising, changing the way they worked etc. to cause a mix of actions for progress in this field. Usability has politics:
    • Systematics: a community for whom data and resources must be shared Multiply accountable, highly reflexive Grand visions are a source of pressure and opportunity (e.g. house of lords select committee investigation, Darwin initiative,
      • Big name ecologist (Charles Godfrey) saying that the discipline should become web-based. Don't worry about the way things were done before. Also science should be consensus driven (e.g. new taxonomies are built using consensus over the web). Big change. This became an influential grand vision.
      Grand vision was a driver for technological change AND cultural change within the research community.
    Back to e-Science (and usability).
    • Too much focus on particular technologies can be counter productive - they have to participate in communication ecologies. Belief in the inevitability of digital solutions can sideline users. Users need to be motivated to self-identify as users (themselves). Celebrate diverse and crafted solutions, even the non-digital.

• Jeremy Frey, Southampton - The Usability of Smart Notebooks in Smart Laboratories

  • The CombeChem Project

    • Collect data with regard to how it could eventually be used.
      • Metadata must be of high quality
    Chemical Grids
    • Start in the laboratory - pervasive physical grid (Smart Tea) Used by chemists so must be simple to support and maintain (simple stable secure and autonomic)
    What are the people up to? The lab notebook is a permanent, documented and primary record of laboratory observations. A lot of pressure on researchers to use this notebook and not to use scraps of paper which they then later transfer. They have used lab books with carbon paper and now have moved on to tablet machines. The 'pens' get melted a bit too often though. Really entertaining talk! Have managed to capture margin notes (informal observations) too and these can be resurrected later and added to the searchable record at a later date, if the information therein is seen to be important.

• Andy Gibson, Bioinformatician - Providing an In Silico Lab Book for In Silico Science
  • Smart Tea project. What is bioinformatics. Tough question. Skills, bio-science, computer science, statistics, physics etc. Behavioural observations:
    • Bioinformaticians do not keep detailed notes of their work. The term 'lab book' may put some off.

    Lab book paradigm. Hypothesis -> test > repeat > conclusions etc. etc. can be traced chronologically throughout a lab book. Bioinformaticians tend to do these cycles really quickly (before coffee break) and therefore the chronology may be lost or irrelevant. Magic blanket approach to capturing their work: not that successful. Hard to filter out repetitious tasks and browsing the BBC web site! Bagic bullet approach: like jigsaw analogy, user performs analyses through a software workbench. System tries to capture the work as the user goes along and changes tools. Didn't introduce any extra work for the bioinformaticians.

• Discussion Session: Planning for the Call for the Special Issue, IJHCS for Usability Design in e-Science
  • Papers coming in throughout the early summer and papers accepted later on in the year. mc typed up lots of emerging themes.

• Russell Beale - Issues in designing for e-science
  • Russell having some difficulty defining e-science. Because it's so many things to so many people, it means that it's hard to generalise or apply usability concepts.
    • Context is important.
      • Used to be single users and single systems, then multiple users, single systems. Now multiple users and multiple systems. Users affected by other users, the task, political, publication pressures, collaborators etc. Systems affected by distributed nature, QoS, etc. etc.
      The science is affected by the technology - it isn't just doing the same thing better, using ICTs, but the way scientists work may be different.
      • Also is science changing. We used to look for patterns - now a lot of science is looking for anomalies.
    • Only from embedding design in the wider context is it effective* Need to learn from CSCW work. Model users and systems (e.g. UML). Little science, big science.
      • e-science democratises science. Mass participation (even outside academia). Non-scientists as sources, need appropriate systems for this.
        • Usability needs of collectors often very different from usability needs of scientists.
        Ethical duty (duty to convey results back to them). Also raw data, anonymity of participants.
        • Have to be careful about feedback (e.g. cancer near pylons etc.)
      People say that the aim of design is usability. But is it? No, there's more to it.
      • Russell calls this "slanty design" (Comes from the design of sloping desks in libraries so that people don't put their drinks next to the books). Slanty design
        • design that reduces functionality (deliberately) makes it simple to do simple things makes it hard to do unwanted things (e.g. gmail)
      Slanty design process
      • Identify the user goals. Look at the big context and then come up with designs.
      Problems with e-Science as well as problems with usability Need to design for the complex situation from the start Scientists and non-scientists working together.

• Chris Hinds - Position Paper: Designing for Usability in e-Science
  • Looked at e-DiaMoND, national mammography national database. Complex medical work, conducted on a grand scale. Technology can be disruptive, especially at trial stage Usability: system needed to be efficient and easy to learn
    • Needed to be a fit for real medical work. Did field work in clinics etc. Ethnographic studies.
    "Practical Ethical Action"
    • a notion that came out of their fieldwork studies. a critical dimension of day-to-day medical work
      • but how is this related to usability?
    • Access control in e-diamond. They need unique one-off ethical decisions (e.g. showing a mammograph to another clinician who 'should' not be authZed to see that record). If system is too restrictive, then it is a barrier to everyday work.

• John Rooksby (Lancaster University) Usability and / as / in Mundane Practice
  • Ongoing use and ongoing usability.
    • Usability is not static, need to put some notion of time into there. Usability AND practice (the practice of the researchers/scientists) Usability IN practice - it would be good to get ongoing design into the normal way of working (possibly owned more by the users themselves). Usability is often treated as a separable feature/issue/profession.
    Talked quite a bit about how ethnomethodology should be used.

• Cherri Pancake: e-Science Tsunami prediction
  • Best case: quickly rising tide Worst case: wall of water with rocks and debris Tsunami means "Bay wave" as the worst aspects in Japan were often the ricochets with debris from the sides of bays after the main wave has come in. Distant (ocean-wide) tsunami often causes more widespread damage. Warning guidance
    • Real time instruments detect earthquakes and tsunamis. Inform 5-15 mins. Risk reduction - need preparedness from numerical models and scenarios.
    Emergence of e-Science (last 10 years)
    • Real-time deep ocean data (and giving direct confirmation of tsunamis) Collaboration among key communities
      • Research, community planning, relief agencies etc. etc.
      New generation of inundation models
      • Better mapping Wave height forecasting
      Paleo-tsunami studies
      • More complete longer tsunami histories Recognition of "slump"-induced tsunamis This category has added a lot to the general understanding
        • Slump-induced big landslip into the sea. e.g. 1958 in Lituya Bay Alaska (rel. uninhabited) Slump induced tsunamis have the greatest effect.
      E-science enables "community-based science" (in this sense a community of different scientifc disciplines).
      • Tsunami Computational Portal - example of infrastructure for collaborative model.
      • *This looks really interesting - completely application led. Uses grid at back end**
      To date long term preservation of data is a problem. Data collected in an idiosyncratic way. Very hard to put this together. Tsunami Reconnaissance Data Repository.
      • To collect the data for the Dec 26th 04 Tsunami and make sure that it survives and is accessible.
        • Will have the ability of the broader community to add commentary.
      Scenario-based human response simulation
      • Visualisation models to help design how to build emergency response plans. And possibly to educate the population

END OF WORKSHOP

• ############# See below for my notes on the security panel session, which I've sent to mc at: uwsATmspaceDOTfm #############

Notes on security panel session from day one On the panel were, Brock Craft, Ivan Fleschais, Bruce Beckles and me

We were asked to tackle the question as to whether there was any opposition between usability and security (i.e. were these two concepts in tension within interface/system design?). And secondarily, if there was a tension, how could it be reduced?

We all briefly talked about our position papers that we submitted for the workshop (not actually tackling this question directly, of course).

I talked about our attempt to categorise users very coarsely to highlight that each user type has a different security/threat profile. And, depending what the user was doing, it is probably appropriate to have different access management 'solutions' for different types of users. There had earlier (in the workshop) been criticisms of any attempts to categorise users, but I believe that it can be useful, and only if we recognise that users can move between categories.

• Before appearing on the panel, I had given some thought as to how to answer the question posed, and here are some notes regarding that - despite the fact that I didn't really get the opportunity to say some of this...
  • Firstly, I would have liked to define what we meant by security in this context. As a very gross separation, we could break it down into 2 separate areas:
    1. The myriad of data, privacy, confidentiality, provenance and traceability issues.
    2. Initial access management (i.e. logging on, identification and authorisation)
    Regarding area #1, these access policies are very hard and the interfaces to them necessarily complex. For example, displaying categories of data or activities and blocking some users' access is difficult to present, and will give rise to many usability (and understandability) issues. It's unavoidable, but there should be ways of making the interfaces as friendly as possible. This particular 'area' is so huge that I imagined that we wouldn't tackle it in our short panel session. However, see my notes below, and especially my reaction to Marina Jirotka's assertion of the 'messy' (or informal) real world, which made the discussion a lot more interesting, Regarding area #2, I'd say that this is just another usability factor. If we wanted to give the user a lot of options, s/he may see this interface as complex and possibly unnecessary, just like any other complex interface that the user would prefer to be more "google-like"! If you have an on-line application that does something useful for the user with one click of the mouse, adding security will involve at least two clicks. Therefore, it impacts usability. The problem with grid middleware is that the impact on usability for this step is far too high: a huge amount could be done to make it easier.
  I managed to say that I thought that it was appropriate to categorise users, depending upon what they were doing, and to present them with an (initial) access management interface that was appropriate for their level of use. Non-IT-savvy users who are doing basic things require a basic interface. Power users who are doing advanced things can cope with a more sophisticated (or onerous) interface/experience. I also mentioned that there is an endemic expectation, held by both non-technical users and by developers, that security experiences should be hard or onerous. This is widespread and it also dampens our resolve to improve the terrible usability associated with security.
  • I could use direct examples of how identity management via digital certificates in the UK e-Science community is conducted poorly in an over-centralised (and insecure) fashion, but let me give a better example. If you work in an organisation that demands that you have a security pass to get through every doorway, you may find that you have to go to distant 'Central Security' to get a security pass. This sounds sensible, and more secure, as the 'Central Security people' will have been on training courses. HOWEVER, to issue a pass to a new member of staff in your group, they will probably require you, or the head of the group to vouch for them when issuing the pass. So, why couldn't you - or your group head - issue the pass directly? The onerous trip over to central security (and the delays that usually occur in issuing the passes) seem good security, but they are not. They are often the opposite. What usually happens is a culture develops of lots of people being in the building who are waiting to receive their security passes, and everyone else holds the doors open for them because everyone else recognises the widespread lack of passes to honest, decent individuals. Or (with the parallel of on-line applications), people just choose not to work there, as it's too hard to get started.
  I was really interested with Marina highlighting the "informal" real world where people often break security policies in order to 'do their job' where the breaking of the catch-all policy is justified. Her example was a clinician showing a fellow clinician a patient's data (where clinician #2 should not have been authorised) because, in this case, it is clearly in the patient's best interest to obtain a second opinion or diagnostic advice. I'm fascinated by the idea that, as ICT systems are designed to be inflexible (as often flexibility=poor security), the hard coding of security/authorisation represents the formalisation of the real world's security policies (or someone's interpretation of them). In the real world, we can break the policies, but - more importantly - they are often not formalised at all. Thus flexibility in security policy may be a fundamental requirement. After someone from the audience mentioned the scenario of a nurse forgetting his password and another nurse 'lending' her credentials to him (with the assumption of this being bad), we talked about delegation. I gave the example of a nurse leaving her store-room key in her car and being lent another nurse's store-room key temporarily. This is fine in the real world because there is a degree of trust or confidence in people realising their responsibilities. Modern sophisticated software should therefore model the real world a little better than it does, with respect to security. The panel discussed this briefly and identified that - in many cases - an audit trail was more important than an absolute blocking of access to functionality or devolution of authorisation (or even identity management) to others. There are strong arguments for some 'systems' allowing one user to grant access to another, but a flag is set for the investigation of this 'breach of security' later. Very interesting concepts. End users should be more vociferous in demanding more usable and more appropriate security. But the issue of 'formalising' existing (and often unspoken) security policies remains very difficult, and potentially divisive.