LTGPublicWiki: Diff for "OxCLIC_MDID"

OxCLIC_MDID

TableOfContents([2])

Overview

MDID is a web application designed to deliver digital images for Teaching and Learning. It was designed and developed at James Madison University and has been continually refined since 1998. It is used by around 80 colleges and made available under an open source GPL license.

From the MDID site http://mdid.org/

"The Madison Digital Image Database is an online image database and multimedia instructional system designed to create and show Internet-based lectures using digitized images. The system permits instructors to remotely generate "slideshows", which can be annotated, placed online for student study, or archived for testing or future use. Though many image databases provide flexible faculty and student access to online images, those systems generally do not provide a tool via which faculty can teach and student can learn. MDID brings the digital image and data library into the teaching and learning process, in and outside the classroom."

• Remote Collections
• Personal Images
• IPTC Injection

• ImageViewer:classroom application

• Integration with RLG CAMIO
• Image Moderation
• Support for multiple collections
• Custom catalog data structures
• Search and browse functions
• Cross-collection searching
• Personal collections in "My Images"
• Slideshow light table
• Tools for managing slideshows
• User image notes and annotations
• Web-based slideshow viewer
• Packaged slideshows for offline presentation
• Printable flashcards
• Tools for managing user accounts and authentication
• Data exchange through XML

OxCLIC MDID resources

• Setting up MDID on the server ["OxCLIC_MDID_server"]
• An overview of importing material catalogued in Portfolio into MDID - ["OxCLIC_MDIDimporting"]
• Customising MDID look and feel - ["OxCLIC_MDID_customising"]
• Connecting between collections at Oxford - ["OxCLIC_remote_collections"]

Further References

• The main MDID website wiki:http://mdid.org/

• A-Z list of all pages/topics on MDID - http://mdid.org/mdidwiki/index.php?title=Special:Allpages

• The MDID web site has a series of [:http://mdid.org/mdidwiki/index.php?title=Tutorials:tutorials], online videos and guide books for both user and administrator.]

• http://mdid.org/mdidwiki/images/b/bb/MDID2CuratorWorkshop.pdf

Importing images and catalogue metadata into MDID

Material can be imported on an image by image bases via the broswer. However this process is too slow and time consuming for more than a few odd images. MDID has support for multiple images and their catalogue information being imported through a simple batch off-line import process on using MDID tools on the server. The MDID Curator Workshop manual, section 3 and 4 explains the step by step process for doing this. Briefly the steps are

• Create a new collection in MDID
• Convert the .csv file to XML via a MDID helper tool
• Import the XML file into MDID
• use the MDID image importer tool to import the images

The Curator Workshop manual

The Curator Workshop manual available for the MDID web site is a key guide for importing, administrating and maintaining collections in MDID.It covers the step by step process of converting and moving a .csv metadata catalogue (e.g. Exported from Portfolio) and the associated images into MDID.

Links

The Curator workshop manual - http://mdid.org/mdidwiki/images/b/bb/MDID2CuratorWorkshop.pdf

The process of importing a catalogued collection in Portfolio is also covered in the OxCLIC Importing document, ["importing"]

An visual overview of how image material can be added to MDID

attachment:OxCLIC_Workflow.jpg

Suggestions for work by OxCLIC to improve MDID

Work for ACDT

There is a fair amount of IIS and ASP work that could be done locally.

Investigate running MDID under WebAuth, probably using the new IIS module rather than the proxy solution http://www.oucs.ox.ac.uk/webauth/index.xml.ID=body.1_div.3 WebAuth. This requires that we:

• Purchase a blade server with a large hard drive
• Install a windows server on it.
• Install MDID onto the server

• Configure and test WebAuth under IIS

• Modify MDID such that it utilises WebAuth (login page accessed by https, remaining pages accessible by standard http as per WebLearn - I have no idea whether this is remotely possible under IIS or ASP )

• Determine whether there is any point in drawing on the oxford LDAP information, and if so implementing this ( or whether we can make this information available to departments outside OUCS)
• User testing of the MDID interface to determine its useability and accessibility. This should probably be the first task undertaken, to see how well end users are able to comprehend MDID.
• Perhaps rework the search interface to be a bit more friendly to end users rather than cataloguers.
• Add in thumbnails to search/browse results. This should be a quick and easy task.
• Add a 'browse by thumbnail' option.

• Investigate the feasibility of guest access to the collection by default, offering unregistered users access to items in catalogues based upon each item's Rights field. This will be tied in with the first item above, WebAuthing the server.

• Produce a wizard for publishing a slideshow/collection from the private sphere to the public sphere, based upon a Rights field within the collection metadata as above.

Work for MDID Developers

rewrite the import script to generate better XML files.

MySQL database, table and column names have certain restrictions:

max Length 64 characters May not end with a space, and may not contain the characters: ‘/’, ‘\’, ‘.’

The utility currently converts field names from the CSV file into XML element names. XML element names have a different set of restrictions:

• Names can contain letters, numbers, and other characters
• Names must not start with a number or punctuation character
• Names must not start with the letters xml (or XML, or Xml, etc)
• Names cannot contain spaces
• Names cannot contain the : (colon) character as this has a special meaning.

This means that it is inappropriate to convert column names that could contain spaces and other characters into XML entity names. Rather the solution would be to do the following

CSV file: Title,Description,ID,File Name,DC:Date.created,Author/Creator,Catalogued By, etc.

The headers should be made MySQL-safe by removing / \ and . from the column names and limiting the length to 64 characters. The whole file should be made XML safe by converting < > & to < > and & respectively.

• <data>

  • <row>

    • <column>

      • <name>Title</name> <content>The Christmas Album</content>

      </column> <column>

      • <name>Description</name> <content>Textual description & information</content> # & converted to &

      </column> <column identifier="Y"> # use an attribute to show that this is the Identifier field?

      • <name>ID</name> <content>ARTH_aa9999</content>

      </column> <column resource="Y"> # use an attribute to show that this is the resource field?

      • <name>File Name</name> # spaces are allowed in MySQL column names. <content>ARTH_aa9999.jpg</content>

      </column> <column>

      • <name>DC:Date_created&lt;/name&gt; # . converted to _ <content>Not > 50 years ago</content>

      </column> <column>

      • <name>Author_Creator</name> # / converted to _ <content>Nana Mouskouri</content>

      </column> <column>

      • <name>Catalogued By</name> # spaces are allowed in MySQL column names. <content>Jonathan's Mum</content>

      </column>

    </row>

  </data>

MDID Work Packages

Work Packages for ACDT

• Installing it on a new OUCS server
• Authentication
• Authorisation
• Customisation
• External work by the US MDID developers

MDID under WebAuth

Any authentication work needs to be designed and documented in a way that the goal is a department or division can install MDID and the Authentication work as a complete package.

Thoughts on authentication and WebAuth

We need to work out the best practical solution to getting the system working under Oxford Authentication in a manner that is sustainable and could be recreated with a departmental installation. Authorisation information is harder to achieve in a devolved manner as there is no access to lookups of the LDAP from outside of the OUCS domain.

Installation on a high end server.

It would be sensible to buy and install a new server, possibly a Blade for all this new work and try and replicate the WebAuth setup and MDID work on another departmental server too.

Authentication

Method a) Using WebAuth proxy server. As MDID needs to run under Windows IIS server which doesn't support the normal WebAuth Apache module then the workaround is to use a "proxy server" that passes the username in the URL. See

This might mean two stages,

• setting up with the SysDev team the proxy server

• writing as piece of code to pull in the webauth username variable. Karl Harrison and Robin McCleary have both done this IIS hack on their web databases.

Method b) MDID has a way of using what it calls "SSO" but is really configuring it to expect the username in a URL with a security token from a customised php script that is under say a WebAuth server. See attached my attached graphic. .pdf

• MDID Authentication documentation

<http://mdid.org/mdidwiki/index.php?title=Authentication_section>

• MDID Single Sign-ON mechanism

<http://mdid.org/mdidwiki/index.php?title=Single_Sign-on_Integration>

Method c) IIS WebAuth solution

Stanford have a IIS WebAuth solution. ( Update Aug 2006 tested by ACDT )

Authorisation

LDAP work

Some work could be done on pulling in LDAP user ID information after gaining the WebAuth username for the test system.

Is there a way that we could pass any group or attribute details from the LDAP from within OUCS or if the server is located outside of OUCS. Yes, perhaps using the single sign-on method built into MDID. This trusts another server to be the authentication stage and to pass to it the username and IT attributes such as roles. If this trusted server was in the OUCS domain then it could have access to the LDAP details. The mechanism suggests that this trusted MDID auth server would be able to support multiple instances of MDID. this federated set-up might just be to difficult to support in terms of resources and expertise and the central model might win out.

MDID has an LDAP authentication configuration script where you specify the IP address of the LDAP server but at first glance it looks like it uses LDAP to cover both authentication and authorisation. Can we decouple it presuming the authentication is done elsewhere, could this be built in as a config script for the application.

Customisation

a) Notes and Annotations / Browsing and Searching

Gathering user requirements might discover some needs here especially for the non-HOA users. They may wish to template out the search display in some way. Although I believe each individual collection has different fields displayed depending on the preferencess for those fields.

b) Video and multimedia support

We need to gather some Oxford requirements for video and audio and see how they map to the new features in this area coming shortly within MDID

The developer has indicated that there is a draft version with support for media player on the website for pulling in Podcasts from another server.

c) Linking to Weblearn

Is there any work to be done here? SSO obviously helps, simple links between the two would help, the slideshow links can be placed in the VLE for student access. What would be a standard use case for this.

d) The ability to have a public open version a collection would appeal to research projects who want a quick fix for getting visual material such as research collections or digitisation projects on the net with open access and with a search interface. At the moment you can do this with MDID as guest access to the collection but there isn't a thumbnail display view once you are in, just a Search and/or Browse view.

External work by the MDID developers

a) Adding authentication config script to the bundled version of MDID that tells it to check the WebAuth usernname session variable to see if someone is login in. i.e when they click login it uses a different script for checking credentials. (LDAP look up is difficult here in a federated model.)

MDID under Linux/Mac OS X: Fun and Games

It is apparently possible to run MDID under Linux by making use of the http://www.mono-project.com/ MONO software. MONO is a port of ASP to Linux, and includes an apache module mod_mono that will allow ASP pages to be run under apache httpd.

The basic process should be as follows:

1. Obtain and install MONO for your platform I used the Mac OS X Framework installer, and it worked very nicely, and includes xsp, so you don't need to worry about installing that.
2. Install mod_mono into your current apache setup. This was essentially a case of download, ./configure make and make install, I just had to point it to the appropriate apxs ( /opt/apache2/bin/apxs under OSX ) and apr-config (/opt/apache2/bin/apr-config) and tell it where to install ( /opt/apache2 )
3. Install nant on your system (requires pkg-config to be installed, and that you set PKG_CONFIG_PATH=/path/to/your/mono/installation/pkgconfig/directory and export $PKG_CONFIG_PATH before running make
4. The current runtime framework 'mono-2.0' is not correctly configured in the NAnt configuration file.:

   NAnt.Core.BuildException: The current runtime framework 'mono-2.0' is not correctly configured in the NAnt configuration file. ---> Unable to locate 'mono' module using pkg-config. Download the Mono development packages from http://www.mono-project.com/downloads/.: NAnt.Core.BuildException: Unable to locate 'mono' module using pkg-config. Download the Mono development packages from http://www.mono-project.com/downloads/.

I also had to edit the nant Makefile:

• install: bootstrap # $(NANT) -f:NAnt.build install -D:install.prefix="$(prefix)"
• $(NANT) -f:NAnt.build install -D:install.prefix="/usr/local"
• * unpack MDID under your apache setup

• set config.xml <database><upgradeable>true</upgradeable></database>

• use the 'nant' tool to compile MDID into the necessary dll files by running 'nant go' in the MDID directory. I am currently trying to sort out the following error:

  • [csc] Starting '/Library/Frameworks/Mono.framework/Versions/1.1.13.8/bin/mono ("/Library/Frameworks/Mono.framework/Versions/1.1.13.8/lib/mono/2.0/gmcs.exe" @"/var/tmp/tmp4b361d76.tmp")' in '/Library/WebServer/ASP/MDID/lib/packages' [csc] /Library/WebServer/ASP/MDID/lib/packages/seclib-1.0/seclib/Security/Ssl/Tls1/ExpansionDeriveBytes.cs(116,11): error CS0104: HMAC' is an ambiguous reference between System.Security.Cryptography.HMAC' and `Org.Mentalis.Security.Cryptography.HMAC' [csc] Compilation failed: 1 error(s), 0 warnings BUILD FAILED - 0 non-fatal error(s), 1 warning(s) /Library/WebServer/ASP/MDID/lib/packages/default.build(89,4): External Program Failed: /Library/Frameworks/Mono.framework/Versions/1.1.13.8/lib/mono/2.0/gmcs.exe (return code was 1): NAnt.Core.BuildException: /Library/WebServer/ASP/MDID/lib/packages/default.build(89,4): External Program Failed: /Library/Frameworks/Mono.framework/Versions/1.1.13.8/lib/mono/2.0/gmcs.exe (return code was 1) in <0x00454> NAnt.Core.Tasks.ExternalProgramBase:ExecuteTask ()

This appears to be because nant is building against the mono 2.0 framework, rather than the 1.0 framework. The only way that I can see to alter this setting is by editing Nant.exe.conf and changing <platform name="unix" default="auto"> to <platform name="unix" default="mono-1.0"> and then running "nant go' again - all seemed to work.

• use apache or mono's xsp webserver to configure the database via a web browser... unfortunately after entering the username and password Apache hengs, or returns a 500 Error, and trying the same thing under xsp also gives a hang or a 500 error, and the following output
  • ancalagon:/Library/WebServer/ASP/MDID/wwwroot jmiller$ xsp xsp Listening on port: 8080 (non-secure) Listening on address: 0.0.0.0 Root directory: /Library/WebServer/ASP/MDID/wwwroot Hit Return to stop the server.
  • * (/Library/Frameworks/Mono.framework/Versions/1.1.13.8/lib/xsp/1.0/xsp.exe:4226): WARNING **: _wapi_handle_unref: Attempting to unref unused handle 0x51
  • * (/Library/Frameworks/Mono.framework/Versions/1.1.13.8/lib/xsp/1.0/xsp.exe:4226): WARNING **: _wapi_handle_unref: Attempting to unref unused handle 0x50
  • * (/Library/Frameworks/Mono.framework/Versions/1.1.13.8/lib/xsp/1.0/xsp.exe:4226): WARNING **: _wapi_handle_unref: Attempting to unref unused handle 0x50
  • * (/Library/Frameworks/Mono.framework/Versions/1.1.13.8/lib/xsp/1.0/xsp.exe:4226): WARNING **: _wapi_handle_unref: Attempting to unref unused handle 0x51
  • * (/Library/Frameworks/Mono.framework/Versions/1.1.13.8/lib/xsp/1.0/xsp.exe:4226): WARNING **: _wapi_handle_unref: Attempting to unref unused handle 0x51
  • * (/Library/Frameworks/Mono.framework/Versions/1.1.13.8/lib/xsp/1.0/xsp.exe:4226): WARNING **: _wapi_handle_unref: Attempting to unref unused handle 0x51
  • * (/Library/Frameworks/Mono.framework/Versions/1.1.13.8/lib/xsp/1.0/xsp.exe:4226): WARNING **: _wapi_handle_unref: Attempting to unref unused handle 0x51

    System.Threading.ThreadAbortException: Thread was being aborted in <0x001d8> System.Web.UI.Page:ProcessRequest (System.Web.HttpContext context) in <0x00000> <unknown method>

  in (wrapper xdomain-dispatch) Mono.WebServer.XSPApplicationHost:ProcessRequest (object,byte[]&,byte[]&,int,long,int,long,int,string,string,string,string,byte[],string)

  • in (wrapper xdomain-invoke) Mono.WebServer.XSPApplicationHost:ProcessRequest (int,long,int,long,int,string,string,string,string,byte[],string,intptr,Mono.WebServer.SslInformations) in (wrapper remoting-invoke-with-check) Mono.WebServer.XSPApplicationHost:ProcessRequest (int,long,int,long,int,string,string,string,string,byte[],string,intptr,Mono.WebServer.SslInformations) in <0x006f4> Mono.WebServer.XSPWorker:InnerRun (System.Object state) in <0x0003c> Mono.WebServer.XSPWorker:Run (System.Object state)

  • * reset config.xml <database><upgradeable>false</upgradeable></database>

• make use of MDID under your apache setup (I wish )

Importing a catalogue from portfolio to MDID ...more fun and games

In Portfolio you should select the records for which you would like to export the metadata. Choose File -> Export Field Values. You will then get to choose which field values you would like to export to a tab-separated text file.

You will need to open the file up and convert it to CSV. This is most conveniently done using a spreadsheet such as excel.

Then you will need to manipulate the first row of the file, which contains the headings from the Portfolio catalogue. You will need to replace certain characters (in the first line only) for the inport filter to be able to proceed (see Section 5.2 above ).

• spaces with an underscore _
• colons : with an underscore _
• semicolons : with an underscore _
• parentheses : with an underscore _

• ampersands & with an underscore _

If you do not remove these characters, any column that contains such a character will be discarded silently by the convert.exe program, and will not be imported into MDID. It is arduous to add the data in later, so you must get this operation correct right now.

The file: [MDID2 Curator's Workshop .pdf http://mdid.org/mdidwiki/images/b/bb/MDID2CuratorWorkshop.pdf] gives instructions from this point onwards, and is quite good.

Locate the program convert.exe, which comes with the MDID2 Tools download (avaliable from sourceforge with the MDID2 server software). Run the program (does not seem to run over remote desktop on Windows server 2003) and point it at the csv file that you have just created. Apparently it is possible to work from an Excel worksheet instead of a CSV file, but I have not tested how well this works.

There are a number of settings that you need to make before you generate the XML file.

Choose the Identifier field (this should be the accession number e.g. arth_aa1994) and Resource field (this should be the filename e.g. arth_aa1994.JPG ) from the drop down menus.

Under Options you should choose to split field values at newline characters rather than semicolons.

Click Start Conversion and you will be prompted for the name and destination of the XML file.

The next step is to log into the MDID2 application and create a new collection to hold the images and metadata. Fill out the name, Description and image path (which you will presumably have to know by this stage) Then click on Field Definitions at the top of the page. Choose to get your field definitions from the XML file you have just created, and navigate to locate it.

Next you choose Import Data

Choose your XML file again, and click the import data button.

The page will refresh, showing you the progress it has made in importing your data.

{{{Import Status

The last data import for this collection successfully finished. 1107 of 1107 records processed: 1107 records added 0 records replaced 0 records already existed and were skipped 0 records did not have an identifier and were skipped }}} Assuming this completes satisfactorily you now need to deal with the images in your collection.

On the MDID server, run Imagemanager. Again, it does not seem to work over remote desktop on our blade so I am stuffed at this point.

Cross your fingers and hope it worked.