The plan
How exactly do I expect the SWORD2 application to work? From the point of view of a Sakai user, I think that the first version should do the following:- add a new action to the drop down menu which appears in a resources page, which would have a configurable name but basically be "Submit to archive";
- when files and/or directories are selected, this action may display a metadata form which would need to be completed to continue to submit;
- the items chosen are submitted using SWORD2.
So the development tasks consist of:
- work out how to add a new action and give it configuration information;
- work out how to pick up pointers to the selected items and display an error if none are selected (and a list of what has been selected and a confirmation that the list is correct);
- work out how to create a new form for the metadata and configure its contents and when it is displayed;
- integrate with the SWORD2 java library for deposit using configured information about the archive (and obtain any other information about the submitted items which is required by SWORD2 for deposit);
- install patch on server as well as on laptop;
- test against at least EPrints and DSpace.
Figuring Things Out and the First Code Updates
The first thing to do is to figure out where the specific action drop down is actually set up. The easiest way I can think of to do this is to carry out a very simple search of the source code. There are 15,826 files in the source code, according to eclipse, so there may well be lots of hits for words likely to be common in the source code of any application, such as "action". Eclipse does provide a comprehensive interface to search the contents of the files, but it does't appear to do anything really useful for a large number like this, such as indexing them. (It's probably possible to set this up, maybe with an eclipse interface to a tool such as apache's solr, but I don't know how and I'm not sure it would be worth it for a small task like this.) Running a search goes through about 8,000 files and then creates a pop-up which tells me "Problem occurred with search", with a button "Show details" which freezes eclipse when pressed. So this is less useful than it appears to be.Linux has a large number of command line tools which can be used to search files. There are file indexers around, but I don't run one on my desktop as the indexing process is usually slow and memory intensive. Other tools are rater complicated to use - the find command, for example, has a syntax I can never remember how to use properly even though I first came across it in 1990 in a Unix version which is close to that still available in Linux in 2012 and which appears in several shell scripts I have written over the years. I know what the name of the graphics file which is used to display the button which expands into the drop-down menu I want to change is (because I've found it in the web page source code): icon-dropdn.gif. So I can search for that in the source code, as it's likely to appear near the place I want to work on. (I actually did this before, when trying to get the resources tool to work properly.) Then a simple series of greps finds the file I want in tomcat:
% cd $CATALINA_HOME/webapps
% grep "icon-dropdn.gif" */vm/*
% grep "icon-dropdn.gif" */vm/*/*
sakai-content-tool/vm/content/sakai_filepicker_select.vm: img alt="$tlang.getString(" border="0" class="dropdn" gen.add="gen.add" icon-dropdn.gif="icon-dropdn.gif" nbsp="nbsp" sakai="sakai" src="#imageLink(" /
sakai-content-tool/vm/content/sakai_filepicker_select.vm: img alt="$tlang.getString(" border="0" button.add="button.add" class="dropdn" icon-dropdn.gif="icon-dropdn.gif" nbsp="nbsp" sakai="sakai" src="#imageLink(" /
sakai-content-tool/vm/content/sakai_resources_list.vm: img alt="$tlang.getString(" border="0" button.add="button.add" class="dropdn" icon-dropdn.gif="icon-dropdn.gif" nbsp="nbsp" sakai="sakai" src="#imageLink(" /
sakai-content-tool/vm/content/sakai_resources_list.vm: img alt="$tlang.getString(" border="0" button.actions="button.actions" class="dropdn" icon-dropdn.gif="icon-dropdn.gif" nbsp="nbsp" sakai="sakai" src="#imageLink(" /span class="Apple-tab-span" style="white-space: pre;" /span>
sakai-content-tool/vm/content/sakai_resources_list.vm: img alt="$tlang.getString(" border="0" button.add="button.add" class="dropdn" icon-dropdn.gif="icon-dropdn.gif" nbsp="nbsp" sakai="sakai" src="#imageLink(" /
sakai-content-tool/vm/content/sakai_resources_list.vm: img alt="$tlang.getString(" border="0" button.actions="button.actions" class="dropdn" icon-dropdn.gif="icon-dropdn.gif" nbsp="nbsp" sakai="sakai" src="#imageLink(" /
(I searched the vm directories because they contain the files which generate the interface for Sakai; I've removed some of the HTML markup from the output lines as a quick way to get the results to display properly in blogger.) The results make it clear that the file we want ends up as sakai-content-tool/vm/content/sakai_resources_list.vm. But what is the file which is used to generate this one in the source code? There is no sakai-content-tool directory there. For this, a very simple find command will do the job perfectly:
% find work/sakai-src -name "sakai_resources_list.vm"
./work/sakai-src/content/content-tool/tool/target/sakai-content-tool-2.8-SNAPSHOT/vm/content/sakai_resources_list.vm
./work/sakai-src/content/content-tool/tool/src/webapp/vm/content/sakai_resources_list.vm
and there we have it: work/sakai-src/content/content-tool/tool/src/webapp/vm/content/sakai_resources_list.vm. Now to actually look at this file in eclipse - but it turns out that the listing in eclipse doesn't follow the directory/file structure of the source code itself and doesn't contain the vm file at all (I should perhaps have expected this, but it's about five years since I last used the IDE). And even on a fairly new laptop bought specifically to have as much RAM as financially possible (3 GB), it's using 50% of the available memory just sitting there open but doing nothing. For reading the source code files, it's definitely going to be easier to use a normal text editor, of which my choice tends to be gedit. So that's where I now open the file. (Note: I will want to create a patch eventually, so I'm going to need to have two copies of the source code, as well as the one in the Eclipse workspace so that the changes can be extracted.)
Now, vm files are not a type of HTML generating script I've used before (the older JSPs being more familiar in my Java development experience), so I want to do a little work understanding the format before going on with this. A quick search on filesuffix.com tells me that the program used to parse them is Apache Velocity, and on that site is a user guide.
While the graphic was useful to find the file, and indeed the part of the file which generates the drop down list which I want to alter, it doesn't do any work of itself. The graphic appears several times on each row of the table containing the directories and files managed by the resource tool, with the various menus being different types of actions, such as ones which are only applicable to folders (e.g. create new folders) and some to both files and folders (copy/paste, etc). Two of the drop down lists apply to single items at a time (Add, Actions) and appear next to each item in the list, rather than letting the user select a group of items and then carry out an action on all of them at once. However, what I want is more an action which is applied to every selected item in the table, whether it is a file or a folder, and these appear at the top of the table, greyed out unless at least one item is selected. (This set of buttons consists of text, rather than a drop down list.) The buttons are "Remove", "Move" and "Copy", by default.
A bit of fiddling with "Inspect Element" in the web browser (a really useful feature for understanding complex web pages), and I see that the relevant lines of code are those near the HTML class definition "navIntraToolLink viewNav" (line 178), which displays a button for each member of an array called listActions. This is not itself set in this vm file, but comes from the relevant Java class which is listed at the top of the file in a useful comment: "sakai_resources_list.vm, use with org.sakaiproject.content.tool.ResourcesAction.java". So that is the next file to look at, and here it is possible to see the array being created and populated (lines 4315-4336). It's a little difficult to work out what these lines do without a significant amount of context, as they basically turn one type of list into another, and the file is really too big to be readily understood without a great deal more commentary (I've removed much of the spacing to make the code display better in a fairly narrow style of blog post):
ContentCollection collection = ContentHostingService.getCollection(collectionId);
ListItem item = ListItem.getListItem(collection, null, registry, need_to_expand_all, expandedCollections, items_to_be_moved, items_to_be_copied, 0, userSelectedSort, false, null); MaplistActions = new HashMap (); List items = item.convert2list(); for(ListItem lItem : items){ if(lItem.hasMultipleItemActions()){ for(String listActionId : lItem.getMultipleItemActions().keySet()){ ServiceLevelAction listAction = registry.getMultiItemAction(listActionId); if(listAction != null){ listActions.put(listActionId, listAction); } } } }
This then indicates that the I need to understand the ListItem class to work out how this is populated, because this is what is used to create the initial list which is then manipulated. This class is defined in the sakaiproject/content/tool/ListItem.java file, another 4000+ lines of code with fairly minimal commenting. I worked through the relevant code (the getListItem method starts on line 151) adding temporary comments to ensure I remembered what I had already worked out. The first important bit is to work out what permissions the user has over the items in the collection, which in fact has an extremely useful comment:
* calculate permissions for this entity. If its access mode is * GROUPED, we need to calculate permissions based on current user's * role in group. Otherwise, we inherit from containing collection * and check to see if additional permissions are set on this entity * that were't set on containing collection...
and it's a sensible way to do it, dynamically changing the list of actions depending on what the user is allowed to do, but isn't something that I'd thought of. Stupid of me, especially as access control is my main field of expertise... So the place to look is not here at all, but the code which lists the permissions available to be set. The form which does this looks like this in a default setup (presumably the creation of custom groups of users would add new columns):
So to find the vm file which generates the table, I need to find a fairly distinctive looking piece of the HTML source for this table. The name attribute of the form tag, "permissionForm" is an obvious one, and finds me the authz/authz-tool/tool/src/webapp/vm/helper/chef_permissions.vm VM file. The line which generates the rows of the table, which correspond to actions and which we want to add to, is line 65:
#foreach($lock in $abilities)
so next I need to find out where the list $abilities is set. Unlike the VM I looked at earlier, this one doesn't have a useful comment indicating which java file is responsible for its processing, but I think I can assume it will be authz/authz-tool/tool/src/java/org/sakaiproject/authz/tool/PermissionsHelperAction.java. Having done so, I can see that it is from line 120:
private static final String TEMPLATE_MAIN = "helper/chef_permissions";
The abilities variable is exported to the VM in line 385, which is set between lines 368 and 382 from a list of functions (potentially filtered, which is why it takes 14 lines of code). The code is set by lines 357-363:
// in state is the list of abilities we will present
List functions = (List) state.getAttribute(STATE_ABILITIES);
if (functions == null)
{
// get all functions prefixed with our prefix
functions = FunctionManager.getRegisteredFunctions(prefix);
}
which will first get data from the class attribute STATE_ABILITIES, and, if this is empty, it will ask the FunctionManager for a list of registered functions. I suspect that it is the latter which is needed, as the script then goes on to write the list of functions back into STATE_ABILITIES. The function manager is defined in the class org.sakaiproject.authz.cover.FunctionManager, so that is where I need to look next. However, finding it is more problematic. There is no file in the source code named FunctionManager.java, and no file containing a class definition for FunctionManager. However, I did eventually find some documentation on the use of the FunctionManager which seems to explain how I should add new functions (it appears when searching using google for "sakai FunctionManager" but not when searching the Sakai WIKI through Confluence's own search box...). It's clearly out of date (linking to non-existent files), but I hope still helpful. I also found a document in the reference documentation section of the source code, reference/docs/architecture/sakai_security_reg.doc, which explains how the function manager works. However, what I really need is an overview of how the security in Sakai works, and I eventually found a useful collection of WIKI pages, https://confluence.sakaiproject.org/display/~markjnorton/Sakai+Security+Model and the four linked to from it.
My understanding of the Sakai security model is now this. There are four components: users, (permission) groups, functions, and objects. New groups can be created (using the Security Service); users can be placed in groups (using a GroupProvider); functions can be associated with objects (using the function manager); and users or groups can be granted permission to carry out a function (using a Permission Helper). Resolution of a user's permission to carry out a function on an object is done by the AuthzGroup service.
So what I need to do is to add a new function via the ListItem.java script (not the Permissions Helper as I previously thought) and work out how to add that permission to the owner of a collection of files. The function manager service documentation tells me that the preferred method to do this is by using Spring to inject the service, via an XML file; however, I quickly discover (with my old friend grep - in this case, the command "grep -R --include *.xml FunctionManager .") that there are no XML files which include the string FunctionManager in the Sakai source. So: should I be good and follow the documentation (last edited in 2010), or should I do this the way that everyone else seems to have done? In fact, it looks on close inspection as though the content manager tool doesn't do things like this at all, because it inherits its permissions from elsewhere (and I've now basically gone round a circle to ResourcesActions.java again). But this time I have more idea what I'm looking for, and can actually start making changes. I'm not sure how or indeed if I can create a configurable name for it, however.
In what follows, line numbers refer to the line number displayed in the editor as the file is changed, so once a line has been added, all those with higher numbers are one greater than they were in the original file. Don't worry: at the end of this process, I'll publish a patch file with all the changes included.
Change line 348 to have ARCHIVE as a member of the list of actions:
CREATE, DELETE, READ, REVISE, SITE_UPDATE, ARCHIVEAdd new line 415 to be similar to the other actions here:
public static final ListCONTENT_ARCHIVE_ACTIONS = new ArrayList ();
There probably need to be some constants set in this section to govern the behaviour of the archiving process, but at the moment I'm not entirely sure what they should be. Perhaps one to indicate the requirement for metadata, and another to give the status of the archiving process (similar to line 510, "private static final String MODE_DELETE_FINISH = "deleteFinish";" - if that is indeed what this constant indicates!). I have to revisit this. I could really do with finding proper javadocs for this part of the code, but Sakai is exceptionally unhelpful for this. For example, https://confluence.sakaiproject.org/display/DOC/Sakai+CLE+2.8+Release+Notes lists some sources for javadocs, but there are separate javadoc locations for each of the projects which make up sakai, and chasing links in from this list ends up at 404 not found errors in many cases. The Sakai project is desperately in need of a tidier documentation collection for developers, but creating one will be an enormous job. Looking at way that MODE_DELETE_FINISH is used later on, it is used to set up what is displayed after the deletion has occurred, and I need some sort of equivalent, a message indicating the archive submission has been made, as well as a similar confirmation message. So I add a new line 512 and 514 (with a blank line between them):
private static final String MODE_ARCHIVE_FINISH = "archiveFinish"; private static final String MODE_ARCHIVE_CONFIRM = "archiveConfirm";There is already a "MODE_REVISE_METADATA" constant, which appears to make Sakai display a metadata form, though I presume that this is for a single resource at a time rather than for a collection. So I'm going to want to have a mode for adding archive metadata, which forms a new line 531:
protected static final String MODE_ADD_ARCHIVE_METADATA = "add_archive_metadata";
Further down, there are more constants to add, with comments similar to those already there (lines 605-11):
/** The name of the state attribute containing a list of ListItem objects corresponding to resources selected for submission to the archive */ private static final String STATE_ARCHIVE_ITEMS = PREFIX + REQUEST + "archive_items"; /** The name of the state attribute containing a list of ListItem objects corresponding to nonempty folders selected for submission to the archive */ private static final String STATE_ARCHIVE_ITEMS_NOT_EMPTY = PREFIX + REQUEST + "archive_items_not_empty"; protected static final String STATE_ARCHIVE_SET = PREFIX + REQUEST + "archive_set";
This is proving quite complicated. I think that, if I had more time, I'd probably want to build a generic method for adding new actions, by configuring the action name and a Java class (implementing some interface, say) to carry it out. The difficulty with that approach would presumably be how to handle actions which need to take the control away from the current page, as happens with deletion confirmation and with the archive metadata I'll need.
Next, I will set the VMs to use to handle the deposit confirmation, completion, and the metadata form, at line 785.
private static final String TEMPLATE_ARCHIVE_FINISH = "content/chef_resources_archiveFinish"; private static final String TEMPLATE_ARCHIVE_CONFIRM = "content/chef_resources_archiveConfirm"; private static final String TEMPLATE_ARCHIVE_METADATA "content/sakai_resources_archiveMetadata";
The need to set these in the code rather than making them configurable details seems rather poor design to me, but never mind.
Still more constants need to be added. Archiving should make no change to the resources themselves, so it is an action which should be in the same category as copy. So at line 816, I add:
CONTENT_READ_ACTIONS.add(ActionType.ARCHIVE);
and at 836:
ACTIONS_ON_FOLDERS.add(ActionType.ARCHIVE);
and at 850:
ACTIONS_ON_RESOURCES.add(ActionType.ARCHIVE);
There's a lot more to do, but that will have to be in the next post, I think.
Posts
No comments:
Post a Comment