Writing Custom Save Logic for ItemFileWriteStore

This post focuses on a question reincarnated most recently by cbarrett1 in the Dojo IRC channel:

• How do I implement custom save logic on an ItemFileWriteStore?

This was always something I wanted to become familiar with, and upon initial inspection it seemed easy enough to implement an example. It’s also a prime opportunity to put across a few other general points as well. While the main intent of this post should be clear, it also briefly touches upon various other topics along the way, so hopefully there’s a little something for everybody who’s learning Dojo.

Note: as always, I’m foregoing links for the most part until the Further Reading section at the end of the article, and there will be plenty there this time around for those interested.

dojo.data.ItemFileWriteStore

The class dojo.data.ItemFileWriteStore implements all four of the major dojo.data APIs: Read, Identity, Write, and Notification. (It inherits most of its functionality for the Read and Identity APIs from dojo.data.ItemFileReadStore.) However, it leaves the Write API’s save method mostly unimplemented out of the box. This method is normally responsible for committing modified (“dirty”) data items back to a server endpoint to persist modifications to the data model; however, ItemFileWriteStore makes no assumptions about expectations of server-side logic, and thus does not implement this method (other than mark all items as “not dirty” and fire any onComplete callback passed to the save method).

ItemFileWriteStore doesn’t leave developers high and dry, however; it allows for the addition of custom save logic via one of two optional methods:

• _saveEverything(successCallback, failureCallback, newContentString): can be used to blindly re-push the entire data set back to the server en-masse (the newContentString parameter contains the entire data object of the store in JSON format).
• _saveCustom(successCallback, failureCallback): is expected to iterate through the store’s data internally, performing server requests as necessary. This allows for more flexibility and potentially far greater efficiency (i.e. send data for only the changes), but generally involves more work to implement.

Either of these methods can be simply monkey-patched onto an ItemFileWriteStore instance (any time before save gets called), or could be coded into a new class extending ItemFileWriteStore. Below, I walk through a basic example which involves declaring a class extending dojo.data.ItemFileWriteStore, implementing the _saveEverything method.

Example Implementation

If you’ve got a local web server lying around (if you’re a windows developer like me, I’d suggest xampplite), I encourage you to follow along with the example below if you’re interested. Just create a new directory under your htdocs folder for it and put everything directly under it.

The Data

First of all, let’s determine exactly where and how we’ll be retrieving, and saving back, our data. The following are two ridiculously simple PHP files which respectively read and overwrite a file named data.json in the same folder.

getdata.php

<?php
  header('Content-type: application/json');
  echo(file_get_contents('data.json'));
?>

postdata.php

<?php
if ($_POST['data']) {
  file_put_contents('data.json', $_POST['data']);
}
?>

And here’s some data to get us started. Keep in mind this follows the format understood by ItemFileReadStore and ItemFileWriteStore.

data.json

{
  "identifier": "id", 
  "label": "name", 
  "items": [
    {
      "id": 1, 
      "name": "Foo"
    }, 
    {
      "id": 2, 
      "name": "Bar"
    }, 
    {
      "id": 3, 
      "name": "Zaphod"
    }, 
    {
      "id": 4, 
      "name": "Beeblebrox"
    }
  ]
}

Note: If you’re on *nix of any sort, chances are you’ll need to make data.json writeable by the user your web server runs under. The simplest way to tweak this would be to chmod 666 data.json, but if the idea of making something world-writeable makes you cringe, you could do something along the lines of sudo chown apache data.json, where apache is the user your web server process runs under. If you’re on Windows, it may “just work” without any tweaks necessary.

The Custom Implementation

Now let’s get to the interesting part: the custom implementation, which will actually save our writes to the server. You may have already guessed how this is going to work, but let’s outline it below before looking at any code.

1. Create a class which extends dojo.data.ItemFileWriteStore
2. Define a _saveEverything method within the extension, which will send a POST request to postdata.php via XHR, with the new content to write to data.json

With that in mind, let’s take a look at the example.

Note: the example is presented without comments below for brevity, and I will break it down to explain it afterwards; check the zip file available at the end of this post for a version with inline comments.

dojo.require('dojo.data.ItemFileWriteStore');

dojo.declare('SimpleSaveWriteStore', dojo.data.ItemFileWriteStore, {
  _postUrl: 'postdata.php',
  
  _saveEverything: function(saveCompleteCallback, saveFailedCallback, newFileContentString) {
    dojo.xhrPost({
      url: this._postUrl,
      content: {
        data: newFileContentString
      },
      load: saveCompleteCallback,
      error: saveFailedCallback
    });
  }
});

…Yes, it’s over already. You might be surprised at how little it took, but this is all we need for a simple example that just makes toast. If you’re reading this while relatively uninitiated to dojo, however, there may be a few things going on in here that you don’t fully understand, so let’s step through it.

1. We first dojo.require('dojo.data.ItemFileWriteStore') to ensure that the class we’re extending will be loaded. We can’t extend something that Dojo doesn’t know exists.
2. We then dojo.declare a new class named SimpleSaveWriteStore, extending dojo.data.ItemFileWriteStore, and specify members to include/override in the extension. Any members present in the object passed as the third parameter to dojo.declare will be mixed in to the prototype of the class, on top of the prototype inherited from its parent (in this case dojo.data.ItemFileWriteStore, which itself extends from dojo.data.ItemFileReadStore).
3. We define a _postUrl property, identifying the URL to which we will fire POST requests to persist the modified data. (The leading underscore is a common naming paradigm suggesting this property is for internal use within the class only.)
4. We define the _saveEverything method, which uses dojo.xhrPost to POST data back to the _postUrl. The logic inherited from ItemFileWriteStore’s save method will call this function now that it is present.

Before moving on, I’d also like to take a moment to explain the notion behind the callbacks passed to the _saveEverything method, as these trace back to a significant common feature present in several functions across the dojo data APIs. The dojo data APIs are entirely callback-driven, since implementations are not guaranteed to be synchronous in nature – in fact, it’s far more likely they won’t be, since server communication will often be involved. Many of the most commonly-used functions in the data APIs rely on callbacks to perform logic once data is retrieved or stored, such as:

• the Read API’s fetch method (onItem, onComplete, onError)
• the Identity API’s fetchItemByIdentity method (onItem, onError)
• the Write API’s save method (onComplete, onError)

When writing custom implementations of the dojo data APIs, it is important to be fully aware of the contracts these APIs uphold, and to properly hook up the callbacks that are expected to be supported. ItemFileWriteStore’s custom save handler extension points receive two functions: successCallback and errorCallback, which among other things, delegate to respective onComplete and onError callback functions supplied in the original save call. Therefore, when implementing _saveEverything it’s important to hook up these two callbacks accordingly. We do this in our implementation by simply passing the two callbacks as the respective load and error callbacks in the dojo.xhrPost call.

Test-driving the Example

The above code could easily have been placed in its own js file and dojo.required into a test page, but for the sake of simplicity (and ending this article sometime today), we can put it directly to use in one shot. (Modularizing it isn’t really much work, though in the case of working with a CDN or otherwise cross-domain build it takes a bit more effort; maybe in another post…)

Here is a js file that we can directly source into an HTML page. It includes the above code, as well as some simple code to load up an instance of our SimpleSaveWriteStore using getdata.php to initially retrieve the data, and then display it in an instance of dojox.data.StoreExplorer – a widget that is as useful as it is undocumented, unfortunately.

SimpleSaveWriteStore.js

dojo.require('dojox.data.StoreExplorer');
dojo.require('dojo.data.ItemFileWriteStore');

dojo.addOnLoad(function() {
  
  dojo.declare('SimpleSaveWriteStore', dojo.data.ItemFileWriteStore, {
    _postUrl: 'postdata.php',
    
    _saveEverything: function(saveCompleteCallback, saveFailedCallback, newFileContentString) {
      dojo.xhrPost({
        url: this._postUrl,
        content: {
          data: newFileContentString
        },
        load: saveCompleteCallback,
        error: saveFailedCallback
      });
    }
  });
  
  var explorer = new dojox.data.StoreExplorer({
    store: new SimpleSaveWriteStore({url: 'getdata.php'})
  }).placeAt(dojo.body());
  dojo.style(explorer.domNode, {
    width: '100%',
    height: '400px'
  });
  explorer.startup();
});

There are a couple of things worth pointing out in the additional code in this example.

Notice that everything that references any modules that were dojo.required (in this case, everything beyond that point) is placed in a function passed to a dojo.addOnLoad call (or dojo.ready for you 1.4-savvy and/or jquery-loving types). This is insanely important, and is a point that probably merits reiterating in its own post, not just buried in the middle of this one. Especially when you’re working with cross-domain Dojo, any logic that expects a dojo.required module to be loaded MUST be placed within a function passed to dojo.addOnLoad!

Also notice that I manually call explorer.startup() – since I’m instantiating this widget on its own programatically, nobody else is ever going to tell this thing to move on with its lifecycle. (If you instantiate widgets declaratively, the parser calls startup for you, and parent widgets that function as containers call it recursively on their children.) I also manually add a height style to the widget’s DOM node, since the dojox grid doesn’t tend to decide height for itself (at least not by default).

Now all that’s left between us and a working example is a simple HTML page to throw it all together. I use the Google CDN here, to avoid the need to host dojo locally and muck with file paths. (Since it’s cross-domain, it’s all the more important that the code above is properly nested in a dojo.addOnLoad call.)

index.html

<html>
 <head>
  <title>Custom ItemFileWriteStore Save Logic Example
  <link rel="stylesheet" type="text/css" href="http://ajax.googleapis.com/ajax/libs/dojo/1.4/dojo/resources/dojo.css"/>
  <link rel="stylesheet" type="text/css" href="http://ajax.googleapis.com/ajax/libs/dojo/1.4/dijit/themes/tundra/tundra.css"/>
  <link rel="stylesheet" type="text/css" href="http://ajax.googleapis.com/ajax/libs/dojo/1.4/dojox/grid/resources/tundraGrid.css"/>
  <script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/dojo/1.4/dojo/dojo.xd.js">
  <script type="text/javascript" src="SimpleSaveWriteStore.js">
 </head>
 <body class="tundra">
 </body>
</html>

With this final piece in place, if you point a web browser at the folder where you placed all these files, you should see something resembling the following:

StoreExplorer viewing an instance of SimpleSaveWriteStore, whose data was initialized from getdata.php

Using the StoreExplorer, you can easily manipulate the data in the store:

• Edit existing values by doubleclicking a cell in the DataGrid
• Add a new value by clicking Create New, then entering an item in JSON format; for example, {"id":5, "name":"Slartibartfast"}
• Delete an item by clicking the Delete button
• Save changes by clicking the Save button
• Revert changes since the last save by clicking the Revert button

If you’ve been following along, try messing around with some values and clicking Save. Assuming your web server was able to write to your data.json file, you should see that its content has been updated. Now even if you refresh the page, you should see your data persist! Before ItemFileWriteStore had been told how to communicate changes to the server, refreshing the page would have lost your changes.

Congratulations! You now have a ridiculously simple customization of ItemFileWriteStore that persists its data to the server when save is called.

Encore

Maybe some of you are curious (or skeptical) as to whether the saveCompleteCallback and saveFailedCallback callbacks to _saveEverything will really work. This is simple enough to test, but before doing that, perhaps it’d be interesting to examine how they work. For those inclined, here’s a look at part of the source in ItemFileWriteStore’s save method, as it pertains to the _saveEverything and _saveCustom extension points:

        var saveCompleteCallback = function(){
            self._pending = {
                _newItems:{}, 
                _modifiedItems:{},
                _deletedItems:{}
            };

            self._saveInProgress = false; // must come after this._pending is cleared, but before any callbacks
            if(keywordArgs && keywordArgs.onComplete){
                var scope = keywordArgs.scope || dojo.global;
                keywordArgs.onComplete.call(scope);
            }
        };
        var saveFailedCallback = function(err){
            self._saveInProgress = false;
            if(keywordArgs && keywordArgs.onError){
                var scope = keywordArgs.scope || dojo.global;
                keywordArgs.onError.call(scope, err);
            }
        };
        
        if(this._saveEverything){
            var newFileContentString = this._getNewFileContentString();
            this._saveEverything(saveCompleteCallback, saveFailedCallback, newFileContentString);
        }
        if(this._saveCustom){
            this._saveCustom(saveCompleteCallback, saveFailedCallback);
        }
        if(!this._saveEverything && !this._saveCustom){
            // Looks like there is no user-defined save-handler function.
            // That's fine, it just means the datastore is acting as a "mock-write"
            // store -- changes get saved in memory but don't get saved to disk.
            saveCompleteCallback();
        }

The source above, in the context of ItemFileWriteStore’s save method, accomplishes the following:

1. save is called, passed a keywordArgs object which may or may not contain onComplete and/or onError properties pointing to callback functions.
2. Two callbacks are defined on the fly within the save method: saveCompleteCallback and saveFailedCallback.
   • saveCompleteCallback will perform store-specific logic, then call the function specified by keywordArgs.onComplete in the proper scope if it was provided.
   • saveFailedCallback will perform store-specific logic, then call the function specified by keywordArgs.onError in the proper scope if it was provided.
3. The save method now checks to see whether either _saveEverything or _saveCustom exists in this store instance.
   • In the case of dojo.data.ItemFileWriteStore out of the box, neither method exists, so the code simply calls saveCompleteCallback immediately, which effectively convinces the store’s internal state that it no longer has dirty items. However, no data would be persisted anywhere.
   • In the case of our extension, _saveEverything exists, so the process continues…
4. _saveEverything or _saveCustom is called if it exists, passing in saveCompletedCallback and saveFailedCallback so that they can be properly hooked as success/error callbacks to whatever logic must take place within. In our case, we hook them up directly to the load and error properties of a dojo.xhrPost call, which will be executed when the XHR completes successfully or fails, respectively.

Now that it’s hopefully clear what’s going on, let’s put it to the test. To make sure both callbacks are evident, we can add some default callbacks ourselves in save if they aren’t passed in. Replace the dojo.declare in SimpleSaveWriteStore.js with the following…

Note: Again, the version of this code in the zip available at the end of the article contains inline comments. In the paste below, modified/added lines since the original above have been highlighted (assuming you have JavaScript enabled).

  dojo.declare('SimpleSaveWriteStore', dojo.data.ItemFileWriteStore, {
    _postUrl: 'postdata.php',
    
    _saveEverything: function(saveCompleteCallback, saveFailedCallback, newFileContentString) {
      dojo.xhrPost({
        url: this._postUrl,
        content: {
          data: newFileContentString
        },
        load: saveCompleteCallback,
        error: saveFailedCallback
      });
    },
    
    save: function(keywordArgs) {
      if (!keywordArgs.onComplete) {
        keywordArgs.onComplete = function() { console.log('complete!'); };
      }
      if (!keywordArgs.onError) {
        keywordArgs.onError = function() { console.error('error!'); };
      }
      this.inherited(arguments);
    }
  });

If you’re testing in IE < 8, you may wish to replace console.log and console.error above with simple alerts, or set isDebug: true in djConfig so that the console messages will be visible.

The code above adds an implementation overriding ItemFileWriteStore’s save method, adding onComplete and onError callbacks that simply log a message to the console in the event that no callback was actually specified to the original call. This implementation then simply lets its parent implementation do the usual work, by invoking this.inherited(arguments).

Now, have Firebug or your respective developer tools open to the console, and click Save – you should see a “complete!” message logged.

If you’d like to test the error callback, the easiest way is to either rename postdata.php or change the value of _postUrl temporarily. Under these circumstances, clicking Save will hit the error callback, which in this case pops an alert due to logic in StoreExplorer.

Conclusion

This was probably hands-down the longest post I’ve written yet… if you managed to come along for the whole ride, I hope it was in some way enjoyable and worthwhile.

Admittedly, the length of this post likely belies the nature of the actual task it describes; writing this up took several hours, but putting together the example took a fraction of one.

If it looks like I still managed to screw up and/or miss something important, or if something’s still unclear, shoot me a comment.

Example Download

The example discussed in this article is available as a zip.

Feel free to use it as a basis for your own implementations, if it helps.

Further Reading

This post touched upon many pieces of Dojo which are further explained in the official documentation.

• dojo.data.ItemFileWriteStore: API DojoCampus
• dojo.data.ItemFileReadStore: API DojoCampus
• dojo.data APIs: API DojoCampus
• dojo.xhrPost: API DojoCampus
• dojo.xhrGet: API DojoCampus
• dojo.declare: API DojoCampus
• dojo.require: API DojoCampus
• dojo.addOnLoad: API DojoCampus