Local Storage in PhoneGap

A Summary of Local Storage Options for PhoneGap Applications



·

Quite often when creating mobile applications you will want to store data that the user can retrieve later. A lot of the time this is done by storing the data somewhere that can be accessed through some server (think Facebook, Twitter etc.) which would require an Internet connection, but we may still want to store some or all of that data locally.

There’s a few reasons we might want to store some data locally:

  • The application is completely self contained and there is no interaction with other users, so all data can be stored locally
  • We need to store data locally for some specific functionality like a “Remember Me” feature
  • We could skip unnecessary calls to a server by storing preference settings locally
  • We could skip unnecessary calls to a server by caching other data locally
  • We want to sync online and offline data so that the user can continue using the application even when they are offline (Evernote is a good example of this)

HTML5 applications run in the browser, so we don’t have access to the storage options that native applications do. We do still have access to the usual browser storage options that websites have access to though like Local Storage, Web SQL (deprecated) and IndexedDB. These options might not be ideal for reasons which we will go through in this article, but we can also access native data storage by using PhoneGap which can overcome the shortfalls of browser based data storage.

There’s actually quite a few different options out there, and it can be pretty confusing, so in this article I will provide a brief overview of the different storage options, with quick examples and a summary of when you might want to use each one.

Local Storage

This is the most basic storage option available, which allows you to store up to 5MB worth of data in the users browser.

Local storage gets a bit of a bad wrap, and is generally considered to be unreliable. I think the browsers local storage can be a viable option and it is reasonably stable and reliable, but, it is possible for the data to be wiped, which means for a lot of applications it’s not going to be a great option.

In general, you should only use it where data loss would not be an issue, and it shouldn’t ever be used to store sensitive data since it can be easily accessed. One example of where local storage might be a suitable option is if you wanted to store something like a temporary session token. This would allow you to create a “Remember Me” type feature, but if the data is lost it’s not really a big deal because the user will just need to enter in their username and password again.

If you are just using local storage to cache data from a server it would also not be a big issue if the data is lost since it can just be fetched from the server again.

Local Storage is a simple key-value system, let’s take a look at how you might go about setting and retrieving data:

localStorage.setItem('someSetting', 'off');
var someSetting = localStorage.getItem('someSetting');

This is the native way to set and retrieve local storage data, but some frameworks have their own inbuilt ways to access local storage.

One important thing to keep in mind when using local storage is that it can only store strings. If you wanted to store an object or array in local storage you would first need to convert it to a JSON string, and then when you retrieve it later you can decode that JSON string back into an array or object.

Local storage is super simple to use and doesn’t require installing anything, but it is very basic and not 100% reliable.

SQLite

SQLite is basically an embedded SQL database. In the context of mobile applications it allows each application to have its own SQL database to work with. Unlike a normal SQL database it does not need to run on a server, and does not require any configuration. SQLite can be utilised by both iOS and Android applications (as well as others), but the SQLite database can only be accessed natively, so it is not accessible by default to HTML5 mobile apps.

We can however use a SQLite PhoneGap Plugin to easily gain access to this functionality. Simply run the following command in your project to install it:

cordova plugin add cordova-sqlite-storage

The main benefits of using SQLite are that it:

  • Provides persistent data storage
  • There is no size limitation on how much can be stored
  • It provides the SQL syntax, so it is a very powerful tool for managing data

Although there are some differences in supported commands between SQL and SQLite, it is almost exactly the same. Here’s an example of how you might execute some simple queries in SQLite:

  var db = window.sqlitePlugin.openDatabase({name: "my.db"});

  db.transaction(function(tx) {
    tx.executeSql('DROP TABLE IF EXISTS test_table');
    tx.executeSql('CREATE TABLE IF NOT EXISTS test_table (id integer primary key, data text, data_num integer)');

    tx.executeSql("INSERT INTO test_table (data, data_num) VALUES (?,?)", ["test", 100], function(tx, res) {
      console.log("insertId: " + res.insertId + " -- probably 1");
      console.log("rowsAffected: " + res.rowsAffected + " -- should be 1");

    }, function(e) {
      console.log("ERROR: " + e.message);
    });
  });

If you’re using Ionic then I recommend checking out $cordovaSQLite that is provided by ngCordova. The ngCordova plugin makes SQLite play much more nicely with AngularJS applications, here’s an example from the documentation:

  var db = $cordovaSQLite.openDB({ name: "my.db" });

  $scope.execute = function() {
    var query = "INSERT INTO test_table (data, data_num) VALUES (?,?)";
    $cordovaSQLite.execute(db, query, ["test", 100]).then(function(res) {
      console.log("insertId: " + res.insertId);
    }, function (err) {
      console.error(err);
    });
  };

SQLite is essentially the only way to persistently store unlimited data for PhoneGap applications, however other options I will cover in the rest of this post can also be made persistent and have unlimited data storage by using SQLite as a backend.

Web SQL

Web SQL is basically SQLite but embedded in the browser, which would make it accessible to HTML5 mobile applications without PhoneGap. However, the Web SQL specification is no longer being maintained and although it is supported on iOS and Android browsers, support may be dropped in the future.

Since Web SQL is deprecated and there are alternative options for PhoneGap applications, there’s no reason to use Web SQL.

IndexedDB

Where SQLite provides a typical relational database approach, IndexedDB provides a NoSQL approach. You can read more about what exactly NoSQL is and why it exists here, but essentially this means that data is stored as objects rather than in tables. NoSQL databases are generally considered to be faster and more scalable than SQL databases, and since there is no pre-defined schema it is really easy to add new data.

If I were using SQL and wanted to add a new Drinks table, I would need to figure out how that would relate to the rest of my data, modify the schema, and then add my data. With a NoSQL approach I can just store a new Drinks object. This is a bit of a double edged sword, this less structured approach isn’t necessarily a good thing in all cases, nor is it a bad thing – it all depends on the circumstance and preference.
In short, IndexedDB:

  • Provides key-value storage
  • Has asynchronous storage and retrieval
  • Is object oriented

Raymond Camden has written an excellent summary of working with IndexedDB (note: support for IndexedDB is much better now than when that article was written, but it is still not supported universally)), but in short you can store data with IndexedDB like this:

    var transaction = db.transaction(['drinks'],'readwrite');
    var store = transaction.objectStore('drinks');

    var drink = {
        name: 'Tequila',
        alcoholic: true,
        price: 8.90
    };

    var request = store.add(drink);

and retrieve it like this:

    var transaction = db.transaction(['drinks'], 'readonly');
    var store = transaction.objectStore('drinks');

    var object = store.get(1);

    object.onsuccess = function(e) {
         console.log("My Result", e.target.result);
    }

One of the main benefits of IndexedDB is that it provides an API for searching through the data you have stored (something that would be very difficult to do with plain old local storage).

Unfortunately, like local storage and Web SQL, IndexedDB is stored in the browser so it has the same size limitation and persistence issues that they have. You can however use something like localForage which I will discuss in a moment, to provide a NoSQL approach with persistent data by using SQLite in the background.

Third Party Libraries for Local Storage

There’s quite a few JavaScript libraries that have popped up to help deal with the local storage problem, and also to sync both offline and online data. I’ll cover a few of the more popular ones now.

localForage

localForage was created by Mozilla as a fast and simple storage library for JavaScript applications. Basically it provides all the benefits of IndexedDB with the ease of use of plain old local storage. Behind the scenes, localForage uses IndexedDB to store data by default, but you can make it use SQLite instead by implementing the localForage-cordovaSQLiteDriver PhoneGap plugin.

One big benefit of localForage is that you don’t need to worry about all the hassle of converting everything to JSON before storing it, and converting it back afterwards – localForage will magically handle everything for you.

Here’s some simple examples of setting and retrieving data with local forage:

localforage.setItem('key', 'value', callbackFunction);
localforage.getItem('key', callbackFunction);
localforage.removeItem('key', callbackFunction);

We supply callback functions above because reading and writing data with localForage is asynchronous. Nobody likes callbacks though, so fortunately localForage also allows you to use promises that look like this:

localforage.getItem('key').then(function(value) {
    console.log(value);
});

For more advanced localForage examples, take a look at this Team Treehouse tutorial.

PouchDB

PouchDB is another local storage option that takes a NoSQL approach, but this one comes with the HUGE benefit of being able to easily sync with an online data store.

Keeping data from two different data sources in sync sounds like quite a task, and it certainly is if you try to do it yourself, but it’s kind of ridiculous how easy PouchDB makes it. Take a look at this example of adding some data to a new PouchDB database and then syncing it to an external database:

var db = new PouchDB('dbname');

db.put({
  _id: '[email protected]',
  name: 'David',
  age: 68
});

db.changes().on('change', function() {
  console.log('Ch-Ch-Changes');
});

db.replicate.to('http://example.com/mydb');

Of course you also need to set your server up correctly as well, but that’s quite straight forward as well and Nic Raboy has an excellent tutorial on how to do that.

PouchDB can also take advantage of SQLite by using the SQLite plugin. It won’t require any changes to your code, you just have to install the SQLite plugin as I described earlier in this post and PouchDB will automatically be able to detect and start making use of it.

LokiJS

This is an interesting one I’d never heard of until recently when Ashteya of gonehybrid.com posted a tutorial on how to use LokiJS.

LokiJS doesn’t have the synchronisation functionality that PouchDB has and it seems to be a bit tricker to set up data persistence, but it claims to be faster as it uses an in memory database. In most cases I would imagine this performance difference wouldn’t be noticeable, but if you have an offline application where fast reading and writing is imperative it will likely be a good option.

Summary

I certainly haven’t covered every possible option out there, but I have covered the core technology that sits behind most options and also the most popular third party libraries in use today. You might find yourself at the end of this article still unsure what to use, so to summarise:

  • If data loss does not matter, and you will be storing less than 5MB of data, use Local Storage

  • Local Forage is a good option as your default local storage approach. It will make use of the best data storage technology available and provides a simple interface. It is a good idea to use the SQLite driver for this.

  • If you want to store relational data and would like to use SQL, use SQLite

  • If you need to sync online and offline data use PouchDB (again, with SQLite installed)

These aren’t hard and fast rules, I haven’t used all of these technologies so it wouldn’t be fair for me to make judgements on which is the best, but if you don’t know where to start this should be a pretty good guideline.

What do you use for storing data? Have you tried any of the technologies listed in this article? I’d love to hear your thoughts in the comments.

What to watch next...

  • Awesome! I never knew there were so many different options available. I’ve always been doing localStorage for apps. The 5mb is a big issue, I wrote a little script called fileStorage (https://github.com/CookieCookson/cordova-filestorage) which mimics the localStorage functionality and stores the string data into .txt files using the File plugin. This is a much slower solution but I needed a quick fix without massively changing my data storage method on an application.

    Local Forage looks like a much better solution in the future! Keep posting stuff Josh, your articles are great.

  • Pingback: A Summary of Local Storage Options for PhoneGap Applications | HTML5 Mobile Tutorials | Ionic, Phaser, Sencha Touch & PhoneGap | Blog eloylinus()

  • Pingback: Build a Todo App from Scratch with Ionic 2 [VIDEO TUTORIAL] | HTML5 Mobile Tutorials | Ionic, Phaser, Sencha Touch & PhoneGap()

  • Vijith Nair

    Great article. Thanks for sharing the info. This is my first article here. Have already bookmarked your website 🙂

  • Pingback: A Summary of Backend Options for HTML5 Mobile Applications | HTML5 Mobile Tutorials | Ionic, Phaser, Sencha Touch & PhoneGap()

  • Pingback: Syncing Data with PouchDB and Cloudant in Ionic 2 | HTML5 Mobile Tutorials | Ionic, Phaser, Sencha Touch & PhoneGap()

  • divulgo

    Could you please provide some help on how to use SQLite driver for localforage in Ionic 2? It would help immensely.

  • Robin Kramer

    Why do such tutorials always require the reader to be Git and Cmd MASTERS?

    “Simply run the following command in your project to install it:”
    Simply run… off course ’cause that’s how every normal being does everything on the computer, per command line~

    But okay… lets consider that everyone knows how to open the console and to cd… (’cause I luckily do know this… and most people would be able to google it faster than loose interest)
    -bash: cordova: command not found

    But why? Doesn’t it say, simple run that command?!

    Well let’s check if there’s any help… okay there is a link to that plugin…

    Okay… lots of files… *scrolling*… a Tutorial!!! Starting on the programming part…

    Way down the file finally also comes a part about “Recommended prerequisites” and “Quicki nstallation” containing the “Use the following command to install this plugin from the Cordova CLI:”
    AHA!!! How the F*ck should any reader know that?
    And also… it’s only talking about cordova, never about phonegap…
    what even is that… or the difference? I still don’t know…

    But this site implies that I should know throw phonegap away, install cordova and just leave this website and learn somewhere else…

    but wait… wasn’t this a list for phonegap?
    Yes was it, but who cares, these websites are obviously only for people who are already knee deep into the subject, why should there be any logical arrangement for newbies~

    • Adam

      Are you mad at the author for assuming you know what a computer is?

    • zzzxtreme

      overwhelming isn’t it ?

    • Pat

      I know your comment’s getting old, but hopefully this’ll clear up some of the questions asked.

      Phonegap vs Cordova
      This is always difficult to answer because there really isn’t a big difference. The best way to think of it is Cordova is the engine, Phonegap is the platform. Phonegap is a distribution of Cordova.
      In -most- tutorials, the words “Phonegap” and “Cordova” are interchangeable, so just use whichever one you’re currently using.

      cordova: command not found
      Cordova hasn’t been installed. To double check if you have phonegap, just type “phongap” in cmd, if you don’t have it it’ll say “command not found” again.

      If you have neither, you’ll have to get one (obviously), but to do that, you’ll need NodeJS.
      To check if you have node, type “npm -v” in cmd. If it returns a version number, hooray, you’ve get node.
      If it did not return a version number, aww, you don’t have node. Download it from here: https://nodejs.org/en/download/

      Once NodeJS has been installed, or if it alreadys was, the time to install Phonegap or Cordova is here!
      To install, open cmd in the Phonegap directory (you should see the “www”, “plugins”, “platforms” directories). An easy way to open the cmd on Windows is to hold shift then right click, then select “Open command window here”. I’m not sure if a shortcut like that exists for Mac, so you’ll probably be stuck with using cd.
      Once in the directory, type “npm install -g phonegap” (note if you want cordova, it’d be “npm install -g cordova”).
      It’ll take a little bit to install, but when it’s congrats! The hard parts done.
      Type either “phonegap” or “cordova”, whichever you installed, in cmd. It should no longer say “command not found”.

      Installing a plugin
      Make sure youre still in the correct directory (should be to see the “www”, “plugins”, “platforms” directories) in your cmd.
      Type “phonegap plugin add cordova-sqlite-storage” (replace phonegap with cordova if you installed cordova).
      Now you’re ready to go! Probably!

      —-

      If you’re just starting out, there’s a few other things that may catch you up.

      Somewhere on your HTML page, make sure you include the code “http://cordova.js” exactly like that, no file path.
      Phonegap will generate a cordova.js for you, but you need to connect to it.
      Now you have access to some events that’ll be triggered by your device. The main one to care about is “deviceready”.

      To use the event:
      document.addEventListener(“deviceready”, onDeviceReady, false);
      function onDeviceReady() {
      console.log(“Device now ready”);
      // code here!
      }

      Where it says “code here!” is where the can be sure the plugin is ready to be used (unless the plugin has its own ready event).

      —-

      (Apologies for bad grammar, been a long day there boys)
      That’s a lot to read, but should help to explain a bit. If you have more questions, feel free to reply and I’ll answer best I can.

  • Tuan Jinn Nguyen

    Great article

  • Altaf Hussain

    great, can we use local forage for more than 5mb limit ?? I need to store 50mb data !!!