How to Store Unlimited* Data in the Browser with IndexedDB

This article explains the fundamentals of storing data in the browser using the IndexedDB API, which offers a far greater capacity than other client-side mechanisms.

Storing web app data used to be an easy decision. There was no alternative other than sending it to the server, which updated a database. Today, there’s a range of options, and data can be stored on the client.

Why Store Data in the Browser?

It’s practical to store most user-generated data on the server, but there are exceptions:

• device-specific settings such as UI options, light/dark mode, etc.
• short-lived data, such as capturing a range of photographs before choosing one to upload
• offline data for later synchronization, perhaps in areas with limited connectivity
• progressive web apps (PWAs) which operate offline for practical or privacy reasons
• caching assets for improved performance

Three primary browser APIs may be suitable:

1. Web Storage

   Simple synchronous name-value pair storage during or beyond the current session. It’s practical for smaller, less vital data such as user interface preferences. Browsers permit 5MB of Web Storage per domain.

2. Cache API

   Storage for HTTP request and response object pairs. The API is typically used by service workers to cache network responses, so a progressive web app can perform faster and work offline. Browsers vary, but Safari on iOS allocates 50MB.

3. IndexedDB

   A client-side, NoSQL database which can store data, files, and blobs. Browsers vary, but at least 1GB should be available per domain, and it can reach up to 60% of the remaining disk space.

OK, I lied. IndexedDB doesn’t offer unlimited storage, but it’s far less limiting than the other options. It’s the only choice for larger client-side datasets.

IndexedDB Introduction

IndexedDB first appeared in browsers during 2011. The API became a W3C standard in January 2015, and was superseded by API 2.0 in January 2018. API 3.0 is in progress. As such, IndexedDB has good browser support and is available in standard scripts and Web Workers. Masochistic developers can even try it in IE10.

This article references the following database and IndexedDB terms:

• database: the top-level store. Any number of IndexedDB databases can be created, although most apps will define one. Database access is restricted to pages within the same domain; even sub-domains are excluded. Example: you could create a notebook database for your note-taking application.

• object store: a name/value store for related data items, conceptually similar to collections in MongoDB or tables in SQL databases. Your notebook database could have a note object store to hold records, each with an ID, title, body, date, and an array of tags.

• key: a unique name used to reference every record (value) in an object store. It can be automatically generated or set to a value within the record. The ID is ideal to use as the note store’s key.

• autoIncrement: a defined key can have its value auto-incremented every time a record is added to a store.

• index: tells the database how to organize data in an object store. An index must be created to search using that data item as criteria. For example, note dates can be indexed in chronological order so it’s possible to locate notes during a specific period.

• schema: the definition of object stores, keys, and indexes within the database.

• version: a version number (integer) assigned to a schema so a database can be updated when necessary.

• operation: a database activity such as creating, reading, updating, or deleting (CRUD) a record.

• transaction: a wrapper around one or more operations which guarantees data integrity. The database will either run all operations in the transaction or none of them: it won’t run some and fail others.

• cursor: a way to iterate over many records without having to load all into memory at once.

• asynchronous execution: IndexedDB operations run asynchronously. When an operation is started, such as fetching all notes, that activity runs in the background and other JavaScript code continues to run. A function is called when the results are ready.

The examples below store note records — such as the following — in a note object store within a database named notebook:

{
  id: 1,
  title: "My first note",
  body: "A note about something",
  date: <Date() object>,
  tags: ["#first", "#note"]
}

The IndexedDB API is a little dated and relies on events and callbacks. It doesn’t directly support ES6 syntactical loveliness such as Promises and async/await. Wrapper libraries such as idb are available, but this tutorial goes down to the metal.

IndexDB DevTools Debugging

I’m sure your code is perfect, but I make a lot of mistakes. Even the short snippets in this article were refactored many times and I trashed several IndexedDB databases along the way. Browser DevTools were invaluable.

All Chrome-based browsers offer an Application tab where you can examine the storage space, artificially limit the capacity, and wipe all data:

The IndexedDB entry in the Storage tree allows you to examine, update, and delete object stores, indexes, and individual record:

(Firefox has a similar panel named Storage.)

Alternatively, you can run your application in incognito mode so all data is deleted when you close the browser window.

Check for IndexedDB Support

window.indexedDB evaluates true when a browser supports IndexedDB:

if ('indexedDB' in window) {

  // indexedDB supported

}
else {
  console.log('IndexedDB is not supported.');
}

It’s rare to encounter a browser without IndexedDB support. An app could fall back to slower, server-based storage, but most will suggest the user upgrade their decade-old application!

Check Remaining Storage Space

The Promise-based StorageManager API provides an estimate of space remaining for the current domain:

(async () => {

  if (!navigator.storage) return;

  const
    required = 10, // 10 MB required
    estimate = await navigator.storage.estimate(),

    // calculate remaining storage in MB
    available = Math.floor((estimate.quota - estimate.usage) / 1024 / 1024);

  if (available >= required) {
    console.log('Storage is available');
    // ...call functions to initialize IndexedDB
  }

})();

This API is not supported in IE or Safari (yet), so be wary when navigator.storage can’t returns a falsy value.

Free space approaching 1,000 megabytes is normally available unless the device’s drive is running low. Safari may prompt the user to agree to more, although PWAs are allocated 1GB regardless.

As usage limits are reached, an app could choose to:

• remove older temporary data
• ask the user to delete unnecessary records, or
• transfer less-used information to the server (for truly unlimited storage!)

Open an IndexedDB Connection

An IndexedDB connection is initialized with indexedDB.open(). It is passed:

• the name of the database, and
• an optional version integer

const dbOpen = indexedDB.open('notebook', 1);

This code can run in any initialization block or function, typically after you’ve checked for IndexedDB support.

When this database is first encountered, all object stores and indexes must be created. An onupgradeneeded event handler function gets the database connection object (dbOpen.result) and runs methods such as createObjectStore() as necessary:

dbOpen.onupgradeneeded = event => {

  console.log(`upgrading database from ${ event.oldVersion } to ${ event.newVersion }...`);

  const db = dbOpen.result;

  switch( event.oldVersion ) {

    case 0: {
      const note = db.createObjectStore(
        'note',
        { keyPath: 'id', autoIncrement: true }
      );

      note.createIndex('dateIdx', 'date', { unique: false });
      note.createIndex('tagsIdx', 'tags', { unique: false, multiEntry: true });
    }

  }

};

This example creates a new object store named note. An (optional) second argument states that the id value within each record can be used as the store’s key and it can be auto-incremented whenever a new record is added.

The createIndex() method defines two new indexes for the object store:

1. dateIdx on the date in each record
2. tagsIdx on the tags array in each record (a multiEntry index which expands individual array items into an index)

There’s a possibility we could have two notes with the same dates or tags, so unique is set to false.

Note: this switch statement seems a little strange and unnecessary, but it will become useful when upgrading the schema.

An onerror handler reports any database connectivity errors:

dbOpen.onerror = err => {
  console.error(`indexedDB error: ${ err.errorCode }`);
};

Finally, an onsuccess handler runs when the connection is established. The connection (dbOpen.result) is used for all further database operations so it can either be defined as a global variable or passed to other functions (such as main(), shown below):

dbOpen.onsuccess = () => {

  const db = dbOpen.result;

  // use IndexedDB connection throughout application
  // perhaps by passing it to another function, e.g.
  // main( db );

};

Create a Record in an Object Store

The following process is used to add records to the store:

1. Create a transaction object which defines a single object store (or array of object stores) and an access type of "readonly" (fetching data only — the default) or "readwrite" (updating data).

2. Use objectStore() to fetch an object store (within the scope of the transaction).

3. Run any number of add() (or put()) methods and submit data to the store:

   const
   
     // lock store for writing
     writeTransaction = db.transaction('note', 'readwrite'),
   
     // get note object store
     note = writeTransaction.objectStore('note'),
   
     // insert a new record
     insert = note.add({
       title: 'Note title',
       body: 'My new note',
       date: new Date(),
       tags: [ '#demo', '#note' ]
     });
   

This code can be executed from any block or function which has access to the db object created when an IndexedDB database connection was established.

Error and success handler functions determine the outcome:

insert.onerror = () => {
  console.log('note insert failure:', insert.error);
};

insert.onsuccess = () => {
  // show value of object store's key
  console.log('note insert success:', insert.result);
};

If either function is not defined, it will bubble up to the transaction, then the database handers (that can be stopped with event.stopPropagation()).

When writing data, the transaction locks all object stores so no other processes can make an update. This will affect performance, so it may be practical to have a single process which batch updates many records.

Unlike other databases, IndexedDB transactions auto-commit when the function which started the process completes execution.

Update a Record in an Object Store

The add() method will fail when an attempt is made to insert a record with an existing key. put() will add a record or replace an existing one when a key is passed. The following code updates the note with the id of 1 (or inserts it if necessary):

const

  // lock store for writing
  updateTransaction = db.transaction('note', 'readwrite'),

  // get note object store
  note = updateTransaction.objectStore('note'),

  // add new record
  update = note.put({
    id: 1,
    title: 'New title',
    body: 'My updated note',
    date: new Date(),
    tags: [ '#updated', '#note' ]
  });

// add update.onsuccess and update.onerror handler functions...

Note: if the object store had no keyPath defined which referenced the id, both the add() and put() methods provide a second parameter to specify the key. For example:

update = note.put(
  {
    title: 'New title',
    body: 'My updated note',
    date: new Date(),
    tags: [ '#updated', '#note' ]
  },
  1 // update the record with the key of 1
);

Reading Records from an Object Store by Key

A single record can be retrieved by passing its key to the .get() method. The onsuccess handler receives the data or undefined when no match is found:

const

  // new transaction
  reqTransaction = db.transaction('note', 'readonly'),

  // get note object store
  note = reqTransaction.objectStore('note'),

  // get a single record by id
  request = note.get(1);

request.onsuccess = () => {
  // returns single object with id of 1
  console.log('note request:', request.result);
};

request.onerror = () => {
  console.log('note failure:', request.error);
};

The similar getAll() method returns an array matching records.

Both methods accept a KeyRange argument to refine the search further. For example, IDBKeyRange.bound(5, 10) returns all records with an id between 5 and 10 inclusive:

request = note.getAll( IDBKeyRange.bound(5, 10) );

Key range options include:

• IDBKeyRange.lowerBound(X): keys greater than or equal to X
• IDBKeyRange.upperBound(X): keys less than or equal to Y
• IDBKeyRange.bound(X,Y): keys between X and Y inclusive
• IDBKeyRange.only(X): a single key matching X

The lower, upper, and bound methods have an optional exclusive flag. For example:

• IDBKeyRange.lowerBound(5, true): keys greater than 5 (but not 5 itself)
• IDBKeyRange.bound(5, 10, true, false): keys greater than 5 (but not 5 itself) and less than or equal to 10

Other methods include:

• .getKey(query): return a matching key (rather than the value assigned to that key)
• .getAllKeys(query): return an array of matching keys
• .count(query): return the number of matching records

Reading Records from an Object Store by Indexed Value

An index must be defined to search fields within a record. For example, to locate all notes taken during 2021, it’s necessary to search the dateIdx index:

const

  // new transaction
  indexTransaction = db.transaction('note', 'readonly'),

  // get note object store
  note = indexTransaction.objectStore('note'),

  // get date index
  dateIdx = note.index('dateIdx'),

  // get matching records
  request = dateIdx.getAll(
    IDBKeyRange.bound(
      new Date('2021-01-01'), new Date('2022-01-01')
    )
  );

// get results
request.onsuccess = () => {
  console.log('note request:', request.result);
};

Reading Records from an Object Store Using Cursors

Reading a whole dataset into an array becomes impractical for larger databases; it could fill the available memory. Like some server-side data stores, IndexedDB offers cursors which can iterate through each record one at a time.

This example finds all records containing the "#note" tag in the indexed tags array. Rather than using .getAll(), it runs an .openCursor() method, which is passed a range and optional direction string ("next", "nextunique", "prev", or "preunique"):

const

  // new transaction
  cursorTransaction = db.transaction('note', 'readonly'),

  // get note object store
  note = cursorTransaction.objectStore('note'),

  // get date index
  tagsIdx = note.index('tagsIdx'),

  // get a single record
  request = tagsIdx.openCursor('#note');

request.onsuccess = () => {

  const cursor = request.result;

  if (cursor) {

    console.log(cursor.key, cursor.value);
    cursor.continue();

  }

};

The onsuccess handler retrieves the result at the cursor location, processes it, and runs the .continue() method to advance to the next position in the dataset. An .advance(N) method could also be used to move forward by N records.

Optionally, the record at the current cursor position can be:

• updated with cursor.update(data), or
• deleted with cursor.delete()

Deleting Records from an Object Store

As well as deleting the record at the current cursor point, the object store’s .delete() method can be passed a key value or KeyRange. For example:

const

  // lock store for writing
  deleteTransaction = db.transaction('note', 'readwrite'),

  // get note object store
  note = deleteTransaction.objectStore('note'),

  // delete record with an id of 5
  remove = note.delete(5);

remove.onsuccess = () => {
  console.log('note deleted');
};

A more drastic option is .clear(), which wipes every record from the object store.

Update a Database Schema

At some point it will become necessary to change the database schema — for example, to add an index, create a new object store, modify existing data, or even wipe everything and start again. IndexedDB offers built-in schema versioning to handle the updates — (a feature sadly lacking in other databases!).

An onupgradeneeded function was executed when version 1 of the notebook schema was defined:

const dbOpen = indexedDB.open('notebook', 1);

dbOpen.onupgradeneeded = event => {

  console.log(`upgrading database from ${ event.oldVersion } to ${ event.newVersion }...`);

  const db = dbOpen.result;

  switch( event.oldVersion ) {

    case 0: {
      const note = db.createObjectStore(
        'note',
        { keyPath: 'id', autoIncrement: true }
      );

      note.createIndex('dateIdx', 'date', { unique: false });
      note.createIndex('tagsIdx', 'tags', { unique: false, multiEntry: true });
    }

  }

};

Presume another index was required for note titles. The indexedDB.open() version should change from 1 to 2:

const dbOpen = indexedDB.open('notebook', 2);

The title index can be added in a new case 1 block in the onupgradeneeded handler switch():

dbOpen.onupgradeneeded = event => {

  console.log(`upgrading database from ${ event.oldVersion } to ${ event.newVersion }...`);

  const db = dbOpen.result;

  switch( event.oldVersion ) {

    case 0: {
      const note = db.createObjectStore(
        'note',
        { keyPath: 'id', autoIncrement: true }
      );

      note.createIndex('dateIdx', 'date', { unique: false });
      note.createIndex('tagsIdx', 'tags', { unique: false, multiEntry: true });
    }

    case 1: {
      const note = dbOpen.transaction.objectStore('note');
      note.createIndex('titleIdx', 'title', { unique: false });
    }

  }

};

Note the omission of the usual break at the end of each case block. When someone accesses the application for the first time, the case 0 block will run and it will then fall through to case 1 and all subsequent blocks. Anyone already on version 1 would run the updates starting at the case 1 block.

Index, object store, and database updating methods can be used as necessary:

• .createIndex()
• .deleteIndex()
• .createObjectStore()
• .deleteObjectStore()
• .deleteDatabase()

All users will therefore be on the same database version … unless they have the app running in two or more tabs!

The browser can’t allow a user be running schema 1 in one tab and schema 2 in another. To resolve this, a the database connection onversionchange handler can prompt the user to reload the page:

// version change handler
db.onversionchange = () => {

  db.close();
  alert('The IndexedDB database has been upgraded.\nPlease reload the page...');
  location.reload();

};

Low Level IndexedDB

IndexedDB is one the of the more complex browser APIs, and you’ll miss using Promises and async/await. Unless your app’s requirements are simple, you’ll want roll your own IndexedDB abstraction layer or use a pre-built option such as idb.

Whatever option you choose, IndexedDB is one of the fastest browser data stores, and you’re unlikely to reach the limits of its capacity.

Frequently Asked Questions (FAQs) about IndexedDB and Data Storage