Suggestion for interaction with IndexedDB

Request Feedback
1

I am working on a small Elm-Web-App where I want to use IndexedDB.
I decided to use a ‘ports-only’ solution without Native Modules, but realized that with growing complexity, the number of ports I had to use also grows considerably.

So I was looking for a way to communicate with IndexedDB via ports only, without having to write ports for every database interaction of my app.
The problem here is not so much to send requests to the DB but to direct back the answer to the right place. For me a natural solution was to introduce a new communication layer on top of the Elm <-> JS communication via ports.
For example, one part of the elm app sends a package of the form

{ address : 'todo'
, processBy : 'updateList'
, request : { method : 'GET_ALL'
            , storeName : 'todo'
            }
}

through the port sendPort. On the JS-side the request is processed and a result of the form

{ address : 'todo'
, processBy : 'updateList'
, result : '[ ... some objects ... ]' 
}

is send back via the answerPort. Where the address-field identifies the sub-model that sends the request and the processBy-field specifies the DbMsg that should be ‘used’ to process the result.
All sub-models that want to communicate with IndexedDB subscribe to the answerPort. Based on the address - field they decide whether they process or discard incoming answers.
If the database request is a modifying operation (add, store, delete) then in addition to the answer a broadcast is send through the broadcastPort.

I modified Evan’s todo-mvc example to demonstrate the basic idea. You can find the code here and try it out over there. You can read a slightly more detailed explanation in the Readme.MD

My questions are:

  • Do you think this approach is at all reasonable or is there maybe a much easier way to achieve what I want?
  • Do you think this approach can (reasonably) be applied in big(ger) applications as well?
  • What kind of improvements would you suggest?
2 Likes
2

Check out Murphy Randle’s talk at Elm Conf 2017, which is about using ports in this way.

He takes a similar approach of building a kind of protocol on top of ports. I’ve used this technique too, and it worked pretty well for me. I definitely feel like it’s more scalable than creating tons of ports!

3 Likes
3

I just started using indexedDb for the first time and found a pretty simple solution with a single port pair.

With this solution its really easy to make a package that implements like a type Persistent Dict…
So when you insert or removes things from that one it will automatically sync the actions to/from storage.
In elm you just use it as any other Dict…

Solution:
There is no Javascript involved when adding a new ObjectStore, any update to the stores or store names is only done on the Elm side.

Usage is really simple, and somewhat protected by types:

type Store
    = ProfileStore StoredProfile
    | PlantsStore (List Plant)

Storing something:

writeToStorage : Store -> Cmd Msg
writeToStorage store =
    Ports.toStorage <|
        imsg "Write"
            (withData (stringFromStore store)
                (case store of
                    ProfileStore storedProfile ->
                        profileEncoder storedProfile

                    PlantsStore plants ->
                        E.list plantEncoder plants
                )
            )

Javascript side:

// ELM REQUESTS
    app.ports.toStorage.subscribe(msg => {
        let x = msg.payload
        switch (msg.topic) {
            case "Read": read(x.stores, x.storeName); break;
            case "Write": write(x.stores, x.storeName, x.data); break;
            default: console.warn("Request from Elm (" + msg.topic + ") is not handled in switch statement")
        }
    })

Any read or write request from Elm will open the db, do the transaction and close db again.
To handle problems with stale ObjectStores and upgraded schema, I just set the DB Version number to Date().getTime() when the application loads.
The first Elm transaction to db after page-load will trigger the onupgradeneeded function,
and since every read or write request from Elm contains a list of valid Objectstores its easy to delete obsolete Stores, and add any new ones to keep it in sync with Elm-side.

function onupgradeneeded(stores) {
        return function (event) {
            let db = event.target.result
            if (dbAlreadyExists(event)) {
                deleteObsoleteStores(db,stores)
                createMissingStores(db,stores)
            } else {
                createAllObjectStores(db,stores)
            }
        }
    }

This solution is probably not the most efficient and you loose the transaction feature, but it is really simple and keeps everything 100% in sync. I find the performance is great, even with loads of data… Because I only read on startup, end then save each store now and then when appropriate.

1 Like
4

I haven’t worked with IndexedDB yet in Elm, but it’s next on my list after WebSockets (in progress) to add to elm-port-examples. I have used the approach of using a logical source/destination in the port messages for other integrations though, similar to your address field and it worked reasonably well, although in general I’ve found that it’s nicer if you can find a more natural key that’s already part of the existing process.

The thing I plan to explore when I add IndexedDB to the port examples is to see if it works well to use the actual DB request structure in Elm as a natural key to match against incoming port messages, using a Dict (or similar structure) to track the state of pending/completed requests. I’m not 100% sure it will be simpler than addressing part of the app, but I think it’s worth trying. In a more complex app, there might be more than one place tracking DB requests (per page, perhaps), since you probably don’t want that data to just continuously grow.

I’m not sure if that’s directly useful to you now, but I’ll try to report back on how things turn out once I try it.