How to model filter conditions as data

Learn
1

Hi all,

I am stuck on how to model a set of filtering conditions as data types in my Model, and I was hoping the smart folks here would have some insight.

What I am trying to achieve

  • I have some tabular data stored as a list of records. Each record contains String, Int and Float fields.
  • I want users to be able to add/remove filters on this data, where only records matching the filter are shown.
    • For int and string fields, users should be able to define 2 types of filter. (1) the field is equal to a given value; (2) the field is contained within a set of values
    • For float fields: (1) the field is less than or equal to a given vale; (2) …more than or equal to…

For reference, my type definitions look something like this:

type alias Model =
    { data : List DataRecord
    , filters : List Filter
    }

type alias DataRecord = 
    { id : Int
    , name : String,typetype
    , someOtherId : Int
    , metricX : Float
    , metricY : Float
    -- etc
    }

What I am struggling with

I am having difficulty coming up with a Filter definition that enables me to easily:

  • Expose filter creation to the user
  • Display active filters (most likely as String, but could be any HTML)

What have I tried

What I have so far is

type FilterField
    = FilterId (DiscreteComparison Int)
    | FilterName (DiscreteComparison String)
    | FilterSomeOtherId (DiscreteComparison Int)
    | FilterMetricX (ContinuousComparison Float)
    | FilterMetricY (ContinuousComparison Float)


type DiscreteComparison a
    = Eq a
    | In (List a)


type ContinuousComparison a
    = Gte a
    | Lte a

This works relatively well in modelling the filters, but makes it difficult to display active filters (because we need a function that takes *Comparison a -> String with no way to convert a -> String) and to enable user input.

An alternative idea I had was storing the record values with a custom type like

type RecordValue = RecordInt | RecordFloat | RecordString

This would enable me to write an exhaustive RecordValue -> String function and I think solve my problems, with the downside of some additional boilerplate and Json Encode/Decode wrangling.

Another option would be to define the filters as functions directly:

type Filter = Filter (DataRecord -> Bool)

However, including functions in the Model and Msg are against the principles of the elm architecture - for good reason!

Has anyone else tried and achieved something similar? I would be interested in discovering any elegant ways to approach this.

2

To put a term on this, it might be the same thing as “defunctionalization”! (Links: 1, 2, 3)

So I think you’re on the right track, and I think you actually do have a way to convert a -> String: passing the right function down from the nesting context ought to do it. For example, you can tell it to represent the discrete comparisons by supplying String.fromInt or String.fromFloat:

toString : FilterField -> String
toString filterField =
    case filterField of
        FilterId comparison ->
            "filtered by ID " ++ discreteComparisonToString String.fromInt comparison

        FilterName comparison ->
            "filtered by name " ++ discreteComparisonToString identity comparison
        -- the rest of your constructors here

discreteComparisonToString : (a -> String) -> DiscreteComparison a -> String
discreteComparisonToString stringify discreteComparison =
    case discreteComparison of
        Eq a ->
            "equal to " ++ stringify a

        In values ->
            "in " ++ String.join ", " (List.map stringify values)

Now you can get:

In Out
FilterId (In [ 1, 2, 3 ]) filtered by ID in 1, 2, 3
FilterName (Eq "Dave") filtered by name equal to Dave

If you want to convert this to HTML instead, you can make:

toHtml : FilterField -> Html Msg
toHtml filterField =
    case filterField of
        FilterId comparison ->
            Html.div []
                [ Html.text (toString filterField)
                , Html.button
                    [ Events.onClick (RemoveFilter filterField) ]
                    [ Html.text "Remove Filter" ]
                ]

         -- et cetera

editing is a little tricker, but Html.map may come in handy for wrapping and unwrapping the inner types. You can then represent interactivity with an arbitrary HTML structure.

2 Likes
3

That’s very useful Brian, thank you. Knowing how to describe my issue is a huge help.

discreteComparisonToString : (a -> String) -> DiscreteComparison a -> String

Handily, I have come to the same signature after a bit of head-banging this afternoon. Good to know that it’s an “accepted” pattern and that I haven’t done something horribly wrong.

1 Like
4

One issue I can see here is that there’s 1:1 duplication between the record fields and the Filter types. This might just be the price we pay for stability and simplicity, though.

5

Yep! It’s quite like JSON encoders or decoders in this respect: you are defining a way to serialize and deserialize to a format that you find helpful, except in this case you’re deserializing to a function!

6

I think I will continue to bash my head against the wall a little longer to see if I can find a solution that can incorporate the JSON encode/decoders and the view function, in order to minimise duplication. That level of code generation might be a bit beyond Elm (by design), so I may come back to this anyway.

Thanks again for your help, Brian!

7

I went for a slightly different structure in the end, that I think turned out slightly cleaner.

Instead, of using the type definitions shown earlier, I created a record type that mirrored the DataRecord type more closely:

type alias Model =
    { data : List DataRecord
    , filters : Filters
    }

-- DataRecord is as before

type alias Filters =
    { someOtherId : DiscreteFilter Int
    , metricX : ContinuousFilter Float
    , metricY : ContinuousFilter Float
    -- & any other fields from DataRecord that we are filtering on
    }


type alias ContinuousFilter a =
    { min : Maybe String
    , max : Maybe String
    , coerce : String -> Maybe a
    }


type DiscreteFilter a
    = Any
    | Is a
    | Within (List a)

I then chose to display the DiscreteFilters as dropdowns, and the ContinuousFilters as ranges where the user can input upper and lower bounds (Nothing being no bound).

The reasoning behind storing the ContinuousFilter's min and max fields as Strings (rather than a) with a coerce function, is because HTML input happens as text, and we need some way to keep that state. I suppose the coerce function will need to be defunctionalised at some point, too.

1 Like
8

Oh, that’s quite nice! Good design!

9

Thanks - I think the conclusion I came to is that the structure of Filters has to mirror the structure of the data. So if the data is a list of records, then the filters should be expressed as a record, too. If I want the filters to be a List Filter, then the data should be something more list-like (maybe a Dict Field Value, for example).

Thanks again for the help and links, Brian. I thought Jimmy Koppel’s work was particularly enlightening. Interestingly, I came across him last year in a podcast episode , that may be of interest to you too:

1 Like
10

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.