I want to explore something a bit unusual. What would it look like to implement classic machine learning algorithms in pure elm, without any javascript or external libraries?
So far I have experimented with Naive bayes, KNN, K-Mean, Vector utilities, evaluation metrics, etc.
This has raised a big doubt and concern for me: Is this worth doing, using elm, if there are already alternative tools? There are already tools for numerical computing and ML visualizations, so this is not about competing with those tools, but instead to explore if:
What machine learning algorithms will look like in pure functional elm style?
If this can bring more clarity to what elm gurantees (purity, safety, immutability)?
If this can be helpful for teaching or learning ML concepts?
Wether a small ELM ML PLAYGROUND would be interesting for the community?
And above all, If this belongs in elm’s ecosystem?
Before i continue working on this further, I would really appreciate feedback from more experienced elm developers.
This sounds really cool! A part of me has wanted to learn these things, and I do sometimes find that learning these concepts in Elm is easier for me. As you pointed out, Elm can’t really compete in terms of performance, though if that’s not your concern then I don’t think there’s much else to be concerned about.
I guess the thing to be concerned about is that it is not possible to implement all of the algorithms with full fidelity in Elm because some of them will require mutability. That said, an exploration of how to implement these algorithms under the constraint of immutability and how that might differ from the mutable versions could be an interesting project.
I think it sounds interesting and worthwhile experimenting with.
I am sure all the algorithms can be implemented in an immutable language.. but.. they will not be efficient as a result.
They will also not compile down to a binary that will run on a GPU, and that is where the real number crunching power is that drives todays AI boom.
That said, its definitely worth exploring for educational purposes. Also to start thinking about whether and how compiler transformations could take algorithms written in an immutable functional language can in fact be transformed into algorithms that do in-place modifications and can be lowered down into something that can run on a GPU. Or at least be vectorised to run on CPU SIMD vector instruction sets.
See https://rise-lang.org/ for an example of some research in that direction. There are others too.
Implementations of compute graphs for machine learning usually provide automatic differentiation. This is because the most common training algorithm, back-propagation, needs to compute derivatives with respect to the estimated error function at all points in the graph, in order to adjust the weights in such a way as to reduce the error.
It could be very interesting to see how compute graphs and differentiable functions might be represented in Elm. If it were Haskel, I imagine that we would be talking about a type class of differentiable functions. That raises the question of whether a description of such things in Elm would be satisfactory without the concept of type classes.
Elms main focus is for interactive visual applications so it could be great if your aim is to teach machine learning with interactive visuals, even if it can only run tiny examples in practice.
The idea of representing compute graphs and differentiation in Elm is also interesting. Elm doesn’t have type classes, so the design space looks very different from Haskell. But maybe that’s a feature, not a bug (I think we had something similar to the type classes in older versions). The constraints might force a clearer or more visual model.
It won’t be possible to use that in an Elm program without using ports, so while it may be possible to write an Elm program that makes use of it, the compiler can’t generate code on its own that will take advantage of it.
I wrote an experimental simple neural model to try an alternative to back propagation learning using Futhark, a language that can run on GPU.
I wrote another experimental simple neural model in F#, a language running on CPU, but really using all the cores and threads in parallel.
In AI, Python is king. PyTorch allows you to run your Python code on GPU. With Google’s Colab, you can run Python notebooks using the TensorFlow API for free.
If we are talking Python - also worth mentioning Mojo + Max here - Mojo is a language which extends Python with stronger typing, and enables very high performance code to be written that will take advantage of all CPU cores or compile to GPU. Max is its compute library, and can also be invoked from Python, and I think C++ too.
The exciting thing about it is that it does not use nVidia’s CUDA at all, but can compile down to equally performant code on the GPU. And it can do the same for AMD GPUs too.
(I think we had something similar to the type classes in older versions). The constraints might force a clearer or more visual model.