# Your problems writing Elm code with AI/LLM

**URL:** <https://discourse.elm-lang.org/t/your-problems-writing-elm-code-with-ai-llm/9685>\
**Category:** Request Feedback\
**Created:** [February 26, 2024, 4:15pm UTC](https://discourse.elm-lang.org/t/your-problems-writing-elm-code-with-ai-llm/9685 "2024-02-26T16:15:00Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![axelbdt](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/axelbdt/32/4404_2.png) [@axelbdt](https://discourse.elm-lang.org/u/axelbdt)\
**Post date:** [February 26, 2024, 4:15pm UTC](https://discourse.elm-lang.org/t/your-problems-writing-elm-code-with-ai-llm/9685/1 "2024-02-26T16:15:00Z")

</div>

Hi,  
The [“Elm cookbook by AI”](https://discourse.elm-lang.org/t/elm-cookbook-made-by-ai/9682/) thread highlighted the poor quality of AI written Elm code.

As it happens, I am starting a master’s thesis, and the prospective subject is code generation. The aim is to complement LLM generation with symbolic methods (namely [constraint programming](https://en.wikipedia.org/wiki/Constraint_programming)) to improve the code quality in languages that have a tiny corpus to train from.

Elm is a good candidate as a language that is relatively niche, syntactically simple, pure functional, and with great tooling to assess quality like the compiler or elm-review.

To help me assess the potential depth of such a project, I would like you to share your pain points when using LLMs to generate Elm code. Do you often encounter (from low-level to more subjective) :

- Syntactically incorrect code?
- Code that doesn’t typecheck?
- Hallucinated core or lib functions?
- Unidiomatic code?
- Any other thing that frustrates you ?

I am also interested in the relative performance of different models, especially the self-hosted ones, so if you have used GPT-4, Gemma, llama.cpp, ect… Please share as well!

Thanks in advance.

---

<div class="post-metadata">

**Author:** ![wolfadex](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/wolfadex/32/2461_2.png) [@wolfadex](https://discourse.elm-lang.org/u/wolfadex)\
**Post date:** [February 26, 2024, 7:39pm UTC](https://discourse.elm-lang.org/t/your-problems-writing-elm-code-with-ai-llm/9685/2 "2024-02-26T19:39:44Z")

</div>

I mostly use Copilot for my AI still, and based on that:

- Syntactically incorrect code?

Yes, it most often assumes I’m writing Haskell when I start writing a new module.

- Code that doesn’t typecheck?

Sometimes, though I can’t say with what frequency.

- Hallucinated core or lib functions?

Quite a bit.

- Unidiomatic code?

Sometimes, though I couldn’t say how often

- Any other thing that frustrates you ?

I find that non-Copilot AI tools tend to be even worse regardless of the language. I do use them sometimes but always in a situation where I know roughly what I need and can work out the kinks. Every language I’ve tried, including the popular ones like JS, still have issues with code that’s just broken or not valid syntax.

---

<div class="post-metadata">

**Author:** ![axelbdt](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/axelbdt/32/4404_2.png) [@axelbdt](https://discourse.elm-lang.org/u/axelbdt)\
**Post date:** [February 27, 2024, 1:08am UTC](https://discourse.elm-lang.org/t/your-problems-writing-elm-code-with-ai-llm/9685/3 "2024-02-27T01:08:39Z")

</div>

Thank you for sharing!

Generating syntactically correct code seems like a low hanging fruit, good to know that it hasn’t been picked yet 🙂

---

<div class="post-metadata">

**Author:** ![hrnxm](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/hrnxm/32/5116_2.png) [@hrnxm](https://discourse.elm-lang.org/u/hrnxm)\
**Post date:** [February 27, 2024, 7:00am UTC](https://discourse.elm-lang.org/t/your-problems-writing-elm-code-with-ai-llm/9685/4 "2024-02-27T07:00:48Z")

</div>

When using ChatGPT (3.5) for Elm, I don’t expect any helpful responses anymore. I just check it from time to time to see if something changed, or I use it for rubberducking. Again, with very low expectations.  
My biggest gripe with it is hallucinated libs/functions and impure functions. Like it will straight up use `Random.int` as a function that doesn’t take anything and returns an `Int`. Usually, when I present it with a problem it will abstract the actual complicated part into some lib function that either doesn’t exist or it expects me to write it myself.  
It consistently takes less time to write the solution myself from scratch than to lead ChatGPT to the right answer.

---

<div class="post-metadata">

**Author:** ![lydell](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/lydell/32/178_2.png) [@lydell](https://discourse.elm-lang.org/u/lydell)\
**Post date:** [February 27, 2024, 7:20pm UTC](https://discourse.elm-lang.org/t/your-problems-writing-elm-code-with-ai-llm/9685/6 "2024-02-27T19:20:37Z")

</div>

I also use GitHub Copilot for Elm. I find that it does a good job when I already have a bunch of code (when I’m not writing stuff from scratch). Elm code is pretty “repetitive” and a lot about “just follow the pattern”, which is where Copilot shines. Many times where my instinct is to copy-paste some similar code, and then adjust it, Copilot now often just suggests the end result immediately. For example, today I wrote code for a “previous” button. Copilot then suggested the correct code for the “next” button. I also needed a `map4` for a custom type. I already had `map`, `map2` and `map3`. Copilot suggested a correct `map4` before I wrote a single character. It’s also pretty good at helping with refactoring code, like changing a `case of` on a `Maybe` into a `Maybe.unwrap` (or the other way around).

I really like that way of using Copilot. Sometimes I can imagine exactly all of the code I need to write for a feature (“it’ll be like _that_ feature in most places, except …”), and then using Copilot feels is a bit like pouring my thoughts directly into the code without having to type it all out. It feels a bit faster and less energy draining than doing it all by hand.

The biggest quirk I notice in Elm is that Copilot _really_ likes to close lists. If I’m in the middle of a list of HTML attributes, Copilot often suggests the correct attribute I wanted to add, but also decides to close the list with a `]` even though I’m clearly in the middle of the list. I have not noticed that problem when writing TypeScript.

Another thing I noticed is that Copilot is not as good when your types are in one file, and the code in another. Then I often think “that’s a good guess, Copilot, but if you would have looked in the other file you would have seen that it’s called `phoneNumber` not `phone`, and that there are _three_ fields in this record, not just two”.

I’ve also noticed that sometimes it helps adding parameters to a function _before_ trying to use them. When I write code manually, I often realize that I don’t have the data I need available, so I need to add that as a parameter. Copilot sometimes likes to “invent” those values instead. For example, instead of suggesting using a to-be-defined `customer` variable, it can come up with `Customer.emptyCustomer` instead.

---

<div class="post-metadata">

**Author:** ![timinou](https://avatars.discourse-cdn.com/v4/letter/t/e9c0ed/32.png) [@timinou](https://discourse.elm-lang.org/u/timinou)\
**Post date:** [February 27, 2024, 10:46pm UTC](https://discourse.elm-lang.org/t/your-problems-writing-elm-code-with-ai-llm/9685/7 "2024-02-27T22:46:33Z")

</div>

Same issues here, and also has a hard time using certain patterns I tell it to, or remember I use Elm UI.

Side note: did you check out elm-dev? [GitHub - mdgriffith/elm-dev](https://github.com/mdgriffith/elm-dev). It uses the elm compiler to give you more info about your code. As a context feeder, it might be very useful. Using elm-dev to make an Elm-specific co-pilot is a side idea of mine, so I’m curious to know more about your thesis as it grows!

---

<div class="post-metadata">

**Author:** ![axelbdt](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/axelbdt/32/4404_2.png) [@axelbdt](https://discourse.elm-lang.org/u/axelbdt)\
**Post date:** [February 28, 2024, 1:41am UTC](https://discourse.elm-lang.org/t/your-problems-writing-elm-code-with-ai-llm/9685/8 "2024-02-28T01:41:34Z")

</div>

Thanks for reminding me about elm-dev, definitely something I need to check out!

---

<div class="post-metadata">

**Author:** ![Birowsky](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/birowsky/32/3298_2.png) [@Birowsky](https://discourse.elm-lang.org/u/Birowsky)\
**Post date:** [March 1, 2024, 1:32pm UTC](https://discourse.elm-lang.org/t/your-problems-writing-elm-code-with-ai-llm/9685/9 "2024-03-01T13:32:05Z")

</div>

This would be absolutely awesome Axel. Thanks for pursuing it!

---

<div class="post-metadata">

**Author:** ![paulh](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/paulh/32/5022_2.png) [@paulh](https://discourse.elm-lang.org/u/paulh)\
**Post date:** [March 3, 2024, 8:29am UTC](https://discourse.elm-lang.org/t/your-problems-writing-elm-code-with-ai-llm/9685/10 "2024-03-03T08:29:11Z")

</div>

I’ve been using Copilot for a while and like the way it cuts down on boilerplate, however, (maybe it’s just my suspicious mind), but it doesn’t appear to be 100% honest.

I’ve just started a new project, and 5 minutes ago I started hard coding the clients address into fields of a newly created `Address` type - Copilot autogenerated the `postCode` **value** for me, exactly correct.

So I asked copilot: Where did you get the postCode from?

This is its response:

“”"  
As an AI, I don’t have access to any external databases or specific codebases to “get” a postCode from. If a postCode is mentioned in a code example I provide, it’s likely a placeholder or example value. In a real-world application, postCodes (or postal codes) would typically come from user input or be retrieved from a database or API.  
“”"

The post code copilot provided is clearly **not** a placeholder or example value, it is my clients exact post code. I have just created the Address module, and the clients address does not exist anywhere in my codebase so it couldn’t have already ‘learned’ it - I was retrieving the address for the first time from an internet search engine.

Something similar happened in a previous project just after I started using copilot, but as it was something new to me I put it down to ‘user error’.

This particular ‘incident’ is not user error - AFAIK 😀 😡

---

<div class="post-metadata">

**Author:** ![system](https://canada1.discourse-cdn.com/flex035/uploads/elm_lang/original/1X/50a05e53677a2c3b47776d7abd0f113eb50193a1.png) [@system](https://discourse.elm-lang.org/u/system)\
**Post date:** [March 13, 2024, 8:29am UTC](https://discourse.elm-lang.org/t/your-problems-writing-elm-code-with-ai-llm/9685/11 "2024-03-13T08:29:36Z")

</div>

Closed on March 13, 2024, 8:29am UTC

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.