Help test the new npm elm package!

Me, @supermario and @evancz (with the help of some more folks) are working on a new version of the elm npm package.

  • Changed: Binaries are downloaded by npm instead of custom code in the elm package. No more dependencies or deprecation warnings!
  • New: Native macOS ARM binary. No more Rosetta!
  • New: Linux ARM binary. Useful for people with such computers, and for Docker on macOS ARM!

Please try it out and let me know how it goes! We’re testing using a temporary npm package until we know everything works.

npm install @lydell/elm

Or replace "elm": "0.19.1-5" in your package.json with "@lydell/elm": "0.19.1-9" and run npm install.

I’m especially interested if you:

  • Use a package manager other than npm.
  • Use a very old Node.js or npm for some reason.
  • Use Linux ARM.
  • Use --ignore-scripts.
  • Are behind a corporate proxy or something like that and configure things like HTTPS_PROXY.
  • Use a “probably not the most common setup” in general.

See the #core-coordination channel on Slack if you want all the details.

39 Likes

I installed it on my M1 iMac. elm make src/Main.elm works great for https://mammudeck.com. Faster, though improving three seconds to one second isn’t a big deal. Main.elm has 20444 lines, 630K bytes, and a lot of dependencies.

4 Likes

Really naive question: how does this relate to / interop with Elm Tooling CLI?

2 Likes
  • This provides two new binaries for it! I’m going to update it later.

  • This makes the elm-tooling CLI less relevant. I intend to open issues for elm-format and elm-json as well later, about copying the npm approach, which would make it even less relevant.

    • Package size: I haven’t measured yet, but I expect the new elm package to be slightly smaller than elm-tooling (which is already tiny).
    • Installation speed: Will probably be basically no difference for uncached installs. Technically elm-tooling might be faster since it needs exactly 1 npm package request, while the elm package would need 1 + number of different binaries (it won’t download them all, but needs to make a metadata request for all of them). But we’re talking milliseconds here.
    • Disk space usage: With elm-tooling you get exactly one of each binary on your computer; with npm you get one per project. You can solve this more generally by using pnpm though.
    • Execution speed: elm-tooling will be slightly faster on Windows. An npm package is forced to execute the “binary” it provides with node (which in turn executes elm in our case), which adds around 100 ms of execution time. elm-tooling can instead use a .cmd file which executes much faster.
    • Security: elm-tooling verifies downloads with SHA256 hashes; npm verifies all packages (including the ones where we distribute elm binaries) with SHA512 hashes.
    • elm-test-rs: Currently has no npm package – elm-tooling is “the way” to install it. However, this new way of installing is pretty lightweight so maybe elm-test-rs will get an npm package now.
    • Ease of use: Probably easier to use the new elm npm package than elm-tooling. But maybe not if you count setting up pnpm (if you choose to use that for disk space savings).
    • Mirrors: elm-tooling is currently hardcoded to download from github.com, while there are npm mirrors you can use if npmjs.com is not available for you.
9 Likes

I’m running a Mac with an M1 Pro chip and am doing elm development inside a VS Code dev container (via Docker) running aarch64 Debian GNU/Linux 11 (bullseye). I installed the package inside the container and it worked great! Thanks!

1 Like

Fantastic news, thanks for this.

Sounds good!

As far as I’m concerned, it doesn’t work with my current setup with either elm 0.19.1-5 or @lydell/elm 0.19.1-9, so nothing new here. My setup is Gulp with Yarn 2, so I guess the gulp-elm plugin that I’m using doesn’t know about Yarn 2. (It only works if elm is absent from my package.json, and the plugin then relies on my global elm install.) Looking forward to switching to elm-watch on this project :smile:

So I tried updating globally: removed global elm with npm, installed global @lydell/elm with npm. I see no change when I build so I guess it’s working!

(In case the fix to my unrelated issue above is obvious to anyone, the error I’m getting is “SyntaxError: Invalid or unexpected token” on the beginning ELF characters. I guess that means it’s trying to read a binary as JavaScript? Sorry, that’s totally irrelevant to the post)

Yep, neither elm nor @lydell/elm work with Yarn 2/3. (It might work if you don’t use PnP or whatever – I’m not that into Yarn 2/3.)

@lydell/elm took a lot of inspiration from esbuild’s npm installer. They have some very elaborate workarounds for Yarn 2/3. Since the current elm package does not work either with Yarn 2/3 I decided to scope that out for some other time. Interesting to hear from a Yarn 2/3 user though!

(What happens is that the package replaces its JS entrypoint with the executable itself as an optimization. Yarn 2/3 does not seem to like that.)

Oh okay, so Yarn (with PnP) is the issue then, not Gulp or the gulp-elm plugin. Might consider going back to npm at some point then :confused: I would argue it’s a pretty big issue that it doesn’t work with yarn, but maybe it’s just yarn’s fault, not anticipating enough potentially breaking use cases. Also it’s not blocking since you can just run the global binary.

Thanks for the info!

Just wanted to clarify for other’s reading that Yarn 1 works fine, it’s Yarn “Berry” (v2+) that are problematic.

Is it Yarn’s fault? Well, read this issue: [Bug] SyntaxError when running a "bin" file that isn't authored in JS · Issue #882 · yarnpkg/berry · GitHub (it mentions Elm a couple of times!)

2 Likes

Wow! This is great news!!! I’m sitting behind a corporate firewall and have not been able to install elm through npm due to then elm binary not being downloaded through internal npm proxy.

I have just tested with "@lydell/elm": "0.19.1-9" and it worked great. Thanks!!! :smiley:

If the same solution could be applied to elm-format and the other elm-tools it would be awesome.

2 Likes

Just did it on my work computer, an Intel Mac, and I can confirm that it actually works pretty seamlessly to install and run the elm binary.

Thank you everyone for testing! Much appreciated.

Follow this PR to get updates about the progress of the upcoming new official elm npm package: Free the npm package from third party dependencies by lydell · Pull Request #2287 · elm/compiler · GitHub

It’ll take a while though because there is a lot to review and December can be a busy month.

7 Likes

I have used my 8 years old MacBook Air macOS Mojave for this purpose successfully.
I have first removed the official version with npm -g uninstall elm and installed globally the temporary version npm install @lydell/elm
(To be sure, only the new elm is running, elm must be deleted also from /usr/local/bin folder.)
All seems to work excellent, at least no new problems appeared so far.
Here are some little examples of my tests with elm reactor and elm-spa.



(See also here!)

Working with great with pnpm on my Ubuntu 22.10 machine. Note that I have the following lines in my .npmrc file:

shamefully-hoist=true
auto-install-peers=true

(Adding a comment to keep this thread alive for a bit longer.)

6 Likes

Hello,
I tried on a prooted linux install which is on my android phone. It failed with following output and logs.

system
lscpu
Architecture: aarch64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 8
On-line CPU(s) list: 0-7
Vendor ID: Qualcomm
Model name: Kryo V2
Model: 4
Thread(s) per core: 1
Core(s) per socket: 4
Socket(s): 1
Stepping: 0xa
CPU max MHz: 1612.8000
CPU min MHz: 633.6000
BogoMIPS: 38.40
Flags: fp asimd evtstrm aes pmull sha1
sha2 crc32
Model name: Falkor V1/Kryo
Model: 2
Thread(s) per core: 1
Core(s) per socket: 4
Socket(s): 1
Stepping: 0xa
CPU max MHz: 1804.8000
CPU min MHz: 1113.6000
BogoMIPS: 38.40
Flags: fp asimd evtstrm aes pmull sha1
sha2 crc32

output
npm install @lydell/elm
npm ERR! Exit handler never called! npm ERR! This is an error with npm itself. Please report this error at:
npm ERR! https://github.com/npm/cli/issues
npm ERR! A complete log of this run can be found in: npm ERR! /home/hwan/.npm/_logs/2022-12-16T22_09_11_976Z-debug-0.log

logs
0 verbose cli /usr/lib/nodejs/node-v18.12.0-linux-arm64/bin/node /usr/lib/nodejs/node-v18.12.0-linux-arm64/bin/npm
1 info using npm@9.1.1
2 info using node@v18.12.0
3 timing npm:load:whichnode Completed in 4ms
4 timing config:load:defaults Completed in 11ms
5 timing config:load:file:/usr/lib/nodejs/node-v18.12.0-linux-arm64/lib/node_modules/npm/npmrc Completed in 4ms
6 timing config:load:builtin Completed in 6ms
7 timing config:load:cli Completed in 9ms
8 timing config:load:env Completed in 1ms
9 timing config:load:project Completed in 6ms
10 timing config:load:file:/home/hwan/.npmrc Completed in 1ms
11 timing config:load:user Completed in 2ms
12 timing config:load:file:/usr/lib/nodejs/node-v18.12.0-linux-arm64/etc/npmrc Completed in 2ms
13 timing config:load:global Completed in 2ms
14 timing config:load:setEnvs Completed in 7ms
15 timing config:load Completed in 50ms
16 timing npm:load:configload Completed in 50ms
17 timing npm:load:mkdirpcache Completed in 3ms
18 timing npm:load:mkdirplogs Completed in 2ms
19 verbose title npm install @lydell/elm
20 verbose argv “install” “@lydell/elm”
21 timing npm:load:setTitle Completed in 9ms
22 timing config:load:flatten Completed in 18ms
23 timing npm:load:display Completed in 24ms
24 verbose logfile logs-max:10 dir:/home/hwan/.npm/_logs/2022-12-16T22_09_11_976Z-
25 verbose logfile /home/hwan/.npm/_logs/2022-12-16T22_09_11_976Z-debug-0.log
26 timing npm:load:logFile Completed in 27ms
27 timing npm:load:timers Completed in 0ms
28 timing npm:load:configScope Completed in 1ms
29 timing npm:load Completed in 123ms
30 silly logfile start cleaning logs, removing 2 files
31 timing config:load:flatten Completed in 7ms
32 timing arborist:ctor Completed in 6ms
33 silly logfile done cleaning log files
34 timing idealTree:init Completed in 114ms
35 timing idealTree:userRequests Completed in 7ms
36 silly idealTree buildDeps
37 silly fetch manifest @lydell/elm@*
38 verbose shrinkwrap failed to load node_modules/.package-lock.json out of date, updated: node_modules/@lydell
39 http fetch GET 200 https://registry.npmjs.org/@lydell%2Felm 96ms (cache hit)
40 silly placeDep ROOT @lydell/elm@0.19.1-9 OK for: want: *
41 silly fetch manifest @lydell/elm_darwin_arm64@0.19.1-0
42 silly fetch manifest @lydell/elm_darwin_x64@0.19.1-1
43 silly fetch manifest @lydell/elm_linux_arm64@0.19.1-2
44 silly fetch manifest @lydell/elm_linux_x64@0.19.1-0
45 silly fetch manifest @lydell/elm_win32_x64@0.19.1-0
46 http fetch GET 200 https://registry.npmjs.org/@lydell%2Felm_darwin_x64 65ms (cache hit)
47 http fetch GET 200 https://registry.npmjs.org/@lydell%2Felm_linux_arm64 76ms (cache hit)
48 http fetch GET 200 https://registry.npmjs.org/@lydell%2Felm_linux_x64 79ms (cache hit)
49 http fetch GET 200 https://registry.npmjs.org/@lydell%2Felm_win32_x64 82ms (cache hit)
50 http fetch GET 200 https://registry.npmjs.org/@lydell%2Felm_darwin_arm64 101ms (cache hit)
51 timing idealTree:#root Completed in 228ms
52 silly placeDep ROOT @lydell/elm_darwin_arm64@0.19.1-0 OK for: @lydell/elm@0.19.1-9 want: 0.19.1-0
53 silly placeDep ROOT @lydell/elm_darwin_x64@0.19.1-1 OK for: @lydell/elm@0.19.1-9 want: 0.19.1-1
54 silly placeDep ROOT @lydell/elm_linux_arm64@0.19.1-2 OK for: @lydell/elm@0.19.1-9 want: 0.19.1-2
55 silly placeDep ROOT @lydell/elm_linux_x64@0.19.1-0 OK for: @lydell/elm@0.19.1-9 want: 0.19.1-0
56 silly placeDep ROOT @lydell/elm_win32_x64@0.19.1-0 OK for: @lydell/elm@0.19.1-9 want: 0.19.1-0
57 timing idealTree:node_modules/@lydell/elm Completed in 32ms
58 timing idealTree:node_modules/@lydell/elm_darwin_arm64 Completed in 0ms
59 timing idealTree:node_modules/@lydell/elm_darwin_x64 Completed in 0ms
60 timing idealTree:node_modules/@lydell/elm_linux_arm64 Completed in 0ms
61 timing idealTree:node_modules/@lydell/elm_linux_x64 Completed in 0ms
62 timing idealTree:node_modules/@lydell/elm_win32_x64 Completed in 0ms
63 timing idealTree:buildDeps Completed in 309ms
64 timing idealTree:fixDepFlags Completed in 1ms
65 timing idealTree Completed in 439ms
66 timing reify:loadTrees Completed in 447ms
67 timing reify:diffTrees Completed in 7ms
68 silly reify mark retired [ ‘/home/hwan/node_modules/@lydell/elm_darwin_x64’ ]
69 silly reify mark retired [ ‘/home/hwan/node_modules/@lydell/elm_linux_x64’ ]
70 silly reify mark retired [ ‘/home/hwan/node_modules/@lydell/elm_win32_x64’ ]
71 silly reify mark retired [ ‘/home/hwan/node_modules/@lydell/elm_darwin_arm64’ ]
72 silly reify mark retired [
72 silly reify ‘/home/hwan/node_modules/@lydell/elm’,
72 silly reify ‘/home/hwan/node_modules/.bin/elm’
72 silly reify ]
73 silly reify mark retired [ ‘/home/hwan/node_modules/@lydell/elm_linux_arm64’ ]
74 silly reify moves {
74 silly reify ‘/home/hwan/node_modules/@lydell/elm_darwin_x64’: ‘/home/hwan/node_modules/@lydell/.elm_darwin_x64-ZZ9dWhvB’,
74 silly reify ‘/home/hwan/node_modules/@lydell/elm_linux_x64’: ‘/home/hwan/node_modules/@lydell/.elm_linux_x64-jUyVDcq9’,
74 silly reify ‘/home/hwan/node_modules/@lydell/elm_win32_x64’: ‘/home/hwan/node_modules/@lydell/.elm_win32_x64-isxq6iMb’,
74 silly reify ‘/home/hwan/node_modules/@lydell/elm_darwin_arm64’: ‘/home/hwan/node_modules/@lydell/.elm_darwin_arm64-0kX48DKK’,
74 silly reify ‘/home/hwan/node_modules/@lydell/elm’: ‘/home/hwan/node_modules/@lydell/.elm-2Ei54wkB’,
74 silly reify ‘/home/hwan/node_modules/.bin/elm’: ‘/home/hwan/node_modules/.bin/.elm-FTZ31yKf’,
74 silly reify ‘/home/hwan/node_modules/@lydell/elm_linux_arm64’: ‘/home/hwan/node_modules/@lydell/.elm_linux_arm64-L4tvrRUI’
74 silly reify }
75 timing reify:retireShallow Completed in 24ms
76 timing reify:createSparse Completed in 4ms
77 timing reify:loadBundles Completed in 0ms
78 silly audit bulk request {
78 silly audit tslib: [ ‘2.4.1’ ],
78 silly audit ‘upper-case’: [ ‘2.0.2’ ],
78 silly audit ‘@lydell/elm’: [ ‘0.19.1-9’ ],
78 silly audit ‘@lydell/elm_darwin_arm64’: [ ‘0.19.1-0’ ],
78 silly audit ‘@lydell/elm_darwin_x64’: [ ‘0.19.1-1’ ],
78 silly audit ‘@lydell/elm_linux_arm64’: [ ‘0.19.1-2’ ],
78 silly audit ‘@lydell/elm_linux_x64’: [ ‘0.19.1-0’ ],
78 silly audit ‘@lydell/elm_win32_x64’: [ ‘0.19.1-0’ ]
78 silly audit }
79 verbose reify failed optional dependency /home/hwan/node_modules/@lydell/elm_darwin_arm64
80 silly reify mark deleted [ ‘/home/hwan/node_modules/@lydell/elm_darwin_arm64’ ]
81 verbose reify failed optional dependency /home/hwan/node_modules/@lydell/elm_win32_x64
82 silly reify mark deleted [ ‘/home/hwan/node_modules/@lydell/elm_win32_x64’ ]
83 verbose reify failed optional dependency /home/hwan/node_modules/@lydell/elm_linux_x64
84 silly reify mark deleted [ ‘/home/hwan/node_modules/@lydell/elm_linux_x64’ ]
85 verbose reify failed optional dependency /home/hwan/node_modules/@lydell/elm_darwin_x64
86 silly reify mark deleted [ ‘/home/hwan/node_modules/@lydell/elm_darwin_x64’ ]
87 timing reifyNode:node_modules/@lydell/elm_darwin_arm64 Completed in 108ms
88 timing reifyNode:node_modules/@lydell/elm_win32_x64 Completed in 109ms
89 timing reifyNode:node_modules/@lydell/elm_linux_x64 Completed in 109ms
90 timing reifyNode:node_modules/@lydell/elm_darwin_x64 Completed in 109ms
91 timing reifyNode:node_modules/@lydell/elm Completed in 456ms
92 http fetch POST 200 https://registry.npmjs.org/-/npm/v1/security/advisories/bulk 720ms
93 timing auditReport:getReport Completed in 725ms
94 silly audit report {}
95 timing auditReport:init Completed in 0ms
96 timing reify:audit Completed in 730ms
97 timing reifyNode:node_modules/@lydell/elm_linux_arm64 Completed in 1215ms
98 timing reify:unpack Completed in 1216ms
99 timing reify:unretire Completed in 1ms
100 timing build:queue Completed in 6ms
101 timing build:link:node_modules/@lydell/elm Completed in 20ms
102 timing build:link Completed in 21ms
103 timing npm Completed in 2086ms
104 verbose unfinished npm timer command:install 1671228552266
105 verbose unfinished npm timer reify 1671228552307
106 verbose unfinished npm timer reify:build 1671228554032
107 verbose unfinished npm timer build 1671228554033
108 verbose unfinished npm timer build:deps 1671228554034
109 verbose unfinished npm timer build:run:install 1671228554064
110 info ok
111 error Exit handler never called!
112 error This is an error with npm itself. Please report this error at:
113 error https://github.com/npm/cli/issues
114 error A complete log of this run can be found in:
114 error /home/hwan/.npm/_logs/2022-12-16T22_09_11_976Z-debug-0.log

As you seem to have issues with npm itself, you could instead download the elm binary for Linux/ARM64 manually, e.g.

wget https://unpkg.com/@lydell/elm_linux_arm64/elm

Thanks! It works. Now I can play with Elm on my phone.

Someone on Discord reported Android not being supported either. (Just to be clear: Neither the latest official elm package nor @lydell/elm does, so it’s not a regression.)

However, when they tried compiling a “Hello World” Elm program it didn’t work, so I didn’t look into supporting it in my package.