Tags will be used to describe what the runtime requirements of packages are. Tags will be inherited from package dependencies, by forming the super-set of all tags on the dependencies.
The following packages will be root tagged as core : elm/core
The following packages will be root tagged as browser : elm/browser
The following packages will be root tagged as vdom : elm/virutal-dom
All other packages will inherit some combination of core , browser and vdom tags.
A potential alternate compiler for the Elm language, as opposed to The Elm Architecture, may support only a sub-set of these tags. This feature is not useful at the present, but is included now, so that it is part of the package server API from the start.
Motivation for the package domain tags:
The intention is to be able to have an Elm build system that only supports say core , for running Elm code outside of the browser, with no virtual DOM to render as a view .
A future version may cater for private tags. For example ACME Corp can tag all its packages as acme , making it obvious when code depends on that private package domain. The private package domain at the head of those packages will not allow them to be published upstream, unless the private tag is removed by using only packages with public tags.
A build can be set to fail, if it includes packaging domains that it is not configured to allow. For example, ACME Corp might publish a popular open source library that gives it kudos alongside its regular business. This would not include the acme domain, and if a developer accidentally starts including that domain, the build will fail.
The compiler seems to honor the environment https_proxy, like env https_proxy=http://my.proxy.local:3128 elm init, this could be a way to go without compiler patching, maybe someone could use node.js or an other app-server as a proxy, but also TLS needs to be broken…
Sometimes I think, I have something like a “stockholme syndrome” regarding elm.
What do you mean by that? You mean it checks it gets a valid cert because its over HTTPS, and a local package server will have to serve up a self-signed cert, so maybe the compiler complains at that point?
Its a good idea anyway, I’ll try it out and see what happens.
For what it’s worth, there are a couple of use cases where I would find a tool like this to be useful.
Alternate ecosystems (for example, packages intended for use with platforms that build on Elm such as elm-pages, lamdera, etc.)
Commercial (paid) packages
I think that what you’re describing about being able to “tag” packages as having certain runtime requirements could pair nicely with ecosystem use case. It would be nice to be have a way to avoid cluttering up the main Elm package repository with packages that depend on specific runtime requirements.
For the commercial use case, I know there are some people working on some commercial Elm packages, and right now there isn’t a good distribution channel. (I happen to be working on a commercial NPM package for Elm at the moment, but at some point I could imagine wanting to work on a paid Elm package, too). It could be a really great thing to have a tool that could help with distribution of commercial packages.
I hope those use cases are helpful, I’d be happy to discuss them more if you have any questions!
Yes, that is precisely my thinking too - that is why I put:
“If a hacked version of the compiler is needed, it must have the publish command removed, so as to ensure the package.elm-lang.org site is treated as read-only, and source of truth.”
It is also the intended use case for tags.
An alternate compiler with different runtime requirements should use its own set of tags to describe them. The idea being that it is very clear that these are not part of the Elm kernel.
It should also be very clear that having a different runtime carries a risk, if depending on Elms undocumented kernel interface.
It should also be very clear that there is absolutely no obligation on Elm to avoid breaking changes to that interface, or to port third party packages using it, to new versions of Elm.
Another possibility is that an Elm implementation could arise that compiles to a completely different back-end. A known example is Morphir, which compiles Elm to Scala. As it is focussed on business rules, I am guessing its runtime requirement is just core, and it does not need vdom or browser at all:
With the tags it would be easy to set up a mirror of the Elm package site, just for core, and get all packages that can compile on core alone - mostly data structures/algorithms/update helpers/etc. Anyone using Morphir would then have a clear view on what is available to them.
Also worth noting that Morphir is not an identical implementation of Elm as the original - it has 64-bit integers for one thing. Some kind of formal language spec and test suite might eventually be needed for Elm. The beauty of Elm is that it is such a small language, so this is a less daunting proposition than for many languages.
elm-pages is an interesting example too. I originally assumed it would not depend on browser, but I see it does - well of course it does, because it runs client side too. I think it would be possible to split the client side out into a separate package, if you ever wanted too. But elm-pages was what got me thinking that vdom and browser should be separate tags - since you need vdom for server side rendering, but not browser. I can imagine a future where elm-pages has a different implementation when doing server side rendering, but runs with pure Elm on the client side.
Also note, in the spec I put:
Confirm the package does not contain ports.
Confirm the package does not contain kernel code.
No intention at the present time of overriding these restrictions. I just want to focus on doing private packages with minimal disruption to the current system. The reason for introducing the tags idea now, is that its going to be easier to get that in right from the start, even though it is not immediately useful.
Depending on how elm compiler does HTTPS, it might not accept self-signed certificate and might require proper certificate instead (meaning you need local CA so you can create proper spoofed certificate for any website).
I have such a setup on localhost, but I havn’t tried playing with HTTPS proxies before …
This does not make sense to me. You would need a fork of the compiler in order to have it use a different (private) domain for the packages. With this new domain for packages running, it would make sense to keep the publish functionality because all the publishing will be done ONLY to the private packages server. The packages server would have to also be forked and logic implemented that would treat differently the packages belonging to the private company.
If one needs to publish to the official package repository, they would use the official elm binary.
But for publishing to a private repository, the idea is to use the new command line tool I am speccing out. Something along these lines:
> eco-install publish
Reading `elm.json` ...
Publishing to `packages.acme.com`.
... Package uploaded.
... Git tag correct.
... Checksum calculated.
... API version is correct, 2.1.0.
... Compilation succeeded using 'elm 0.19.1'
... Publishing succesful.
Its not 100% clear yet if a fork is needed (if HTTP_PROXY works, for example).
The reason I put this in, is that I just want to make absolutely sure that it is not possible to publish a private package to package.elm-lang.org accidentally. For example, there is already an Elm compiler out there that removes native code restrictions, and someone has used it to publish a package with kernel code to package.elm-lang.org. I don’t think that is right, although it is not really causing any harm since the official compiler will not build that package. However, if it happened a lot, it might start getting hard to understand what packages you can use, and what needs some special compiler.
So my policy is - leave the existing Elm system alone, and make sure I don’t do anything that might break it. Treat it as an upstream system that is a read-only source of truth, so far as eco is concerned.
You are right though, in that a forked compiler could have the publish command, and publish to a private repository. Personally, I think tight coupling of compiler and package system is not a good design. I am thinking a system like Java and Maven is a better way - Java is the compiler, Maven is the build/packaging tool.
As I read the documentation of mitmproxy, a CA-certificate from mitmproxy must be installed on the client. This mitmproxy-CA-certificate is working system-wide. That’s what I mean by breaking TLS, as an example.
The Elm-compiler probably checks the certificate of package.elm-lang.org, created by mitmproxy in this case, using the system-certificates. It trusts the mitmproxy-CA, because the system trusts the mitmproxy-CA.
(O.T.: This is nearby a common way used in companies to find malware encrypted in HTTPS-traffic, causing often web-sockets not to work, as the web-browsers use the same “CONNECT”-method to the proxy for requesting a connection a server, independent of the used protocol, like HTTPS or WSS, while the used proxy can only handle HTTPS)
There are more advanced checks, but not used very often, like HPKP and now CAA.
yes, the reason why I need --ssl-insecure is that the certificate for box (my synonym for localhost) is signed by my personal CA, which is in system-certificates, but for some reason mitmproxy doesn’t seem to be using system certificates and it complains that certificate of box isn’t trusted.
@rupert I haven’t touched elm-pubgrub for a while since development switched to rust, so there are still bugs in there. But if you need a robust dependency solver in elm at some point let me know and I could take some time to update it.