Oh sorry, the repo is private actually, I completely forgot about that!
Anyway, problem fixed now! I just ran yarn global upgrade create-elm-app, and all is well now. create-elm-app was apparently broken in my environment since it was also unable to create new projects.