ElmX - find and delete unused modules

[paulh]

Hi all,

I’ve been learning some c# to build some cross platform CLI’s and Apps, and after reading this post a couple of days ago about removing unused modules in a project it seemed like a good problem to try and solve.

I’ve been wanting to give something back to this great community for some time, so this is the start of what I hope will end up being a feature rich CLI to help with developing Elm projects.

At the moment it enables you to search a project and either:

1. Show all the unused modules.
2. Rename all the unused modules by adding a tilde (~) to the front of the filename. Allowing you to run the compiler and check all is still ok, before removing them.
3. Interactively delete the unused modules one at a time, choosing whether to delete each file or not.
4. Delete all unused modules in a single operation.

Of course, the safest way to use this would be to check all your files into version control, run the tool, and if it deletes a file it shouldn’t - which I haven’t encountered - you can then simply checkout your files to recover them.

I want to add lots more features, but decided to put this out there now to see what you all think.

The repo is here where you can find more info, and releases for OSX and various Linux distros. I don’t have a Windows machine at the moment, so haven’t been able to compile for Windows, but you can compile from source if you are working on a Windows machine.

I hope this is useful, and look forward to adding more features over the coming weeks, months and hopefully years if people like the idea.

Thanks to everyone in this community for all the help I’ve received over the years, hopefully as this CLI matures it will help others.

Paul

[paulh]

Just released v1.0.2 - Major speed improvements.

I’m testing on a project with 215 files and 273 unique imports and the processing time has dropped from an average of 15 seconds to an average of 0.5 seconds

[lydell]

Out of curiosity, I searched for “source-directories” in your code:

https://github.com/search?q=repo%3Aphollyer%2FElmX%20source-directories&type=code

I expected to find code that reads elm.json files and then the "source-directories" field (defaulting to "src" for packages). But it return zero results! (Maybe the search was too bad though.)

How do you know what file a module name maps to without source-directories?

[paulh]

Hi Simon,

I’ve taken the view that an import statement can be mapped to a file path.

So given a directory to search, I first look for elm.json, and then bring in a list of all the files ending in .elm in all subdirectories from there.

All the import statements in the list of .elm files are then read, converted into a file path, sorted into alphabetic order, and all duplicates are removed. This list is then compared with the original list of file paths ending in .elm.

If one of the files in the original list is not found in the transformed list of import statements then it is deemed as being unused.

So, given the following files:

Main.elm
Pages/Home.elm
Pages/Contact.elm
Shared/Utils.elm
Shared/Extra.elm

And the following imports are extracted from those files:

import Element as El
import Pages.Home
import Pages.Contact as Contact
import Shared.Utils exposing (func1, func2)
import SomePackage

Then the imports can be transformed into:

Element.elm
Pages/Home.elm
Pages/Contact.elm
Shared/Utils.elm
SomePackage.elm

From there it is simple enough to deduce that the real file Shared/Extra.elm is not imported by any other files and therefore is not used.

I did consider reading in “source-directories” but figured I didn’t need to and that I should keep it simple to start with by using the known module structure of Elm.

At the moment any external files that are brought into a project via “source-directories” are not part of the search. I’m still getting to grips with c# and wanted to focus on the basic/simplest structure of an Elm project first. But this is certainly something to consider going forward.

If there is something else I’ve overlooked please do let me know, and I hope the explanation above makes sense (It’s nearly 1:30am here so a bit tired )

EDIT: I’m only targeting Elm projects at the moment, not packages. I should probably add that to the README tomorrow morning.

[lydell]

Given these three files:

module Main exposing (main)
-- no imports

-- unused
module Shared exposing (thing)
import Extra

-- “used” by Shared, which in turn is unused so this one is in effect unused too
module Extra exposing (thingy)

Will the first elmx run delete only Shared.elm? And the next run Extra.elm? Or am I misunderstanding your algorithm?

[paulh]

Yes you are correct, I hadn’t yet considered that scenario, so thank you for bringing it to my attention.

Initial thoughts might be:

1. To stat the imports per file, and the files per import as each file is read, so that as a file is marked as unused, I may then be able to determine where else it’s imports are used, and react accordingly.

2. It doesn’t have to exit after the first pass. It could rerun internally with the updated results of the previous run until zero files are found.

The second option is clearly the simplest to implement, but the first is probably the most efficient.

I’ll spend some time on this today and over the weekend to see what I can come up with. (I’m full time carer for my mum, so I spend a lot of time indoors with her and this little project gives me something else to focus on ).

[lydell]

Here’s why source-directories matter.

Let’s say you find these three files (among others):

• src/Foo/Helpers.elm
• src/OtherFoo/Helpers.elm
• src/Foo/Page/Foo/Helpers.elm

And you find this import (among) others:

• import Foo.Helpers

You turn Foo.Helpers into Foo/Helpers.elm, and then do a “string contains” check on each file name. I understand where “contains” comes from – you don’t know the source directories (src/ in this case).

However, all the above files match Foo/Helpers.elm, so all three count as used just from import Foo.Helpers, which is incorrect. This can lead to some unused files not being detected.

---

Another way that unused files might not be detected, is if import Something is inside comments and strings (both of which can be multiline!)

{- Commented out for now:
import Something.Cool
-}

templateCode = """
import Something.Cool
"""

---

A – luckily unlikely – way that your tool might remove too many files, is when import statements have unusual formatting so that your tool does not extract them correctly:

• It is legal to have a newline between import and the module name:

  import
    Something.Cool
  

  Luckily, elm-format never does that.

• It is legal to have comments between import and the module name:

  import{-{-{--}-}-}Something.Cool
  

  Luckily, I’ve never seen Elm code with comments even near imports.

---

One final edge case thought – what happens if the code accidentally has a circular import? If 5 unused files import each other in a circle, they won’t be removed? And the user might not notice that circular import, since the files are unused so elm make never runs on them.

[paulh]

Thank you so much for such great feedback/insight. I’m going to enjoy solving the cases you outline.

[wolfadex]

At work and in personal projects I’ve seen the source-directories point to directories outside of the one containing elm.json. It will likely be worth using the information in elm.json for determining where to look for modules.