Project size and minifications

My application is written with Phoenix 1.5.16 backend and Elm 0.19 frontend. I have a suspicion that the bundled code is larger than it should be, with Webpack informing that the parsed and statsize of the elm-file is 1.42 MB and the output of the script in this post, Asset size is:

Compiled size: 0 bytes (elm.js)
Minified size: 319 182 bytes (elm.min.js)
Gzipped size: 87 436 bytes 

When I run find src/ -type f -name '*.elm' | xargs cat | wc -l, I have only written ~23k lines of code. This post, Small assets without the headache, talked about an elm project with ~50k lines of code ending up as 100kb. Am I missing something or mixing something up, or is the compiled file abnormally large? And if so, how can I debug it to try to figure out why/how?

My elm.json files look like this, so I don’t think the dependencies should cause the file size.

{
"type": "application",
"source-directories": [
    "src",
    "elm-form/src",
    "date-picker"
],
"elm-version": "0.19.1",
"dependencies": {
    "direct": {
        "NoRedInk/elm-json-decode-pipeline": "1.0.0",
        "ccapndave/elm-update-extra": "4.0.0",
        "cuducos/elm-format-number": "6.0.2",
        "elm/browser": "1.0.1",
        "elm/bytes": "1.0.7",
        "elm/core": "1.0.2",
        "elm/file": "1.0.5",
        "elm/html": "1.0.0",
        "elm/http": "2.0.0",
        "elm/json": "1.1.2",
        "elm/parser": "1.1.0",
        "elm/regex": "1.0.0",
        "elm/time": "1.0.0",
        "elm/url": "1.0.0",
        "elm-community/json-extra": "4.0.0",
        "elm-community/list-extra": "8.2.2",
        "elm-community/maybe-extra": "5.0.0",
        "justinmimbs/date": "3.1.2",
        "myrho/elm-round": "1.0.4",
        "rtfeldman/elm-iso8601-date-strings": "1.1.2"
    },
    "indirect": {
        "avh4/elm-color": "1.0.0",
        "elm/virtual-dom": "1.0.2"
    }
},
"test-dependencies": {
    "direct": {
        "elm-explorations/test": "1.2.0"
    },
    "indirect": {
        "elm/random": "1.0.0"
    }
}

}

Here some data for one of our projects:

LoC: ~90k
Size: 2 374 513
Minified: 560 949
Gzip: 174 323

1 Like

Are you compiling Elm with the --optimize flag and without the --debug flag? That’s supposed to reduce the final asset’s size.

Yes, I am.

My rule for elm i webpack is

{
    test: /\.elm$/,
    exclude: [/elm-stuff/, /node_modules/],
    use: [
      {
        loader: 'elm-webpack-loader', //?verbose=true&warn=true&', //debug=true',
        options: {
          cwd: path.resolve(__dirname, 'elm'),
          optimize: true
          }
      }
     ]
  },

and I use Uglify:

optimization: {
    minimizer: [
      new UglifyJsPlugin({ cache: true, parallel: true, sourceMap: false }),
      new OptimizeCSSAssetsPlugin({})
    ] 
 }

Given that when I run uglify manually, the size is a lot smaller than when webpack does, maybe uglify isn’t ran on the elm-code when building with Webpack?

I made a little script to get some rough statistics on the origin of all code in the compiled js and how much each elm module contributes. Hope it can help you to get some insights.

For examle:

node count.js < compiled-from-elm.js

will print out something like this:

45838 author/project
    1445 Webbhuset.Ecom.Data.Currency
    1358 Webbhuset.Checkout.Cart
    ...
7815 mdgriffith/elm_ui
    3815 Internal.Model
    1941 Internal.Style
    1216 Element.Input
     540 Element
      87 Element.Border
      82 Internal.Flag
      45 Element.Lazy
      36 Element.Font
      35 Element.Keyed
      10 Element.Background
       5 Element.Region
       3 Element.Events
2514 hecrj/html_parser
    2129 Html.Parser.NamedCharacterReferences
     385 Html.Parser

The numbers are the line counts per package and module in the compiled javascript file.

5 Likes

It seems, there is some problem with dead-code elimination, or mangeling, causing your code to be too big.

Here was a discussion about using webpack and Elm.

Ha funny. I made something similar: https://github.com/matheus23/elmjs-inspector

I ultimately abandoned the project, as I struggled deploying a binary typescript npm package. Maybe I should try again. There seems to be a need for such a tool.

@teatimes, I suspect the thread linked by @hans-helmut may have your answer:

It seems like the root issue there was that the default uglifyOptions are not right for Elm, so @rupert gave a recommendation of how to make those options match the recommended script here:

But it looked like the author ended up using a β€œseparate plugin” for Elm code such that the same uglify options were not used on JS code from different origins:

Curious to hear what the root cause was if you figure this out!

2 Likes

I looked at that thread earlier today, and tried to add the uglifyOptions, but don’t have the impression it makes any difference. I also looked at elm-minify, since it seemed like it was the solution, but it is abandoned.

The script didn’t work for me, since it seems like the .js had only three β€œ\n” in it. Not sure if there is any configurations that determines how the .js file is created, and how it is split up in lines?

Did you run it on the minified file?

I suspect people will be able to help more if you can share more of your code. For example, is there a way to make a gist of the whole webpack setup? Directory structure? Etc.

Given that you can get the small sizes calling uglifyjs directly, it seems like its is something in webpack or some configuration detail. Or maybe that extra code is getting thrown in by the server code. It’s really hard to pin it down with the information we have so far.

Talking it through on the Elm slack may also be a better fit because the feedback loop on sharing additional information can be faster.

We have a webpack setup with both regular js and Elm. The advanced uglify options mentioned above only work on the js code compiled by Elm, but should not be used for regular js. So what we did is create a custom minimize loader which is only included in the elm loaders.

So in the webpack config you get something like this:

rules: [
	{
		test: /\.elm$/,
		use: [
			{
				loader: [path to elm minimize loader]
			},
			{
				loader: "elm-webpack-loader"
			}
		]
	}
]

And the minimize loader looks like this. Note that it uses terser instead of uglify. But the idea is the same. Also note that at the moment terser is the default optimizer for webpack.

const terser = require("terser");

const terserOptions = {
    mangle: false,
    compress: {
        pure_funcs: [
            "F2",
            "F3",
            "F4",
            "F5",
            "F6",
            "F7",
            "F8",
            "F9",
            "A2",
            "A3",
            "A4",
            "A5",
            "A6",
            "A7",
            "A8",
            "A9"
        ],
        pure_getters: true,
        keep_fargs: false,
        unsafe_comps: true,
        unsafe: true
    }
};

module.exports = function(source) {
    if (this.mode !== "production") {
        return source;
    }

    const result = terser.minify(source, terserOptions);

    if (result.error) {
        this.emitError(error);
    }

    if (result.warning) {
        this.emitWarning(warning);
    }

    if (!result.code) {
        throw new Error("no result code");
    }

    return result.code;
};

Yes, I ran t on the minified file. Did I misunderstand which file to run it on?

Not all lines of code are created equal. Sometimes you have content in your code that simply cannot be minified.

For example, I have a bunch of SVG icons that are generated from some path data and some live variables. That single module ends up being twice as big as the entire runtime when minified and gzipped even if it only has a couple hundred lines.

In my use case, I decided for this implementation out of convenience and because the way this webapp is being used allowed for this.

For a comparison, tokei reports 22776 LOC in my app and the final gzipped asset size is 116.5k . (the original elm.js is 1.37 Mb, minified to 369.89k) These sizes are obtained by following the exact commands from The Guide.

I have managed to get it down to ~419 kb by changing the UglifyJs configs with uglifyOptions. It might also be possible that some of the issues stemmed from me not knowing that you had to force minifization when in dev.

But still, does ~400kb sound a lot for Μƒ23k lines of code?

Webpack also warns me that the entry file is above the recommended 244kb. Mine is currently around 1.03MB with 419kb from Main.elm, 550kb from node_modules and the other ~60kb I am not sure of and is not displayed with webpack-bundle-analyzer (as I can see). I import and init Elm in the app.js with is the entry file. Is there another and better way to do it, given that the entry file becomes so huge? Or shouldn’t it be a problem, because the elm-code shouldn’t be minifed more?

I wasn’t very clear about which file to run it on, sorry for that.
The idea is to run it on the non-minified file from elm (output of elm make). Maybe it’s not very helpful but it can sometimes give a hint on how much every Elm module contributes to your build size. Right now the script is very crude, it can for sure be improved.

I am not familiar with gist, but here is the webpack file and a simplification of the directory structure.

Webpack file:

const path = require('path');
const glob = require('glob');
const MiniCssExtractPlugin = require('mini-css-extract-plugin');
const UglifyJsPlugin = require('uglifyjs-webpack-plugin');
const OptimizeCSSAssetsPlugin = require('optimize-css-assets-webpack-plugin');
const ClosurePlugin = require('closure-webpack-plugin');
const webpack = require('webpack');
const { CleanWebpackPlugin } = require('clean-webpack-plugin');
const merge = require("webpack-merge");
var BundleAnalyzerPlugin = require('webpack-bundle-analyzer').BundleAnalyzerPlugin;
const HTMLWebpackPlugin = require("html-webpack-plugin");
require('dotenv').config()

var MODE = process.env.NODE_ENV === "prod" ? "production" : "development";

console.log(
	"\x1b[36m%s\x1b[0m",
	`** elm-webpack-starter: mode "${MODE}"\n`
);

var common = {
	mode: MODE,
	entry: {
		app: "./js/app.js",
	},
	output: {
		filename: 'js/[name].js',
		path: path.resolve(__dirname, '../priv/static/')
	},
	plugins: [
		new HTMLWebpackPlugin({
      // Use this template to get basic responsive meta tags
      template: "./index.html",
      // inject details of output file at end of body
      inject: "body"
    }),
		new MiniCssExtractPlugin({
		   filename: 'css/app.css',
		}),
	   new BundleAnalyzerPlugin(),
	   new CleanWebpackPlugin({
		 dry: true,
	   }),
	],
/*resolve: {
		modules: [path.join(__dirname, "src"), "node_modules"],
		extensions: [".js", ".elm", ".less", ".css", ".png"]
	},*/
	module: {
		rules: [
			{
			  test: /\.(ttf|eot|svg|png|jpg|gif|ico|woff|woff2)$/,
			  use: ['url-loader']
			},
			{
				test: /\.js$/,
				exclude: [/node_modules/, /elm/, /elm-stuff/],
				use: {
					loader: "babel-loader"
				}
			},
			{
			  test: /\.css$/,
			  use: [
				MiniCssExtractPlugin.loader,
				{loader: 'css-loader', options: { minimize: true } }
			  ]
			},
			{
			  test: /\.less$/,
			  use: [
				{
				  loader: MiniCssExtractPlugin.loader,
				},
				'css-loader',
				'less-loader'
			   ]
			}
		]
	},
	watch: true,
	node: {
	  __dirname: false
	},
	devtool: "source-map"
};

if (MODE === "development") {
	module.exports = merge(common, {
		optimization: {
		  minimize: true,
		  minimizer: [
			  new UglifyJsPlugin({
			  	cache: true, parallel: true, sourceMap: false,
				  uglifyOptions: {
				  	compress: {
			      pure_funcs: ['F2','F3','F4','F5','F6','F7','F8','F9','A2','A3','A4','A5','A6','A7','A8','A9'],
			      pure_getters: true,
			      keep_fargs: false,
			      unsafe_comps: true,
			      unsafe: true,
			      passes: 3
			    }
			  } }),
/*			  new ClosurePlugin(
				  { mode: "STANDARD" },
				  {
					  // compiler flags here
					  //
					  // for debugging help, try these:
					  //
					  formatting: 'PRETTY_PRINT',
					  //debug: true
					  // renaming: false
				  }
			  ),*/
			  new OptimizeCSSAssetsPlugin({})
		  ]
		},
		plugins: [
			// Suggested for hot-loading
			new webpack.NamedModulesPlugin(),
			// Prevents compilation errors causing the hot loader to lose state
			new webpack.NoEmitOnErrorsPlugin()
		],
		module: {
			rules: [
				{
					test: /\.elm$/,
					exclude: [/elm-stuff/, /node_modules/],
					use: [
						{ loader: "elm-hot-webpack-loader" },
						{
							loader: "elm-webpack-loader",
							options: {
								// add Elm's debug overlay to output
								debug: true,
								//
								forceWatch: true,
								cwd: path.resolve(__dirname, 'elm'),
								//optimize: true
							}
						}
					]
				}
			]
		},
		devServer: {
			inline: true,
			stats: { colors: true },
		}
	});
}

if (MODE === "production") {
	module.exports = merge(common, {
		optimization: {
		  minimize: true,
		  minimizer: [
			  new UglifyJsPlugin({ cache: true, parallel: true, sourceMap: false }),
			  /*new ClosurePlugin(
				  { mode: "STANDARD" },
				  {
					  // compiler flags here
					  //
					  // for debugging help, try these:
					  //
					  // formatting: 'PRETTY_PRINT',
					  // debug: true
					  // renaming: false
				  }
			  ),*/
			  new OptimizeCSSAssetsPlugin({})
		  ]
		},
		plugins: [
			// Delete everything from output-path (priv/static) and report to user
			new CleanWebpackPlugin({
				root: __dirname,
				exclude: [],
				verbose: true,
				dry: false
			}),
			new MiniCssExtractPlugin({
				// Options similar to the same options in webpackOptions.output
				// both options are optional
				filename: "[name]-[hash].css"
			})
		],
		module: {
			rules: [
				{
					test: /\.elm$/,
					exclude: [/elm-stuff/, /node_modules/],
					use: {
						loader: "elm-webpack-loader",
						options: {
							optimize: true,
							cwd: path.resolve(__dirname, 'elm')
						}
					}
				}
			]
		}
	});
}

Directory structure:

β”œβ”€β”€ assets
β”‚   β”œβ”€β”€ css
β”‚   β”‚   β”œβ”€β”€ alert.less
β”‚   β”‚   β”œβ”€β”€ app.less
β”‚   β”‚   β”œβ”€β”€ chart.less
β”‚   β”‚   β”œβ”€β”€ form.less
β”‚   β”‚   β”œβ”€β”€ globals.less
β”‚   β”‚   β”œβ”€β”€ modal.less
β”‚   β”‚   β”œβ”€β”€ responsive.less
β”‚   β”‚   └── table.less
β”‚   β”œβ”€β”€ cypress.json
β”‚   β”œβ”€β”€ elm
β”‚   β”‚   β”œβ”€β”€ date-picker
β”‚   β”‚   β”œβ”€β”€ elm-form
β”‚   β”‚   β”œβ”€β”€ elm.js
β”‚   β”‚   β”œβ”€β”€ elm.json
β”‚   β”‚   β”œβ”€β”€ elm-stuff
β”‚   β”‚   β”œβ”€β”€ optimize.sh
β”‚   β”‚   β”œβ”€β”€ src
β”‚   β”‚   └── tests
β”‚   β”œβ”€β”€ elm-constants.json
β”‚   β”œβ”€β”€ elm.min.js
β”‚   β”œβ”€β”€ images
β”‚   β”‚   └── loading.svg
β”‚   β”œβ”€β”€ index.html
β”‚   β”œβ”€β”€ js
β”‚   β”‚   β”œβ”€β”€ app.js
β”‚   β”‚   └── socket.js
β”‚   β”œβ”€β”€ package.json
β”‚   β”œβ”€β”€ package-lock.json
β”‚   β”œβ”€β”€ semantic
β”‚   β”‚   β”œβ”€β”€ dist
β”‚   β”‚   β”œβ”€β”€ fonts
β”‚   β”‚   β”œβ”€β”€ gulpfile.js
β”‚   β”‚   β”œβ”€β”€ src
β”‚   β”‚   └── tasks
β”‚   β”œβ”€β”€ semantic.json
β”‚   └── webpack.config.js
β”œβ”€β”€ _build
β”œβ”€β”€ config # Phoenix
β”œβ”€β”€ lib # with Elixir code
β”œβ”€β”€ mix.exs
β”œβ”€β”€ mix.lock
β”œβ”€β”€ priv
β”‚   β”œβ”€β”€ repo
β”‚   β”‚   β”œβ”€β”€ migrations
β”‚   β”‚   └── seeds.exs
β”‚   └── static
β”‚       β”œβ”€β”€ app-ca86cff7da3d6ea30457.css
β”‚       β”œβ”€β”€ css
β”‚       β”œβ”€β”€ index.html
β”‚       └── js
β”œβ”€β”€ rel
β”œβ”€β”€ test

Thank you, then it managed to count the lines!

It seems like I have ~39k lines myself and an additional 5k lines from other elm packages that I use), so a total of 44k lines.

Compared to the Instructions, webpack does no β€œβ€“mangle” here. I tested using uglify-es 3.3.9 with the following params. The good news is, that in my case all three ways create exact the same output. So the mentioned bug (Note 1 in the instructions) seems to have gone. This could help in cases where uglify.js / terser runs only once due to a limit in the packager.

uglifyjs build/Main.max.js --compress 'pure_funcs="F2,F3,F4,F5,F6,F7,F8,F9,A2,A3,A4,A5,A6,A7,A8,A9",pure_getters,keep_fargs=false,unsafe_comps,unsafe' --mangle --output=build/Main.js1
uglifyjs build/Main.max.js --compress 'pure_funcs="F2,F3,F4,F5,F6,F7,F8,F9,A2,A3,A4,A5,A6,A7,A8,A9",pure_getters,keep_fargs=false,unsafe_comps,unsafe' | uglifyjs --mangle --output=build/Main.js2
uglifyjs build/Main.max.js --mangle --compress 'pure_funcs="F2,F3,F4,F5,F6,F7,F8,F9,A2,A3,A4,A5,A6,A7,A8,A9",pure_getters,keep_fargs=false,unsafe_comps,unsafe' --output=build/Main.js3
1 Like