Reuse Webpack plugins without a webpack build

Webpack ecosystem is likely the richest for the frontend development today (even ng of Angular 9 or react-script use it).

However, sometimes, they encapsulate some logic which can be neat to reuse outside of a webpack build. To illustrate it, you can take the example of an Angular 9 library which uses ng-packgr. It relies on rollup instead of webpack for the build and is not that extensible today. However, if you want to chain features it would be a pain to create a webpack build for what is already well done with rollup.

To illustrate the reuse of a webpack plugin without webpack, i will try to compress assets created with rollup (with ng-packagr for example) and create gz/br archives  from the output assets using webpack CompressionPlugin.

The plugin configuration itself is quite common for anyone familiar with webpack. What we want to run is:

const plugins = [
    new CompressionPlugin({
        test: compressedAssets,
        filename: '[path].gz[query]',
        algorithm: 'gzip',
        compressionOptions: {level: 9},
        threshold: 10240,
        minRatio: 0.8,
        deleteOriginalAssets: false,
    }),
    new CompressionPlugin({
        test: compressedAssets,
        filename: '[path].br[query]',
        algorithm: 'brotliCompress',
        compressionOptions: {level: 11},
        threshold: 10240,
        minRatio: 0.8,
        deleteOriginalAssets: false,
    }),
];

Now the question is how to execute them without webpack? The answer is pretty straight forward if you ask yourself what does webpack do? Webpack is a chain of plugins (built-in or not). it can be seen as an event bus. So all the work is to extract the contract CompressionPlugin relies on and mock it.

CompressionPlugin takes a map of assets as input and enrich this map from the compressed assets. An asset is put in a javascript object ("map" behavior) and its value is mainly a source function loading the content of the asset as a Buffer (or some other "content" types we don't care here).

So our first step is to create this asset map. Using node built-in library it can look like:

const folder = 'path/to/dist';
const compressedAssets = /\.(js|css)$/;

if (!fs.existsSync(folder)) {
    console.error('no folder ', folder);
    return;
}

function listFiles(from) { // 1
    const assets = {};
    const files = fs.readdirSync(from);
    for (var i = 0; i < files.length; i++) {
        const filename = path.join(from, files[i]);
        const stat = fs.lstatSync(filename);
        if (stat.isDirectory()) { // 2
            return {
                ...assets,
                ...listFiles(filename),
            };
        } else if (compressedAssets.test(filename)) { // 3
            assets[filename] = { // 4
                source: () => {
                    return fs.readFileSync(filename, 'utf8');
                },
            };
        }
    }
    return assets;
}
const assets = listFiles(folder); // 5

1.  We create a function listing nested files from a folder,
2. If we encounter a directory, we call recursively the function merging current and recursive results,
3. If it is a regular file, we ensure it matches on of the patterns of the files we want to compress,
4. If it matches, we create a fake asset with a source function just reading the file with fs,
5. Finally we call this function on the folder we want to process to initialize the list of assets.

Now we have the input data for our plugin, we must simulate the webpack lifecycle used by CompressionPlugin.

First a plugin must be applied (calling its apply function) on webpack compiler. Then CompressionPlugin reads from the compiler the outputPath (our folder), and the tapAsync emit hook (which enables to process produced assets). This callback gives two parameters to CompressionPlugin, first one is the plugin spec (name) which is not used and second one which is a callback of the plugin, to call with a compilation.

A compilation is just an object with the asset object and a list of errors.

Complicated? let's see what it looks like in terms of code:

async function doCompress(assets) {
    return new Promise((resolve, reject) => {
        let done = 0;
        plugins.forEach(plugin => {
            plugin.apply({ // 1
                outputPath: folder,
                hooks: {
                    emit: {
                        tapAsync: function (pluginDscIgnored, fn) {
                            const compilation = { // 2
                                assets,
                                errors: [],
                            }
                            fn(compilation, () => { // 3
                                if (++done === plugins.length) {
                                    if (compilation.errors.length === 0) {
                                        resolve();
                                    } else {
                                        reject(compilation.errors);
                                    }
                                }
                            });
                        },
                    },
                },
            });
        });
    });
}

1. We fake the compiler argument with all entries the plugin uses,
2. We fake a compilation result by populating it from the scan we just did manually,
3. We finally call the plugin itself.

Note that webpack being callback based, it is convenient to wrap it in a Promise.

Finally, we just need to call our doCompress function, do a diff between assets before and after, and dump all assets added by doCompress on the disk:

const originals = Object.keys(assets); // 1
doCompress(assets) // 2
    .then(r => {
        Object.keys(assets)
            .filter(k => originals.indexOf(k) < 0) // 3
            .forEach(k => { // 4
                console.log(`Creating ${k}`);
                fs.writeFileSync(k, assets[k].source());
            });
        return r;
    })
    .catch(r => console.log(`Compression failed: ${r}`));

1. We save the list of assets we have before the processing to be able to diff after,
2. We call our compression function,
3. We extract all new assets (archives),
4. We dump all new assets on the disk.

And here we are, we use CompressionPlugin without a webpack pipeline :).

This small trick enables to add some webpack goodness in not webpack based builds quite easily. The only pitfall is that the plugin contract is rarely explicit and you must read the sources to find it, but maybe a day webpack would provide it in a normalized way.