Webpack and SCP: write your own plugin!

If you use webpack and a dev server you can desire to update the server on the fly with the development iteration: a.k.a watch process.

I didn't find a built-in plugin doing it for webpack but there is an awesome npm package doing almost it: scp2.

Only missing piece is the webpack integration. Let's see how to make both working together.

SCP from NodeJS

One you added scp2 to your package.json with:

{
  //...
  "devDependencies": {
    "scp2": "0.5.0"
  }
}

You can run this code to scp the folder dist to the folder /server/input on the machine dev.server.company.com:

var scp2 = require('scp2');
var fs = require('fs');

var sshConfig = {
  host: 'dev.server.company.com',
  username: process.env.DEV_INSTANCE_USER || process.env.USER,
  privateKey: fs.readFileSync(process.env.DEV_INSTANCE_KEY || (process.env.USER_HOME + '/.ssh/dev-instance.pem')),
  path: '/server/input/'
};

scp2.scp('dist', sshConfig, function(err) {
  console.log('[ERROR] ' + JSON.stringify(err));
});

This code is pretty straight forward:

1. we import scp2 and fs libraries
2. we build our ssh configuration: which server, which SSH user, for the privateKey the trick is to configure the content of the key and not its path (with 0.5.0, next version should support the path) and finally the path where to upload the data. Note that you can replace privateKey by password if your server relies on that kind of authentication.
3. We SCP dist folder using previous configuration and log an error if it happens.

Tip: for variables you can use process.env to read it from your environment instead of hardcoding the values.

Wrap SCP2 in a Webpack plugin

A webpack plugin is a class with an apply method:

var MyPlugin = (function () {
    var plugin = function (options) {
    };

    plugin.prototype.apply = function(compiler) {
    };

    return plugin;
})();

The options parameter for the constructor is optional and depends if your plugin needs some configuration when instantiated or not. For this post we'll keep it since it is generally a good idea to let it be configurable.

The apply method is called with  the compiler and let you register hooks to each phase of the build. For our case we want the build to be done so we'll register on emit phase:

plugin.prototype.apply = function(compiler) {
  compiler.plugin('emit', function(compilation, callback) {
    callback();
  });
};

The compiler.plugin method allows you to listen for a phase and register a hook. For our case we just need to ensure to call the callback hook once we are done to finish webpack cycle/iteration.

Once we are there the plugin is almost written, we just need to put it all together:

var ScpPlugin = (function () {
    var scp2 = require('scp2');
    var fs = require('fs');

    var sshConfig = {
      host: 'dev.server.company.com',
      username: process.env.DEV_INSTANCE_USER || process.env.USER,
      privateKey: fs.readFileSync(process.env.DEV_INSTANCE_KEY || (process.env.USER_HOME + '/.ssh/dev-instance.pem')),
      path: '/server/input/'
    };

    var plugin = function (options) {
        // optionally override sshConfig with passed options
    };

    plugin.prototype.apply = function(compiler) {
      compiler.plugin('emit', function(compilation, callback) {
        scp2.scp('dist', sshConfig, function(err) {
            console.log('[ERROR] ' + JSON.stringify(err));
        });
        callback();
      });
    };

    return plugin;
})();

Register the plugin in webpack configuration

Finally you just need to register the plugin in main webpack configuration:

var ScpPlugin = ....;

module.exports = {
    // ...

    plugins: [
        // ...,
        new ScpPlugin({});
    ]
};

Going further

This code works well for small projects but for big ones you will get synchronization issues. A better way is to integrate mode deeply with webpack and its chunks. Here is the code based on the same dependencies:

var webpack = require('webpack');
var scp2 = require('scp2');
var fs = require('fs');

// config
var fromFolder = 'dist/';
var targetPath = '/opt/app/';
var sshConfig = {
    host: process.env.THE_HOST,
    username: process.env.USER,
    privateKey: fs.readFileSync(process.env.THE_KEY_LOCATION),
};

// plugin vars
var scpCount = 0;
var chunkVersions = {};

var client = new scp2.Client(sshConfig); // no close since we "watch"
client.on('error', function (e) {
    console.log(e);
});

var scp = function (file, content, hook) {
    console.log('Updating ' + file + ' on dev instance');
    client.write({
        destination: targetPath + file,
        content: new Buffer(content, 'utf-8')
    }, function (err) {
        scpCount--;
        hook();
        console.log('Updated ' + file + (err ? '. Error: ' + err + '.' : '') + '. Remaining uploads: ' + scpCount + '.');
    });
};



var scpSynchroPlugin = {
  apply: function(compiler) {
      compiler.plugin('emit', function (compilation, callback) {
          var filtered = compilation.chunks.filter(function(chunk) {
              var oldVersion = chunkVersions[chunk.name];
              chunkVersions[chunk.name] = chunk.hash;
              return chunk.hash !== oldVersion;
          });
          if (filtered.length == 0) {
              callback();
          } else {
              scpCount += filtered.map(function (chunk) {
                  return chunk.files.length;
              }).reduce(function (a, b) {
                  return a + b;
              }, 0);
              filtered.forEach(function (chunk, idx) {
                chunk.files.forEach(function(filename) {
                    scp(filename, compilation.assets[filename].source(), function () {
                      if (idx == filtered.length - 1) {
                          callback();
                      }
                    });
                });
              });
          }
      });
  }
};


// and add it in webpack plugins:[..., scpSynchroPlugin]

Conclusion

Now if your webpack setup relies on --watch, each time you will update the dist folder, you will also update your dev server.

Note that it is also easy to extend this code to just run a simple npm (or gulp) task doing a deployment of an application on any server.

Happy frontend hacking!