Brief

One day, an idea came to me: I want to write a toy parser, whatever it is.

Antlr is a great tool of such kind to help you create a rich-featured parser in minutes, but I'm not here to advertise something :) It's such a lovely tool that I soon fall in love with it, mentally for sure.

But I soon got into big trouble because it enforces you to name the filename the same as the grammar name, but the problem is that my FS (file system) is case-insensitive! They provide a tool grun to debug your grammar, but it requires you to compile the grammar to the Java target. That's OK, it only requires an extra line in Makefile, how hard could it be, I thought.

It turns out that I have oversight something, while my FS is case-insensitive, it outputs the Java source files in camelCase without surprise. What could it mean? it means that javac won't be happy to compile them.

Well, I'll write some bash lines in Makefile to transform those filenames before feeding them into javac, sounds doable right? And yeah, it soon becomes cumbersome, and the code is getting hard to understand. Most importantly, it doesn't work :(

Gulp to rescue

I have JavaScript background, I know there are tons of awesome build tools, and Gulp is quite the one, simple and lightweight.

About the task

The task is the basic unit of a Gulp file, you define tasks, either to serialize them in a row or to parallelize them in an asynchronized way, it's on your needs.

Go build

In Makefile, to build a Go binary is just one line code, in Gulp, on the other hand, we are in the JavaScript world or, more precisely, the NodeJS world.

Node has a built-in child_process module, it provides the interface to create the Node process, run some shell commands, etc. That's what I need.

const exec = util.promisify(require("child_process").exec);

const { stderr, stdout } = await exec("go build -o app .");
stderr && console.log(stdout);
stdout && console.error(stderr);

Extract variables

It's a common practice that people define the command name and build flags as variables in Makefile, it's also possible and natural in Gulp:

const GOBIN = "app";
const TMP_DIR = "tmp";
const GO_BUILD = "go build";
const GCFLAGS = "all=-N -l";

// ...
exec(`${GO_BUILD} -v -o ${GOBIN}`)

And there is an already full-featured language server, which supports jump to definition in modern IDE, awesome!

A helper runner

It's cumbersome to write the template code everywhere, it's better being DRY:

function exec_template(cmd, name, ...options) {
  const fn = async function (cb) {
    try {
      const { stderr, stdout } = await exec(cmd, ...options);
      stderr && console.log(stdout);
      stdout && console.error(stderr);
    } catch (error) {
      cb && cb(error);
    }
    cb && cb(null);
  };
  if (name !== undefined) {
    fn.displayName = name;
  }
  return fn;
}

fn.displayName is used to configure the task name, since the exec_template is a high-order function and it returns an anonymous function. To give it a name will make the outputs more clearly.

name goes for fn.displayName

So...Antlr?

Let's get down to the business! The steps are listed below:

Empty the tmp directory
Generate Java files
Transform the Java files to PascalCase
Run javac to compile

cleanup

I'll use the del package for the task:

// for generated go parser files
const GRAMMAR_OUT_GLOB = "pkg/parser/**";

const del = require("del");

function clean_tmp() {
  return del([TMP_DIR]);
}

function clean_gen_parser() {
  return del([GRAMMAR_OUT_GLOB]);
}

gulp.task("clean", () =>
  del([
    // debugging resources
    TMP_DIR,
    // go binary
    GOBIN,
    // generated go parser files
    GRAMMAR_OUT_GLOB,
  ])
);
gulp.task("clean:tmp", clean_tmp);
gulp.task("clean:gen", clean_gen_parser);

Done! if you run npx gulp --tasks, it will show in the tree.

Generate

Use the previously created helper runner:

const GRAMMAR = "Arithmetic";

exec_template(
  `antlr -Dlanguage=Java ${GRAMMAR}.g4 -o ${TMP_DIR}`,
  "java target" // annotate task name
)

(It's a part of a complete task, I'll talk about it later).

Transform

I use pascal-case for the purpose:

const { pascalCase: pascal } = require("pascal-case");

function capitalize_java_class() {
  return gulp
    .src("tmp/*.java")
    .pipe(
      rename((p) => {
        p.basename = pascal(p.basename);
      })
    )
    .pipe(gulp.dest(TMP_DIR));
}

It reads all Java files in tmp dir, and transform them to PascalCase.

That's a self-contained task, it's ok to leave it be. (Keep it in mind that it is for debugging, so I put the artifacts in tmp dir).

Javac? javac for sure

Like the way we build go:

exec_template(`javac *.java`, "compile java", {
  cwd: TMP_DIR,
})

I can pass a cwd option, no more cd /xxx && javac ...

All together

gulp.task(
  "antlr:debug",
  gulp.series(
    "clean:tmp", // cleanup first
    exec_template(
      `antlr -Dlanguage=Java ${GRAMMAR}.g4 -o ${TMP_DIR}`,
      "java target"
    ),
    function capitalize_java_class() {
      return gulp
        .src("tmp/*.java")
        .pipe(
          rename((p) => {
            p.basename = pascal(p.basename);
          })
        )
        .pipe(gulp.dest(TMP_DIR));
    },
    exec_template(`javac *.java`, "compile java", {
      cwd: TMP_DIR,
    })
  )
);

gulp.series will make them run in a row, and the whole task is named antlr:debug, a common naming convention for npm scripts.

Antlr for Go

const GRAMMAR_OUT = path.normalize("pkg/parser");

// served as a prerequisite
gulp.task(
  "antlr:go",
  exec_template(
    `antlr -Dlanguage=Go ${GRAMMAR}.g4 -o ${GRAMMAR_OUT}`,
    "generate go parser"
  )
);

Modified Go build

const build = gulp.series(
  "clean:gen",
  "antlr:go", // see above
  exec_template(`${GO_BUILD} -v -o ${GOBIN}`, "build in local env")
);

gulp.task("build", build);
exports.default = build; // make it a default build task

Complete Gulpfile

// Std lib
const util = require("util");
const exec = util.promisify(require("child_process").exec);
const path = require("path");

// util
const { pascalCase: pascal } = require("pascal-case");

// Gulp
const gulp = require("gulp");
const rename = require("gulp-rename");
const del = require("del");

// Go build args
const GOBIN = "app";
const TMP_DIR = "tmp";
const GO_BUILD = "go build";
const GRAMMAR = "Arithmetic";
const GRAMMAR_OUT = path.normalize("pkg/parser");
const GCFLAGS = "all=-N -l";

// globs
const GO_SRC_GLOB = "*.go";
const ANTLR_SRC_GLOB = "*.g4";
const JAVA_SRC_GLOB = `${TMP_DIR}/*.java`;
const JAVA_CLASS_GLOB = `${TMP_DIR}/*.class`;
const GRAMMAR_OUT_GLOB = "pkg/parser/**";

function exec_template(cmd, name, ...options) {
  const fn = async function (cb) {
    try {
      const { stderr, stdout } = await exec(cmd, ...options);
      stderr && console.log(stdout);
      stdout && console.error(stderr);
    } catch (error) {
      cb && cb(error);
    }
    cb && cb(null);
  };
  if (name !== undefined) {
    fn.displayName = name;
  }
  return fn;
}

// clean targets
function clean_tmp() {
  return del([TMP_DIR]);
}

function clean_gen_parser() {
  return del([GRAMMAR_OUT_GLOB]);
}

gulp.task("clean", () =>
  del([
    // debugging resources
    TMP_DIR,
    // app build
    GOBIN,
    // generated go parser files
    GRAMMAR_OUT_GLOB,
  ])
);

gulp.task("clean:tmp", clean_tmp);
gulp.task("clean:gen", clean_gen_parser);

// served as prerequisite
gulp.task(
  "antlr:go",
  exec_template(
    `antlr -Dlanguage=Go ${GRAMMAR}.g4 -o ${GRAMMAR_OUT}`,
    "generate go parser"
  )
);

// build java target, for debugging purpose
gulp.task(
  "antlr:debug",
  gulp.series(
    "clean:tmp",
    exec_template(
      `antlr -Dlanguage=Java ${GRAMMAR}.g4 -o ${TMP_DIR}`,
      "java target"
    ),
    function capitalize_java_class() {
      return gulp
        .src("tmp/*.java")
        .pipe(
          rename((p) => {
            p.basename = pascal(p.basename);
          })
        )
        .pipe(gulp.dest(TMP_DIR));
    },
    exec_template(`javac *.java`, "compile java", {
      cwd: TMP_DIR,
    })
  )
);

// local build
const build = gulp.series(
  "clean:gen",
  "antlr:go",
  exec_template(`${GO_BUILD} -v -o ${GOBIN}`, "build in local env")
);

gulp.task("build", build);

// deployment build
const build_prod = gulp.series(
  "clean",
  "antlr:go",
  exec_template(
    `GOARCH=amd64 GOOS=64 ${GO_BUILD} -gcflags="${GCFLAGS}" -v -o ${GOBIN}`,
    "build in linux"
  )
);

gulp.task("build:prod", build_prod);

exports.default = build;