dissoc

Developing Clojure patches with Pull Requests

In 2021, I proposed around a dozen patches to Clojure. I developed many of them concurrently and created a pull-request based workflow that might be useful for others.

Here’s my usual process for contributing significant work to GitHub repositories:

  1. Create a meta-repo with git worktree helpers.
  2. Clone main branch to the main directory in the meta-repo.
  3. Create feature branches in directories via git worktree add.
  4. Create a tmux session for every feature branch. Via tmux-ressurect, these may be open for days to years depending on the task.
  5. Submit pull-requests during development for continuous integration.
  6. After merge, delete tmux session and directory for feature.

Contributing to Clojure is not immediately compatible with this workflow because:

  1. Code patches must be manually uploaded to Jira.
  2. The Clojure repository has no continuous integration for pull-requests.

The workaround

While Clojure has no continuous integration for pull-requests, we can work around this by forking the repository.

A fork of Clojure on GitHub.

Now we can send pull-requests to our own fork that include GitHub Actions instructions to run continuous integration tests.

A pull request on the frenchy64 fork of Clojure with continuous integration.

But we need to strip the build.yml file from the patch before submitting the patch to Jira.

git format-patch master --stdout -U8 -- \
  . ':!.github/workflows/build.yml' > clj-12345.patch

This works, but it’s too fiddly. I created a meta-repository that packages up this workflow to be more streamlined.

Here’s how to use it.

Setting up

Say you’ve followed Clojure’s contributing guidelines for some hypothetical Jira ticket CLJ-123456, and all that’s left is to code up one or more solutions.

Fork Clojure to your GitHub user (we’ll use your-github-user throughout as a placeholder for your actual GitHub user):

A prompt to create a fork of the Clojure repository in the your-github-username user.

Now clone my Clojure meta-repository clojure-local-dev, tell it where your fork lives, create a new branch for CLJ-123456 development, and switch to that branch’s directory:

git clone https://github.com/frenchy64/clojure-local-dev.git
cd clojure-local-dev
echo "your-github-user" > github-user.edn
./new-branch clj-123456-first-approach
cd clj-123456-first-approach

Coding

We can start coding a solution to CLJ-123456 in the current directory. In most cases, you can stick to mvn clean test for running tests, but the meta-repo contains a few ideas to demonstrate how you might achieve similar results from Clojure CLI.

For example, you can start an interactive REPL via Clojure CLI by running ../repl.sh, which will also run a user-defined :nrepl alias.

clj-123456-first-approach$ ../repl.sh
...
clean and compile Clojure
...
nREPL server started on port 55482 on host localhost - nrepl://localhost:55482
nREPL 0.9.0
Clojure 1.12.0-master-SNAPSHOT
OpenJDK 64-Bit Server VM 18.0.1+10
Interrupt: Control+C
Exit:      Control+D or (exit) or (quit)
user=>

In fact, your branch will have its own deps.edn file that you can customize as needed. But be careful of stale class files—run ../prep-clojure-cli.sh to get a clean slate.

There are some ideas for test runner scripts like ../test-example.sh for unit tests, ../test-generative.sh for generative tests, and ../watch.sh for kaocha.

Continuous Integration

You’ve committed your solution to CLJ-123456 and now you’re ready to run the full CI test suite by pushing it to your fork. Two git remotes were installed to your local Clojure git repo when it was created: one for your fork, and one for Clojure.

$ git remote -v
clojure https://github.com/clojure/clojure.git (fetch)
clojure https://github.com/clojure/clojure.git (push)
your-github-user       git@github.com:your-github-user/clojure.git (fetch)
your-github-user       git@github.com:your-github-user/clojure.git (push)

Choose your fork, and push.

$ git push -u your-github-user clj-123456-first-approach

This will trigger a new matrix build on your branch that you can find via github.com/your-github-user/clojure/tree/clj-123456-first-approach. If (like me) you track ongoing work using pull requests, go ahead and create one. But don’t send it to clojure/clojure—it will be closed. Send it to your own fork your-github-user/clojure.

Submitting patches

Once you’re ready to propose your solution to CLJ-123456, you can use ../format-patch.sh at the root of your branch’s worktree directory to generate the patch. It takes one argument: the file to name the patch.

../format-patch.sh CLJ-123456-my-patch-id.patch

Now go to CLJ-123456 in Jira and upload CLJ-123456-my-patch-id.patch as a new patch.

Resubmitting patches

If your patch no longer applies to Clojure master, it often needs resubmission. Instead of dealing with git patches, you can just rebase can generate a new patch:

  1. To sync your local and fork’s master branch with clojure/clojure, call ./sync-master.sh in the meta-repo root.
  2. Now cd into your feature branch, rebase on master, and resolve any conflicts.
  3. Finally, generate and submit a new patch with ../format-patch.sh as before—just choose a new name for the patch.

I intend to maintain the clojure-local-dev meta-repo as its own project until it becomes obsolete—please open an issue or non-breaking pull-request to help out.

29 Aug 2022