Tugboat

give your bug fixes a URL.

lullabot-logo

This is not a UI

build-passing

Full Steam Ahead

tizen-logo

msnbc-logo

lullabot-logo

First

a little bit of history

Production
Staging
Development

hook_update_N()

Features

drupal-8-logo

Configuration

Management

Initiative

The Cost of Slow

Lessened Visibility

Release blockers

Trade-offs

Visibility for Testability
Agility for Stability

"It worked fine on my local!"

I get it.

Now what?

What if…

Immediate visibility
Avoid regressions
Die, buggies!
Analyze before code gets merged

This is what we built Tugboat to do…
About three years ago…
We have a super technically savvy stakeholder…
But Mike is a busy guy…

This is what we built Tugboat to do. About three years ago, we started work on the Tizen project for Intel, the Linux foundation, and Samsung. Tizen is a Drupal 7 multisite setup comprised of three different websites, tizen.org, developer.tizen.org, and source.tizen.org. We had a super technically savvy stakeholder and project owner named Mike Shaver who totally could have taken the time to build each branch of our code that he needed to test, to make sure you did what we said it was going to do. But Mike is a busy guy, and he doesn't need to be importing three databases five times a day in order to test everything that we were doing.

In order to save hime time, we wrote a set of very simple bash scripts that glued…
Anytime anyone created a new pull request, Jenkins would get triggered…
Anyone here familiar with simplytest.me?
It's much like that, except that it happens automatically, and it builds the Tizen sites with fresh database dumps and files…

In order to save time, we wrote a set of basic bash scripts coupled with some Jenkins jobs and he github integration plug-in to create that glue that we needed. Anytime anyone created a new pull request, Jenkins would be triggered by the plug-in, our bash trips would run, and a clone of the three Drupal sites with the new code merged into master Live on the site and deployed. These bash scripts would place a comment on the github pull request with links to these three sites so that Mike could go there and see the work that is being proposed as added, check to make sure it did what it was supposed to do, and that it didn't break anything else.

msnbc-logo

Tugboat hands out URLs like candy to:

External teams
Internal teams
Individual developers

Thanks James. After all of James' work, putting the initial version of Tugboat together for the Tizen project, we shamelessly _stole_, er I mean **leveraged** it for use on lullabot.com. Like the typical cobbler's children the process of building the latest lullabot.com, to be polite, _took a while_. For quite a while the project trudged forward with whichever developer resources had spare time. This often led to jointly working on, or finishing each other's features, and lots of time sunk into updating / re-syncing a local development environment in order to do some peer review, or test simple bug fixes. Soliciting feedback, or getting peer reviews was a difficult process, because of the scattered nature of development on the project. Using tugboat allowed us to have shared, canonical, urls for in progress features. I know, I at least, used this a couple of times to push code changes to a development branch without actually having my local in sync, or up to date. I could simply hand edit a view configuration, push my commits and see / share the work incredibly quickly. This made it easier for us to solicit peer review, since the reviewer only needed the tugboat environment link in a github ticket rather than a full-blown local development environment. It also made it easier to solicit feedback and iterate on new features. Several times we would use the tugboat environment to share and discuss possible UI implementations throughout the project.

Then last spring, after our successful launch of lullabot.com, I started working on another project at Intel. Our client, Mike, wanted to have another instance of Tugboat set up for use with that project too. One of the challenges with the Tugboat, as it existed at that time, was in the actual setup process. While it was still worth doing, there was a day or two of getting everything installed and configured just right before we'd have a working service. There were several interdependent jobs to handle the production data downsync, keeping the staging sites up to date, and building the actual Tugboat environments. The code we had created was quite DRY, in the sense of don't repeat yourself, but that complicated the initial setup. Jenkins plugin dependency management, and discrepancies between plugin versions didn't help matters any either. In addition, because our Jenkins server was behind a VPN kicking off manual jobs required a bit of extra work (we all have at least a little bit of hate for VPN clients, right?).

I created a command line wrapper for our tugboat scripts and the Jenkins API interface. Now, once a Jenkins instance is setup and the github plugin is configured (in other words Jenkins can access the github repository of interest) a couple of simple commands setup the skeleton jobs needed to generate automated pull request builds. This wrapper code also allows us to backup, diff and update the Jenkins job configurations in much the same way we do with Feature modules now using drush. This ensures that we can keep our Tugboat settings in version control as well. We also don't have to rely on github (and cron) to kick off job builds. Tugboat can build environments for any arbitrary branch, commit hash or ticket number manually from the command line (interface coming soon-ish?). There's also, now, a basic plugin system, so the same Tugboat utility wrapper that's being used to spin up dev environments can be used for backing up existing sites, as a test runner, deployment helper, etc.

resemble.js

Setting this up for yourself

Pull Request Builder blog post

lullabot/jenkins_github_drupal

Jenkins Github Plugin

Where to next?

(this isn't really a UI either)

Welcome To

The Future

Rather than continue to rely on this UI, we're working on our own. In the plans so far we have a per project dashboard, that will provide at a glance all of the Tugboat environments across a particular project. This will give nontechnical stakeholders a bird's eye view to the work in progress code at any given moment in time. In addition, each tugboat build environment will also have their own landing page. This will be a handy place for us to aggregate information about the environment. When was it built? Does it have the most recent code from the relevant branch? Should we manually trigger a rebuild because some potentially conflicting code has changed elsewhere? What do the test results look like? How about performance profiling (both cachegrind and the ???? front end equivalent ???? waterfall). _https://github.com/gmetais/grunt-devperf/_ Can, or should, we use vagrant along side puppet or chef to provision environments and perhaps, in the process, convince developers to use the same stack locally to more closely match the target stack? Perhaps we will use containers, with something like docker, to make each tugboat project a bit more resource efficient while cleanly silo'ing each tugboat build environment.

Q&A

blake@lullabot.com

james@lullabot.com