The Perils of Auto-Merge and the Advent of Semantic Merge

Just because a VCS tool can auto-merge, that doesn't mean there isn't one!

What can be auto-merged by a tool unaware of the programming language?

• add file
• delete file
• rename file: http://stackoverflow.com/questions/2701790/git-merge-with-renamed-files
• change parts of a file which don't overlap with other changes

Trivial examples of how these auto-merges can break the app:

add

• The new Jasmine spec has an error.
• A new configuration file overrides the default configuration in a dysfunctional way.

delete

• The main module is gone so the app won't run.
• A CSS file is gone so the pages aren't styled.

rename

• require("Counter") fails because Counter.js is now TheCounter.js.

"no-conflict" changes

function square(x) {
- return x * x; #
+ return x * x * x;
}

A careful engineer reviews all changes (his changes & others') to increase the chance of a problem-free merge:
git diff <branch> <branch>
git diff master rough

Unfortunately:
git mergetool
only shows conflicts.

Unit tests provide an additional check.

Now, there's a "semantic" merge tool which understands the programming language:

What We Look For

Check out what we look for.

google's Dart Language

Javascript is a language invented by Netscape. It is the only language which all browsers can execute. (IE adds Basic & perhaps others.) Netscape's LiveWire product allowed Javascript to execute in the web server. node.js is the second coming of this isomorphic language.

Javascript has dynamic typing, simple formal parameters (including varargs), and an inheritance mechanism known as prototypes. There are several languages that aim to be more expressive & compile into Javascript.

Coffeescript: It uses Python-style indentation. Its class feature is the main reason to consider it. It has some other nice features like: destructured assignment as in [x, y] = 1 2, existential operator as in x ?= 1,  embedded JS as in `var x = 3;`, a switch statement which supports strings, string interpolation, multi-line strings, function binding (aka fat arrow =>) whereby this's value can be preserved in the call. Coffeescript is supported by Rails 3's asset pipeline.

Dart: Some notable features:

• types: optional static typing as in var i; vs. int i;, typedefs (an evolving feature), constants as in const double PI = 3.14159;
• operators: divide returning an integer result as in 4 ~/ 3
• strings: multi-line, concatenation as in s1 s2
• collections: Map literals as in { 'color': 'red', 'n': 13 } -- too bad the keys must be quote-delimited
• functions: anonymous, syntactic sugar for single-expression methods/functions as in sum = (a, b) => a + b;, closures, optional positional & named formal parameters
• inheritance: library-level instance variable visibility via _ identifier prefix as in _privateI, named constructors -- more readable than overloading, factory constructors, constructor initializers as in Person(this.name), getters/setters, abstract methods/classes, implicit interfaces (if you know Java, there is an implements keyword but not an interface keyword), operator overloading, generics as in Map<String, Integer> 
• modularity: libraries (a refinement of Java packages)
• comments: documentation comments (as opposed to a convention imposed by a document generator)
• multiprogramming: isolates (instead of threads)

Notable omission: enumerations.

There are others which I won't cover here.

Reflection on Ruby on Rails 3

In 1976, I took a course at Tulane University entitled Comparitive Languages. I recollect it covered Snobol, Fortran, Cobol, APL, Algol, Pascal, & Lisp but my memory is fuzzy. It was taught by a lecturer from industry. There was no computer science major at the time. I loved this course because it explored very different ways of programming.

I transferred to CMU where faculty created Alphard (in the hopes of winning the DoD's language contest), Bliss (a systems programming language), & OPS (a rule-based language). At a summer job at Texas Instruments, I learned Jovial (the Air Force's language). A famous professor taught us the moniker F***tran.

In grad school at UC Berkeley, I worked with declarative languages for circuits: EDIF, CIF, STL, STIF. The awful language tcl (Terrible Computer Language :)) was invented there.

In my first job (SDA which became Cadence Design Systems), one of our products was a Lisp dialect dubbed Skill. I programmed in C through my first three jobs. I used C++, Perl, & Javascript in my 4th job, excite, during the early days of the web. In my 5th job, IGN Entertainment, I used Java & Perl. We bought companies which used C# & Php (Pretty Horrible Programming language :)). In my 6th job, vudu, I used Java but Python, Lua, Ruby, & C++ were used by teammates. I used Python for Collectrium's proof of concept.

My most recent experience is using Ruby on Rails for Hands for Science. Ruby is a clean, elegant language. Interesting features include blocks/lambdas/Procs, default arguments, collecting hash arguments, dynamic types, symbols, mixins/modules, multi-line strings, string interpolation, eval, hash initializers.

Having used Python, Ruby, & Javascript heavily, I'll never understand the appeal of dynamic types. Dart is a new language which offers static & dynamic types. Related to this is collecting hash arguments. Keyword arguments are more robust & Dart has them.

Rails is a vast web framework. Declarative model validation & the asset pipeline are nice features. The worst problem is the long load time slows the iteration cycle. Other big problems include the form builder's weak support for relationships & the deep, obtuse stack traces. I'm not a fan of the verbose template language but there are many alternatives.

The asset pipeline supports Coffeescript. Perhaps its best feature is the addition of classes to Javascript.

I doubt I'll use this stack in the future. Currently, I'm studying node.js & Dart.

You can’t just have a sysadmin on call

A good article on the importance of Dev Ops: Server Density Blog

At IGN & vudu, a dev team member for each stack was on call for a week at a time. He was free to enlist a substitute for short periods such as a commute, dining out, or a medical appointment. A notebook PC was supplied.

Let's be realistic: It is unlikely that the on-call engr will be familiar with the code which breaks. The company should pay cell phone bills to guilt engrs to keep their phones charged & nearby. For a newly deployed app which has the highest chance of failure, the original author must be on call. If there's an upcoming busy period, all authors of code under stress must be on call.

htmlburger & 99designs

Hands for Science is a non-profit helping teachers do hands-on science activities more easily. I programmed the web site www.handsforscience.org, visual design was outsourced to 99designs.com, and web coding was outsourced to htmlburger.com.

As expected, 99designs.com was a cost-effective way to get several concepts. Many of the designers don't understand CSS so they suggest unrealizable designs such as a paginated float & colored auto-numbering. Don't expect one designer to excel at everything. The logo and page templates were done in separate contests. Be sure to point out that the logo must look pleasing in the global nav. One template is for the description of the hands-on science activity. Unfortunately, the designers did not seem to excel at printed page design.

It took some doing to adapt the htmlburger.com coding of the 99designs.com PSDs. The client needs to give htmlburger specific instructions or the code will be too specific to specific pages instead of being reusable style/html frags/templates. Insist that all directory, file, & style names be prefixed w/ hb- or the like. This makes it easy to distinguish their stuff. They improved my jquery slideshow. They even wrote JS to shrink a long caption's font-size.

On the marketing pages, I wrote custom tags (rails view helpers) which alternate sides for the graphics.

HandsForScience CMS Uses mongodb

HandsForScience is a non-profit which helps teachers do hands-on, science activities more easily. I'm using mongodb for the HandsForScience CMS: http://www.handsforscience.org/activities

The stack is rails 3, mongoid, cancan, & jquery hosted via Heroku, mongolab, S3 (for attachments).

Any CMS will have User, Article, Link, & Attachment classes--in our case, Activity takes the place of Article. Our schema adds:

• Standard: A curriculum standard which the activity cross-references. So far we include the California K-5 standards.
• Material: An activity includes materials. The list is crowdsourced.
• Term: An activity includes vocabulary terms. The vocab is crowdsourced.

Instead of a Grade class, an activity's grades are an array-valued field. Instead of a Subject class, an activity's subject has a string-valued field validated as an enum: life, earth, physical sciences.

The most interesting tradeoff is the Activity/Material relationship. There must be enough material for each activity. The quantities are classroom, small group, student. Instead of creating a join class ActivityMaterialQty, there are three many-many relationships.

Another tradeoff is object creator fields are string-valued instead of many-many-User relationships. I should probably switch to BSON::ObjectId.

It would be nice if mongodb supported case-insensitive sorting. I'm working around this by sorting in memory. I should switch to denormalized, lowercase fields.