Caffeinated Simpleton

After the first two posts in this series, we have a site that displays the twitter public timeline and twitter searches. Now let’s customize this by allowing users to access their own twitter accounts.

To do that, we need to allow users to log in to their twitter accounts. Twitter hasn’t rolled out their OAuth solution yet, so we’ll have to ask users to trust us with their twitter names and passwords. For now, we won’t store the password, so all we need to do is get it out of a form and store it in a session.

Processing Forms in Compojure

This is pretty easy. Compojure provides a “params” hash-map that has the parameters of the servlet, which we will pass to a “login” controller. There are several things provided to the servlet. I won’t bore you with details of creating the form itself, so here’s the server side code that processes “twitter-user” and “twitter-password”.

As you can see, we have some new syntactic goodies to learn about. “session” is a map provided by Compojure. It is thread safe, which in clojure, is done through transactional memory. Dealing with that transaction is what most of this code does. dosync is a macro that allows many expressions to be executed in one transaction. alter takes a reference (session), a function that will alter the reference (assoc) and the arguments for the altering function. We finish our dosync call, which executes the transaction, and our session is all set.

After we set up our session, we redirect to the url of the user. We pull the user name out of the session. The “@” symbol is a macro that refers to the deref function, which allows us to get values out of references. If no name was provided, we kick back to the home page.

Pretty clean and painless. Clojure isn’t so bad.

After yesterday’s introduction to the syntax of clojure, now I’m going to start organizing my code a bit more and pulling in real data.

Templating

The first thing I want to do is abstract out a base page so that I don’t need to worry about headers, JS libraries, standard CSS, and all that when going from page to page. To do this, I’ll create a “page” function in a new namespace.

Then I add this into our original page.

Now I can call the “page” function, pass it a title and some contents, and it outputs a full page. Perhaps later on we’ll make it so we can derive templates from other templates, but I have no need for that yet.

Getting My Twitter Feed

Let’s make it so that getting “/justin_tulloss” will give me my twitter feed. To do that, I add a handler to our servlet.

Now when I go to http://localhost:8080/justin_tulloss, it will say “Welcome justin_tulloss”. Pretty high tech stuff.

Now let’s see if I can change that justin_tulloss parameter into my actual twitter data. First, I head over to the twitter API docs. The public timeline doesn’t require any authentication, so let’s pull that down first. The public timeline is a list of “status” messages, which we will map to html representations of the same. To do this, I used the clj-http-client library by Mark McGranaghan. It was a pain to get installed because I don’t understand Java JAR files fully, but once I got it figured out, it worked great.

Hopefully this code isn’t hard to follow, but let’s take a look. The “Welcome” part is the same. Then we map everything that we read from the twitter URL into a twitter-status function, which just grabs some data out of the statuses and displays it.

Referring back to the non-templating code, the “read-json-string” function is in clojure.contrib.json.read, which you can get here. The *twitter-url* is a global constant. Apparently it’s a lisp thing to surround globals with asterisks. I wonder if that also applies to global constants.

Well, that’s all for tonight. Next time we’ll fetch some more relevant data from twitter and see what we can do with it. As always, the complete code is on github.

I’ve decided I want to learn clojure. Clojure is a lisp dialect that runs on the Java Virtual Machine. It’s all the rage these days, and I want to be cool. I’m going to post my trials and tribulations here for others to read and hopefully learn from. If you are a better hacker than I and see problems with what I’ve posted, please comment! This is going to be a multi-day process, and a slow one as I don’t have much time, but I hope to finish a project within the next six months.

There are a couple problems with Clojure and I. First of all, I’ve never programmed in any lisp dialect. Second of all, I’ve never programmed in Java. Clearly I do not have the tools to get into this, but I’m going to go for it anyway.

The Project

I’ve been thinking a lot about Twitter recently, and there are some really cool applications of Twitter that I find kind of exciting. However, Twitter’s signal to noise ratio can quickly become unbearable. So, I thought I’d come up with a Twitter feed app that will organize people you care about, conversations you want to monitor, tweets around where you are, and the regular twitter feed into a sort of Twitter portal. It should be fun and easy, and it’s not the project that’s important. It’s learning Clojure. I’m going to call it Flockr for maximum Web 2.0 street cred. You can find the source on github.

Getting Started

I’m using Compojure as my web framework. I like web frameworks, especially for little projects like this that don’t need any specialized plumbing. Compojure isn’t terribly mature, but it should be good enough to get me off the ground.

Installing compojure is easy. Just download, compile with ant, and put the resulting jar file in your classpath. Being a vim guy, I wanted clojure syntax highlighting and auto-indenting. There’s a vim package available to do that stuff, and I recommend it.

The First Page

This is basically copied straight off the compojure “Getting Started” page. However, this is the first time I’ve really looked at lisp, so let’s see what this does.

Clojure Basics (or at least, what I’ve inferred about Clojure thus far)

Parentheses indicate a list. I’m a big C/Python/JavaScript guy, so this a bit strange, but I can deal. Evaluation is simply evaluating every list that is passed into the REPL. This first program has three things that are evaluated. The first is the namespace. The second is the servlet, and the third is the server. Let’s that a closer look at the first line.

Returns the result of the “ns” function, which creates a new namespace called “index” (in this case). Within this namespace, we are dependent on the “compojure” namespace. We pass the “:use” key to indicate this.

The colon syntax is something I’m still getting used to. It means a key, which makes sense in a map, but it would appear that you can also put keys in other data structures.

The only other things you need to know to understand the rest of the code is “[]” and “{}”. “[]” is a vector. I’m not certain I understand when to use vectors versus lists except for performance. “{}” is a map. A colon indicates a key (IE :port), the thing following it is the value (IE 8080). Commas are optional, and don’t seem to be used in examples at all.

Doing something real

Ok, so we’ve got something up and running. That’s good. Now let’s grab some twitter statuses. Unfortuneatly, it’s past 4 in the morning. This will have to wait for another day.

I’ve been a big fan of Mercurial for a while now. Harmonize is all kept in Mercurial, and most of my side projects are as well. Mercurial is great for a number of reasons. It’s easy to use, not only for former subversion users, but also at a conceptual level. Understanding how DVCSs work is easy when you’re using Mercurial. The vocabulary is sparse and makes a lot of sense. As added bonuses, it runs on Windows just as well as it runs on every other OS, and it’s written in Python (my favorite language).

However, I’ve decided to allow git into the very exclusive club of version control systems that don’t suck. Git makes two, and it might become my favorite as I get more comfortable with it.

To pause for a moment, one of the things git does not do better is github. Github has, perhaps, a slight advantage over bitbucket because of the size of the user base, but that doesn’t make it inherently better. It’s just a better social experience for now. From what I can tell, bitbucket matches github feature for feature and outdoes it in some areas. I’m a huge fan.

That being said, there are a few things git does better than mercurial. The first is editing commit histories. Mercurial supports a similar feature through extensions, but git has a leg up by coming with the ability to alter any part of your commit history built in. This allows things like cherry picking, merging batches of commits into one, and cleaning up commit logs. Mercurial can kind of do these things, but it’s much better supported in git. Plus, as a relative newcomer to git, I’m sure I haven’t even scratched the surface of what’s possible.

The second advantage that goes to git is local branching. Now let’s be clear here. Mercurial does support branching in much the same way git does. However, if you ever push to a remote repository, your branches must exist there or you must create them there. Local branches are nice because they don’t need to clutter up whatever master repository you have. You can have your own branches for whatever you’re up to and don’t need to mess with what the rest of the world is doing. I use this all the time and love it.

In the end though, there is one feature that has pulled me toward git and kept me there, and that’s git-svn. My current employer uses Subversion, and since that is not one of the version control systems that don’t suck, I quickly became agitated. I desperately tried to use Mercurial with SVN, to no avail. There is pretty good support for mirroring a SVN repository in Mercurial, but pushing back to svn quickly became a nightmare. I even found a script that did a pretty good job, but in the end, the support wasn’t there.

Git-svn, however, is a different story. It utilizes git’s amazing ability to manipulate commit histories to allow you to pull down changes from an SVN server, do your own thing, commit as much as you want in yout git repository, and then when you’re ready, push it all back to the SVN server. It’s incredible. I can do all the local branching I want, play with things, finalize a feature, whatever, and in the master branch keep up with the day to day bug fixes. When I’m ready, I just merge and push. Now I don’t even think about dealing with straight SVN. The first thing I do if I’m going to be doing anything beyond reading code is pull it into a fresh git repository. From there, I’m free to use version control as it should be used without bothering anybody else. At the end of the day, I can either push all my commits out, or modify them into a few larger, more well defined commits. I pretty much never fight with subversion anymore, and that makes me a happier person.

Despite what I’ve said here, I still love Mercurial. It’s such a simple and well designed piece of software that it makes me smile a bit. It’s like the Mac where git is like Linux. I work on Linux because I need the raw power, but at the end of the day, I really want a mac to be sitting there beside me. So make your own decisions, but at this point I’m willing to admit it. Git is awesome.

After posting cobra, one of the things I heard most was “don’t try to make JavaScript be something it’s not”. This is good advice, but I feel that in this case it was given too hastily. Cobra did not come out of a desire to make JavaScript more like Python, even though that was the result. Cobra came out of careful consideration of how to make JavaScript better.

Problems with JavaScript

I think most people who happen upon this lonely corner of the internet are already pretty familiar with the flaws of JavaScript, but I’ll mention a couple of the most glaring faults briefly.

Scope

JavaScript’s scoping rules are dumb. Variables default to the global scope unless told otherwise and the this keyword, which is supposed to point to the current instance, actually points to the global object unless told otherwise. These scoping rules cause huge problems for beginning JavaScript programmers and trip up the most experienced programmers if they aren’t paying attention. Since these odd scoping rules default to the global object, and not to some error state, errors can go by unnoticed for long periods of time.

Object Syntax

JavaScript claims to be a prototypal language. It lies. It’s a language that also has prototypes. In a true prototypal language, there are objects. Objects can be derived from other objects, either by copying the parent object or by linking to it. JavaScript does something similar with its objects’ built in prototype attribute. Its prototype points to an object that it replicates the behavior of. However, it doesn’t replicate this behavior until after you create a new object. Let me demonstrate with some unsupported features of JavaScript:

//This is my base object. It's a pretty simple 4 step dance.
Dance = {
    danceAround: function() {},
    steps: 4
}
 
// Let it be known that I wouldn't know what a tango was if I saw it.
// (Unless there was a rose in somebody's mouth, of course)
Tango = {
     steps: 34,
     dip: function(){}
}
// Now we pull some true prototype magic.
Tango.__proto__ = MyDance;
 
// This works then:
Tango.danceAround();

That’s true prototypal behavior. There’s no “new” keyword, just behaviors that can be stolen by other objects. Unfortunately, JavaScript was thrown together in (I believe) about 15 minutes, minus a 5 minute coffee break. The writers realized that people were going to flip if it didn’t resemble the popular languages of the time, so they implemented a prototype-based language, pulled the prototypes into a separate property, and added a “new” keyword. This led to syntax like the following:

//This is my base object. It's a pretty simple 4 step dance.
Dance = {
    danceAround: function() {},
    steps: 4
}
 
Tango = function() {
    this.dip = function();
    this.steps = 37;
}
Tango.prototype = Dance;

That’s not quite as pretty. Not only is it not as pretty, but there are some fundamental flaws. The dip function gets recreated every time a new Tango object is instantiated. This isn’t a big deal most of the time, but once a year the king has a ball and all of a sudden you have 1000 partners tangoing about, and with them, 1000 identical copies of the dip function.

Another flaw is that the prototype is not in the lookup path of the object until a new instance is instantiated. In this example, Tango.danceAround is not defined. This is because prototypes are not applied to objects until a new instance of the object is instantiated. A “class” in most languages is a definition of object behavior. Instances of a class are objects that behave as defined by the class. This is very close to how prototypes behave in JavaScript.

To summarize, JavaScript isn’t quite a prototype based language, it isn’t quite a class based language, and the syntax for doing either is ugly (for more on the ugliness, check out one of my previous posts).

Fixing the Problems

Fortunately, JavaScript is awesome in most ways. It’s so flexible that fixing the issues I’ve spelled out above is no problem.

Fixing Scope

You can’t entirely fix JavaScript scoping. Local variables which aren’t declared with var become global, and there’s nothing you can do about that.¹ You can, however, fix the this object. You can wrap any given object method in another method which asserts that its this will be set to a specific object.

MyDance = {
     danceAround: function() { console.log(this.cheer) },
     cheer: "WHOOO!"
}
 
MyDance.danceAround = function() {
    MyDance.danceAround.apply(MyDance, arguments);
}

This is called binding, and it makes this be what you would want it to be in most instances. My initial thought was to just bind all object methods to their instances at the time of instantiation. However, I don’t really like this idea. For one thing, it changes the language. For another, some libraries (I’m looking at you jQuery) like to mess around with this. If this is expected to be something, I do not want to change that. What I needed was a shadow this, a variable that was always present and always pointed to the instance of the object. Luckily, there was an easy solution. Python always passes its instance object as the first parameter of any method of a class. I could replicate this behavior easily using essentially the same binding code and have the code look familiar to Python programmers everywhere. So I did. Every instance method in Cobra has “self” passed to it as the first parameter, which is automatically guaranteed to be the instance, no matter what. No binding required.

Fixing Object Syntax

It is fairly easy to get JavaScript to behave like a true prototypal language. However, I don’t much care for true prototypal behavior since it still leaves an ugly syntax. My solution was to create a “Class” object that will implicitly apply prototypes.

Instead of:

MyNewThingy.prototype.doSomeStuff = function () {};
MyNewThingy.prototype.doMoreStuff = function() {};

We can do:

MyNewThingy = new Class ({
    doSomeStuff: function() {},
    doMoreStuff: function() {}
});

These end up being exactly the same, except the latter is clearer and cleaner in my opinion.

Inheritance is a bit tricky in the first case. To achieve prototypal inheritance, I have to do some magic.

Base = {
   basicStuff: function() {}
}
ThingyPrototype = new function() {
      this.doSomeStuff = function () {};
      this.doMoreStuff = function() {};
}
ThingyPrototype.prototype = Base;
 
ChildThingy.prototype = ThingyPrototype;

Now ChildThingy inherits from Base and has some of its own functions in its prototype. Cobra takes care of all of this for you:

Base = new Class({
    basicStuff: function() {}
});
ChildThingy = new Class({
    __extends__: Base,
    childsOwnStuff: function() {}
});

Again, I think this is a lot clearer and cleaner.

If you put both these fixes together, you end up with Cobra, which you can read all about here.

To wrap things up, augmenting JavaScript to fix its flaws is not a bad thing. The question is what to add. I haven’t used Cobra for anything yet, but it’s my current pet project. We’ll see if it really makes JavaScript that much more pleasant.