Pairing Tour: Day 7

Today I got to pair with Doug Bradbury on some Rails work. Doug is currently working on a new project, so we got the chance to lay some of the foundation of the app. All of the stories for the iteration are currently blocked because they are dependent on work being done by other developers, so we got they rare opportunity to spend the day refactoring. For this Rails app we are trying to use the Boundary, Interactor (Control), Entity pattern inspired by Uncle Bob. The goal of this is to make our tests much faster. By decoupling our business logic from our delivery mechanism, we will produce a better design that is easier to test. So we started by refactoring a controller action to not depend on an Active Record model. The required us to isolate the calls and return values from active record in our interactor (business logic module). Once we finished this we decided that we wanted to remove the dependency on Active Record from our interactor.  So we started to abstract the calls to Active Record into Database transaction objects. These objects represent one transaction with the database in the same way that application interactors represent one business rule. So, we created one Transaction class title FindAuthorizationByNameAndUid. The name of transaction itself clearly represents what this module is supposed to do. It can be implemented using Active Record or any other persistence mechanism. Most importantly, it can be easily mocked out in tests my an in-memory equivalent. By the end of the day we were able to remove Active Record models from the interactor and put them into transaction objects. The refactoring of the code was really easy. Since our interactor was already tested, we made all the refactors that we wanted and the tests still passed. However, we were unable to refactor our tests to not use active record objects. This is little bigger of a task because the in-memory test double must be built first. So, I'm going to pair with Doug again in the morning to see if we can find a good way to test the interactor without using the Database. All in all, it was a very enlightening day on how to decouple application logic away from Rails.

Pairing Tour: Day 6

Today I got the opportunity to pair with Craig Demyanovich. The first part of the day we attended a project retrospective facilitated by Uncle Bob. The retro provided a lot of context as to the history of the project, the team members, feelings about progress and the goals of the project. Essentially, the project is a "great big redesign in the sky". They have decided that their current application is no longer meeting the needs of the business, especially in the area of extensibility. As their business has grown, there has been an obvious desire to extend their application to serve more markets are more consumers. However, the architecture of their current system isn't able to support the growth of their business. So they have decided to redesign a new one. In some ways the new system is rebuilding the old application, but in many other ways, the new system is a "platform" in which to build other applications on top of. I say platform in a very loose way, because it's not really a platform. However, they have utilized service oriented architecture with the mindset that many other applications will need to consume the business logic that they are building right now. This is intelligent way to approach the problem of future extensibility. The complexity of the SOA has caused a few pain points along the way, however, the team seems to be handle the application very well by practicing the disciplines of TDD, pair programming, and continuous integration, all of which are new to the company. The team has done a remarkable job of coming into a fresh project and practicing disciples that they did not use on previous projects. The second half of the day, Craig and I were able to work on a few stories for the application. Close to the end of the day, we ran into an issue with the Ruby Date class. Our problem was simple, we needed to check that a given date was less than or equal one year from the current date. Here was our first try.

(Date.current - given_date) < (Date.current - 1.year)

However, this didn't work at all. The expression on the left side takes the current date and subtracts the given date, with yields a Rational, representing the amount of days between them. The right hand side takes the current day and subtracts one year, which yields a Date one year ago. So in the end we were comparing a rational to a Date which blew up. Our next try was 

(Date.current - given_date) < (Date.current - 1.year.ago)

This time the right hand side takes the current day and subtracts it from 1 year ago. However, this doesn't work either because 1.year.ago yields a DateTime object as opposed to a Date object, so this blew up too. So we massaged this a little to get it to work.

(Date.current - given_date) < (Date.current - 1.year.ago.to_date)

Finally! Both sides of the expression produce a Rational object which can be compared to each other!

Pairing Tour: Day 5

Today I got to pair with Brian Pratt. Since Brian was transitioning from one client project to another today, I got to sit in on the IPM for Brian's new project in the morning. Most of the stories picked out for the next iteration were front-end features and refactorings to prepare for a new UI design. So, in the afternoon we picked up one of the UI stories. This is a shift from my normal work, so it was a good learning experience. We had to use mock ups and a demo app as a template in which to redesign the header and footer of their website. This doesn't sound terribly complicated, but every one knows that HTML and CSS never like to play nice. We got caught up on a few styling issues throughout the day, but by the end of the day were able to successfully implement the design. I definitely learned a lot about working with designs and mock ups, as well as a few CSS caveats today. Although, one thing is for sure, I'm glad that I'm a developer and not a designer. On another note, I'm making some great progress in my refactoring of Jasmine, the BDD framework for Javascript. Over the weekend I was able to get all of the preexisting tests passing again. Now that the system is working as it was before, I believe that my next steps are going to be redesigning the internals (refactoring out the "Env" God object) and rewriting all the tests. Currently, Jasmine is tested with Jasmine. The source is used to test the source. This seemed cool to me at first, but now I'm convinced that it is a really bad idea. For instance, I can break all of the tests in the system by commenting out one line (the "describe" definition). This will essentially turn describe into a noop, leaving the system with no tests. So, I'm going to rewrite the tests without Jasmine. Also, there are some pretty nasty SOLID violations in there as well, so I would like to take a golden hammer to a few of those files. Should be fun.

Pairing Tour: Day 4

Today I had the opportunity to spend the day with Dave Moore on some client work. We were able to work on a few interesting stories throughout the day. Dave is currently on a team that is working on internationalizing a Rails application. I was able to pair with Dave while we tried to internationalize the social media widgets on the site, specifically the Facebook login, button, like button, and login window. This can be accomplished through calls to the FB API. However, we were seeing a very specific bug with Facebook's Api. When we opened the application in a specific language, say German, the Facebook button would render in German, no problem. Also, we could click on the login button and the FB Connect window would show up in German, no problem. Also, once we changed languages on our application, say we switched to French, the buttons would render find in French, all through the FB API, no problem. However, after changing languages in our application, the FB API login window would not change languages with us, even though the FB button was changing languages with us. After some searching we deduced that FB renders it's buttons solely off of the API calls you make to it, which can change on every page refresh. However, it renders it's login window based on the language that was specified when we first logged in, so we would see the window in German even though we had switched to French. We were able to track that FB was storing the locale information that we first gave to it within a cookie on the browser. So, our first instinct was to attempt to mutate the FB cookie every time we changed languages in our application. However, after some searching we found that altering cross domain cookies is not allowed. So, at this point we were essentially stuck. Facebook is going to render the login window based on the first API made to it. I definitely learned a lesson today about 3rd party dependencies. Even though they are extremely convenient at times, they come at a price of freedom. When you use a 3rd party library, you lose the freedom to manipulate every detail, and as I learned today, that can be a high price to pay.

Javascript Woes

Ever since I attended GeekFest at Groupon on Tuesday, I have been interested in working with Javascript a little more. The talk was about sharing code between client and server via Node.js. It really got me curious about what that would look like on real project. However, I've not had the opportunity to try it out yet. My exploration into Node.js and Javascript over the past week has led me down a very dark and desperate path. The first woe that I encountered in my exploration was modules. As I tinkered, I began to realize that defining and loading modules was a very fragile process. I was quickly dismayed. I did some googling and found and library called RequireJS. This library implements AMD protocol for Javascript modules, which is great because it offers a standard definition that Node and Browsers can use. So, in theory, one could write one application that runs in both places. I was happy again. Then I decided to explore some previously existing Javascript projects to see how they handle modularization. I took a look at Jasmine, the BDD framework for Javascript. As I was browsing their code, I realized that they didn't attempt to use any sort of module loading like I hoped. Instead, they compile everything into one file, test against it, and distribute it as such. This isn't a bad method per se, but I think that it is possible to do better now that the toolset for Node has matured. Also, I was really displeased to see that Jasmine has a lot of Ruby files floating around their repository (mostly rake tasks). This also isn't bad per se, but once again, the Node toolset has matured and I think its possible to do better now. So, with that in mind, I decided I wanted to hack on Jasmine a little bit. First, I deleted all the Ruby files. Second, I decided to reorganize the project structure to separate Node specific files from Browser specific files. Now, I'm trying to convert all of Jasmine core into Node modules via RequireJS. This is when the headaches started. As it turns out, it is hard to test a testing framework. For instance, when I change the test Runner, not only does that break the tests, but it breaks the runner that which shows me that the tests are broken. I know, headaches. Anyways, I'm continuing to refactor Jasmine and learn more about Node. I'm hoping that soon I will have enough grasp to write a simple Todo's application where code is shared between client and server.

Pairing Tour: Day 3

Today I continued my pairing tour with Steven Degutis. I worked with him on a client's Rails app. However, this particular client has a very interesting setup with their application. They have brought service oriented architecture to the Rails world. Their application is actually multiple applications communicating with each other via HTTP requests. This may sound pretty messy, but they have managed to isolate the service calls into gems, which are managed and tested on their own. So each service (i.e. Rails app) has its own gem that wraps all the HTTP calls so that the user of the service is none the wiser. This really doesn't seem to bad. However, in practice it hasn't turned out so well. The have experienced some pain in adding features to the services. Adding a feature to a services means updating all of it's clients as well. This is not an inherent aspect of SOA, but it is a sign that they have broken the Common Closure Principle. They are starting to see that many of these separate modules all change together, which means that they should be kept together in the same package. Unfortunately, there is no simple answer to solving this problem. Bring the clients and services together into the same package is going be very difficult, especially since both clients and services are Rails apps. So, I definitely learned a lesson about the CCP today. On another note, pairing with Steven was a fun time. He is very discipled. He sticks to his principles well. And, he is very fast. I don't think I have ever seen anyone type and maneuver around Vim like he does. I definitely leaned a lot by pairing with him. I'm not pairing with a Craftsman tomorrow, because it's Friday, so I get to work on some Open Source stuff. I've been interested in Jasmine, the BDD framework for Javascript lately, so I'm planning to work on that.

Pairing Tour: Day 2

Today I had the opportunity to pair with Mike Jansen on some client work. He is currently working on a Rails app with some heavy client side Javascript via Backbone.js. We worked on a mix of bug fixes and new stories. Most of the bug fixes were in the Javascript code, while the new stories were in the Rails app, so I got to work on a variety of interesting stories. However, I didn't get to work on any Backbone code. In general, I'm a fan of client side Javascript frameworks and I like the way the our client is using Backbone in their app. I believe it has awarded them with a really rich UI. The Rails app I worked on today was nowhere near as large as the app that I worked on yesterday (1,500 specs vs 15,000 specs). This made a large difference in development feedback and turn around. We were able to fix a bug in the morning, push it to the CI server, and QA gave us feedback with in a few hours. Another plus was that we were able to run the entire test suite (acceptance + unit) locally. Even though this is better than having to wait 45 mins to get feedback from the CI server, it still took 5-10 minutes to run the suite. I think I'm starting to get a more clear picture of the dream world I have been living in the past few months of only Clojure development. The only thing limiting the feedback time on those projects was the JVM boot up time (~3 secs). At any rate, I'm grateful that I have been able to get a better understanding of what development will be like on client projects. Continuing my theme from yesterday about developer's personalized environment, I noticed that Mike has a very similar setup as Eric, besides that he uses MacVim, which is fortunately very easy for Textmate users like myself to pick up on. I'm starting to see that the Craftsmen at 8th Light really take ownership of their environments and customize them so that they can work as fast as they can. I'm hoping to take note of what other Craftsmen are using a start personalizing my environment a little more, albeit cautiously with version control. Tomorrow, I am off to pair with Steven Degutis on some more client work. Should be fun!

Pairing Tour: Day 1

As part of my Apprenticeship, I get the pleasure of going on a pairing tour with four 8th Light Craftsmen. While pairing, the Craftsmen will be working on client projects, so I will gain some exposure to our current clients and an idea of what client work is like. This also gives me a great opportunity to learn from the Craftsmen I will be pairing with. The tour also gives a few Craftsmen some exposure to me, which will allow them to evaluate me fairly at the end of my apprenticeship. Today marked my first day pairing with a Craftsmen, Eric Meyer, whom I met with at a client office. We worked on two interesting stories that kept us busy all day. A few things a took away from the day. 1) Eric really takes ownership of his development environment. He was able to navigate around his system very quickly. He has lots of bash aliases set up. He has even set up a special windowing system. Recently, I made a conscious decision to avoid extravagant customization of my development machine because it makes context switching (changing machines, pairing on someone else's environment, etc...) a lot harder. I want to stay flexible with my set up. However, after watching Eric today, I'm starting to rethink that decision. 2) Continuous Integration servers (done right) can be pretty sweet. The CI server at the client had a lot of cool features, such as testing a branch based on the branch name, testing and then merging branches automatically. However, the downside to the really cool CI server is that it takes 45 minutes to run the test suite (Ouch!).  This makes it hard to get instant feedback. Yet another example of why Rails needs to be contained. 3) The client we were at had a really cool talk over lunch today about reusing client and server code via node.js. It was extremely intriguing. It looks great in theory and I would love to build an application to see how well it works out in reality. Hopefully someday soon.

Artisan: Week 2

It was a pretty quiet day at the 8th Light office today. As such, I was able to study the Artisan (our project management tool) code base more in depth along with completing some small clean up stories. Looking at Artisan today reminded me of my first week at my previous job. Diving head first into an existing code base can be overwhelming regardless of how large it is. Artisan's code base is by no means bad and is fairly well maintained, but it is still daunting to try and grasp everything that is going on. I've used two methods to help me try and understand features in the code base. 1) I look at specific use case in the UI and then try and trace the feature the entire way through the system. 2) Look at the tests. This method may be harder for those who don't have a large suite of tests wrapping their application, but lucky for me Artisan is well tested. However, some tests with in the code base are really hard to read, especially tests that make heavy use of stubbing. This had made me realize that I probably shouldn't use much stubbing in my tests because it is really hard for others to understand. Setting up five stubs, then calling one method isn't really too helpful to read and doesn't always reveal the intend of the test. So, this is yet another reason that I prefer to use hand rolled mocks over framework stubs. Overall, I think I'm learning a lot about Rails development through working on Artisan, but there are still some pain points (like slooooow tests).

First Live Kata

I had the pleasure of presenting the Ruby to JSON kata that I posted a few days ago at 8th Light University today. I had a few issues towards the end of my kata, but overall it went well. When I first started practicing the Ruby to JSON kata, it took me around 10-12 minutes to complete. The video I posted a few days ago took me about 8 minutes. Today it took me about 6. In order to cut the time in half I had to practice this kata relentlessly for a few days. I think that I have probably written this kata close to 50 times now. But, that is what it takes to perform a cool kata in 5ish minutes. One cool thing that I did for my kata was playing music in the background. I made this decision because my kata did not really need any specific narration and I didn't want the silence to be awkward. Through this I discovered that it is really hard to perform a kata along with music. I don't just mean playing music in the background while you code, but actually pacing yourself to play along with the music. Towards the end of my practicing for the kata, I was able to do this pretty well. I picked a song throughout that lasted 5:30 and was able to track myself with certain markers in the song. However, during the live presentation, it flopped because I was so nervous. At first, I was typing really slow and got really behind in the song. Then, I had a few typing mistakes that put me behind further. In the end, I finished about 30 seconds after the song, which stinks, because in my practice I would usually end the kata about 15-30 seconds early (the song fades at the end). Overall, I believe that I benefitted a lot from performing the kata and I think (hope) that the views did as well.

Working with Rails again

After working with Clojure for the majority of my apprenticeship, I'm now working on a Rails project. The transition has been pretty smooth, but nonetheless it has certainly been a different experience. In may ways, it feels like I am learning Rails all over. In my previous experience with Rails projects I never practiced TDD. So, now I am learning how to write good tests with rspec. Which, as it turns out, is kind of hard in Rails. Also, I'm learning how to write testable (i.e. good) code in Ruby, something I have never done. One of the ways we are attempting to write testable Rails code is abstracting all the actually application logic away from Rails. This is great. It makes it really easy to test the core application logic without being coupled to Rails. What sucks about this is that there really isn't that much core application logic, or at least there is not that much abstracted. The bulk of the application is the UI. Therefore, most of features are implemented with a combination of Haml, Coffee Script and Ruby, wherein lies the difficulty in testing. For each feature, we have to write rspec tests, Jasmine tests, and cucumber acceptance tests. Something about this doesn't sit quite right with me, however, I'm not quite sure why yet. Perhaps I will write a future blog analyzing what about this perturbs me. However, the transition to Rails has made me realize a one things about Clojure development that I took for granted. The Speclj auto runner. Writing tests an getting instant feedback is a complete game changer. I didn't realize this until we started writing tests a few days ago and had to manually run the tests every time we changed something. It was a complete waste of time. So we started using rspec-guard, a nifty gem that auto runs rspec tests upon change. However, this was only a partial solution. After getting guard up and running with our project, I realized that I wasn't getting the instant feedback loop that Speclj provided me. This is because every time guard detects a change, it has to reload the spec helper file, which essentially reloads Rails. So, every time I changed a spec file that has two tests in it, I have to wait 10 seconds for Rails to load. This is huge time waster. I'm guessing that the solution to this problem is to not use Rails, or at least isolate it as much as possible. However, that doesn't seem like too feasible of a solution with the current project I'm working on because as I said previously, there isn't really that much application logic. So, what can I do? Anyways, moving back in to Rails code has been interesting to say the least.

Chicago Clojure Meet Up: Overtone

Last night I had the pleasure of attending the Chicago Clojure meet up on Overtone. This was actually the first meet up I've ever attended. Over all, the experience was really cool. First, I was pleased to see 20 people gathered about Clojure. I love Clojure and I was excited to see other people passionate about Clojure. Second, the software demoed, Overtone, was really cool. Kevin Neaton, one of the contributors, gave an hour and a half demo on the capabilities of the Overtone, and a brief preview of where the project is going. For those unfamiliar, Overtone is an open source project that allows users to create musical synthesizers. So, Kevin started the night by showing us a very simple sine wave and then incrementally built upon it throughout the night until he created a dub-step beat. Pretty cool. The Overtone project offers the benefit of being written in Clojure, which offers some cool features, for instance, recursive beats. The built-in recursion of Clojure allows users to create recursive beats with ease. Also, it offers users the REPL. Users can fire up the REPL and create, test and demo beats on the fly without a ritzy setup. Overtone seems like a great tool for musicians and audiophiles alike. However, being neither of the two, I don't see myself using Overtone much, except of course to play around a see what comes out. Overall, I had a great time at the meet up and I hope to keeping attending.

Ruby to JSON Kata

I spent a large portion of my day today practicing for the Kata that I will be presenting on Friday. My original intention for this Kata was to take a well known kata that has a commonly accepted solution and rewrite it without using mutable data. Unfortunately, I was unable to find an existing kata that would showcase immutable data well. So, I decided to use the kata written by a fellow apprentice, Patrick Gombert, which does showcase immutable data rather well. Following is the Ruby implementation of this kata. Originally, this kata was written using Clojure, which really showcased immutable data. I believe that my implementation could be greatly improved by Clojure multimethods, which is what I would like to try tomorrow.

# Acceptace Criteria # ------------------ # nil -> null # Fixnum # Float # Booleans # Strings # Symbols -> converted to strings # Arrays -> all values converted to JSON # -> values can be anything # Hashes -> all values and keys converted to JSON # -> ignores keys that are not Symbols or Strings # -> values can be anything def str_or_sym?(data) data.is_a? String or data.is_a? Symbol end def convert(data) if data.nil? "null" elsif str_or_sym?(data) "\"#{data}\"" elsif data.is_a? Array "[#{data.map {|entry| convert(entry)}.join(", ")}]" elsif data.is_a? Hash "{#{data.map {|key, value| "#{convert(key)}: #{convert(value)}" if str_or_sym?(key)}.select {|x| !x.nil?}.join(", ")}}" else data.to_s end end describe "Ruby to JSON" do [ [nil, "null"], [1, "1"], [1.0, "1.0"], [true, "true"], [false, "false"], ["a", "\"a\""], [:a, "\"a\""], [[nil, :a], "[null, \"a\"]"], [{:a => nil, :b => :c}, "{\"a\": null, \"b\": \"c\"}"] ].each do |value, json| it "convert #{value.class}" do convert(value).should == json end it "array values can be #{value.class}" do convert([value]).should == "[#{json}]" end unless str_or_sym?(value) it "ignores hash key type #{value.class}" do convert({value => 1, :a => 1}).should == "{\"a\": 1}" end end it "hash values can be #{value.class}" do convert({:a => value}).should == "{\"a\": #{json}}" end end end

Prime Factors Kata with Immutable Data

Here's my take on the Prime Factors kata in Ruby. I set out with the goal to use immutable data instead of the normal implementation which mutates an array. Check it out here.

Dependency Inversion Principle

In a recent project I have been working on, I've been learning a lot about the Dependency Inversion Principle. There were some places in our code base were Micah showed us how we were violating the DIP and showed us how to refactor. In short, the DIP states that no module should be dependent on a concrete class, but rather an abstraction. Violations of this principle are commonly seen in higher level modules depending on lower level modules, such as a UI depending on a Controller or Model. This means that the modules within the UI become coupled and are not able to be reused later. In our case, our UI layer was directly depended on the concrete implementation of our application classes. In order to adhere to the DIP, we needed to refactor so that our UI was not dependent on our application logic. This required that we build two abstractions. We built an abstraction for the application (called an Interactor) and an abstraction for the UI. The UI interface talks to the Interactor interface and the Interactor interface talks to the UI interface. This allows the UI to be decoupled from the application logic. So, in theory, we could build another implementation of the application underneath of the UI and the UI would be none the wiser. By the same token, we can now build any number of UI's on top of the application and the application is non the wiser. The second scenario is not really a theoretical possibility, but reality. There are tentative plans to build a web application on top of the current architecture (in addition to the desktop app we are building now). Without these abstractions, building the new Web UI would turn the code base into a tangled mess that no one could maintain. So, in summary, always program to an interface or abstraction, never a concrete class.

Test Generation with speclj

Let's start with a simple test definition in speclj, the BDD testing framework for Clojure.

(describe "my module" (it "has the correct value" (for [item {:item1 "value1" :item2 "value2"}] (should= (val item) (get-value item)))))

This test simple asserts that my module has the correct value for item1 and item2. Pretty simple. Ok, let's say that we add 15 more fields to my module. Our test will now look like this...

(describe "my module" (it "has the correct value" (for [item {:item1 "value1" :item2 "value2" :item3 "value3" (etc.)}] (should= (val item) (get-value item)))))

Not too bad. For every test that we add all we have to do is add a key and value to the for loop. Seems pretty extensible to me. So later, one of the assertions fails. However, we only get one error. Thats not good. That makes it seem like our entire module is broken. We first have to identify will field the test broken on, then we have to go fix the bug. This could be especially painful if there were more like 1000 fields! Ok, so lets find a better solution. Consider this example.

(describe "my module" (for [item {:item1 "value1" :item2 "value2"}] (it (str "has the correct value for " (key (name item))) (should= (val item) (get-value item)))))

So what is different in this example? Instead of generating assertions inside of the (it...) block, we loop over it and generate a bunch of (it...) blocks. So what is useful about this? If we consider the example above were "my module" has 1000 fields. Using this new method, we will be able to tell exactly which field this test failed on. This is extremely helpful because it cuts out the entire time it takes to find which assertion was the culprit of the failure. Also, when this module fails, not every test will fail. We will get better feedback on what is actually being tested and breaking. So, in summary, if you need to test a large list of items, it's better to generate tests for it rather than generating assertions.

ns-resolve in Clojure

One interesting feature of Clojure that I was able to make use of today is ns-resolve. This feature allows you to dynamically find a var within a specific namespace. This is especially useful for dynamically calling a function, given a string. I have also used this method in the metis validations project to dynamically load validation functions. Consider the snippet from metis,

(defn get-validation [validatior-key] (if-let [fn (ns-resolve 'metis.validator.validations (symbol (name validatior-key)))] fn (throw (Exception. (str "Could not locate the validator: " (name validatior-key))))))

Here I'm using ns-resolve slightly differently because this method is within the metis.validator.validations namespace already. This function is used to retrieve a validation function based upon a keyword. For instance, (get-validation :inclusion) returns the function inclusion from the validations namespace. This is allows us to not violate the open/closed principle. Users have access to the entire namespace by only using one method on the namespace. This means that if a new validation is added, the user does not have to change their namespaces imports to use it, they just start using it. In this way, the validator is closed to modification and open to extension.