Historically, I typically studied only one item at a time, whereas I had a friend who would do two or three.  I believe his approach to be better and have adopted it.

When you study a single technical book, all items are not equal.  The earlier items provide a required base for the later items; you have to fully understand and recall them in addition to the later material you are studying.  What helps with that fuller comprehension?  Removal and time.

If I read part A of a book and wait until the next day to read part B, my subconscious mind has a chance to process that material and master it.  Thus, coming back the next day makes reading and comprehending part B all that much easier.  It the same science that allows me to suddenly make connections on problems I have been working on when taking a shower.

Sometimes you may be aware of that processing.  For example, some nights I may recall having intense problem solving dreams around something I am studying as my brain forms new neural networks.  This also works for new video games.

Now, I am studying about five different subjects simultaneously.  Not only is the process less taxing and stressful, but I am mastering it more effectively.

Nonetheless, it is really interesting to think about information architecture, organization, structure and search as abstract concept independent of an actual application, as well as applying to the real world of grocery stores, department stores, libraries, etc.

A couple of things really stuck out for me in addition to what I learned in “Don’t Make Me Think”:

There are two main ways to organize based upon the needs of your users:

• Exact organizational scheme. When the user knows exactly what they are looking for (e.g., white pages)
• Ambiguous organizational scheme. When users don’t know what they are looking for.  Some types include organizing by task or topic, among other things

Hierarchical organizational structures can be tricky as many things do not neatly fit into a strict taxonomy.  A taxonomy that allows cross-listing is referred to as polyhierarchical.  However, if too much of this takes place, the value of the organization is reduced.

In addition, there is tension between the breadth and depth of the hierarchies.  It is generally better to go for greater breadth, particularly as your site grows.

My knowledge grew the most in the area of search.  I have a friend whose site needs search capability.  I had thought of plugging something in for him for the entire site, and until did not realize the complexities involved in matching the users needs to the search capabilities.

For one, your users may need recall or precision, but they can’t have simultaneously as these are mutually exclusive:

• Recall. Recall is oriented toward finding a greater number of relevant matches (e.g., doing due dilligence on a company you are considering joining)
• Precision. Precision is oriented toward finding just a few very high quality matches (e.g., instructions on deck staining)

You will need to configure or choose your search engine accordingly.  In addition, choosing to take advantage of automatic stemming (e.g., using thesauri) in your searching will result in greater recall at the expense of the precision.  Your users’ goals needs to play an important part in this configuration.

In addition, you can choose to index your entire site, or break it up different search zones.  Once again, the former increases recall, while the latter increases precision.

Finally, there are numerous ways and levels of details to provide around search results.  Once again, this will be based upon your users goals.

In the end, you need to really understand your users’ goals to be able to create the appropriate Information Architecture.

I (as Scrum Master) had already worked with the PM (e.g., Product Owner) to prioritize the features. We did not have them in user story format yet, but that was okay as we had a set of one to two sentence feature descriptions.

How could that possibly be okay?  Well, the real purpose of user stories is not so much to fully document the desired functionality but to serve as placeholders to drive face to face discussions between developers and PM.  From the Agile Manifesto:  “Individuals and Interactions over processes and tools”.  With only a few sentences, you have to talk.  You can add a few notes and assumptions, but I like the general advice about how if your user story notecard is not large enough to hold all the information, try a smaller notecard.

In other companies I had worked unsuccessfully with PM’s trying to leverage Use Cases as the new solution.  They were taking too long to write, the PM’s did not have time to do it, and often the PM’s would write use cases based upon the UI design they had in mind for a feature as opposed to addressing the harder task of writing it based upon users’ needs.  User stories force you to address the latter.

Make no mistake, designing features based upon the users’ real needs is challenging, hard, and painful.  The work we spent the last 1.5 days was much harder than the coding I expect to be doing in support of these features – I would go home exhausted!

First, we gathered the developers, QA and Doc together.  PM and UX served as the customer representatives in the Estimating Game.  We have our upcoming release, and needed to get a rough estimate for each feature for initial release planning and re-prioritization based upon effort.

The PM would grab the next highest priority user story (feature) and present it.  Then, all of us would ask a number of questions about the feature.  After that, developers would write their estimates down, and then they would reveal all at once.  The estimates would often differ wildly, so the developers would discuss their rationale.  We would repeat a couple times and come to consensus.  QA and Doc would give their estimates, and then it was tracked.

The meeting went well at first, but then started to bog down as the PM had to leave early and we were missing the business value.  So, for our next meeting, I asked the PM to convert each feature to an epic user story on the fly.  It was epic because it was too large in term of what would need to be done, and required being broken down into multiple user stories.

We would then raise a number of questions, continue to take the initial user story and break it down into other user stories based upon user end-to-end functionality.  We all worked extra hard to keep UI out of the stories, and to use implementation only as examples to flesh out desired behavior.  What was particularly helpful was using this format from the book:

I (as role) want to be able to do (feature) because (business value)

Since our PM is busy, I decided to use the meetings with him to focus on user stories, and then have separate meetings with the implementers to play the estimating game (and later sync back together).

Now that we had this, it was much easier for the developers to work with UX on how to approach the feature, and how to design it at a high-level for estimation in separate sessions (we’ll pull the PM back in to these as well as his time frees up).

For one user story, a developer pointed out that the user story presented a solution for solving a problem rather than stating the problem that needed to be solved.  It was subtle, but by being able to back the user story up to the more abstract representation of the problem to be solved, we can design a much better feature.

In general, it takes longer for the development team to complete the estimating game for the user stories associated with each feature than it does to work with PM to break them up.

Initially, I had hoped to approach the features planned for the release in a Just in Time (JIT) approach on each Sprint, but given dependencies between the features and user stories, it was important to spend a few days up front establishing the user stories associated with the release, even if all may not end up making it.

It was painful, but necessary and effective.  Sitting down together and talking through these features was highly effective.  We have a much better understanding of how to design the features in a way that will best meet the end user’s needs.  The Developers picked up a much deeper domain knowledge of the problems our customers are facing, as well as an increased respect for the PM (as did I).  I think the PM (by participating in the initial estimating game) had a better understanding of why the estimates were as high as they were.  All in all, we are much better positioned for success.

The book begins by using an analogy of physical building architecture, which builds nicely on the mental model I started to build up from Steve Krug’s “Don’t Make Me Think”.  Different building architectural styles serve different user purposes, labeling and classification enable users to navigate effectively, and the importance of search.  While there are also similarities with physical libraries, the multi-dimensionalities associated with the web present a different set of problems.

What I found interesting is their discussion on Information Needs and Information Seeking Behaviors.  They discussed four types of Information Needs:

• The Perfect Catch. You know exactly what you are looking for – someone’s telephone number, a fact about the population of the state of Louisiana, etc.  Basically, you are looking for “the right answer”
• Information Exploration. You might looking for the best apartment swapping services in Paris, or different investment options in your online 401K service (as I was recently).  There are multiple good matches.
• Exhaustive Research. You might be doing research for your thesis, or conducting medical research about a disease a friend may have acquired.  You want to leave no stone unturned.
• Refinding. This is where services like del.icio.us come in handy, or the “Favorites”  link in YouTube

The point is, how you design search capability and organize your site is going to differ vastly for these purposes, and an understanding of how your users will want to use your site will play a major role in your information architecture.  How you organize search, links, content and navigation will either enable or befuddle your users in their goals.  In other words, you want to set things up in such a way that your users do not need to think.

Another thing I found interesting (I am only through Part 1) is the Berry Picking Model by Dr. Marcia Bates of USC:

1. Start with an information need
2. Formulate an information request (query)
3. Move through the site(s) in different ways
4. Pick up important bits of information (berries)
5. Refine your query based upon what you already found and repeat

This stuck out because I was just doing this this morning before reading this:

1. Searching for help on using a particular technique
2. Finding some helpful articles (and either adding to del.icio.us or Evernote) while browsing then rejecting unhelpful ones
3. Altering the search query in the hopes that it would offer more links that better fit my need

Finally, even though the product I work on a web-based application as opposed to a site, an understanding of Information Architecture can also be helpful in terms of how we present information and work with user requests.

The WATIR site has a page on installation which is pretty straight-forward (I didn’t bother with supporting Safari on my Mac, and the Windows install was pretty straight forward as well).  Installing the plugins for FireFox referenced on the install page was also straightforward.

You do need to start Firefox up initially using the -jssh option (for IE on Windows, it comes up automatically).  Here is how I did it for Mac:

cd into /applications/Firefox.app/Contents/MacOS
./firefox-bin -jssh

For Windows 7:

cd into Program Files (x86)\Mozilla Firefox
firefox.exe -jssh

I brought up Ruby’s IRB to start playing around.  On my Mac, I could execute commands in the IRB, but in my ruby script file, it was failing on the following:

require ‘watir’

For the scripts, I needed to add require ‘rubygems’ first.  On Windows, I needed to do this for both IRB and ruby scripts:

require ‘rubygems’
require ‘watir’

To bring Firefox, I did the following (you can just comment out the first line to bring up IE):

Watir::Browser.default = “firefox”
b = Watir::Browser.start “http://mysite.com”

The browser popped up to this site.  I needed to login, so I specified the name and password by finding the element by id and then specifying the text to be typed in:

b.text_field(:id, “user”).set(“name_of_user”)
b.text_field(:id, “password”).set(“the_password”)

In the browser, it was almost as if an invisible person typed in the text.  Next, I needed to the “Sign In” button, but there was no id associated with it, so I had to click it after finding it by its value:

b.button(:value, ‘Sign In).click

Now the home page came up.  I wanted to create a new Foo, so I needed to get to the Foo page, which is referenced by a link and is called “New Foo” on the page:

b.link(:text, ‘New Foo’).click

I was now on the new page, and starting following the steps to fill out the fields to create the new Foo.  However, I was getting the following error:

 C:/Ruby/lib/ruby/gems/1.8/gems/watir-1.6.5/lib/watir/element.rb:56:in `assert_ex ists': Unable to locate element, using :id, "username" (Watir::Exception::Unknow nObjectException) from C:/Ruby/lib/ruby/gems/1.8/gems/watir-1.6.5/lib/watir/element.rb:288 :in `enabled?' from C:/Ruby/lib/ruby/gems/1.8/gems/watir-1.6.5/lib/watir/element.rb:60: in `assert_enabled' from C:/Ruby/lib/ruby/gems/1.8/gems/watir-1.6.5/lib/watir/input_elements .rb:327:in `set' from watir_fun.rb:6 

I double checked the id’s; all looked well.  It turned out that that page was doing some additional javascript after having been loaded, so these fields were not ready for me to access.  By adding ‘sleep 2′ to the script prior, the page had time to load; and I was able to follow the steps to create the entity.

But was the entity created successfully?  Because none of the displayed fields had a unique id (and this was only a quick experiment), I simply checked with something like the following:

b.text.include?(“Hot Stocks”)

Obviously, for production readiness this is not acceptable, so we would likely had keys or some easier way to access via XPATH and so forth.   The point is, WATIR gives you an easy way to automate interacting with a browser and seeing what the results are.

In the end, I found picking up WATIR to be quite straightforward.  Back when I was at my current company previously, I used to go through a short manual test script before checking it to make sure my changes didn’t break anything.  This weekend (for fun), I hope to code up this former script in WATIR in just a few hours and start having us use it in Development next week.

Additionally, the WATIR web-site is helpful, well-organized, and I hear the help and mailing lists are quite responsive and friendly.

Two weeks or so ago Stuart Halloway gave a presentation on Clojure at another NOVAJUG meeting held at Oracle. Stuart is one of those guys who makes you marvel how fast his brain works. Unfortunately, his one hour presentation probably could have used 90 minutes to two hours to fully absorb. I went by the book store the next night, and all copies of his Clojure book were gone.

For the last five months or so, I have been trying to figure out whether to study Clojure or Scala. Given the fact that I am in chapter 2 of Stuart’s Clojure (as well as the subtle title of this post), I have decided to go with Clojure.

Scala looked very promising, but Clojure seemed far more different than Java. I view that as a good thing. Getting experience in a variety of language styles as opposed to those that seem closer to Java can only be a good thing, No, I am not saying the Scala is just like Java.

Clojure does have a lot of common with LISP, which I enjoyed programming in in college. But it appears to have made some improvements over standard LISP.

Its functional and transactional approach to concurrency (as opposed to manual locking) seems interesting.

Finally, as Ruby is a more expressive language than Java, Clojure appears to be even more expressive than Ruby. Clojure may be very different than Java, but it is easier to program in (as claimed by Stuart).

Will Clojure be the next big language? Possibly. Will I be a much better developer by learning a language such as Clojure? Definitely.

def sed_result(input_str, sed_str)
     %x[echo #{input_str} | #{sed_str}]
end

It worked well, until I did this:

assert_equal("howdy\n", sed_result("howdy from Texas", "sed 's/\([a-zA-Z]*\).*/\1/'"))

Assigning the latter string to a variable in ruby produces a different string, because strings in double quotes are processed: “sed ‘s/([a-zA-Z]*).*/01/’”

For example, when I want to do string interpolation, I have double quotes instead of single quotes as follows:  “The value of foo is #{foo}” to the string “The value of foo is bar”.  This time, I don’t want any processing of the string, but want to pass it as is.  Thus, I need to use single quotes, except my string contains single quotes. What to do?

Well, the following use of %q[] is the same as using single quotes, and worked well:

assert_equal("howdy\n", sed_result("howdy from Texas", %q[sed 's/\([a-zA-Z]*\).*/\1/']))

Additionally, you can use %Q[] in place of double quotes.

Right now, I have been going through SED, which does not have an associated unit test framework.  To test this, I want to use Ruby and then spawn a process to execute SED and then get the results back for the assert_equal call of all my tests.  I want to do this while avoiding the overhead of using files and file redirection (where possible).

I obviously can’t use Kernel::exec, as that will replace my currently running process and end my testing real quick.  I want to do the following:

     system("echo #{input_str} | #{sed_str}")

Where input_str is the input SED is processing, the sed_str is the SED command str.

Of course, I can’t do this as this is two processes being piped together.

If I use read and write files, I can do the following:

     system("#{sed_str} output.txt")

In the end, I got it to work as follows:

def sed_result(input_str, sed_str)
     %x[echo #{input_str} | #{sed_str}]
end

There are other options to look into – IO.pipe, IO.popen and Open3 (among others).  But I have a lot of SED unit tests to write first!

The major premise of the book is that we often grow up with the myth that people either have a talent or they don’t, and that that is what their determines their successful (or its corollary – you must have a great inborn talent and practice).   It vigorously tears down this myth in regard to Mozart, and then Tiger Woods.  In a nutshell, sustained practice is what determines your level of success.  And not just any practice – deliberate practice, which is often not very fun.

It reminded me of “Asians are intrinsically better a math” myth that is debunked in Malcolm Gladwell’s Outliers.  A hard work culture that results in lots of hard work and practice in math, a discipline that requires a lot of practice.  I have shared this with my daughter many times vis-a-vis math as she studies to get into college and become a doctor.  But have I shared it with myself enough?

How many times have you worked with a programming wonderboy and attributed it to a natural talent that very few people have?  How often have you let yourself off the hook due to this?

This last year has been an intense year of deliberate practice for me.  Outside of work, I spend 15-20 hours a week studying computer science – learning new languages, mastering old ones, new OS’s, techniques, technologies, etc.  It has been a lot of deliberate practice, and my skills are far superior to what they were a year ago.

I have also noticed that the more I study, the easier it is to pick up new concepts I come across.

In addition, I have developed an absolute passion for programming that I had not experienced in a while.  All of this is leading to more deliberate practice and greater proficiency, as well as widening of the areas I am interested in and studying.

My latest thing is going through the Ruby Quizzes from Pragmatic Programmers.  I am not doing it to learn Ruby, but to write better Ruby.  The exercise in writing the best Ruby code I can, comparing it to example, and making changes it making me a much better Ruby developer.  I am learning about my strengths and weaknesses as a developer, and focusing on improving those weaknesses instead of doing the same old thing over and over.

So I picked up his book – Clean Code, and summarized the contents to be able to repeatedly refer back it.  A number of the things he covered I already knew from an earlier study group on Martin Fowler’s Refactoring book.

Understanding it intellectually was not going to be enough so I picked out one of the quizzes from the Ruby Quiz (Pragmatic Programmers) and tried to apply it.  At the same time, I also decided to do full “Orthodox TDD”, where I did not do any development until I had a failing test.  I worked through some design alternatives up front, and then proceeded with the one I liked.

It was slow going.  I tried to be as true to the book as possible, and then meant a lot of refactoring as I thought of new ways to create cleaner code based upon the book.  As much as I like Ruby, Java is definitely an easier language to refactor (but thank heavens for all the unit tests – a necessity in Ruby).  I have seen a book out there on Refactoring in Ruby, and I may pick that one up.

But slow going is okay.  When you study martial arts, you are supposed to go slow as you apply the techniques.  With practice, speed comes later on top of perfect technique.  In this practice scenario, I was doing just that.  This was the major reason I was doing this on a home project as opposed to work, where speed is obviously more important.

More challenging than I expected was creating extremely well named entities – a lot of rename refactoring took place.  Nonetheless, the number of comments I had in my code was extremely small – it was unnecessary as the naming told the story.  I tried to follow his newspaper model of letting my code tell the story in various levels of detail – this worked out well in the end.  Plus, the lack of comments that were not there that typically were “because they are supposed to be” made the code much easier to understand.  It was better, and liberating.

I did struggle with some things.  The breaking down of all the methods into small methods ended up with a lot of lower level methods.  While the methods were extremely cohesive, there were a lot of them.  A good signal to me that I had broken them down into too many sub-methods was the extreme difficulty I had in creating names for them to show their intent without using any type of synonyms.  Based upon that, I recombined some of the methods.

I also struggled with his advice on not using return codes or error codes.  In his statement, if your method performs a state change on the object and returns a return code, it is doing both a state change and a query, violating the cohesiveness of the method.  By throwing an exception instead, the method is more cohesive and the consumer can write less complex code.  Plus, if you refactor complex methods into multiple more cohesive methods, having to check for return codes permeates through each invocation, increasing code bloat.

Nonetheless, this seemed to run against Joshua Bloch’s advice in Effective Java on using Exceptions only for exceptional conditions as well as the overhead associated with them.  I also did not like having to create a new Exception class as this seemed to increase the necessary mental model for code.

Then, I thought on the pattern typically found in Enumerators – a cohesive querying hasNext() method along with a next() method that either returns the next entity or throws an exception if one is not found.  How many API’s had I seen lately where they had been modified to offer a similar query method, or offer variations of methods that either returned an error code or threw an exception.  I am seeing the value, but I am continuing to work through this.

Overall, this is an excellent book to study.  I will continue to apply it as I work with other technologies at home to improve my general coding abilities.