Feeds:
Posts
Comments

I wrote a shell script that used SED to make context-specific changes along with copying files (among other things).  Out of the six scripts only one of them consistently experienced a problem – the last line was missing in all of the copies.

I wondered if there was some kind of dependency between the steps where something wasn’t getting properly written out, but that seemed odd for Unix.  I tried to do line by line comparisons to find the error in my shell scripts.  Finally, I came up with a different solution – simply add an additional blank line to the file to be copied.  After that, everything worked fine.

It wasn’t until I was studying SED in more detail yesterday that I realized that this was a necessary step.  SED seemed to ignore transforming or working with the last line of input that did not end with a newline.   I find this odd – if the last line of a file does not end with a newline, SED will ignore it?  There has to be more to the story than that.

Anyway, I am about halfway through my study with SED and will blog on that and then move on to AWK.

I recently listened to “Uncle ” Bob Martin giving a podcast on Software Craftsmanship on Software Engineering Radio, and decided to pick up his book Clean Code – A Handbook of Agile Software Craftsmanship.  I was struck by the forward by James O. Coplien of Denmark on the 5S Principles and how they tied to maintenance instead of production in Total Productive Maintenance (TPM).  Furthermore, these principles as applied to the maintenance floor apply equally well to software:

  • Seiri (organization). You need to be able to find things easily.  In software, proper naming helps you find things easily.
  • Seiton (tidiness; systematize). Everything has its appropriate place.  Code should be where it is expected to be, and if not, refactor to get it there.
  • Seiso (cleaning; shine). Keep the workplace clean for easier thinking (think grease, wires, etc.).  In code, leaving commented-out code and inappropriate comments increases mental clutter that has to be overcome.
  • Seiketso (standardization). The group agrees on how to keep the work place clean.  Consistent coding standards and patterns apply here.
  • Shutsuke (self-discipline). Having the internal discipline to follow the practices and to reflect on one’s work.

If you take a moment to really think about these parallels between the factory maintenance floor and the code we have to work in, these practices that reduce mental clutter on the factory floor apply equally well to developers working with code.  Less clutter means higher productivity and later mistakes.  For me, it was like a light bulb turned on.

One of his major points is that in coding, we spend 90% of our time reading, and only 10% of our time writing.  Thus, what we do to improve the efficiency of reading carries further weight.  Unfortunately, we are measured (subjectively or objectively) in terms of writing productivity, passing the costs on to fellow programmers later on.

It reminds me of what I wrote earlier while musing on Steve Krug’s Don’t Make Me Think as applied to web site design and usability.  By writing clean code, we require later developers to not have to think more deeply than necessary when reading and using our code by reducing misdirection and mental clutter.

I’m on page 21 of the book now, and looking forward to working through it in continuing to improve my own software craftsmanship.

Fluent Interfaces in Ruby

I had a chance to look at what Neal Ford had to write on Fluent Interfaces in his book The Productive Programmer.  Granted, I heard him cover it before at one of his presentations at the No Fluff Just Stuff conference in Reston, VA.

Basically, fluent interfaces have setter functions return self (or this in Java).  By doing this, you can chain the setters together into what looks more like a composed english sentence instead of multiple single-line setter statements.  I find it easier to code and read code written in this manner (particularly in the jQuery style).

Sitting here watching Olympic couple’s figure skating with my wife, it seems like the perfect time to put some example ruby code together for this (as well as the Self Yield spell from Paolo Perrotta’s Metaprogramming Ruby):

class Car
    attr_accessor :speed, :wheels, :direction

    def initialize
        self
    end

    def moving_at(mph)
        self.speed = mph
        self
    end

    def with_wheel_count(count)
        self.wheels = count
        self
    end

    def heading_toward(toward)
        self.direction = toward
        self
    end

    def display
        puts "speed is #{speed}"
        puts "moving in direction #{direction}"
        puts "number of wheels is #{wheels}"
    end
end

car = Car.new.moving_at(65).heading_toward("NE").with_wheel_count(4).display

attr_accessor would not have returned the object instance, requiring a different line for each mutator.

I was writing a KSH script with the following snippet:

exercises="ex1 ex2 ex3"
for exercise in exercises; do
    mkdir $home/$exercise
    filename=Target${exercise}Config.xml
done

The problem was I did not want the file to be named Targetex1Config.xml, but TargetExercise1Config.xml, respectively.  So, I wondered if it was possible to do double-dereferencing (i.e., go from exercise to ex1 to Exercise1, such as in the following:

exercises="ex1 ex2 ex3"
ex1=Exercise1
ex2=Exercise2
ex3=Exercise3
for exercise in exercises; do
    mkdir $home/$exercise
    filename=Target${${exercise}}Config.xml
done

That did not work, but a friend showed me the following use of SED with a regular expression to do the replacement:

exercises="ex1 ex2 ex3"
for exercise in exercises; do
    mkdir $home/$exercise
    newName=`echo $exercise | sed -e s/ex/Exercise/`
    filename=Target${newName}Config.xml
done

This worked like a charm (at least once I figured out that the ‘`’ above is not an apostrophe but the backward apostrophe underneath the ‘~’ key).  Later that day when I needed to copy and rename a bunch of files using a boring, repetitive pattern, I used SED to help make that easier:

for file in Provider*.xml ; do
  mv $file `echo $file | sed 's/Ex/Exercise/'` ;
done

I would like to have used Ruby to modify the contents of the files automatically using regular expressions, but Ruby is not available in this environment, and case-sensitivity in changes is an issue. Perhaps AWK can help me out with this. Either way, I plan to dig deeper into SED and AWK.

Safer Monkey Patching

General Monkey Patching

In Java, classes are defined up front, compiled into bytecode, and remain unaltered.  Every instance created from a class adheres to that class.

Of course, Ruby is different.  In Ruby, you can define a class and later redefine it (and of course, you can modify individual instances of a class so that they differ from the class):

class car
   def drive
      "I'm driving!"
  end
end

# blah blah blah

# Reopen car and monkey patch it by adding a new method
class car
  def brake
     "hitting the brakes!"
  end
end

From the Java point of view, it looks odd to repeat the class keyword twice with the same class.  But in Ruby, the class keyword simply places you back into the scope of the class you are working with (e.g., car).   There is no significant distinction in using it a second time.

What becomes more of an issue is that any plug-in or any code in your Ruby space can monkey patch a top-level entity (e.g., String) and globally change or break behavior.  This could include over-writing an existing method.  Furthermore, it can be difficult to determine what entity made the change.

There are two approaches that were highlighted from Rake and ActiveRecord in Ruby Metaprogramming for safer monkey patching.

Monkey Patch with Module

I lived in Texas for a few years, and the people were really friendly.  In their honor, I could monkey patch String as follows:

class String
    def greeting
        "howdy ya'll!"
    end
end

"abc".greeting # => howdy ya'll!

Unfortunately, there is no easy way to tell what made this change.  This would make it more explicit:

module JohnRagan
    module String
        def greeting
            "howdy ya'll!"
        end
    end
end

class String
    include JohnRagan::String
end

puts "abc".howdy  # => howdy ya'll
puts String.ancestors # "JohnRagan::String" shows up as a module

Now, the modules involved in monkey patching show up via ancestors.

Preventing Monkey Patches

Common lore is that folks in New York City are not as friendly as in Texas.  Whether or not this is actually true, let’s have some fun with it.  Let’s say a NYC developer writes a plug-in your Rails app includes, which contains the following:

class String
    def greeting
        "go screw yourself!"
    end
end

"abc".greeting # => go screw yourself!

Now, your code is suddenly rude and devoid of its previous Southern charm as this monkey patch has overridden the previous one.

While there may not much be you can do to prevent monkey patching by others, you can check to see if a method already exists before you monkey patch it yourself.  Rake has a mechanism it uses to prevent unintended monkey patching overwrites on its part.  Rake itself monkey patches the class Module, adding a method called rake_extension(method), which takes the method name, and method to be extended as a code block, and prevents the monkey patching (with a warning) if the method already exists.  I would have used it as follows:

require 'rake'
class String
    rake_extension("greeting" ) do
        def greeting
            "howdy ya'll!"
        end
    end
end

If my developer friend in New York City had already added a greeting method, this logic would have ignored my monkey patch.

The following code uses a class factory to create a new class along with getter and setters for the fields passed into it:

class ClassFactory
  def self.create_class(new_class, *fields)
    c = Class.new do
      fields.each do |field|
        define_method field.intern do
          instance_variable_get("@#{field}")
        end
        define_method "#{field}=".intern do |arg|
          instance_variable_set("@#{field}", arg)
        end
      end
    end

    Kernel.const_set new_class, c
  end
end

ClassFactory.create_class "Car", "make", "model", "year"

new_class = Car.new
new_class.make = "Nissan"
puts new_class.make # => "Nissan"
new_class.model = "Maxima"
puts new_class.model # => "Maxima"
new_class.year = "2001"
puts new_class.year # => "2001"

In Ruby, classes are simply objects like any other, which are then assigned to a constant.  Hence, to create a new class dynamically we instantiate the class Class with Class.new, and then assign it to a constant via const_set (we invoke it on Kernel so that it is a top-level constant like any other class).  We then add the code that makes up the class in a do-end block.

In that do-end block, for each field we invoke define_method twice, first for the getter method and then the setter method with get_instance_variable and set_instance_variable, respectively.  For each field, we create the instance variables (e.g., for make, we use @make).  Note how we make use of passing the argument in for the setter.

Additionally, if I wanted to make the class a sub-class, I could have used Class.new(parent_class)

I gave a brownbag recently at work on Java Web Services.  To get that to work, I had to run utilities to generate skeletons and stubs in java source that were then compiled into byte-code.  As you can see above, no such steps are necessary as the metaprogramming is naturally built into the language.

The biggest difference really comes down to metaprogramming.  While Java offers introspection (which was a major step up from C++), Ruby offers the ability to dynamically define new classes and methods without recompilation.  Additionally, the ability to hook into a module being included or a new class being sub-classed combined with metaprogramming adds extremely powerful new capabilities.  This is why in Rails I can sub-class from the model class and automatically have all the methods available to me for CRUD operations for my database entities.

In other words, I can create and use frameworks that leverage metaprogramming to make their use by end-users almost trivial.

Nonetheless, my love of Ruby does not cause me to knock Java.  I still enjoy coding in Java, and one should use the right tool for the job.

In order to be a better Rails and Ruby developer, I studied Pragmatic Programmer’s “Metaprogramming Ruby” (thumbs up).  By study, I mean read, took notes, created flash cards, wrote sample code and played, then finally created a suite of RUnit tests to exercise all the various capabilities.  “Mastering” Ruby metaprogramming greatly expanded my understanding of how Rails works under the covers.

I was surprised by how much Ruby metaprogramming had in common with JavaScript metaprogramming (such as dynamically defining methods).  Once again, this is one of the benefits of studying different languages – seeing the common patterns that emerge.

The next few blogs I do will go into Ruby metaprogramming.

I saw the Pragmatic Wetware book by Andy Hunt.  “Hmm”, I thought, “a book on the human brain and how it works.  Interesting, but I have too much else to read and study.” Wrong.

Apparently I had missed the pragmatic part of that title.  As I am averaging three to five technical books a month right now, an investment in becoming more efficient in my learning and study would surely pay off.  So, I got the book, and I am glad I did.

Just as you want to be an efficient coder, if you are doing a lot of studying, you want to be an efficient learner.  I will talk about a few of things I found over a few blogs, but I highly recommend purchasing this book and implementing it.

In the Dreyfuss model, there are five levels of progression of mastery in terms of problem solving and the mental models you form:

  • Novice.  They have little experience to rely on, but they can follow context-free rules (recipes) to guide them (think ISP help-desks and pre-canned scripts).
  • Advanced Beginner.  They can break away from context-free rules a little bit and try things on their own, but they do not have a big picture understanding.
  • Competent.  They have developed conceptual domain models and can work with them and solve problems.
  • Proficient.  They can reflect, improve, and learn from the experience of others.  They can apply maxims (as opposed to just recipes) to problems.
  • Expert.  They work from intuition instead of simply reason and are primary sources of information.

Becoming an expert is more than just collecting a body of skills and knowledge; it is about moving to intuition-based problem solving based upon those experiences.

We have a great cookbook that my wife and I pick out a new recipe from to try each week.  I take the recipe, and I follow it carefully (X amount of flour, Y amount of seasoning X, etc.).  A great chef (such as my grandmother) did not follow a recipe – it was based on intuition gained over experience (a pinch of this, adjust it this much because of the humidity, etc.).  I tried to ask my grandmother once how to make something, and she couldn’t really articulate her responses beyond “add a little bit of this” and “cook it until its ready but not too long”.  This is the typical mark of an expert.

What does this mean in terms of coding?  Well, it means becoming an expert is a little less about collecting skills and more about applying skills intuitively given a particular context.  Any fool can apply a parade of various design patterns (I recall being one of those fools 13 years ago), but an expert will intuitively apply the right pattern given a particular context.  That seriously shifted my paradigm when I read it.

Personally, I have a backlog of skills I am studying and investing in to increase my knowledge and skills portfolio.  However, I plan to spend a lot more time going forward on context.  One of the things I will soon be doing is personally reviewing my entire technical career to determine what I did and did not do right given the particular contexts I was working in.  Am I repeating any poor patterns, what should I continue, what should I drop?  I expect it to be a worthwhile exercise.

I always wanted to read the Anti-Patterns book, but never got to it.  I stumbled across the Anti-Patterns list on Wikipedia at http://en.wikipedia.org/wiki/Anti-patterns and found it to be a worthwhile read.  Obviously, a big part of learning what to do is learning what not to do as well.

I got to the end of it, and discovered a link to Software Development Philosophies at http://en.wikipedia.org/wiki/List_of_software_development_philosophies.  Now, I can Kanban with the best of them!

A worthwhile read – check it out!

Regular expressions were not my strong suit, but I felt it was important to master them (or at least reach competency).  So, on a flight back from Walt Disney World last summer, I studied them from one of my Ruby books and summarized them in detail in Evernote (to refer back to them).  I thought I had a good enough understanding where I could refer back to my Evernote reference material and quickly put it together – I was wrong.

Writing a Rails app I have been working on for fun, it took longer for me to put a working regular expression together than I expected.  I had studied, summarized and even written a few short programs for regular expressions in Ruby.  However, what I hadn’t done was to go from the other direction – take a domain problem and map it back to a regular expression.  That’s when I came up with “Regular Expression Pushups”.

When I was younger, I used to be able to do a massive amount of push-ups.  Rather than continuing to increase the count, I tried to do them more quickly.  Similarly, I reviewed my notes and created thirty-three problems that would use the underlying techniques.  I would do these as quickly as possible, and then later change the questions a little to see if I could do them more quickly.  The idea was to better form the “regular expression neural networks” in my brain

I’ve gone through one round and have seen a huge improvement.  I expect the second round will go a lot faster than the first (how could it not)?!

Here are the ones I put together:

  1. Find out if a certain string exists in as a substring within a document; and if so, where
  2. Replace the contents of a string within a document with something else
  3. Find three keys parts of a document and pull them out
  4. Find the word “foo” in a sentence but it cannot be “foobar”.
  5. Find a string that is a whole word only
  6. Find a string that is not a case-sensitive match
  7. Find the string foo or bar
  8. Match foo and goo (and so forth) but not boo without using these as words in your formula
  9. Find all strings that end in “oo” (3 character, and unlimited characters)
  10. Find any string that ends in “oo” but boo is not valid (REPEATS #8?  Or subtle difference?  I think I reversed them)
  11. See if a word matches that starts with “foo”.  Additionally, one that does not start with “foo”.
  12. See if a string matches that ends with “bar”.  Additionally, one that does not end with “bar”.
  13. Find a substring that begins with “foo” and ends with “bar”.
  14. Find the first and last word in a sentence
  15. Find the first character that is not a number or digit in a string
  16. Find the first number in a string (and last number)
  17. What is the text before the phrase “in the middle”?  What is the text that follows?
  18. Find  “abc” or “abcabc” and so forth in the sentence
  19. Find the string “abc” or “abcdef”
  20. Find the string “abc” or “abc1″ or “abc11″ and so forth
  21. Find all alphabetic words that end with the first “3″
  22. Find the word that starts with an alphabetic character and ends with 17
  23. Find the telephone number in the format 571-217-9451
  24. Find a number (without commas) that is at least 5 digits
  25. Find a number between 4 to 9 digits
  26. Find the word that matches a sequence of 5 instances where there is one to 3 numbers and a single character
  27. Find the string “abc” at the end of the string where there is a newline character
  28. Given a dollar figure, return the portion without the cents.
  29. Given string “#@%# 123bar 23bar 342 siojbar”, find the first word that does not have “bar” in it
  30. Find all the instances of numbers greater than 5 digits
  31. Find the number followed by the a space and word “bang” from “123 howdy 456 wow 789 bang”
  32. Do a greedy match (from “abc!def!ghi!” get the whole thing for .+!)
  33. Do a non-greedy match (from “abc!def!ghi!” get “the whole thing”abc!”" for .+!)

« Newer Posts - Older Posts »

Follow

Get every new post delivered to your Inbox.