Computer Science or Software Development?

The field of “computer science” is much lauded. In this academic domain are such heady topics as order of execution,  computability theories, and even quantum computation. While I applaud the study of computer technology as both necessary and beneficial, I don’t see myself as a computer scientist. Rather, my field is software development; in essence, it is applied computer science.

The difference between computer science and software development is in goals. The ostensible goal of computer science is, like any science, to extend people’s knowledge and understanding. The code produced during these efforts is far less important than the knowledge it represents. By contrast, in software development, the goal is to automate or enable a business function, and the code one creates often lasts for years after the original developer has left.

As I am a software developer, I tend to view scientific efforts with curiosity and the tiniest bit of amusement. Firstly, I find it hard to understand a methodology or technology unless I understand its application. The idea that we have a body of knowledge which we can call computer science is hard to take seriously when you compare the field to more mature sciences like physics, math, or geology. If we are generous, we might place the age of computer science as it is known today in its seventies. Compared to other sciences, we’ve had only a fraction of time to mature our understanding. But I feel that as software developers, we want that prestige that comes from being scientists.

Of course, this is not meant to knock the science. The most important things I’ve learned about programming, I learned in school. I simply happened into a career that isn’t actually about programming. Perhaps it’s counter-intuitive that software development is largely not about writing code. But ask a seasoned developer how many hours a day they spend in meetings and compare that to actual time at a compiler. My experience has been that the former is quite larger than the later.

If we are to be successful software developers, we need to learn when and how to incorporate the lessons of computer science into our work, and to be diligent about increasing our understanding of what our field is really about.

Take for instance, the aforementioned meetings. If one is to be a good software developer, one needs to be good at meetings. What does this mean? How many developers have any sense of what is expected of them as pertains to meetings? These are questions that few degrees in the computer field will cover, yet they are as important as determining the O(n) execution time of an algorithm.

One of the reasons developers get paid so well is because the breadth and depth of knowledge the job requires. It goes far beyond the clerical task of writing code. Developers are expected to make assumptions, suss out requirements, troubleshoot production issues, suggest new features, and know the length and breadth of computer technology. And those are just the explicit responsibilities. The unstated ones include working extra hours the days before a launch, advanced writing skills (in e-mail, reports, and presentations), having a good sense of the market space their products reside in, and others too numerous to list here.

In my experience, the most harm in software companies comes from software engineers who want to be computer scientists more than they want to produce working code. These people have an interest in new frameworks, starting over, designing by configuration, “scalability”, adding code to support business models that the business doesn’t need, doing it “right” the first time, and implementing the latest cool buzzwords (“federated”, “cross-cutting”, “ISP” – anything that looks like they’ve read a computer-centric business magazine). Each of these is a fallacy large enough to deserve its own article.

These poor souls are deluded. They believe that if they can just write code perfect enough that they wouldn’t have to do all the other things that makes software development a difficult job. They often want to be architects for their business – a job role that I personally think should be abolished – because this has the dual benefit of getting to feel very smart and important and of never needing to be responsible for actually coding. But only by balancing our desire for well designed, highly performant code with the needs of the people for whom the code is intended (and who are almost uniformly not actually interested in the code) can we be successful software developers. Forget that, and one soon loses the very reason one is writing code in the first place.

Until next time,

Practice Makes.

Reducing Complexity

As programmers we tend to love and embrace complexity. Towering class trees and data-driven algorithms fool us into thinking we’re doing our jobs. That is, if you think programming is about writing code. I prefer to think of it as creating features. Not all features are the same, however.

Let’s take this method:

public String printCompanyName()
{
    System.out.println("Welcome to Global Dynamics!");
}

and compare it to this method:

public String printCompanyName(String name)
{
    System.out.println("Welcome to " + name);
}

It’s natural to view the second method as better. It has the added feature that the company name can vary. But this feature adds complexity and all complexity costs something. To make a decision about which method to prefer, we need to know how valuable the feature we’re adding is. Unused features are worthless. In this example, if the company name never varies throughout the program, we should use the simpler method.

This is the principle of YAGNI – You Ain’t Gonna Need It. Complexity always introduces cost so we should struggle to resist our desires to “improve” the system until we have a clear understanding of what the system should do. And even then, we should only do the minimum required to implement those features. YAGNI evolved out of the agile community as one technique for eliminating feature creep in programming. By only implementing the features needed today, programmers defer the cost of introducing complexity as long as possible. This same principle is stated in a different way in Test-Driven Development — programmers should do the simplest thing possible to get a test to pass. Complexity, when necessary, should evolve out of simplifying code, rather than complexity driving the creation of code.

We do this because complex code has several associated costs. Firstly, it’s a lot harder to change complex code than simple code. The cruft from complex code must be maneuvered around and creates confusion over where to add new code. This directly increases the cost of adding additional features. Secondly, complex code is harder to understand, which increases the cost of training new developers and also the risk of introducing bugs. And simply writing complex code in the first place is error prone.

There are some guidelines. If we only ever print the company name once, having a method to do so is probably wrong. Likewise, for classes, those that have only one child or that only have one method are likely introducing unnecessary complexity. Obviously these should be evaluated on a case-by-case basis, but these are pretty good rules of thumb. On the other hand creating methods and classes can reduce the overall complexity in a program by centralizing, encapsulating, and abstracting. The salient point is that these techniques should be used to reduce the overall complexity of the program. Refactoring should always be an attempt to reduce complexity by introducing structure.

YAGNI applies even when you’re looking at old code. There’s a comparable principle of You Didn’t Need It. When we speak of refactoring we usually mean the process of extracting generalities and creating structure, but it’s just as valuable to deconstruct over-complex implementations. It’s natural to think of the effort invested in creating a rich structure as a sunk cost but it is not. These unnecessary complexes silently intrude on one’s conceptualization of the program and make it harder to create new code. Often it’s more efficient to defactor code before extending it.

Defactoring is a term that has no clear definition associated with it. By defactoring I mean taking code and reversing the process by which it was refactored in order to return the code into less differentiated state. Defactoring is how I suggest solving the invisible design antipattern. Most instances of defactoring are followed by refactoring along different lines, but YAGNI dictates that step is only taken if there’s some clear need for the extra complexity. You might also think of it as attempting to reduce complexity by reducing structure.

While it is true that I view complexity as a necessary evil, I don’t view it as a villain to be fought at every turn. Rather, I view it much like mechanical engineers view friction. If we tried to build a program with no complexity, that program wouldn’t do anything interesting. But we must always be mindful of complexity as a force to be addressed and managed in its own right, on par with performance or defects.

Until next time,

Practice Makes.

Quick, I need a Topic!

Writing a personal blog has always come naturally to me. Who doesn’t know how to talk about themselves? But writing a professional blog is a lot more challenging, because I’m writing to a (highly informed) audience  making concrete claims about topics they’re already somewhat familiar with. It’s challenging to come up with subjects that are interesting that I can add more to. And frequently, as I’m writing an article for this blog, the topic shifts.

But Friday is my day to let my hair down a bit and write more intimately. I would’ve had something up a little sooner, but yesterday was a banner day at work — one of those random twelve-hour shifts where production problems and new promotions drain all energy for new creativity. It’s alright, however; my company has officially offered to sponsor Talk Like A Pirate day. Between that and the free soda machine, I’m willing to put up with a few early/late days. 

The technical field is full of examples where a little employer-bought gratitude will make up for a lot of work-induced stress. Company-bought lunch for attending meetings, discounts at local stores for employees, annual cook-outs. I’m not talking about stuff like insurance or a 401k — those things are generally considered part of compensation — but things outside the official pay that are nontheless reasons why we stick with it.

So, opening the forum, what’re some of the more interesting things you get out of your job that aren’t strictly part of your official compensation? Here’s my list:

• Free chocolate/soda
• Random company-sponsored events (now including TLAPD!)
• Awesome, awesome whiteboards the size of entire walls
• Festive office decorating throughout the year
• Proximity to downtown
• Cinema display monitors
• Free parking

The Business Life Cycle: The Awkward Phase

Much is made of the software development life cycle in our industry. Less talked about is the business life cycle — how businesses start, mature, and eventually close their doors. Perhaps in the technical field, we spend so much time thinking forward that we forget what happened before. So many business and career decisions are made without this context, and it really is a shame. The phases in the life-cycle of software companies are smaller and more pronounced than in a regular business; to really flourish in the industry we need to understand how that industry works.

A lot has been written about start-ups and the unique challenges they face. I’ve done the start-up thing before a few times and my experiences have been mixed. While the start-up phase is interesting, it’s also fairly simple: a company either makes a sustaining product, or they go under. What I’d like to talk about today is about that period slightly after a company makes its first major success. Some companies seem to grow slowly and organically — shout-outs to Silver Creek Entertainment — but most companies have a phase of rapid expansion, followed by an awkward phase of incorporation.

This awkward phase occurs any time there’s a major shake-up with staffing or a ton of new people are added. (My company is currently in one now.) Sometimes the expansion happens because venture capitalists inject new revenue into a company. Other times a new product happens to hit a sweet spot and bring in a lot more revenue than expected. The reasons for the expansion don’t much change the outcome. New people need to find their place, understand the business goals, figure out how to participate, learn who they’re working with, and so on. Larger corporations talk about “culture” and “team dynamics”. The truth is, culture develops because of peoples interactions with each other. These interactions take time.

How much time depends on a lot of different factors. Hiring outgoing, gregarious people never hurts. Having team-building activities sometimes works, but it needs to be done very carefully. But what seems to be the simplest and most powerful way to gel a team rapidly is to give people ownership of small domains and to make them responsible for setting policy regarding that domain.

This is a hard thing for most companies to do. It seems easier for start-ups that are expanding because of capital infusions, but even then, it’s a challenge. Fundamentally, it’s a trust issue. The people who made the company what it is want to protect what they’ve created, even if that means shutting new people out. And there’s a long term issue of the domains being created becoming prisons or for the business to become over-reliant on individual people. But all of these challenges have solutions.

First, this doesn’t mean handing over the business to the monkeys. We all have a role to play; C-level executives create vision, target markets, and set policy for which products will be made. Product owners set policy for what features will be in the product. Developers set policy for how those features are implemented in the domains they’re responsible for. But within the domain that one is responsible for, one should have ownership of the decisions.

As for the idea of over-reliance on an individual or roles becoming prisons, documentation is critical. In fact, I would argue that the main point of documentation is to give those who are not currently responsible for a given domain the information required to make the right decisions on policy in that domain. But I’ve yet to see a software business that defines a comprehensive documentation policy.

The rewards for structuring the business so that employees have ownership over their domains are tremendous. People gain a stake in the business, and they gain pride in their work. In other words, they care, and that caring will reflect itself in their work. (Unless you hire a bunch of deadbeats, but that’s a topic for another post.) If you can get your employees to care about your business, then they’ll get your customers to care, and when that happens, you get Three Rings Design or Trileet or Fog Creek Software. It’s a beautiful thing.

Until next time,

Practice Makes

Antipatterns: Invisible Design

In my last post, I alluded to the idea of antipatterns. The notion was first surfaced in Koenig’s “Patterns and Antipatterns” article that talked about the GoF book on patterns. Basically, Koenig made a good observation that some proposed patterns look like they work, but in practice fail to solve the problem or create other problems that are worse. Ironically, I think people far more easily understand antipatterns than their converse. At the moment, Wikipedia lists 38 patterns and 52 coding-related antipatterns.

Today I’m going to talk about an antipattern I haven’t seen referred to before that I call the Invisible Design. (The name means that the design of the classes can’t be understood without studying each in depth.) The problem the pattern attempts to solve is “how do I assemble code such that it can be easily changed when requirements change?” The easiest, and first solution is to write a base class that implements the solution you have now and then override parts of that solution in a sub-class later. It’s surprisingly easy for this to go horribly wrong.

In general, the original design is often uninformed. It fails to account for some facet of the problem set, or perhaps the scope of the object changes to acquire new responsibilities. Because code grows organically, it’s usually easy to make the first change by overriding a single method or tweaking a constructor. Occasionally, the original solution is specifically refactored to allow the variation to occur by overriding a method. In another case, a developer realizes that some parts of the solution may need to change in the future and creates a class hierarchy to absorb those variations. But again, because the original design is uninformed, it is rare that the true points of change are encapsulated. It is even rarer that the code that accommodates those changes is placed in the correct place in this hierarchy.

In any case, over some time-frame, there’s a loss of cohesion in the classes in question. In order to understand how an individual unit works, the developer must understand how they all work. New developers struggle with understanding how the parts plug into each other. Old developers are hesitant to change base classes because of uncertainty about which other units will be affected or how much. And unit tests become increasingly difficult to write, if they’re written at all.

This antipattern isn’t just related to classes, though. For instance, in Spring, objects are instantiated by a container, then wired together by the container, and then receive input, often from the container. When these objects are configured through external files, those files can become subject to their own invisible design. For instance, spring.xml may import a database.xml that contains data configuration beans that are used to configure the beans in spring.xml. But what if one of the database beans requires a bean from spring.xml? Then it becomes difficult to say for certain which file to look for any random bean. The problem gets worse as you add more files.

But wait! There’s more! A even more specialized form of invisible design occurs when you introduce the notion of programmatic metadata, such as Java’s annotations or C#’s tags. For instance, Hibernate annotations allow you to embed information about how an object is persisted directly into the object itself. This works particularly well for situations where entity objects are relatively self-contained. But for objects with complex interrelationships, this effectively spreads the configuration of the data objects across every source file.

Some symptoms of an invisible design antipattern:

• deep, narrow class hierarchies — all base classes should have at least two children and hierarchies should rarely go farther than two classes deep
• circular dependencies in configuration files — files should be like a tree, and each file should only use its own resources and those of files further down the same branch
• no or only a few unit tests around a whole set of classes — every class should have its own unit tests that exercises every feature of that class

The forces influencing this antipattern are several. First, developers often want architecture to insulate them from change, rather than to cleanly accommodate variation. There’s a certain amount of ego invested in wanting the system to account for all possible variations. Sadly for software, unexpected change happens, and it is usually better to address those issues as they arise. Secondly, decision makers often want small changes in behavior to correspond to small amounts of time. It seems to reason that if you’re not changing something much, it shouldn’t take long. However, changes accumulate, so the cost to make the same size of change is higher after many other changes have been made. The only way to eliminate that debt is to refactor the code.

Let’s go back to the root problem in this antipattern — distributing cohesive behavior across more than one functional unit. There is a fine line between writing code that is modular with high reuse and spreading one feature across multiple classes. In the case of the invisible design, the refactoring required is to merge units until a cohesive solution emerges. Sometimes this means collapsing a hierarchy of classes, other times it means merging configuration files, and still other times it means moving configuration data out of code and into an external resource. I’ve found that once the code is all forced into the same spot, it becomes clear which pattern should be used to handle the variations that have occurred so far. However, it’s generally only after the code starts to coallese in one place that a refactoring solution can be chosen.

Until next time,

Practice Makes.

Tools of the Trade

What comprises a sufficient office environment for your average software developer? Over the years, I’ve learned that it pays good dividends to supply your own kit. Here’s a list of stuff I make sure to have in my work environment:

• Index cards — for some reason, companies supply all manner of paper products, but not simple index cards
• Stapler, scissors, tape dispenser — for binding print-outs, and the occasional papercraft project
• Books — reference books, sure, but also some off-topic books to read for general education
• Mug — ’nuff said
• Kitsch — small toys, various desk fobbles and doo-dads
• Headphones — I use active noise canceling to drown out the air-conditioner
• Push-pins — also inexplicably not supplied by any companies I’ve been part of
• Cordless optical mouse — every company I’ve owned has paid bargain price for mice so I bring my own
• USB key drive — because sometimes sneakernet is actually easier
• My SCJP certificate — I can be proud

Not all kit is physical. Besides the standard IDE/SCM/Office applications, I have a few software tools I use routinely:

• Chrome — at home I use Flock for the social networking, but I really like the tight Google integration in Chrome while coding
• Last.FM — it’s better than ripping my entire CD library on my desktop or carrying an iPod to work
• Meebo — easy IM without installing Pidgin or Trillian
• Mibbit — likewise, but for IRC
• EditPlus — because I need a text editor that groks regular expressions
• Fiddler — great for troubleshooting web-services
• 7-zip — for opening .jar/.war/.ear files
• Desktops — I download a new desktop about every week to give myself the illusion of change
• Visio — though I’m not happy with it, I simply haven’t found anything else worth using
• WordPress — duh

That’s my list, and it seems to grow every time I change jobs. What’s on your list?

Patterns: Template Method

Among the many things that programmers are convinced of (and there are many, even some that are right) is that code should be written according to common, standard approaches. These repeating ways of doing things in programming are referred to as patterns. The word goes back to the architectural field where it refers to documenting the generally good solutions that people solving the same problems over many years evolved. Note that when “A Pattern Language: Towns, Buildings, Construction” was written in 1977, it had over two hundred and fifty patterns identified, from the most macro level to the most basic. It’s a solid academic work. (Its sister title, “The Timeless Way of Building” is meant to be equally important, concerning the application of the patterns, so we’re really discussing several books.)

Though it is clearly the best work of its kind, the original ‘Gang of Four’ book “Design Patterns: Elements of Reusable Object-Oriented Software” contained only a couple dozen patterns, almost of similar scope. It would be the bottom tenth of the patterns from the work in which it was modeled — with so few patterns, we hardly have enough to describe one building, let alone architecture at large. That said, it was such a monumental work precisely because the rapidly emerging software industry needed some guidence on how to write a program when it wasn’t meant to be thrown away. One could argue that Microsoft’s release of Office, Windows 3.0, and Windows NT signaled a shift from the era of console cowboys to career programmers. It’s in this historical context that patterns should be understood. New work is being done all the time, and yesterday’s favored pattern may become tomorrow’s cruft as our understanding grows.

This post is specifically about the template method pattern. This pattern provides for a module that executes some basic sequence of steps, but leaves it up to another module to decide how one or more of those steps are actually done. Typically this is implemented with an base class that leaves a few methods abstract or virtual. Those methods are protected-scope and are then called by a public-scope “algorithm” method that is itself called by users of the class. Strictly speaking, this is not the only implementation that could qualify, but it’s probably the easiest one to visualize. The most important feature of a template method pattern is that on the whole, the algorithm stays the same, while specific pieces of the algorithm change.

A fairly good example of a template-method use-case is that of controlling a family of devices that all fill the same role. Perhaps on one printer, for the model A, you fill register seven with the number of seconds to wait between commands, but on the model B, you fill register eight with the number of milliseconds to wait. In all cases, the wait gets registered at the beginning and then the data is sent, page by page. As long as the implementations only vary at a few key points, the template method is a good solution. If the points at which the implementations vary differs between implementations, or if the variations in behavior are extreme, you want to look at a strategy pattern.

The template method pattern sucks in one very important way. It’s difficult to test. Specifically, each implementation will either need all the same tests — which sucks, because the base-class code will be tested once per derived class — o one can test the base-class and live with the fact that the sub-classes aren’t being really tested. You can get some milage out of using the same template method in your unit test classes — the same base tests still run once per sub-class, but at least they’re only written once. Depending on how extensive the customization is, it may be worth switching to a less-coupled pattern to make sure that code is adequately tested.

Some signs that you might reconsider the template method pattern:

• if you have more than three abstract methods in your base class
• if your template method base-class’ derived class is itself a template method base-class
• if your customization methods are more than a dozen lines of code

Some signs that you should think about applying this pattern:

• if you have more three or more classes to handle slight variations in implementations
• if you have incremental parallel changes between versions of data or devices
• if you need to support a common algorithm that allows for user extensibility at a few control points

When we use the template method pattern in a situation where it’s not a good fit, the code tends to be difficult to follow. There’s a lack of cohesion in each class; in the base class, most of the real logic is in the sub-classes, and in the derived classes, it’s not clear how the methods relate to each other. I refer to this as the invisible design anti-pattern.

In my mind, the template pattern is something of an exercise in polymorphic excitement. There is a tendency to apply it almost accidentally, because the very nature of polymorphism gives rise to class hierarchies where sub-classes override their parent’s behavior. Yet as with any pattern, applying a pattern blindly leads to unfortunate consequences. In limited circumstances, this pattern may be the best solution as long as one is mindful of the challenges involved. But, in most cases of the template method pattern I’ve seen, a simple dependency-injected strategy would make the system more robust (better testing), easier to understand (more cohesion) and easier to extend (looser coupling).

Until next time,

Practice Makes.

Welcome to Practice Makes!

Greetings and so forth. And let’s make them seasonal while we’re at it. I’ve started this blog to post my own opinions on software development to a larger community. There are almost as many blogs on software development as there are people in the industry; this one is about what happens when programmers actually program, when project managers actually have to make a product, when some poor maintenence engineer gets told to make the e-mail server do billing too. And with luck, there will be some tidbits of information here to help the reader get through some tough spots.

As for me, I’m a professional programmer. My last job, I got suckered into that time-honored position of “team lead” – where you get to manage full time and also program full time. My current job, I’m working with mobile development software under a tight deadline with legacy software and moving from a waterfall development to a agile one. Don’t get me wrong, I love my job. I love my career. But it can get a bit crazy. 

I’m hoping to get a post in a week, maybe two, depending on how work is going. I’ll try to keep articles tagged and respond to comments as best I can. Do keep in mind this is a fun diversion from my real work. All opinions stated here are the author’s only, blah blah blah, though I do hope it educates and more importantly, entertains.

Until next time,

Practice Makes.