I've been researching and playing with some ideas for a blog post regarding team organization and software development.  I hadn't actually gotten around to writing anything, but a post by Abby, a fellow blogger at USD fired me up to push this one out the door.

As Abby pointed out, there is a distinct difference between programming and computer science.  And, I'll confide in my readers that despite my education I can no longer in full faith consider myself a computer scientist.  I am a business administration major who happens to be pretty good at programming.  That said; I'm going to talk about programmers and programming... not computer scientists and the science of programming computers.

What is Computer Programming?

Wiki says computer programming is the process of writing, testing, and maintaining the source code of computer programs.  That doesn't say a whole lot, but if you managed to find this blog I can assume you know what a program is and that it is generated by “programmers” writing code.  I can also assume that you know that many programs are disappointingly late, extremely buggy, and too expensive.

 

Why are programs delivered late?

Simply put, for the same reason any product is delivered late.  Computer programs aren't like a book that you can start writing at one end and end up at the other with a nifty little progress bar.  Most often, you start in the middle and have no known destination.  Even more often, the spec for the project changes as the project grows.  By the time you get to one end, you find the other is no longer applicable.

As if that wasn't bad enough, you might have yourself a p@nda-c0uch client that changes its mind about details in the middle of the project because of changes in circumstances.  Circumstances may be changes in budget and resources, or environment factors such as competitor activities, changes in technology, etc.

Being the needy, impatient people we are, we often want to know when we are going to receive our much treasured prize.  With the pressure on, managers and project leads often pick unrealistic and inflexible deadlines.  With all the afore mentioned scope-creep, it's a wonder that anything is delivered on time.

Why are they buggy?

For nearly all the same reasons mentioned before.  A computer scientist will say because of lack of planning.  But have you ever planned a road-trip for an unknown destination?  It's difficult to know how to get somewhere if you don't know where you want to be in the end.

The real reason is all the complex scenarios that no one could ever possibly anticipate.  Managed code, or object oriented code helps with that a great deal by breaking things down to small pieces and making sure those pieces worry about themselves and only themselves.

But data moves through any number of pieces or modules before getting to the end point.  So by the end, the question is; who remembered to escape the quotes in the string?  Did anybody? Did everyone do it?  If so, we might have a string whose quotes were escaped a dozen times.  That is way too much overhead, no wonder our programs are slow.  So why not let the entrance do the check and everyone else assume it's escaped?  Because our string sorter doesn't know which entrance the string came in and a programmer who is new to a project doesn't know that all new entrances need to escape the quotes of incoming strings.  Add a dozen or so scenarios like this to a project with hundreds of programmers and someone is going to slip up; and BOOM the program crashes and goobers your database. (watch for upcoming post, “OOP; is it Gospel?”)

Why are programs so expensive?

Because a good programmer requires at least some computer science education and demands a pay deserving of the novelty of their skills.  Fact of the matter is, programmers are rare and good ones are even rarer.

Not to mention that in addition to programmers, software development requires the expertise of designers, architects, testers, business analysts, and project managers.  I could write a blog the size of this one on why each of those people is important and deserves his/her $15-$50 per hour pay.

The scope creep issues above also come into play here because time ends up wasted when a team needs to backtrack and reprogram or debug old code.  Wasted time = money.  Again, a computer scientist will say that with good planning you can avoid all that mess.

So Why Don't We Plan?

Well, we do.  Even our cowboy programmers do in their own way.  When a programmer is writing any given line, they are usually thinking about how that line fits into the bigger picture.

Let's plan more!

Yes, let's plan more.  Let's plan out the entire project, then everyone will know what they are going to do and we will just do it.  When we are done we will have excellent documentation and everything will run like a well oiled machine.

All this planning will require a committee.  Perhaps we should send them to dinner.  In fact, we are using some very new technology. We should send our committee to a convention to learn about it; maybe even invite the client.

So, a month and a half later we have a full project spec for what all the code should do.  We may have already spent $40,000 (if we were damn lucky), but we will be ahead in the end won't we?  So what if we don't have a single, tangible line of code to show for our efforts.

Well, the client has just spent a lot of money and wants to see where you stand in the development.  Are you going to show them some drawings?  Maybe they come to the conclusion that the project as it stands is going to cost way too much so they slim it down, change features, and cut others out.  Your design is now nearly useless, it has to be redone.  And you still don't have any code to show for your efforts.

How do we plan right?

My boss started throwing around this buzz-word the other day.  It's called agile development.  Essentially, at the end of every deliverable the development team (and sometimes client) gets together briefly, looks at what they have accomplished and decides what the next deliverable is.  At that checkpoint they make changes, rework billing, and decide exactly what the next checkpoint should be.  A deliverable should be a usable/tangible piece of software.  So, X dollars = Z code.

That way, we don't have to know where the end is.  We just need directions for the next 5 turns before we ask for more directions.  This forces problems to surface before the project gets too far along.  It also provides a medium for changes.

With my experience thus far with this type of planning, after the communication, the programmers return to their desks and cowboy coding ensues.  Nothing gets put on paper outside of an outline for the deliverable and only then for accounting purposes.  The accountants out there know how important it is to define an expense and to place a value on it.  Not knowing what your software is worth can make both the financers and the managers nervous.

Agile development and deliverable checkpoints also makes it easier to stop a project midpoint and switch development teams. I've advised against this in the past and still do, but managers will do what they do.

Why would we not plan?

Many times it's just not necessary.  Most applications out there are routine, run of the mill apps; such as CRUD or content management systems.  Any development team delivers one of these every other day.  Planning one of these would be a waste of time.

Sometimes the budget doesn't support it.  I've worked on projects with budgets so tight that the supervisor can't afford to look at the deliverable or the incoming specs; just merely pass them on.  You end up with a project on auto-pilot full of hastily written cowboy code.  This scenario might seem like it screams disaster, but sometimes it is exactly what a client needs.  You get what you pay for; so why purchase a new Lexus SUV when you need a used 2 passenger VW.  It may have some bugs in it, but it'll still get you to the grocery store.

So Why Are You Attacking Computer Science?

Because one too many computer scientists have scoffed at programmers and the work they produce.  Needless to say, sites like youtube and facebook won't be curing cancer or finding life on mars anytime soon.  But I can guarantee they will have something to show for their efforts in the end and it won't be a bill for the tax-payers.

Don't get me wrong though, I'm not downplaying the accomplishments of computer science research.  But there are some hard working programmers out there who aren't any less accomplished just because their software organizes cat food product images instead of photos of moon craters.