martes, 21 de agosto de 2012

How many lines of code per day should a good programmer produce?

It depends on the programming language and the ability of the programmer, but in general, really, how many lines of code I must expect from my programmers?

First we must get rid of all the noise. The duplicate lines for example. There are tools to detect and eliminate repeated code.

Sonar for example is an excellent tool for detecting repeated code. Also there is CPD. You can run those with your nightly builds.

So, how many lines I can expect from them?

There is so much noise because even if I remove all the duplicate code, I still end up with some duplicated code. Avoiding duplication is hard.

Also there are bugs. Watts Humphrey, of SEI and CMM fame, mentioned that there on average there is a bug every 10 lines of code. I cannot agree more.

Also there is always the question how we do things. I can use a linked list or a hashmap and those implementation details will for sure give different number of lines of code, different execution speeds, different implementation speeds, and so on. Most of the the time, using more abstraction gives better results, on less amount of code per funcitonality, faster execution speeds and faster time to market.

So instead of measuring lines of code we should be measuring functionality. The problem with functionality is that it is so hard to define, so Microsoft for example defines function point in relation the different languages and therefore they think they can calculate the contribution of a developer by the number of lines they write and divide by the average number of lines required by a function point in that language. 

Really? You gotta be kidding me.

Because we all know that if a programmer is measured by the number of lines he writes he is going to copy and paste like crazy. But he will get the job done.

So if he gest the work done, what is the problem with that?

The code becomes unmantainable. It is repeated so many times that a single bug can get copied twice and the resulting code now has 3 bugs. If my code with has 3 bugs is copied along another code which has 2 bugs, now 5 bugs got converted into 10. If the resulting 10 bugs get copied twice with other code with 6 bugs, the result is 48 bugs. Only by copying 3 times. Imagine how many times you can copy code and now think that the modifying that code also introduces some subtle bugs.

That's why code is so expensive. It is full of bugs. Bugs that are hard to find and correct. So avoid coipying and pasting and reuse the code. Eliminate bugs by reducing the clutter (boilerplate code) and the repeated code.

One person can write between 100 and 3,000 lines of code per day. But measuring that is lying to yourself.

When a project is finished, dead code removed, and repeated code is not taken into account, usually every programmer has provided between 10 and 50 lines of code per day.

The way you get programmer productivity is not by increasing the lines of code per programmer per day. That doesn’t work. The way you get programmer productivity is by eliminating lines of code you have to write.

The line of code that’s the fastest to write, that never breaks, that doesn’t need maintenance, is the line you never had to write.


-- Steve Jobs

Sources: 
1. http://www.crosstalkonline.org/storage/issue-archives/2010/201007/201007-Humphrey.pdf
2. http://www.youtube.com/watch?v=3LEXae1j6EY#t=41m26s

lunes, 16 de junio de 2008

Lacking abstraction

Structuring systems around objects or beans and lacking abstraction is like building a building around bricks.

You can do that, but a pile of bricks is not a building.

In order for a building to be functional, well, it has to be able to interact with its environment. For example the entrance must be big enough for humans to enter, and it must be open to a street. Isn't it obvious?

Well, most of the stuff is obvious when mentioned. But it has to be mentioned for people to make the right decisions at the appropriate time. That is, when one of the workers is closing the entrance with a brick wall, he (or anyone realizing this) should speak up and say "we shouldn't be doing this, because we are closing the entrance".

Yet we don't expect people to find out things like that when they are actually putting the bricks, because there is a model of how the building will look like before they start to make a hole in the ground, so a lot of those problems have been ironed out long before somebody starts removing soil.

Yet there are lot of technical decisions to be made inside every apartment, from the point of view of each individual worker. Irrelevant from the point of view of the architect of the building, but important from the point of view of the person who is inhabiting that building if they turn out to be bad decisions.

One of the best tools for an architect is to build a small model of the building, and these days they can easily traverse the Autocad model using software like Doom or Quake.

It turns out that good developers are hired at the best places, because they are willing to learn and they come with a good background (they already learned at other places and they continue learning). Also they stay at their jobs because they are paid more. Eventually all those little differences make a huge difference in income for the company and you can see that in that most successful companies pay more for their people and also the company's income per worker is higher, meaning their selection has better results.

Having worked at those environments I can tell you what it feels like: It feels like you are working with smarter people and that they are therefore more autonomous. They make fewer mistakes and they feel free to steer the way they work when the results are not ok. The results can be seen on a daily basis, so there is always room for improvement and the feeling is very powerful. You have a lot to learn when you arrive and you keep learning for a very long time, while at the same time being productive in a very short period of time (around 2 to 3 months).

The place I'm referring to is Microsoft. Pretty smart people, pretty smart business people, pretty smart bosses and pretty smart developers. There were no architects, therefore architectural decisions turned out to be not so smart decisions.

The architect should concentrate on looks and interactions, for example: will the system have scroll? Most usability experts agree that web applications should never scroll, but sometimes you have so much information that you need a scroll here and there. I would reasonably try to remove those scrolls and exchange them for something else. For example it is typical to have "pagination". Yet what is pagination if not another kind of scroll? Yet pagination is simpler from the point of view of the user, because you don't loose the screen view everytime you paginate.

There is a problem with pagination though. If you mark a row in a page and then move to the next page, should the mark remain in the previous page. How can we make the user remember? You could use a counter ("3 items selected") or move the selected item to another part of the screen, but all solutions look ugly in my mind (we need a usability expert here).

Take Google for example. It has pagination and scroll. Its a terrible user interface, since I can't paginate without using the scroll. Also it could be done using Ajax and it could sort results by different criteria and it could do so much things. Web search is not finished, yet it seems they think they have reached a plateau.

lunes, 5 de mayo de 2008

Architecturally Poor J2EE Applications

Structuring systems around objects or beans and lacking abstraction is like building a building around bricks.

You can do that, but a pile of bricks is not a building.

In order for a building to be functional, well, it has to be able to interact with its environment. For example the entrance must be big enough for humans to enter, and it must be open to a street. Isn't this obvious?

Well, most of the stuff is obvious when mentioned in retrospect. But it has to be mentioned for people to make the right decisions at the appropiate time. That is, when a guy is closing the entrance, he must speak up and say "we shouldn't be doing this, because we are closing the entrance, people won't be able to enter the building anymore".

Yet we don't expect people to find out things like that when they are actually putting the bricks, because there is a model of how the building will look like before they start to make a hole in the ground, the so called blueprints, so a lot of those problems have been ironed out long before somebody starts removing soil.

If we only could build software like that. Most projects are severely underspecified.

Even in buildings, yet there are lot of technical decisions to be made inside every appartment, from the point of view of each individual worker. Irrelevant from the point of view of the architect of the building, but important from the point of view of the person who is inhabiting that building if they turn out to be bad decisions.

It turns out that good developers are hired at the best job, because they are willing to learn and they come with a good background (they either learnt at other places or they came with straight A's from their PhD). Also they stay at their jobs because they are paid more and they get recognition. Eventually all those little differences make a huge difference in income for the company and you can see that in that most successful companies pay more for their developers and also the company's income per developer is higher, meaning their selection has better results.

Having worked at those environments I can tell you what it feels like: It feels like you are working with smarter people and that they are therefore more autonomous. They make fewer mistakes, they do things that seem impossible and they feel free to steer the way they work when the results are not ok according to their own criteria. The results can be seen on a daily basis, so there is always room for improvement. Whe you arrive to such a job, you have a lot to learn and you keep learning for a very long time, while at the same time being productive in a very short period of time (around 2 to 3 months). When you arrive to the worst jobs, they have been doing nothing for 6 months, they have mandatory overtime and it doesn't matter how much time you spend at the job, nothing ever gets done. If something looks like it is going to get finished, it will get redone using a different format, just for the sake of it.

Some pitiful companies are struggling for money so they can't keep their developers long enough for them to learn, they produce mediocre products that no one is proud of and finally either developers quit or they are fired. Therefore this perpetuates bad developers job hoping and good developers stay at the best jobs, learning and improving.

So the first job for a job seeker is to work at companies that consider hiring developers as an investment. The first job for a manager is to hire the right candidates. No, they are not going to learn Java on the job. They need to know Java and algorithms right away, so that they can learn the advanced stuff.

There is a need for an architect on every project, although most probably you need one architect for each 10 projects, if you have hired expereinced developers. If you are hiring unexperienced developers, no architect is going to solve your problem.

The architect should concentrate on looks and interactions, for example: will the system have scroll? Most usability experts agree that web applications should never scroll, but sometimes you have so much information that you need a scroll here and there. I would reasonably try to remove those scrolls and exchange them by something else. For example it is typical to have "pagination". Yet what is pagination if not another kind of scroll? Yet pagination is simpler from the point of view of the user.

Companies can produce technology, like Sun, Bea, IBM and Oracle. When a company, say a bank or a health insurance company wants to develop an information system, it selects one of the providers mentioned and then one integrator to implement the system. The provider does nothing but selling its trademark and gets the lion share of the profits, while the integrator does all the work, gets all the risk and gets almost no profit.

Providers usually seat on their pants with mediocre products.

Take Google for example. It has pagination and scroll. Its a terrible user interface, since I can't paginate without using the scroll. Also it could be done using Ajax and it could sort results by different criteria and it could do so much things. Web search is not finished, yet it seems they think they have reached a plateau.

jueves, 13 de marzo de 2008

Snooze

Snooze (according to Webster at m-w.com):

1. To take a nap.
2. Something boring or uninspiring.

A build machine is a nooze that allows me to snooze. But certainly not something I would skip on my laundry list of things to do in a software development project.

What inspire us? To solve problems no one has solved before, specially if several thousand people have failed miserably while trying to achieve the goal.

Other than that, I bet that using new technology is also inspiring, although I must say that it bores me. After several decades learning new interfaces, it certainly bores me that I have to learn new interfaces to do thing I already was capable of doing and I was even proficient at it, having to remember and most of the time, forget, is certainly draining resources I could well spend somewhere else, doing something that I find productive.

Writing unit tests that are never run is probably the biggest waste. Or realizing that the tests that were running during the whole project, were in fact not run after the month 2. Now they are broken and finding the time to fix them all is not only challenging but close to impossible.

Another waste is trying to solve production problems (by writing SQL scripts for example) and at the same time trying to build the new version of the software.

Why are we having so many production problems? Why aren't we running tests? Why we don't separate production from development?

I must be living in the dark ages, when only a master could direct a job and solving anything in you house meant having at least 20 persons living in your house, like Leonardo Da Vinci did several times.

Those processes were new at the time, so they couldn't mass produce them. Today we all expect to find something at the supermarket and of course it is being mass produced. Otherwise only the super rich could only afford it. And yet we see time and time again that even the super rich go to the supermarket. I haven't heard of many Michelangelos today. At least not in the painting market.

But in the software market of course there are several Michelangelos. Like the Spring framework author Rod Johnson. Spring is as inspiring as it can get, but I'm afraid that giving it to anyone necessarily means that everyone doesn't know what to do with it. In other words, the Mona Lisa now is in the supermarket and people are haging it in their kitchens.

Always fix bugs in the trunk

I just realized this: Keep the number of environment branches to the bare minimum and always fix bugs in the trunk.

What is an environment branch?

It is a branch based on a release (a release branch) or based on a machine (an environment branch). Since every release was tested on an environment, every release branch is an environment branch. But environment branches are not needed, in my opinion. Here is why.

Some development shops keep a separate branch for each environment, not only for production builds, but also for UAT, QA, SIT, development and so forth.

First of all development should be done in the trunk.

Second, if the bug can't be reproed in the trunk, why care? One valid reason is that some customers need the fix.

Third, why have a UAT branch? Do you really think that you will fix a UAT environment and then continue testing? Would't be more reasonable to test everyting again if you are submitting a patch to UAT?

For having the idea of submitting a patch for UAT you should be ashamed, since the patch invalidates all the previous tests. Furthermore, why not wait for a development build to pass QA and then pass UAT?

And since we are testing so much, why not automate the testing (for example using JUnit)?
I've run into a project that keeps separate branches for every environment and when a bug is found in UAT or whatever environment, you are given a branch based on that branch. You fix it and request a merge against that branch.

The only problem is that there are mountains of integration problems when that fix must be merged against the trunk, because you are working on such an old branch (UAT), so that the merge becomes meaningless and totally buggy.

One excuse is that if you fix all bugs in the trunk, you will be using really new code, and therefore you will use code that doesn't exist in the UAT or production enviroment and therefore merge will be impossible. In that case I would prefer a back port (although it may take a while) or redoing the fix for the older enviroment.

Having so many environment branches and keeping track of them is a nightmare.

sábado, 1 de marzo de 2008

At least 5 times faster

Lately XP (eXtreme Programming) has been getting a bad reputation because it is being considered an excuse for not receiving nor writing documentation, even though that was the real intent of the XP inventors.
It is very interesting to note that even though XP as a whole is being dismissed, the individual practices are being incorporated into other competing processes like CMMI and RUP, as for example TDD (Test Driven Development), Continuous Integration and Spikes (Evolutionary Prototyping).
One of the XP practices that I'd like to write today about is the "Make it work, Make it right, Make it fast" (MiwMirMif) best practice, because I think most developers are doing the "make it fast" part with too much effort and very lame results.
The whole idea of the MiwMirMif is that:
1. First you get a working prototype you can manually test and verify it is working as expected. 2. Then you read the code and make sure the code is production quality, which means the code is clear enough, there is no code you don't understand or don't need, it is easily modifyable, etc. Whatever your criteria is for production-ready code. 3. Finally you make sure the code performs as expected.
The only problem I see with this is that most developers, once the second step has been achieved, ruin the second step in order to achieve the third step, and only to get a x1.2 speed increase, which in my opinion, is too lame.
There is a better way, and as always, the best way is far easier than the wrong way.
First, speed increases ideally should be an order of magnitude better. If not, they should be removed, as they are probably not the right speed increases. Second, if you positevely can't get your code to run 10 times faster, it at least should be 5 times faster. Else remove those optimizations and find better ones. Third, the optimizations must not clutter the code.
The functional code must be functional, and all mechanisms must be hidden in libraries that hide all the technical details. If the functional code also has optimizations, then most likely your optimizations are the wrong ones, since you can't test them independently, and anyway the functional code is always changing because business requirements change, therefore if you add optimizations to your functional code you are getting yourself into a problem most likely you will not be able to get out of..
All optimizations should get into libraries to be reused and to make sure your optmizations are correctly coded. Then your functional code will be clear and easy to modify and verify.
Which optimizations are the best?
First of all, make sure you have measured and you are positively sure you need some optmization, let us say, some operation takes 30 seconds and you need it to be under a second.
The first idea to consider is "do we really need to do this?", "can we avoid doing it entirely?". Sometimes it is not necessary at all.
Second, "do we have an optimization already in place?" If that is true, take it out, because it is not working as expected.
Third, "why is it taking so long?" Take a profiler and see for yourself which part of the code is taking long. Do not guess, just look at the profiler.
Fourth, "Is it O(1), O(log n), O(n)"? "Is it that we are going to the database too much?", "Are we performing the same operations over and over?" Make sure you understand why it is taking so long.
Fifth, make a guess about what could improve the speed. A cache? Then use a cache proxy. A buffer? Then use a buffer. Maybe you need a new data structure (highly unlikely), maybe you need shards, maybe you need to handle all in RAM in a distributed cluster using map/reduce. What ever you think can improve the speed, implement it at the library level and test it. Is it 10 times faster? Good! If not just test the next idea.

martes, 8 de enero de 2008

3 instance variables at most

One message that is very hard to get through is that all classes should have 3 instance variables at most.
Why limit the number of instance variables and why 3?
What if you need more?
Let me look at this another way.
Why do you need more then 3?
Why can't you group variables into objects that make sense?
This is very important. While it is rather easy to constraint method length to 10 lines, and to constraint the number of methods to 10, to constraint classes to only 3 instance varialbes seems almost impossible.
Yet, this is the most important thing to achieve, just because it is so hard to achieve it will give a competitive advantage if you do.
Which advantage?
More clases!!!
Having more classes shouldn't be an advantage per se. But if you test every class using JUnit, your code will be better tested.

Which code looks clearer?

class Form
{
String name;
String company;
String address;
int age;
String companyAddress;
String phone;
String companyPhone;
}

Instead of that, consider this:

class Form
{
Client client;
Company company;
}

class Client
{
String name;
String address;
int age;
String phone;
}

class Company
{
String name;
String address;
String phone;
}

Only the Client class has more than 3 instance variables. But address and phone can be stored together:

class AddressPhone
{
String address;
String phone;
}

class Form
{
Client client;
Company company;
}

class Client
{
String name;
AdressPhone addressPhone;
int age;
}

class Company
{
String name;
AdressPhone addressPhone;
}

Isn't this code clearer?