jueves, 20 de diciembre de 2007

Computer Science and Software Engineering

There are 2 main camps in software development:

1. Turing computable computer scientists: All turing computable languages are equivalent. If something goes wrong, RTFM. If everyone RTFM, there would be less defects. This is the main driving factor in Unix and Linux.
2. Software engineers: Writing correct programs is almost impossible because defects are so pervasive, even if you read all manuals in the world. There is cognitive disonance between what the programmer thinks he is doing and what he is actually doing, mostly because every programming expression has limits based on the types of the variables he is using. There are other border conditions that produce the same problem. If the user misuses the system, it is because it is so hard to use. This is the main driving factor in Windows.

For example:

x + y

Is it a correct expression?

It depends.

If we assume that x and y are bytes, the sum will overflow if x + y > 255.

The same can occur with the expression:

x = 1/10.

What is the value of x * 10?

Most developers would expect 1, but a few selected ones would answer that it depends on the type of x, since if it is int, float or double, the result would be different. It would certainly be different from 1 in all those cases, which means languages are broken, from the point of view of what the language user expects.

Turing computable computer scientists would think it is irrelevant: Just hire smarter programmers and write down all requirements, so that if the requirement is not written, it is not the programmer's fault, but if the requirement is written and the program still fails, it is your fault because you didn't hire smart enough programmers. Programmers should be able to code using wires if they are required to, creating abstractions is such a waste of time, because it *derails* programmers from their real work.

Software engineers would rather think that languages need to evolve in order to avoid dumb programmers make idiotic mistakes. It is just the opposite of what turing computable computer scientists think. When it comes to languages, market economics agrees with software engineers in the long run (3 to 10 years), but in the short run (3 to 10 months) turing computable computer scientists are correct. Hire te smarter ones, write requirements down, work longer hours and ask users to RTFM.

When it comes to libraries, turing computable computer scientists are always wrong, because they avoid using and creating libraries. After a few months their projects flop because of the massive amount of copy and paste and the huge number of defects that go undetected and detected but not fixed. They may even try to fix those bugs, but for every 10 lines of code you touch, you introduce another bug.

Software engineers succeed only when using and creating libraries and abstraction layers. Language does matter because market economics is far more important than turing compatibility. Who cares if they are equivalent? The fact is that one of them is easier and therefore it reduces the amount of stress on the developer. The code looks correct when it is correct. There is a lot less code (since most code is in libraries anyway).

But there is a 3rd strain of the virus:

The OO camp.

It doesn't belong in the software engineering field nor in the computer science field, although it is closer to the software engineering field. While software engineering concentrates on software configuration management, Subversion, Jira and the like, the OO camp concentrates on JUnit (automated unit testing), Selenium (automated functional testing) and design patterns.

While the software engineering camp believes that specification is the key and believes strongly in waterfall (finishing the previous step before starting the new one), the OO camp believes you need to specify only what you are going to build in the next few weeks, it negates waterfall and believes in the spiral model, building functionality one step at a time.

The most vocal methodology when it comes to the OO camp is XP. Cynics declare XP to be programming without plans and without documentation. Although XP does not mandate documentation, it doesn't require that you don't write documentation. It just says not to write more documentation that is externally required by the client. All the rest should be explicitly stated in automated tests, so that you can know for sure you are complying after any change. No, "I know this is correct and doesn't affect the rest of the system" is not a valid response according to XP. Some critics argue that XP is too expensive because it requires you to write so many tests. At the beginnig that's true, it is expensive, because you need to get used to write tests. After a few months, though, the project is performing a lot faster. We run a project using XP for several months and we were improving our speed constantly until the project was simply finished. We could have been fed a new project twice the size and we would have finished it in just 2 months. We were coding 10 uses cases per day.

domingo, 28 de octubre de 2007

How do you when you are doing a terrific job?

You could expect some people to be very happy or very angry at you because you have achieved things that nobody else has or nobody believed possible. If they are incredibly happy, it means you haven't done enough. You must continue down that path until they consider it too much and become angry.

Why?

Because otherwise you are doing what is expected of you and you are therefore not delivering something amazing.

But they also could be angry because they don't understand what you have done or if you have done anything. They could for example not understand your achievements or if they are achievements at all. They may think they are a terrible waste of time.

How do I know if I'm doing right or wrong?


One pretty sure way to do that is when they say that you have used a commercial product instead of developing it by yourself. That for sure means that simply can't understand that one person is able to achieve something like that. Good! You have made something unique.

And that's really important. It doesn't matter in which environment you work, you need to strive for people to genuinely perceive you as a bearer of the godly goods. An incredible overachiver. Of course they can always say that's not enough, but then continue down that path until you affect the Excel spreadsheet so that it is unavoidable to realize that it is amazing.


Enterprisey 101

One thing that you must learn is that if you do the same as 99% of the people, it is worthless (from the point of view of the revenue, it doesn't mean you shouldn't be polite for example).

Whenever you or your company enter a market, you must make sure there is a very high barrier of entrance, because if not, you will be in a perfect market, and in a perfect market no producer can make a profit (although the consumer is more than happy with this situation).

Having a trademark and defending it is important. You can sell water and sugar in a bottle and people will pay for it, but you certainly need to protect your trademark. Otherwise your company has a value of zero and a lot of bills to pay.

In the same fashion as a company needs profitability, each individual needs to be profitable, and therefore whatever you do, you must do it in a way that permeates (or communicates) your potential clients with the message: "this is what I deliver". It is good for you, because you get a better name and it is good for your customers because you take care on what you do. This caring and concentration is important because it makes you able to do things that others can't, making you more necessary, because your clients don't have an option.

In the end, what is really important, is to be able to deliver the goods. This may require many years. That's a barrier of entrance and therefore it is good. Whenever you see a market with high barrier of entrance, you have spotted a good business for the producer. If it looks unsurmountable, it is a great business.

Barriers of exit are another thing. Usually markets that have high barriers of exit are rather bad for the producers, since when the tide changes, and it is not a good business anymore, you can't simply switch production and leave this market to enter the new one. Therefore if you are exploring new markets, you shouldn't try the ones in which you won't be able to exit gracefully, because by the time you are in the market, others will have followed you and you won't be able to exit. Neither will they, so all of you will suffer the tragedy of the overproduction, and you will have to resist that pain for a very long time.

Examples

Burger King in Chile circa 2000. Many locals closed, because they were not profitable enough. At the same time, Doggis, a chilean fast food chain, was growing at 10% per year.

How come?

The main difference are the expectations. American companies have very high expectations in terms of revenue. Chilean companies have very low expectations. So a market that for Burger King is lame, for Doggis is an excelent market. They are on the same market, but their perception is different. Only when Doggis see what Burger King was making before, they realize they are not making that much and their perception changes. A saturated market is good for no producer. A saturated market of fast food, one could argue, is not good for the consumer either, just because they think the food is unhealthy, and they are of course right.

Companies need to differentiate and professionals need that too. For professionals, having certifications is the best way to differentiate. Getting a certificate each year is the best way to differentiate.

Too much?

Consider this: each year more and more universities are open. Each university is trying to attract more people into their programs. They open new campuses in other countries, so in a few decades, you will be able to obtain a grade from an american or european university without leaving your city. You think that is not possible? Think again. All you need is to travel one weekend a month to a city nearby (where your professors and your classmates travel on the same dates) and then you simply get information by mail, by wikis, etc.

This is it. Most people with degrees will obtain several degrees (there is not much abilites needed when you already have studied something, so to continue studying is a natural choice). And most people will have degrees, since today even for driving a bus you need a degree and even that job will be performed by robots.

Education is a big business and will only continue to grow. The only problem is that educational institutions do not have a way to differentiate. At least not an obvious one.

To perform a job will eventually mean no revenue, so instead you will have to "think" the job, in order for robots to perform.

Therefore the best rewarding degree will be economics. In the short run, technical people have something, if not everything, to say. In the long run, though, jobs have to be carefully defined by economists, because it is the economic incentives what make the world turn. Even if you don't hire economists, eventually the jobs at your company will follow the rules of economics and since managers fire undersperformers, eventually you can find the optimal way to work through trial and error. If you know economics, you can guess a few months before and improve your revenue, guess a few years before and beat your competitors or guess a few decades earlier and beat the whole market.

If you guess a few centuries earlier, there is nothing you can do with that information, except pass it on to future generations. Those generations may or may not believe you. Anyway, if you are an economist that can guess what will happen in a few centuries, I guess there is no way to know if you are doing a terrific job.

There is a saying: "Manager's errors are their subordinate trials". I guess it works both ways, since "Subordinate errors are manager's trials".

jueves, 25 de octubre de 2007

Structuring systems around objects

Structuring systems around objects or beans and lacking abstraction is like building a building around bricks.

You can do that, but a pile of bricks is not a building.

In order for a building to be functional, well, it has to be able to interact with its environment. For example the entrance must be big enough for humans to enter, and it must be open to a street. Isn't it obvious?

Well, most of the stuff is obvious when mentioned. But it has to be mentioned for people to make the right decisions at the appropiate time. That is, when a guy is closing the entrance, he must speak up and say "we shouldn't be doing this, because we are closing the entrance".
Yet we don't expect people to find out things like that when they are actually putting the bricks, because there is a model of how the building will look like before they start to make a hole in the ground, so a lot of those problems have been ironed out long before somebody starts removing soil.

Yet there are lot of technical decisions to be made inside every appartment, from the point of view of each individual worker. Irrelevant from the point of view of the architect of the building, but important from the point of view of the person who is inhabiting that building if they turn out to be bad decisions.

It turns out that good developers are hired at the best places, because they are willing to learn and they come with a good background (they learnt at other places). Also they stay at their jobs because they are paid more. Eventually all those little differences make a huge difference in income for the company and you can see that in that most successful companies pay more for their people and also the company's income per worker is higher, meaning their selection has better results.

Having worked at those environments I can tell you shat it feels like: It feels like you are working with smarter people and that they are therefore more autonomous. They make fewer mistakes and they feel free to steer the way they work when the results are not ok. The results can be seen on a daily basis, so there is always room for improvement. You have a lot to learn when you arrive and you keep learning for a very loing time, while at the same time being productive in a very short period of time (around 2 to 3 months).

The architect shoudl concentrate on looks and interactions, for example: will the system have scroll? Most usability experts agree that web applications should never scroll, but sometimes you have so much information that you need a scroll here and there. I would reasonably try to remove those scrolls and exchange them by something else. For example it is typical to have "pagination". Yet what is pagination if not another kind of scroll? Yet pagination is simpler from the point of view of the user.

Take Google for example. It has pagination and scroll. Its a terrible user interface, since I can't paginate without using the scroll. Also it could be done using Ajax and it could sort results by different criteria and it could do so much things. Web search is not finished, yet it seems they think they have reached a plateau.

lunes, 22 de octubre de 2007

Project Speed

Always consider realistic expectations (or realistic goals if you prefer) and also consider an improvement of the team performance.

How much can we improve?

You could consider that the team will deliver 10% of improvement per year or per month. 10% per year sounds reasonable, given that China has been growing at 10% per year for the past 10 years. 10% per month sounds like too much. But I've found that in practice that is not the case: you can't improve by 10% per year nor per month because performance doesn't improve cuadraticly but linearly.

I usually divide the teams in layers, one layer for the presentation, one for the business rules and another for persistence. Also there may be other teams for workflow or for whatever communication with other system is needed. This means that initially the team produces zero use cases per week, irrespective of their size, because of the communication problems. Some developer requests something to another developer, but they don't understand what it is. Even when they do understand each other, there are so many unkowns, that they simply fail to deliver a working system.

But eventually they learn and they deliver something usable.

Then each month the speed of the team is increased by 5 use cases per week.

Month Speed per week Done this month Total
Month 0 0 0 0
Month 1 5 20 20
Month 2 10 40 60
Month 3 15 60 120
Month 4 20 80 200
Month 5 25 100 300

Incredibly, our team is able to produce more and more per week, increasing the speed effortlessly each week. But the project is finished at 300 use cases (actually there was some duplication and some returned use cases, so finally only 250 use cases were in the system).
If we allowed other projects to enter our production system we could continue to produce more:

Month Speed per week Done this month Total
Month 6 30 120 420
Month 7 35 140 560
Month 8 40 160 720
Month 9 45 180 900
Month 10 50 200 1100
Month 11 55 220 1320

So in a year we could have finished 3 projects instead of just finishing one in 6 months.

Clear roadmap

Any project is doomed if it doesn't have a clear roadmap. A clear roadmap is a clear understanding of:

1. What has been done.
2. What needs to be done.

All the rest can be considered irrelevant "detail". It does not matter if you have the smarter developers working 25 hours per days, 8 days per week. It doesn't matter if they write clear code that anyone can read and modify, and it doesn't matter if they write a million use cases per day. If you don't know what has been already achieved and what has not been achieved, your project is doomed.

Even more important than that is that the whole team knows where the project stands. So ANYONE on the project can login into an staging machine at any time and test the system. this means the project is a 100% repro. You can repro any functionality and any bug.

Clear specs

Ok, let us suppose you already have all that. You really have a road map. You can repro everything on the staging machine. And you have unit tests and functional tests for every feature. And you do iterations and you demo the features at the end of every iteration, and you do pair programming, collective code ownership, etc.

But you don't have clear specs. So whenever you do the demo at the end of any iteration, you fail, because it is not what the user wanted.

Having clear specs is very important, the problem is that it doesn't matter how many times I tell you so, it doesn't matter that you remember it everyday, if you deliver software systems, by definition you know nothing about the business and everything about the past software project, and your users know nothing about writing documents for software developers and the project sucks.

So in the end, prototyping the user interface is the best way to deliver software that works.

We usually call it "user interface mockup" to distinguish this prototypes from the real prototypes, the kind of prototypes we use to figure out if we can deliver the functionality because it is technically feasible.

martes, 9 de octubre de 2007

If XP is eXtreme Programing, SP must be Stupid Programming.

If XP is eXtreme Programing, SP must be Stupid Programming.

I've been in several software development projects in the last 15 years.

And since for the past few years the economy has been booming (and also many people are attracted to this kind of job), I suppose all the good people are scatered, so the average quality of the projects seems to be fading...

In the last too projects I've been I've seen something that I, for lack of a better term, unapologetically call Stupid Programing.

Or "Just Program It".

Now I understand managers. Developers really solve one thing and destroy another, so they are kind of stupid. They can't see past a few lines of code (or a few thousands if he is really good), but most of the time the process seems to be out of his control.

I do this myself all the time, but with a lot of care. I don't mean that I code meaninglessly, but something completely differently.

What do I do?

Simple: I suppose that resolving the general case is solving the first particular case, then the second and so on. Continue untill all particular cases are solved. When the process is finished, you have built an abstraction. If somehow you arrive at a contradiction, it means the abstraction you have built is wrong, must be discarded and you must invent a new abstraction that solves all the previous cases PLUS the new one, and also you should try to foresee if the new abstraction, at least conceptually will be able to handle the cases left.

The only problem I see is that when they solve the second case, they don't care about the first case, so they break the whole application everytime they touch anything. And since you can't keep a very big application "all in your head", this is a very good way to get job stability, since other developers will have a hard time getting this in their head. So development "never ends". It is always declared to be 95% or 99% and there it remains, no matter how much money you throw at it.

If you haven't seen this, you either lack experience or you are doing XP, in which case you will lack this particular experience in the foreseable future.

XP

XPers got this right. You have to understand the code. Everyone on the team should understand the code.

So XPers insist in that you pass 100% of the unit tests all the time. Make a change, compile everything and pass all the tests. Waste of time? If you are repeating your code all over the place, of course it is a waste of time. But if you are repeating your code all over the place without doing unit tests you are also wasting your time, and maybe even more so.

You do a change and you have no freaking idea of what will work and what will stop working. So you have to restest the whole application, but these manual tests require that you set up the corresponding data, so the results of the tests depend on the mood of the tester.

Stop wasting your time. Stop repeating your code. Code once. Then simply use it.

Avoid repetition so that your code is manageable and maintainable.

Avoid SP altogether. If you are doing SP, we are not understanding the code and therefore the probability for the project to be a failure is simply too high. If you decide to stay with SP I hope you don't care if the project fails. Sometimes projects are so small and irrlevant that you can afford them to fail, because you can completely redo them in a single day.

But let's face it, those projects are uninteresting. Interesting projects are big and require a lot of coordination.

Most shops that claim not to be doing XP do it for the wrong reasons. They think writing documentation is forbidden in XP. They also think doing SP is the correct way to direct projects. If you are flippling burgers, maybe "just do this" is the correct way to do it, but flipping burgers is a contiuous operation, while a software project has a duration and after that the project doesn't exist anymore, because of the very definition of project.

domingo, 7 de octubre de 2007

I usually agree against corporate standards

I usually agree against corporate standards ;-), but this got me thinking:

http://blogs.tedneward.com/2007/09/20/Hard+Questions+About+Architects.aspx

Are you really trying to say that the architect should not consider his job seriously?

I mean, do you realy think this architect should say: "Ok, let us do it your way" I wouldn't seem serious. And although deep down it is not serious at all, it still have to seem serious. This sounds cynical, and maybe it is, because I'm thinking on my convenience, but also an strategy that doesn't seem serious to your partners, even if it is a great strategy, is impossible that it achieves success.

Actually you can be pretty serious, doing unit tests and functional tests all over the place, but you still have to play your role. Your architect role.

Although that I agree that anyone who critics XP better doesn't tell me XP is not about not writing documentation, because in that case it means they have an opinion about something they have not read, and the XP documents are readily available, so it means they:
  1. Advocate writing a lot of documents.
  2. They do not read them.

Lame architects!

And yet the world is full of lame architects.

But imagine an XP architect. One that advocates not writing documentation if it is not needed and one that reads all the documentation and signs off projects once all the documentation is in place... (I suppose you agree the architect is not the only one who writes documentation, but the one who supervises the documentation and the code).

Would you fire him for having a strong opinion about how things should be done? (For those who do not know XP, XP have a very strong opinion about almost everything). Furthermore being an architect means that you have had successes and failures and therefore you can guide a team about certain choices because you have more experience.

An architect needs to have an opinion. A severe opinion, about everything (ok, I'm exageratting here, but in the matters that matter to the project, he needs to be opinionated). From how you document to how you should test. From how to do code reviews to how to incorporate new designs and third party code into the systems you are designing.

And if he thinks your code does not belong to the project, maybe he is right. Also maybe you can do things to prove him wrong, in which case he could be happy because he is learning, or otherwise he could be fired because he is unwilling to learn.

The last project I was on went smoothly in every possible way, except in the Workflow team. The workflow code was crap. It was my opinion and the opinion of most developers. But the project manager and the project director defended this developer, who was a pretty good developer, but did not understand the rest of the architecture. They defended him until it was not possilbe to defend him anymore, so they passed me the problem for me to fix it.

What? >8-O

It is my opinion that before making any changes you need to assess what is working and what is not. This is called "making a diagnosis".

And the only way of knowing if the project is ok is to know what the requirements are, what were the design decisions in order to satisfy those requirements, do the adequate prototypes and then integrate them.

XP doesn't say exactly so, but it is pretty close. In XP you need to work in iterations and show the iterations working (the same happens in Scrum sprints). The main difference is that I care less about iterations and care more about doing prototypes (spikes in XP terms).

But the Workflow code was already embedded in the system and surely most of the business objectives were not met by the design. What is the point then of makiung trivial changes to convoluted code, if what was needed was a massive rewrite?

I mentioned that it would take 3 months and 3 developers. But I was denied the resources, which in my opinion was nothing compared with the 1.5 years and 30 people in the project.

Reasonable?

I stand from my point of view.

As C.A.R. Hoare, the inventor of QuickSort, once mentioned: "I conclude that there are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies and the other way is to make it so complicated that there are no obvious deficiencies."

You can read it here.

I don't endorse XP, although I recognize it would be better for 99% of the projects to apply it. Both XP and Scrum. And I recognize you need an XPer and/or an Scrum master to apply them correctly, because reading books is not enough. The main reason I don't endorse XP is that when you finish a project it becomes boring. And most projects I work on, become boring pretty fast, since the possibility of failure decreases very quickly, while XPers advocate that if a project is boring is because you are not doing it right. I understand why XPers mention that. A project may become boring by repetition and repetition must be avoided at all costs. But anyway, a multitude of different projects all look and feel the same after a while, so even starting new projects becomes boring after a while. You can tell in the first day or two if the project is going to be a raving success or a raving madness.

Time is money

Time is money except in one respect. From wasting money you can recover. From wasting time, you can't. Therefore if you are not going to succeed in your endeavor, it is better not to even start. I've seen many projects fail and most of the time the problem is that no one knows what needs to be achieved. Massive amounts of people are hired and no one has an idea of what the objective is.

Therefore you first need to know the objective. Then do the minimal to achieve success. Once finished, verify you have succeeded. Wash, rinse, repeat.

Make sure you are solving real business requirements. Making sure your objectives are important enough as to get promoted once finished. Or at least to get filthy rich, as a second option. There is no worse waste of time than achieving unimportant goals.

jueves, 4 de octubre de 2007

Closures means everyone must go back to college

"Usability" means the customer is always right, that you don't tell the user what the application is about the first time they use it and yet they are able to figure it all out because the user interface is somehow familiar to them.

If we applied this criteria to computer languages, no one would be programming in other languages that were not Visual Basic, because circa 1996 it was the mainstream language and all the rest was considered niche.

Where are now all those Visual Basic applications?

They are considered crap.

Unmantainable crap. That is what Visual Basic is.

I see no one asking for Visual Basic to make a come back, not even hard core VBers.

Closures

With closures it is the same thing. Now that Java is going to get closures, developers will need to understand that concept, because all languages will come "closure ready".

Java for example makes it impossible to code in Java if you don't understand object orientation. Of course you program a website and you can do a lot of things using IDEs, but if you don't understand object orientation, eventually you are going to face a wall and bang your head for months. I've seen projects fail because of this.

The same with AOP, interception and dynamic proxies. Now with the help of Spring, everyone is exposed to interception. If you don't get interception, you should be looking for the janitor job or the manager job, but eventually you are not going to be able to debug and fix those bugs.

Now closures come to the rescue. The first thing you are going to learn about closures one night in the middle of a debug session is that closures do not see the updates on global variables.

Remember how professors at the University talked endlessly about the bad style of using global variables? Well, now you willl see for yourself that you need to redesign massive amounts of code just because you were using global variables.

And as always, massive amounts of programmers will go the easy route and turn into managers. After all it only requires you to think you are incompetent (which now you will be convinced anyway) and therefore you will think you will need to brown nose upper managers. Some people like the attention and will simply promote you, as long as you keep your act together, that is, you keep brown noising, because after all, politics in an organization is not about who is able to do what, but who is best at the polls.

Most companies, when they have grown enough, can't fill all the positions with geniuses. They need to hire less than average people. And this means geniuses become very bored to work with you. I mean they can replace you using a simple script, so they wonder why you were hired in the first place.

If the geniuses can't replace people as fast as the manager can hire these less than average people, geniuses tend to leave. Which means you get all sorts of internal jokes like "how many developers do you need to change a bulb" and stuff like that. If you hear those jokes, you know management is considering to reduce the headcount, maybe not today, maybe not this month, but certainly this year.

And after the crunch, new and fresh developers come to help. Those that still believe in life and work and who of course have never worked and think that if they perform ok they will be promoted, etc. They now know object orientation, they understand Spring because they used it so many times in their homeworks and they used closures a lot. So they perform 10 times faster than older developers and they almost have no bugs, for 1/8 of the wage.

Would you stay as a developer or look for a promotion?

It doesn't matter how much you know, there will always be smarter kids just arriving from the local university and ready to fix everything.

So you have 2 routes:

1. Go back to college to learn closures and then get the same pay as a rookie.
2. Get a management job, get better pay and forget about the whole thing.

Getting into management is kind of easy. Besides the cosideration that you are worthless and therefore you need to brown nose your bosses, you also need to make those happy developers work, because they are the most valuable resource and they need not to know it, or otherwise they would leave for a better paid job.

Usually managers lie. And they lie so that they make it look like you lie. Let us take a closer look at this, because you probably think this can't be possible.

Managers, by definition, have to have an office with a door. Also they need to meet people individually.

Have you ever talked to a manager? They always tell you there are so many projects, but how come we are not working in those projects?

The real reason managers talk one on one, is that the story they tell one person is not the same story they tell another one. And they know projects last for a short period of time (a few months to a few years), so most people will be fired, and the manager knows developers will get nervous, so they need to say with a straight face a lie ("this person was reluctant to do his job", "... to put more effort", "he was inadequate", "he couldn't work with the rest of the developers", "the client didn't like him", etc.), so that other developers will not run en masse.

And managers will of course lie about the fact that they lied. That's the wonderful thing about lying. The not so wonderful thing is that from time to time people confront you, they discover that you lied and they tell everyone. Those cases are pretty unconfortable for managers, but it is like what developers do with bugs: they debug. When managers are confronted with a developers who call them liers, they promptly fire those developers on the basis that the individual developer is lying when saying the manager lied, and not respecting authority.

Upper management is usually on the side of the manager, unless the manager doesn't know how to pull the ropes.

What do I mean now?

Upper management will not fire a good manager because of a lousy developer. And a good manager is the one that can get his act together in front of several other managers and even with the client (which is usually a company full of managers). So managers tend block and surround good developers with his own people (that is people who is loyal to him), and let people communicate with people in other companies as long as they stay loyal.

And for them to be loyal, they need to understand their real value: worthless. If they are worthless, it means they are ready to lie, because they can't go through life showing their real self. They have to pretend.

This is exactly the same that happens in college when people copy in an exam. Why do they do it? Because they know that they can't perform. Are they doing a disservice by graduating without knowing? Yes, but they do no harm in the market because they know they don't know, therefore they don't perform, but ask people who can really perform to do the job, while they try to sell the services to other managers.

This is really important. People who have companies can't really perform at the job. Most managers in high tech companies can't code, except maybe Bill Gates.

So they need to lie to their employees, letting them think their job is irrelevant. You job can be irrelevant only if the managers can control what you are doing. And developers know this. Therefore developers tend to hide what they are doing (they don't write unit tests or funcitonal tests for example) so that it lloks like they really know what they are doing, while the truth is that they have no idea.

While managers tend to control what developers are doing by testing things, not directly, but with the help of testers. Isn't life great? You design a system and let it run, then get paid monthly for running it, which is not more difficult than pressing a few buttons, but on people's minds.

miércoles, 3 de octubre de 2007

Development Best Practices

Usually managers tell you that the projects spend 60% of the time and 80% of the resources on development.

But what they don't measure is that debugging, not writing code is what takes 80% of the development time.

Writing unit test reduces the need for debugging to almost zero. Therefore it is a cost saving mechanism that can save as much as 40% of the total development time.

Other best practices:

1. SQL DDL (database creation script) in SVN.
2. Reinstall database each time you need to verify the SVN repository (compile + run unit test + run functional tests ), which should be done after any check-in.}
3. Branch for each issue???
4. Each developer with its own copy of the database on his machine.

Hibernate Leaks

The whole idea of Hibernate is that you don't need to know SQL or JDBC anymore to persist your database model.

But it is not so easy as it seems.

You can get this exception when saving:

org.springframework.orm.hibernate3.HibernateSystemException: a different object with the same identifier value was already associated with the session:

Or this exception:

org.hibernate.NonUniqueObjectException: a different object with the same identifier value was already associated with the session:

It turns out that Hibernate can detect when an object has been loaded twice, but it signals the error too late, when you try to save it.

The documentation on this subject is sparse. Sometimes it mentions that you haven't coded appropiate equals and hashCode methods and you have inserted this objects in Sets or Maps that span more than one Hibernate session. So when another session loads the object it can't know the object already exist, but it knows when it saves the objects? Come one!

Why doesn't it simply do the same check when loading the objects (possibly replacing the copy with the original object) is beyond my comprenhesion.

In other postings it is mentioned that Hibernate needs equals and hashCode even for object not stored in other places, since you can have proxies for objects, and the proxies could not detect that the objects were already loaded. Can Hibernate make it more complex?

Complete Abstractions

I think the real problem is that the Hibernate model is not a complete abstraction.

What is an uncomplete abstraction?

It is an abstraction that doesn't handle border cases appropiately. Joel calls it leaky abstractions, but not all abstractions are leaky. The ones which are leaking can always be backed up with new abstractions.

For example, in the case of Hibernate, the object model is persisted automagically, but I have to say when. with saveOrUpdate. Also I can't say save this but not that, all I can do is to configure it in a dumb xml file. It is dumb because it is necessarily hardcoded.

Xml is not for configuration, it was created for applications to talk, not humans to read or write. Developers who relay on xml too much are typically substandard developers.

lunes, 1 de octubre de 2007

Usually the hard part is understanding the problem

Usually the hard part of delivering a working information system is understanding the problem.
When the problem is really understood, creating a working solution is trivial.

Therefore there are 2 approaches:

1. Write extensive documentation about the problem to be solved, so that developers can browse them and then come up with a working system.
2. Do not write documentation about the system, except the absolutely necessary, instead focus on creating the appropiate protoypes for the user to imagne the system at work and clarify ideas.

The first approach is typical of UML and RUP guys. They deliver a lot of documents, but no working system. I mean, look at Rational Rose, it was built using UML and it still doesn't work properly after 13 years.

On the other hand, the prototyping camp have lots of prototypes that work, also they have lots of prototypes that did not work, but they spent no more than 2 days on each prototype, so who cares?

Even the most evolved prototypes, those that were successful and were constantly refined, took 6 months. That is nothing when compared to the RUP and UML camp takes to deliver a mountain of documents and no working system. The documents are deliberately ambiguous and meaningless, sometimes they contradict themselves and most of the time they are not complete and you can see the copy and paste didn't mind to modify all the relevant places.

To be fair, UML and RUP does advocate creating an architectural prototype early in the design of the system, but few organizations follow that advice and I doubt I would have any success with such a trivial prototype.

First, all screens need a prototype (more properly called screen mockup).

Also, for every desing decision, you need to:

1. Document them.
2. Do a little prototype to back your decision.
3. Demo it.

Once all desing decisions have been docmented, prototyped and demoed, you just need to integrate those prototypes and the project is finished.

sábado, 29 de septiembre de 2007

Poor Alan turing

Today, poor software is blamed on Alan Turing's programming model.

Can we liberate from Turing? (In the same fashion as "Can we liberate from Von Newman?")

Actually Alan was trying to demostrate mathematically that using simple tricks like a tape in which you can write characters in it, you can compute any mathematical function. Period. That's it.

Your bugs in your system are yours, not Alan's. Alan was not promoting that you used just one tape and just one head to read and write it, which would be the equivalent of just one RAM and just one processor.

This guy even have a black list of people you should not read in order to get reliable software.

I agree with him, but I think this deserves an explanation.

First, i would not black list people because they make a mistake. It woudl be as blacklisting developers because they introduce a bug. That would be ridiculous.

Also, Louis Savain also makes a mistake: he says we software developers can't for sure test our designs and our code. Of course we can! We can do prototypes and we can use JUnit, respectively!

Prices

Have you ever wondered why managers make tons of money and why individual programmers make so little?

I have a possible explanation to that: In any company there are more developers than managers. Of course this doesn't explain it, because there are also less janitors than developers and janitors make less.

Another possible explanation is that less people are willing to be janitors, therefore people who work as developers have to prove that they can code, so this would explain why janitors, having less mental capacity (or training), would have to accept less well paid jobs. But it doesn't make sense shen it comes to managers, since managers are probably the dumbest, the most demoralizing individuals who can't plan in the following 3 months. Most of the time they ask you to do the plans and then they simply shorten the schedule in half.

Also managers are the most prone to do copy and paste, adding really no insights neither in the business knowledge nor in the technical knowledge. They simply make people hate each other and compete, so that they can get all the benefits.

But why do they make more?

Maybe their inclination to lying, with a straight face, makes them more interesting to upper management.

What does managers' wage have to do with Poor Alan?

I have yet another theory of why managers make more.

Managers are the most damaging people in an organization. If they don't harm the company, they certainly can damage it.

And upper management knows this.

Therefore, managers have more bargaining power.

They typically know what to tell people in order to turn people against each other. I mean, they do it for a living, so they eventually know the tricks and with 3 or 4 words spelled in secrecy, they can turn the best firends in to the worst enemies.

And therefore they are brought to make people communicate and blend, but it would be unfortunate for them if people really blend together, since people must be fired when the project finishes, and therefore they have to outright lie to them telling people "there are so many projects" and stuff like that, while in reality "we just need you kid to fix this stuff and then good bye".

I've been working for more than 15 years, so I would say I've seen it all. What I have not seen though, is a manager that hires you telling you the truth: this is going to last just 6 months. No, they promess that their is an endless train of projects and you will work here for 100 years. Yeah, right!

jueves, 27 de septiembre de 2007

Each explorer windows in a separate process

"On NT, go to this key:

HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer

and create a DWORD called DesktopProcess with a value of 1. This makes the desktop and Taskbar run in one process, and subsequent Explorer windows each run in its own process. This means that you can do your debugging with a single Explorer window, and when you close it, your DLL is automatically unloaded, avoiding any problems with the file being in use. You will need to log off and back on for these changes to take effect."

Nice!

Taken shamelessly from: http://www.codeproject.com/shell/shellextguide1.asp

By the way: COM, COM+, ATL COM, etc. seems to be moronic and painfully unnecessarily hard.

I mean, C is kind of moronic. C++ is like moronic++. COM is a totally different, a totally different programming model. While C++ was born as a better C and object oriented C, then it was "upgraded" to generic programming using STL (which is the equivalent of Java generics), but a very bad implementation, although I admit that it is way better than the original implementation using macros (#define).

COM is "component oriented programming". It says "object orientation is hard, therefore let us not try to use object orientation and use components instead". Components are simply objects that do not use inheritance and that have both runtime behavior and design behavior.

Why would we need runtime behavior and design behavior?

Objects in object orientation have only runtime behavior. In case you are designing a GUI, objects have to be put in a place and objects have to remember that when they are running. But objects are not running if you are coding the application.

At least they are not running in Java and C++, but they are running in Smalltalk, because Smalltalk has a different way to look at the world. Therefore in Smalltalk object oriented user interfaces use object orientation and it was like a charm.

But in our poor languages C++ and Java, developers came up with the moronic idea that we needed this thingys called components.

And holly cow!!! There is so much people totally numbed with the idea that "components" is the next big thing as if they were seeing the light of truth, but of course they can't explain why it is so great. They even mention the typical falacy ad hominen "if Microsoft, Sun and IBM like components, components must be good".

I find myself totally happy with the idea that people are wasting their lives while we people on the object orientation camp have totally mastered and fixed all the bugs and we live totally happy in object land, while all those loosers on the component camp are wasting their lives trying to make those trivial things and fail miserably. At first, around 1992, of course I thought we were going to change the world and we had to be on a cruzade to defeat the infidels. But the infidels are so strong and they waste so much money that we do, so there is no point on wasting any effort on defeating them. They are defeating themselves. Good!

Actually it is good that they spend so much time on irrelevant stuff. This way all that irrelevant people is out of the way for us to build the important stuff. And we can learn in 3 months enough to work as much as 10 years, actually I've been working for more than 15 years with what I learned on the first 3 months of object orientation.

And that's because object orientation is hard. And I don't know if that is good or bad, but at least if you take the time to learn, you will be able to work in the industry for probably the rest of your life without having to learn much (ok, languages change, frameworks change, but the original idea has never been surpassed).

Falling in love with the mechanism

It is typical of development projects that the developers usually fall in love with some particular implementation mechanisms and they repeat the same code over and over again.

When you look at the code all you see is the same patterns over and over again:

mechanism1
mechanism2
mechanism3
mechanism1
mechanism3
mechanism2
mechanism1
...

So in reality this developer has learned 2 or 3 tricks and applies them to no avail.

The only problem with this approach is that several days, months or even years later the developer learns that his tricks has a bug.

Either:

1. A programming bug,
2. A design bug (the design doesn't handle some important cases), or
3. An analysis bug (the requirements were not the correct requirements)

So we need to make some changes, but the code has been copied around in too many places and now making a change is too expensive... because we need to change the code in so many places and when we change the code in one place to fix a bug, we introduce 2 more bugs and we destroy what was originally working. Managers feel really frustrated when they find out this and they think it is better to hire more developers, since the amount of bugs is increasing instead of decreasing.

Inevitably developers think it is cheaper to rewrite the whole thing. Managers tend to think it is waste of time. Unfortunately both are correct.

Developers are correct because actually the mechanisms were buggy, so instead of fixing the same 10 lines of code during 3 months, you can simply rewrite the whole thing in 2 months using the improved mechanism.

Managers are correct because rewriting the same thing is only going to lead to missing cases being managed, they code will look correct but the code won't work as expected and we will wonder why did we rewrite the whole thing anyway. Something that took so much time to tweak, now it will be clean and simple, but it will not handle all the special cases. In order to handle the special cases, developers will have to tweak the code and therefore the code will become a mess anyway. If it is already a mess, it is for a reason, go with it.

So I'm saying both are correct and both are saying just the opposite. What gives?

If developers rewirte the whole system, they will throw away the old mechanism and write the new mechanism over and over again. They think the future will be better, and they are correct, but no mechanism is perfect. So eventually they will realize the new mechanism is also buggy, differently buggy, maybe a lot less buggy, but buggy anyway. so the correct answer is:

Developers have to write the intent, and the mechanism must be hidden inside a library (an abstraction layer), so that whenever something needs to be changed, it is changed at the library level (assuming the business intent was correctly explicited in the business code).

I will say it again in different ways:

1. Do not write mechanisms in your business code, but write only business intent.
2. Hide implementation details in libraries.
3. Create abstraction layers.
4. Write black boxes.