Andy's Mind

I have been a computer freak for longer than I would like to admit (I just realized that when I was writing programs on my first computer – the venerable C64 – most of my current team was not even born yet). The shape of computing has changed a lot over that time. Computers now have infinitely more memory and processing power than my first machine. But one thing didn’t improve as much: overall software quality.

Every new programming paradigm or idea coming along since computers crawled out of government labs had “better quality” as part of its promise. Yet somehow we are still surrounded by unreliable, poorly designed software. Making it more graphical didn’t change it fundamentally. And sadly web applications are no exception.

At first at least “hard to use” part was partially solved, because the web interfaces were simple which made them clean and easy to grasp. It also made them fast. Now, however this is quickly becoming a thing of the past. Modern web apps are full of heavy graphics, JavaScript is being pushed to the limits of its capabilities and the result is heavy UIs browsers barely cope with. Opening a few of those modern web apps in Firefox makes it the most CPU consuming process on my system, usually eating lots of memory (and frequently leaking). Frequently sites display errors, get overloaded or otherwise malfunction.

Why it is so? Let me offer two reasons, that don’t exclude each other so both can be true.

First, I think we are all spoiled by the computing power and memory getting cheaper and more abundant. In the past machines changed very slowly. For example the C64 I mentioned didn’t change at all over a period of 10 years, yet software for it did improve immensely. Programmers studied the hardware and by late eighties made it perform things the machine’s original designers didn’t think it would be capable of. That was truly pushing the limits.

Now if an application is a resource hog it is easy to just give it enough to make it run. That’s why each release of MS Office is slower and bigger than previous with marginal functional improvements. Same happened to web browsers and almost all desktop apps. Increase in speed & memory sizes mask the fact that today’s desktop apps are usually bloated and slow, but only to a certain degree. The subjective perception is that they work as fast as the previous generation and everyone accepts that – even though everything should run way faster considering how much the machines this software runs on improved.

Clusters and now cloud computing made it an easy solution for server-based software too – and that’s what web apps are apart from the UI. You can throw a whole bunch of servers on the problem and forget about optimization. And this works, also economically. Sites still make profit on mediocre codebases, because the computing power is cheap and because usually very little depends on web apps. If they break or run slow no one dies and nothing of importance is lost – the impact of low quality is not easily seen. So, there is no economic incentive to improve.

This and overall acceptance of software as buggy (effect of customers being trained for decades to accept dismally poor software from Microsoft) causes buyers of software products to accept low quality as the norm. They learned to live with it and it takes some effort to bring quality into their prospective at all. In fact, I think even some of our clients don’t appreciate all the effort we put into optimization and testing because they don’t see benefits of having a piece of web software that is robust and scalable.

Second, I think many of the web applications today are created by people with little or no education (formal or otherwise) in computer science as such. I think many of “cool web 2.0 kids” have no idea how the computers they use really work inside. I don’t think many of them know for example what preemptive time sharing is let alone are able to calculate the computational cost of the algorithms they use (if they know this difficult word with Greek roots). Very few know and understand the operating systems they use or ever heard of highly available design and other such practices.

Tim O’Reily’s predicted in late 90-ies that easy web technologies (which back then meant PHP and early CMSes) will allow people with little or no technical knowledge to express themselves on the web. And indeed they do – but much of this “creative expression” is, well, crappy software.

All this means that despite everyone saying a lot about quality there is very little push to actually deliver it in web apps. In fact, quality is squeezed out of prospective from both sides. Client’s don’t care for it when they order their web apps and developers frequently don’t have the knowledge necessary to deliver it.

The upside is that this will change as more and more will depend on IT systems that are based on web technology. Crashes, data loss or hour-long “scheduled downtimes” (like Twitter’s) won’t be acceptable. And also the buyers will with time bear the cost of neglecting the quality from the start and – learning from their mistakes – will insist on it in their next project. Which is great news for us, because we already have the knowledge and practices in place to deliver web apps that are also good software. In the meantime we have to keep on educating people that even in web apps quality counts.

I have been using Twitter for a couple of months now. It is a useful tool to follow people & trends. And a nice toy at times. But there is one thing about it that spoils the experience: it’s just one huge ad.

This tool supposedly invented to help people share what they are doing with friends has become one huge fest of shameless self promotion, a true 21st century global vanity fair. This self promotion seems to be centered on one thing: being “cool”. Maybe it’s the people I follow but in many of the tweets (that are not re-tweets and links) I see a disproportionate amount of words like “great”, “awesome”, “exciting”, “kick ass” and – of course – “cool”. All those messages paint an image that is reflecting our current culture as we already see it through mass media’s twisted lens: everyone is doing something exciting on groundbreaking projects in fancy offices of superb companies, then “chills out” at chic clubs or doing some “crazy” activities. Then goes back again to their “cool” work. Everyone is young, physically attractive, smart and interesting. Everything is “cool”.

This image is not new. It has been the backdrop for most advertising for decades, as vendors promised people entry into the paradise on Earth if they buy a pack of coffee or a fancy car. The change here is that before it was just the advertisers pushing this type of BS out in paid slots and now it is the masses trying to market themselves, their jobs, their lives – and in that and through that live up to the ideal they all try to follow and seem to subscribe to: the un-holy grail of coolness.

The desire to follow this ideal – or anti-ideal rather – is not new, too. Ever since mass media – and especially all-persuasive moving images coming from TV screens – started to push this unreal model of life people wanted to live up to it. But before masses could only buy products advertised and try to imitate the behavior and look of the “stars”. Now they can join the band and broadcast their own version of the “brave new world”. On Twitter. And Facebook. And possibly other sites like it.

In this sense the “social media” both reflect the state of society and amplify the trends that shape it.

I hope this is just a shinny facade people put up there and deep inside they realize life is so much deeper and more meaningful. But some are so good at advertising I start to suspect they believe it.

I just devoted 1.2h of my time to view the Google Wave video in its entirety and I’m impressed. I wholeheartedly agree with glowing comments by Tim O’Rilley and Matt Asay. I’m not sure it will retire e-mail or traditional IMs, but it has a potential to change internal communication in companies. It is the first thing that has really a chance to challenge MS Exchange’s reign in corporate communication. Here is why.

First, it will be opensourced and based on an open protocol unlike most of Google products so far. Most corporations won’t use hosted web tools like Google Apps precisely because they value their data and want to retain control over it. They won’t use tools they can’t host on their own servers behind their own firewalls. So Wave is something that can get adopted where Google Apps (and all other hosted apps for that matter) didn’t stand a chance.

But even more important is the ability to integrate almost everything with Wave through extensions and bots. Here lies the real strength of Wave as a corporate solution.

Typically large businesses have already different specialized systems that are supporting their processes – CRMs, accounting systems, ERPs, logistics & order tracking etc. And much of the communication inside those companies revolves around same processes and data stored in those systems. Currently this communication goes in e-mails with data pasted in or attached as spreadsheets etc. With Wave it is easy to imagine integrating all those systems and creating a customized, comprehensive corporate communication environment. People would be able to talk and discuss invoices, orders, reports & other stuff right in their Wave inbox seeing up to date information fetched into context from other corporate systems.

Of course, it will take a while – Wave is still beta and protocol to connect different Wave servers is still being developed. There are issues to fix before Wave will be used and then it will take a push from both corporate IT and companies like us to create all the integration, build extensions, robots etc. But we are definitely seeing something very interesting here. And we are eager to deploy it.

Tech Crunch claims repeatedly that Last.fm passed user data on to notorious extortion syndicate theRIAA has become. Their reports are based on unnamed sources, but Last.fm’s strong denial strengthens the suspicion that it may be indeed true (Prince Gorchakov, Russian foreign minister in the 19th century used to say that he doesn’t believe rumors that have not been strongly denied).

The prospect of RIAA thugs laying their hands on records of what all Last.fm users listened to over – in many cases – years is something that should serve as a wake up call for all of us. Even if it did not happen yet there is no guarantee it won’t happen in the future. Actually, one can be sure that one day it will. As we move more of our data to the notorious “cloud” – that is machines we have no control over at all – we basically make it available to the highest bidder. And consequently we loose some degree of control over our lives and privacy.

Conversely, the corporations operating the “cloud” get more and more insight into our lives, views, relationships, material possessions and even thoughts. Then they sell it to advertisers, governments and – as it turns out – RIAA thugs. This is, basically, why all those services are free to use. The users pay for them not with currency but with their data they feed into them.

And Last.fm is just the tip of the iceberg. Think of Google. If you use the whole range of their compellingly simple and easy to use services then they can corelate your e-mails kept in GMail with your photos kept in Picasa, with your conversations on Groups, your blog posts on their blogging engine, your documents and spreadsheets in their Docs and even your browsing history if you use their browser toolbar or Chromium browser. And they don’t have to manually browse through all that to use it – on the contrary, they can algorithmically process all this data and single out people that match a certain profile. Or detect trends. Or predict events. Or map people’s connections.

There is a book out now called “The Numerati” written by Stephen Baker that describes in terms understandable for a layman what can be algorithmically derived from all the data we leave about us. It is all great for marketers for pinpointing their advertising Google and others live off. But it’s also great for secret police and others who want to single out people based on their views or associations. And great for governments who want to know about trends in a population even before they are publicly expressed.

You think Google won’t work hand in hand with the government if asked, that they would resist to protect your privacy? Don’t delude yourself – they will do it the instant some types from NSA or another gov agency walk into their offices. And they even do it publicly now. Yeah, I know, it is just for predicting flu outbreaks and who would object to something this good etc. But same technology can be used for detecting and pinpointing, say, tax dissent (like tea parties). Or map support for anti-Obama politicians – especially given Google’s political views being in line with the current’s administration.

Since I have been using GMail for some time now I think Google knows more about me than any of the people close to me in the real life. I should have moved away from GMail already – the reason I didn’t do it yet is that using their services is so easy and in our busy lives there is little space left for setting up private servers, installing spam-fighting software etc. But the more I see and realize what is going on the more I’m compelled to do this.

For now I’ve uninstalled the Last.fm’s scrobbler – and so should you.