Rodrigo De Castro

Absolute Consistency

2012-01-16T21:24:00.000-08:00

Very often I have a hard time explaining to people not very familiar with distributed systems how hard it is to provide consistency guarantees in face of node failures and network partitions.

Today I read an emails posted to Riak users list that gives good examples on why this is a non-trivial problem:
http://lists.basho.com/pipermail/riak-users_lists.basho.com/2012-January/007157.html

Definitely worth reading if you're into distributed systems.

The parable of the sadhu

2011-12-28T19:53:00.000-08:00

Yesterday I read this Harvard Business Review called "The parable of the sadhu", by Bowen McCoy.

This is a very popular HBR article where the author faces a moral dilemma in Nepal. When climbing the mountains, his group finds an Indian holy man - a sadhu - lying on the ice, suffering from hypothermia. The dilemma between taking or not care of the man and their own needs ends when they give him some aid and comfort, but carry and leave him close to a hut, not knowing for sure whether he makes to the hut and, for that matter, if he lived after that.

The discussion that comes up after that is what is limit of their responsibility in a situation like that.

"'Where, inyour opinion', I asked, 'is the limit of our responsibility in a situation likethis? We had our own well-being to worry about.'"

And after analyzing the situation, the author understand that...

"One of our problems was that asa group we had no process for developing a consensus. We had no sense ofpurpose or plan. […] Becausethe group did not have a set of preconditions that could guide its action to anacceptable resolution, we reacted instinctively as individuals. […] Wehad no leader with whom we could all identify and in whose purpose webelieved."

I don't to give away the entire article, but I must quote these paragraphs that summarize the lesson:

"Individualswho operate from a thoughtful set of personal values provide the foundation fora corporate culture. A corporate tradition that encourages freedom of inquiry,supports personal values, and reinforces a focused sense of direction can fulfillthe need to combine individuality with the prosperity and success of the group.Without such corporate support, the individual is lost."

"That is thelesson of the sadhu. In a complex corporate situation, the individual requiresand deserves the support of the group. When people cannot find such support intheir organizations, they don't know how to act. If such support is forthcoming,a person has a stake in the success of the group and can add much to theprocess of establishing and maintaining a corporate culture. Management'schallenge is to be sensitive to individual needs, to shape them, and to directand focus them for the benefit of the group as a whole."

It is interesting to analyze our own corporations and if we provide the foundation for a corporate culture, if any. Do we encourage freedom of inquiry? Do we support personal values? Do we have a focused sense of direction? Oftentimes I think that "the individual is lost" in many corporate environments, like the author says above. And this is harmful to the company.

PS: Photo from Flickr (Creative Commons).

Java: how can a 1Gbit/s attack keep up to 100K i7 CPUs busy?

2011-12-28T14:55:00.000-08:00

This is a hash table vulnerability found in many web application platforms, like PHP, ASP.NET, Ruby, and Java (but not only Java). In one of the cases, 1Gbit/s can keep up to 1 million CPUs busy!

Microsoft has already posted an advisory on this issue. The video has more information about other platforms.

The video below was published today and contains more details on the vulnerability. It is quite interesting, especially if you're into security.

And here you can find a blog post with more details on the talk:

http://cryptanalysis.eu/blog/2011/12/28/effective-dos-attacks-against-web-application-plattforms-hashdos/

StackOverflow clone with RavenDB

2011-12-19T21:07:00.000-08:00

Today I watched the video below on how to build a StackOverflow with RavenDB. This was my first video on a document store database, and it was amazing to show show interesting and easy it is to use it for a site like StackOverflow.
I was surprised to know that RavenDB supports transactions and by its integration with Linq. Although it's a .NET solution only and requires a commercial license if you're not working on an open source project, it was good to start learning the benefits of a document store.
It also supports full text search, which is implemented under the hood using Lucene.NET.

If you're interested in getting your StackOverflow running, do the following:

Download RavenDB from RavenDB website
Extract RavenDB into a directory and run Server\Raven.Server.exe
Download RavenOverflow from its GitHub repository
Extract RavenOverflow in a directory and open RavenOverflow.sln in Visual Studio
Right-click on "RavenOverflow.Web" and click on "Set as StartUp Project"
Hit F5 in Visual Studio

The original post by the presenter can be found here.

DNS domain names: 253 or 255 bytes/octets?

2011-12-09T17:55:00.001-08:00

The question of whether DNS domain names are 253 or 255 is something that is hard to find a good confirmation on, but I hope to be able to provide the answer here in this post.

Let's start taking a look at some RFCs:

"Host software MUST handle host names of up to 63 characters and SHOULD handle host names of up to 255 characters." http://www.ietf.org/rfc/rfc1123.txt
"To simplify implementations, the total length of a domain name (i.e., label octets and label length octets) is restricted to 255 octets or less." http://www.ietf.org/rfc/rfc1035.txt
"A full domain name is limited to 255 octets (including the separators)." http://tools.ietf.org/rfc/rfc2181.txt

So it seems that the domain names should be up to 255 octets, right? That is not what Wikipedia says:

"The full domain name may not exceed a total length of 253 characters" http://en.wikipedia.org/wiki/Domain_name

There is even a long discussion on Wikipedia about the right value here.

And even an RFC mentions 253 octets:

"When the result of macro expansion is used in a domain name query, if the expanded domain name exceeds 253 characters (the maximum length of a domain name) [...]" http://www.ietf.org/rfc/rfc4408.txt

Then you start playing with Microsoft DNS or BIND, and tools on both Windows and Linux and see some interesting behaviors. For instance, nslookup on Windows times out when domain name is 255 characters against a BIND server. The version on Linux, however, you get an explicit error for anything longer than 253 characters (ASCII, hence 253 octets):

host <255-char domain name>
<255-char domain name> is not a legal name (ran out of space)
The answer lies actually in the good old RFC 1035 - thanks to a colleague for finding this definitive answer:

"Each label is represented as a one octet length field followed by that number of octets. Since every domain name ends with the null label of the root, a domain name is terminated by a length byte of zero." http://www.ietf.org/rfc/rfc1035.txt

The answer is that, over the wire, the domain name uses 255 octets. The first one is used to indicate the length, and there's a last byte that is a terminator. So what is left for the actual domain name is 253 octets - which can represent different numbers of characters depending on your domain.

Microsoft Visual Studio Tips and Tricks

2011-12-04T21:46:00.001-08:00

As I am trying to become more productive using Visual Studio - even after some months at Microsoft, I still think I am more productive with Eclipse, I came across this good TechEd talk on Channel 9 on tips and tricks that taught me a bunch of nice things about Visual Studio:

http://channel9.msdn.com/Events/TechEd/NorthAmerica/2011/DEV305

I hope it helps you too.

Visual Studio add-ins for pasting XML

2011-12-03T16:16:00.001-08:00

I compiled and installed a couple of Visual Studio add-insthat I thought quite useful for those dealing with XML, in particular pasting them into VS.

SmartPaster 2010

Allow you to past XML as comments, strings, and as StringBuilder – escaping the text as necessary!

In my case, I had to paste a long XML from my API document and did not want to escape everything.

This is the source site: http://smartpaster2010.codeplex.com/

o Binaryversion did not load in Visual Studio 2010 64-bit (exception was thrown), so I recompiled it and it worked just fine.

Paste XML as Type

This add-in is really cool for REST APIs. Youcopy a XML (like from our MSDN documentation) and it paste it as a serializabletype. All classes are automatically generated.

It was included in Microsoft WCF REST Starter Kit Preview2: http://aspnet.codeplex.com/releases/view/24644

o Itdoesn’t have a binary version, so I compiled this add-in and made it availabletoo

Silverlight, cross-domain issues, and self-signed certificates

2011-11-20T13:18:00.001-08:00

I've been meaning to post this for quite sometime now as I haven't seen others with exactly the same issue.First, some context: when running a Silverlight application, it has some special security measures in place to avoid Cross-Site Request Forgery (CSRF). By default, Silverlight only allows site-of-origin communication - for instance, "http://blog.sacaluta.com/test.aspx" will be able to access "http://blog.sacaluta.com/myservice.svc", but not "http://www.example.com". In order to allow more than site-of-origin communication, a service owner must have a clientaccesspolicy.xml file in the root configuring which domains are allowed to access that service. If you're interested, this is explained in greater detail on this MSDN site.

The issue I ran into is that I had a Silverlight application and also a service, both running locally. My service had a proper clientaccesspolicy.xml configured to allow access from anywhere. And still my Silverlight would fail with the message:

"An error occurred while trying to make a request to URI 'https://MYDOMAIN/MYSERVICE.svc'. This could be due to attempting to access a service in a cross-domain way without a proper cross-domain policy in place, or a policy that is unsuitable for SOAP services. You may need to contact the owner of the service to publish a cross-domain policy file and to ensure it allows SOAP-related HTTP headers to be sent. This error may also be caused by using internal types in the web service proxy without using the InternalsVisibleToAttribute attribute. Please see the inner exception for more details. ---> System.Security.SecurityException ---> System.Security.SecurityException: Security error..."

After debugging the issue further, the problem was that my service had only a secure endpoint (SSL) and its certificate was self-signed (or did not match the domain, can't remember now). In that case, my Silverlight application would not download the service's clientaccesspolicy.xml and therefore declined access to it. Since I was running code within another larger application that I did not have control of, I did not investigate further whether one can configure to allow self-signed or mismatched certificates to be accepted during development. (In case you know if these are possible, please let me know!)

How did I get it solved? If you're running in Internet Explorer:

Before loading your Silverlight application, first access the clientaccesspolicy.xml file. IE will alert about being self-signed or mismatched cert, but you can opt to proceed with it.
In the same tab, access then your Silverlight application. It will be able to access your clientaccesspolicy.xml at that point, and the call will go through.

Simple trick, and effective. I'd love to know if other browsers work the same. By the way, this was tested in Internet Explorer 9.

ReadyNas, WebDav, and "Method Not Allowed"

2011-11-20T12:55:00.000-08:00

I have a ReadyNas Duo network-attached storage, which I access via WebDAV only due to some permission conflicts if I use different protocols to write files to it. Given that Windows does not support WebDav properly in my case, I installed a WebDav client called BitKinex. I configured it to point to my share and guess what: "HTTP: Method Not Allowed (/)" error. This is the dialog:The problem is that BitKinex, by default, points to the root of your server. In ReadyNas case, it has different shares, and you must point to the right share to get it fixed. In order to do that, right click on the WebDav connection and select "Properties". Go to "Site Map" and update "/" with your share name (in my case, "/documents").Then it works fine.

.NET: do not use System.Uri for domain validation

2011-11-17T22:04:00.000-08:00

Last time I talked about System.Uri, I was talking about a bug that prevents trailing dots from being used for REST resources. Now the issue is different: how about relying on System.Uri for domain validation?

It's not uncommon to see System.Uri being used to validate an input that is supposed to be a domain name. I've seen code like this trying to validate domains:

public static bool IsDomainValid(string name)
{
    try
    {
        new Uri("http://" + name);
    }
    catch (UriFormatException)
    {
        return false;
    }
}

Or, besides relying on UriFormatException or on the the Host property, something like this:

public static bool IsDomainValid(string domainName)
{
    try
    {
        if (StringComparer.OrdinalIgnoreCase.Equals(new Uri("http://" + domainName).Host, domainName))
        {
            return true;
        }

        return false;
    }
    catch (UriFormatException)
    {
        return false;
    }
}

Preliminary tests with this code shows that completely wrong domains (like 3@#@@.com) are rejected, so it seems to be a great code. And the best is that we don't have to write any domain validation ourselves.

Now, what about the following domains?

--------.com
-test.com
test-.com

They are all considered valid according to System.Uri(). However, according RFC 1035 or RFC 1123, they are not. According to RFC 1035, not even a digit only domain (like 999.com) is valid, but System.Uri() is fine with all of them.

I played with some of the internal flags and it seems that, if you use E_HostNotCanonical (256), it starts rejecting some of these invalid domain names, but I really couldn't understand the rules it follows. And since there are different RFCs and different interpretations, it would be really hard for System.Uri() to do a precise validation unless one passed the type RFC that the domain is expected to be compliant with.

At the end of the day, you're better off understanding the RFC you want to comply with and implementing the proper regular expression for that. In my case, I wanted it to be compatible with RFC 1123, so this is the regular expression I started with:

"^(?![0-9]+$)(?!-)[a-zA-Z0-9-]{1,63}(?<!-)$

And then relaxed it to the following after learning that digits only domains were accepted by RFC 1123 (there are multiple interpretations, but I read the RFC and was convinced that it was fine).

"^(?!-)[a-zA-Z0-9-]{1,63}(?<!-)$"

This is the regular expression per domain label (text between the dots). It does not apply to the rightmost label as it must not start with a digit - in order to differentiate a domain name from an IP address.

Also, this regular expression requires an explicit check that the entire domain is less or equal to 255 characters.

Regular expressions: backtracking can kill your performance

2011-11-16T20:56:00.000-08:00

Or why you should learn atomic grouping...

After last post on turning off useless backtracking by using atomic grouping, I kept on reading the Regular Expressions Cookbook and ran another experiment to validate the performance difference. Let's start with the results:

Normal Regex - # of loops: 1000, # of matches: 0, time (ms): 66149
Atomic Grouping - # of loops: 1000, # of matches: 0, time (ms): 11196

Note that a normal regex was 6 times slower than atomic grouping to fail to match.

The example from the book is a regex to match a well-formed html page. I saved a Wikipedia page, made a few adjustments to the program used in the last post (e.g. to read the html contents from file), and ran the tests.

The regular expression used is:

<html>(?>.*?<head>)(?>.*?<title>)(?>.*?</title>)(?>.*?</head>)(?>.*?<body[^>]*>)(?>.*?</body>).*?</html>

Every test I ran with this regex and with the version without atomic grouping, I can see the normal regex being 6 times slower. If you want to know more about using atomic grouping or a regular regex, please read my last post.

And this difference was found without setting the regex to be compiled. After setting this flag, these are the values I get:

Normal Regex - # of loops: 1000, # of matches: 0, time (ms): 49319
Atomic Grouping - # of loops: 1000, # of matches: 0, time (ms): 9471

Still a pretty significant change between normal regex and atomic grouping.

Regular expressions: turning off useless backtracking

2011-11-15T17:55:00.000-08:00

Last time I mentioned the nice feature on how to change quantifier to be greedy or lazy, and then help you match what you really want. This time, it is how to make your regular expression more efficient. First, we need to remember last post where greedy or lazy quantifier change how backtracking works. Sometimes backtracking just doesn’t make sense. Look at this example:

\b\d+\b

It is supposed to match integers at word boundaries (\b means boundaries). At first, it may be a bit hard to understand, but backtracking is unnecessary here. For instance, try to run this regular expression on an example: .

789abcdef654 123

At the point when the regular expression fails (when it checks that “a” is a word boundary), it doesn’t make sense to start backtracking to see if 9 is a word boundary (or 8, for that matter). We should just go ahead and move on to the next token.

This is where it pays off to know well the tool you’re using. Different flavors of regular expressions offer ways to avoid keep backtracking positions, so when a match fails, it just moves on. In case of Java and .NET, both support atomic grouping. .

\b(?>\d+)\b

Atomic grouping here is represented by “(?>)”. When the regular expression engine leaves the group, all backtracking positions are lost, so a failed match will not have any recourse other than moving on to the next characters to find further matches. It will give up on the current context.

In the example above, when it matches 789, it will leave the atomic group, so when \b fails to be matched there are no backtracking positions to be tried. From this quick analysis, we see that we avoid a lot of extra computation by avoid these useless backtracking.

The question about saving time is how much we actually save here. I wrote a test code to benchmark these different options and verify whether we are talking about any substantial savings or not.

static void Main(string[] args)
{
    TestAtomicGrouping(1000);
    TestAtomicGrouping(10000);
    TestAtomicGrouping(100000);
}
 
static void TestAtomicGrouping(int numLoops)
{
    Regex regex1 = new Regex(@"\b\d+\b", RegexOptions.Compiled);
    Regex regex2 = new Regex(@"\b(?>\d+)\b", RegexOptions.Compiled);
 
    StringBuilder sb = new StringBuilder();
    for (int i = 0; i < 100; i++)
        sb.Append('1');
    sb.Append('a');
    sb.Append(' ');
    for (int i = 0; i < 100; i++)
        sb.Append('1');
 
    string testString = sb.ToString();
    int firstMatchCount = 0;
    DateTime start = DateTime.Now;
    for (int i = 0; i < numLoops; i++)
    {
        if (regex1.IsMatch(testString))
            firstMatchCount++;
    }
    TimeSpan firstTest = DateTime.Now - start;
 
    start = DateTime.Now;
    int secondMatchCount = 0;
    for (int i = 0; i < numLoops; i++)
    {
        if (regex2.IsMatch(testString))
            secondMatchCount++;
    }
    TimeSpan secondTest = DateTime.Now - start;
 
    Console.WriteLine("Normal Regex - # of loops: {0}, # of matches: {1}, time (ms): {2}", 
        numLoops, firstMatchCount, firstTest.TotalMilliseconds);
    Console.WriteLine("Atomic Grouping - # of loops: {0}, # of matches: {1}, time (ms): {2}", 
        numLoops, secondMatchCount, secondTest.TotalMilliseconds);
}

Now we need to see results:

Normal Regex - # of loops: 1000, # of matches: 1000, time (ms): 48.0028
Atomic Grouping - # of loops: 1000, # of matches: 1000, time (ms): 30.0017

Normal Regex - # of loops: 10000, # of matches: 10000, time (ms): 386.0221
Atomic Grouping - # of loops: 10000, # of matches: 10000, time (ms): 239.0137

Normal Regex - # of loops: 100000, # of matches: 100000, time (ms): 3145.1799
Atomic Grouping - # of loops: 100000, # of matches: 100000, time (ms): 2079.1189

So, at the end of the day, getting rid of backtracking can be quite significant if you’re matching this regular expression quite often. In this test, we could save 32% or more of the matching time just by “turning off” backtracking with atomic grouping.

Regular expressions: greedy vs. lazy quantifier

2011-11-14T21:15:00.001-08:00

I am reading the great book Regular Expressions Cookbook after seeing today a few things that I did not about regular expressions. I will get to the interesting regular expression I had to work on in a future post, but for now I will share something I found quite interesting: greedy and lazy quantifiers.

Let's start by trying to match a paragraph in HTML. A paragraph is typically surrounded by <p> and </p>. So, I would write regular expression as:


<p>.*</p>

This should take care of matching the paragraph, right? Partially right. If you have a long HTML, with multiple paragraphs, this will match from the first paragraph start (<p>) to the very last paragraph end (</p>). This "*" is actually called a greedy quantifier.

If you want to make it behave differently, you will want to use what is called lazy quantifier. This is just the regular question mark placed after another quantifier. Note that, if question mark is placed after a regex token, it means "zero or once". This is not what we are talking about here - question mark here after a quantifier means that it changes the quantifier behavior.


<p>.*?</p>

In the example above, it matches the first paragraph only, not the entire text.

Under the covers, the regular expression engine uses backtracking to match the expression. For a greedy quantifier, it eats up all the content that matches the current regular expression and then moves to the next token. In the case of the paragraph matching example, it reads the entire text until the very end. Then it moves on to the next token (in this case <) - and since it fails as the document finished, it back tracks, and tries to match < again. It keeps going back each character until it matches.

For the lazy quantifier, it repeats as few times as it can, moving to the next regex token (here <). If the token is not matched, then it back tracks and moves forward another time, seeing if the token is matched.

I was happy to learn this, as I had always asked myself how to control this behavior. And quite interesting to understand the regular expression engine behavior, what can come in handy. Just be careful when using the question mark. As said above, it can serve two purposes depending on where it's placed.

Moving Channel 9 to Azure: good design principles

2011-11-13T18:22:00.000-08:00

Today I read a great article on Microsoft Channel 9 moving to Azure talking about the sound design principles in place and lessons the Channel 9 team shares about how to move a web site to run in the cloud.

One of the things that caught my attention is to see a Microsoft project using a distributed cache fleet running Memcache. Using a caching layer is definitely the right thing to do in many cases to make the site more scalable. I wonder why they haven't used Windows Azure AppFabric Caching. And also, after working on Amazon Elasticache before joining Azure, I'd be curious how they monitor their Memcache instances.

I was very glad to see modular code, coding to interfaces, and mostly dependency injection being used. While dependency injection is pretty popular in the Java world, it's still not as popular for many Microsoft developer. They mention dependency injection being used "for testing purposes but also to isolate you from very specific platform details". Very well done.

Division of labor is a right principle for environments where machines are not reliable. This is proper mindset about machines in the cloud: "In practice they tend to run a very long time, but you can’t depend on that fact." And breaking down the tasks and using worker roles to pick them up, connecting them via queues, seems a smart strategy (assuming you have proper monitoring on these queue depths in place). In particular, I like the fact that the Channel 9 did not just thought that instances run for a long time and released an architecture based on that, so potential problems could be addressed in the future. Unfortunately I've seen a lot of people with this mindset, and Channel 9 did very well here.

From the article, though, the only thing that could have been done better was to think about database sharding. Although SQL Azure will provide Federation, there are many things that service owners need to think about: what the database partition key will be, what queries will need to go over partition and impact potential scalability, what queries will need to be federated, etc. I am not very familiar with SQL Azure Federations and don't know if it will repartition automatically hot partitions, but if it doesn't, that's another task service owners need to prepare for. With all that said, you don't need to shard right away, but you need to think of that before you service version 1 goes out, otherwise scaling can be a major headache - and if you can't afford downtime, then that can be an almost impossible task to accomplish in some cases.

All that said, I was very glad to read about their work and their sharing the architecture and lessons publicly.

Link to the InfoQ article:
http://www.infoq.com/articles/Channel-9-Azure

Why high code coverage is not enough

2011-11-12T14:50:00.000-08:00

Managers typically like high "code coverage", and oftentimes think that this means that the code quality is good. I agree that low code coverage definitely means that one doesn't have enough unit tests, but high code coverage may not mean much either. It's required but not sufficient. To prove this, let's take a look at one example.

Once upon a time, I saw the following regular expression in a production code. I will write it in C#, but the language or platform doesn't mean much.


public static bool IsValidIp(string ipAddress)
{
    return new Regex(@"^([0-2]?[0-5]?[0-5]\.){3}[0-2]?[0-5]?[0-5]$").IsMatch(ipAddress);
}

Let's say now that you have one unit test to make sure that your "boundary case" is accepted.


Assert.IsTrue(IsValidIp("255.255.255.255"));

Now you are happy, get the code checked in, and brag that you have 100% code coverage for that IsValidIp method. And so what? A simple "192.168.1.1" IP address is not considered a valid address. Completely buggy code, but 100% code coverage.

That is why managers that really understand what is being developed and have the chance to spend time looking at the code can make a total difference in the final product's quality.

Note: on the case above, it's amazing that the developer did not Google'd for the right regular expression for Ip validation, did not write data-driven unit tests to make sure different Ips are being written, and that code reviewers did not review it properly.

On Zynga and its "give back stock or get fired" story

2011-11-12T13:33:00.000-08:00

This week it's been all over the place the news about Zynga CEO and its executives demanding that you either give back the not-yet vested stocks or face termination. One of the ways that startup companies have to lure employees into taking the risk is to offer equity - that's the currency for startups for the risk taking as well as for offering lower salaries and demanding long hours. Once you offer this equity, I think companies must honor their contracts. At the same time, this can tell you a lot about the company, its value, and whether other people will want to join them in the future.

However, Zynga is correct in trying to be meritocratic. Those who contributed more should have a bigger piece than those who are around but did not contribute much to the company's success. A good compromise would be to have policies stating that the stock grants are dependent on your performance evaluation. Some sort of multiplier would be applied in this case - if you reach the expectations, you will get 1x your stock grants, if you're a rock star, you could get up to Nx (e.g. 2x), and if you're an underperformer, you may get nothing at all. That is much more fair than just demanding stocks back or threat with termination. Of course no system is entirely fair, as it can subjective and politics always play a part there, but it's better than lure people into thinking that they will get their stocks if they stick around long nothing and then fire them primarily for this reason.

On other hand, though, how many other companies may be contemplating or actually firing people with unvested stocks to accomplish the same goal? At least one can say that Zynga was transparent on the reason why they would fire its employees. But this is the kind of transparency that one doesn't see quite often because it lowers all employee's morale and the company's moral values are questioned. A company that does the same, but not that openly, seems to be much better off as employees tend to still believe that the company abide by its moral principles and its worth putting in all the effort to make the company grow.

Coding guidelines and readability

2011-11-10T21:02:00.000-08:00

The more experience I get in the industry, more and more I value great developers that know how to distinguish great from good code. And I am glad that I had a fantastic experience reading and working with the Linux kernel - most of the code I had seen at the time falls into the category of great code.

There are many aspects of a great developer that one can think of, but I'd like to focus here on code readability and maintainability. First, one quite important distinction, especially for those used to following strict "code guidelines". Code readability is not so much about where you place the brackets, or any style that can be verified by a static analysis tool. These are usually what don't really matter much in my opinion.

The real readability is about the art in writing your code, not its science. Things like how to properly break your code into methods, how to name variables, classes, and methods, how to use spaces properly, how to make proper use of the fixed-width characters or align/indent code.

In my opinion, great engineers know that the code must not just work, but it must be a work of art. Something that you and others can read in the future and maintain. It is NOT just about getting it work. A lot of code I read works, but they are not readable, not elegant, and oftentimes not efficient at all.

Digressing a bit, I miss systems where you actually need to get the max performance out of a system. That seemed to require better engineers than nowadays. Now, with web services and cloud computing, oftentimes one doesn't care much about performance as you can always get a faster box to run your code. After working for two cloud providers, I see that this can specially happen with those providing these cloud services. I wonder whether these engineers who just throw more CPU power or memory at a problem actually ever thought of the cost of running a service and that this is one of the things factored in that will make the difference between being profitable or not. Or, on a more philosophical side, if they really feel pride of the engineering in the code they write.

But back to coding guidelines, I really recommend that you read the article below (published in the ACM Queue magazine) if you want to get better at that. It has a few tables that you should print out and put it up on the wall where you can peek when writing your code.

Coding Guidelines: Finding the Art in the Science

http://queue.acm.org/detail.cfm?id=2063168

Today I also came across a new O'Reilly book on the topic called "The Art of Readable Code", which seems to be quite interesting and probably delve into this topic in much greater detail.

The Art of Readable Code

http://shop.oreilly.com/product/9780596802301.do

URI segments, dots, REST, and .NET bug

2011-11-10T19:57:00.000-08:00

These days I learned about a bug in the System.Uri() class that would strip leading dots from URI segments. See an example:

http://host/test.../abc

becomes

http://host/test/abc

That happens if your client or your server is .NET. If your client believes you support the URL RFC correctly, it may send the request with trailing dots, and when it gets to your code, these dots are gone.

The implication is that, if this URI segment is actually a resource name, you may be in trouble. Let me show you a concrete example:

Resource is created by posting to URL: http://host/addresses. At this point, the resource name is passed in the payload and your service will correctly accept these trailing dots. For example, let's say we create an address named "home." So far, so good.
User tries to perform a REST operation on this resource. It could be something as simple as a GET on http://host/addresses/home. (dot included)
In the case you have a .NET client, the request will go out as http://host/addresses/home (no dot). Of course your server will return the wrong data or an error (like 404 - not found)
In case you have a non-.NET client, the request will go out correctly, but if your server is .NET-based, then you may have an issue. For instance, a WCF REST service will have this resource name parsed as "home" (no dots), which will also return the wrong data or an error.

The consequence is that, because of that, your .NET REST service should not allow dots. At least trailing dots. However, allowing dots everywhere but at the end is not desirable and quite possibly you will forbid dots altogether.

There's a workaround for this issue if you control both client and server code. However, in case your customers are generating client proxies, then you must document what they need to do.

MethodInfo getSyntax = typeof(UriParser).GetMethod("GetSyntax", System.Reflection.BindingFlags.Static | System.Reflection.BindingFlags.NonPublic);
FieldInfo flagsField = typeof(UriParser).GetField("m_Flags", System.Reflection.BindingFlags.Instance | System.Reflection.BindingFlags.NonPublic);
if (getSyntax != null && flagsField != null)
{
  foreach (string scheme in new[] { "http", "https" })
  {
      UriParser parser = (UriParser)getSyntax.Invoke(null, new object[] { scheme });
      if (parser != null)
      {
          int flagsValue = (int)flagsField.GetValue(parser);
          // Clear the CanonicalizeAsFilePath attribute
          if ((flagsValue & 0x1000000) != 0)
              flagsField.SetValue(parser, flagsValue & ~0x1000000);
      }
  }
}

The code above clears a flag that is set to canonicalize an URL as a file path. Yes, all URLs are thought to be Windows file locations.

Unfortunately this bug is known since 2008, but has never made into a .NET release. It is marked as fixed, but as of .NET 4 we are still waiting for the fix to be released.

Here you can find more details about this issue:

https://connect.microsoft.com/VisualStudio/feedback/details/386695/system-ur

Internet Explorer vs. Chrome: Recover Session

2011-11-10T19:44:00.000-08:00

Working at Microsoft, I use Internet Explorer, but when I want to separate some sessions, I run Chrome at the same time. And I wanted to tell you about an experience I had last week.

Last weekend, after plugging in my Garmin heart monitor to download my bike run, my PC just rebooted (that happened more than once with this monitor). After coming back up, I was expecting that Chrome would recover my session, but not Internet Explorer (from what I recall, IE hasn't been very good at this). And guess what? Chrome did not recover one single of my tabs, while IE recovered the entire session correctly. That was clearly unexpected and a glad surprise.

In the end, I had to go over Chrome's history to recover some tabs that I hadn't read it, what I could avoid with IE.

WCF + REST + PUT/DELETE = 405 (Method not allowed)

2011-09-26T19:19:00.001-07:00

In short: uninstall WebDAV module from IIS.

If you are getting an 405 (Method not allowed) error back when hitting your WCF REST endpoint, you should check your IIS configuration to make sure you don't have WebDAV. In my case, WebDAV was intercepting requests and returning 405 for PUT and DELETE requests. Note, though, that it is not enough to disable WebDAV, you have to uninstall it.

The design of the Domain Name System

2011-09-25T20:33:00.000-07:00

Good posts on DNS, potential issues and limitations with it, and how to design applications to use DNS.

The design of the Domain Name System (Part I)

http://jl.ly/Internet/dnsdesign1.html

The design of the Domain Name System (Part II) - Exact and approximate name matching

http://jl.ly/Internet/dnsdesign2.html

The design of the Domain Name System (Part III) - Name structure and delegation

http://jl.ly/Internet/dnsdesign3.html

The design of the Domain Name System (Part IV) - Global consistency

http://jl.ly/Internet/dnsdesign4.html

The design of the Domain Name System (Part V) - Large data

http://jl.ly/Internet/dnsdesign5.html

The design of the Domain Name System (Part VI) - Overloaded record types

http://jl.ly/Internet/dnsdesign6.html

The design of the Domain Name System (Part VII) - Related names are not related

http://jl.ly/Internet/dnsdesign7.html

The design of the Domain Name System (Part VIII) - Names Outside the DNS

http://jl.ly/Internet/dnsdesign8.html

Flickr interestingness downloader in Ruby

2011-08-06T15:24:00.001-07:00

And this time this is the Ruby code using Flickraw gem to download large size versions of Flickr interesting photos.


require 'flickraw'

FlickRaw.api_key="api_key"
FlickRaw.shared_secret="shared_secret"

photos = flickr.interestingness.getList( :per_page => 500 ) 

frob = flickr.auth.getFrob
auth_url = FlickRaw.auth_url :frob => frob, :perms => 'read'
  
photos.each do |pic| 
   photo_info = flickr.photos.getInfo(:photo_id => pic.id) 
   photo_url = FlickRaw.url_b(photo_info) 
  
   puts "Downloading #{photo_url}"
   
   open("flickr/" + pic.id + ".jpg", "wb") { |file|
    file.write(Net::HTTP.get_response(URI.parse(photo_url)).body)
   }
end

S3 file bucket downloader in Ruby

2011-08-06T13:27:00.000-07:00

Today I wanted to download files from a website that I happened to find out that stored all files in S3. By accessing the website root, I realized that it was just the response of a S3 ListBucket API call. For instance:


<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
   <Name>foo.com</Name>
   <Prefix/>
   <Marker/>
   <MaxKeys>1000</MaxKeys>
   <IsTruncated>true</IsTruncated>
   <Contents>
      <Key>file/1</Key>
      <LastModified>2011-06-09T06:29:02.000Z</LastModified>
      <ETag>"5cb3930839817ff4a5c1ddf08e3fea1e"</ETag>
      <Size>1440231</Size>
      <StorageClass>STANDARD</StorageClass>
   </Contents>
   <Contents>
      <Key>file/2</Key>
      <LastModified>2011-06-09T06:29:18.000Z</LastModified>
      <ETag>"96fdc94d14b6d9817f80ac1e9e2049b4"</ETag>
      <Size>1310</Size>
      <StorageClass>STANDARD</StorageClass>
   </Contents>
</ListBucketResult>

In order to download all files more quickly, I wrote the following Ruby program that downloads all files from this website, and I hope it can be useful for others:

require 'net/http'
require 'rexml/document'

baseurl = 'foo.com'

# get the XML data as a string
xml_data = Net::HTTP.get_response(URI.parse("http://" + baseurl)).body

# extract event information
doc = REXML::Document.new(xml_data)
titles = []
links = []
Net::HTTP.start(baseurl) do |http|
  doc.elements.each('ListBucketResult/Contents/Key') do |ele|
    puts "Downloading " + ele.text
    resp = http.get("/" + ele.text)
    open("images/" + ele.text.gsub("/", "_") + ".jpg", "wb") { |file|
      file.write(resp.body)
    }
  end
end
puts "Done"

link_to data-method delete not working in IE9?

2011-07-26T08:14:00.000-07:00

I was testing a Ruby on Rails 3 application with devise authentication library and to my surprise, link_to with data-method was not working in Internet Explorer 9 (IE9). In that case, the method was set to delete. It works perfectly in Chrome, though.

There are some bugs reported on that, but the solution I found was

Make my application depend on jquery

Edit your Gemfile and add:

gem 'jquery-rails', '>= 1.0.12'

Run rails generate to install jquery and get rid of other .js files (like prototype.js)

rails generate jquery:install

Run application

I did not see these instructions anywhere, just decided to follow what I had done in another project to test Ajax support for Ruby and it just worked. I hope it works for you.

By the way, I've seen several mentions of this, but since my Rails 3 already had it, I did not bother much. In your case, also make sure that your application.html.erb has these tags in your :

<%= javascript_include_tag :defaults %>

<%= csrf_meta_tag %>

Thoughts on "growing as a developer"

2011-07-23T20:15:00.000-07:00

I just read the following blog post written by Robert Bowen:

How to Grow as a Designer or Developer

These are the highlights:

We should always be moving forward, reaching for the next plateau
Find a way to grow, actively pursing opportunities of growth
Make sure:

Your schedule allows for that
You don't think you mastered the field
You think outside of your day-to-day job world that may have become stagnant and monotonous

Ways to grow:

Step outside of our comfort zones to try something new and experiment to guarantee to learn something new and push our skills to new heights
Study the work of those you admire
Keep up with the field and how it is evolving
Reach out to others for feedback
Collaborate on projects with others that will push you to challenge yourself
Be active in the community, like running a blog, contributing to blogs and discussions

Pay-off

This is a great post and reflects my motto of always keep getting better and pushing myself. I believe that, by getting better, you will grow in your field and, no matter what the current circumstances are, over time that will pay off.

Is it worthwhile?

I've started challenging myself in this regard by asking: is my motto actually something worth living by? This pay-off will definitely be worth in terms of personal satisfaction. Like one that has hobbies and personal projects, oftentimes these are for personal satisfaction more than for some special reward in the future.

Provided value

Other than personal projects, when you are an employee or a freelancer, or even an entrepreneur, this growth pays off to the extent that it provides value to those paying for your time, service, or product. And that is very important to realize sooner than later.

Multiplying effect

Even getting better technically, typically you hit a ceiling at some point, because no matter how much you can accomplish, you can accomplish only so much as an individual contributor. Unless you have a multiplying effect in the organization. Not that multiplying effect is something easy to define, and can be subject to politics and subjectivity, but the general idea is that on can make the entire organization better. Only doing that you can grown as an individual contributor, otherwise you will not grow beyond a certain point in your career.

Is getting better valued?

Besides that, another very important point is how much getting better is actually valued by the organization you are part of. If you are surrounded by people that do not see value in getting better - they may think that the current level is good enough, or just don't value improvement as long as things get done - you will grow frustrated as your growth will not be recognized and you may not even be potentially be rewarded at all by it. In some cases, you can be even penalized by that. This all can lead to potential stagnation and will require a major effort for one to keep sanity in such an environment. Notice that, if you are dealing with customers (as a freelancer or entrepreneur), it can be the same.

Address actual needs

The key is how you can leverage your getting better to provide more value - and find or create the environment that nurtures that mindset. You can make your program/code/design more efficient, more resilient, more secure, etc, and that is where your growth will manifest itself. In reality, the trick is to find the right set of people that really want more efficient, more resilient, more secure, etc, as oftentimes they think the current state of affairs is good enough. It just could be that the challenges faced by the company cannot be fixed or substantially improved by your skills, in which case the relationship can grow sour over time.

Being ahead of the curve

Eventually, if one keeps getting better, it becomes increasingly more difficult to find the right opportunities to leverage this knowledge, and this can be incredibly frustrating for the individual. The personal satisfaction is still there, but as we say that some people are much ahead of their times, some people are just ahead of the curve and must find how to apply their knowledge. Taken to the extreme, only a few places and few people in world may actually value these people, and spending their days doing the "wrong" thing or in the "wrong" place may prove to be a complete waste of time.

More than technical skills

As part of growing as developer, everybody should work on their interpersonal skills if these can be their weaknesses. I don't think that weaknesses must be necessarily completely fixed. One should build on their strengths with the following rule: don't let the weaknesses get in the way. In our field in particular, these weaknesses are typically related to soft skills, and bright people often can work on them. By doing that, nobody will dismiss great ideas or contributions on the basis of soft skills. As part of finding the right environment and right set of people to work with, improving on soft skills can be quite beneficial.

Steps to install MySQL2 GEM (Ruby on Rails) on Windows 7 (64 bit)

2011-07-23T14:26:00.000-07:00

I ran into many issues trying to get it compiled and installed on Windows 7 64-bit and wished I had an "apt-get" that would have fixed all these issues.

The trick here is: you need MySQL 32-bit to have Ruby on Rails MySQL2 gem compiling on Windows 7 64-bit. In short, this is what I had to do:

Install Ruby Development Kit in order to be able to compile C-bindings for Ruby

Error: The 'mysql2' native gem requires installed build tools.

Install MySQL Server 64-bit
Download MySQL Server 32-bit .zip file
Add MySQL 32-bit lib directory to PATH (or copy libmysql.dll to %RUBY_HOME%\bin)
Install MySQL2 2.0.6 GEM specifying --with-mysql-lib and --with-mysql-include options pointing to the 32-bit lib and include directories

gem install mysql2 -- '--with-mysql-lib="c:\Development\MySQL Server 5.5\lib\opt" --with-mysql-include="c:\Development\MySQL\MySQL Server 5.5\include"'

Install Rake 0.9.2

Error: uninitialized constant Rake::DSL

References:

Why use regular expression?

2011-07-22T12:40:00.000-07:00

I never posted code I've come across, but reading a random code today, I started wondering why one couldn't use regular expressions below to accomplish the same result - maybe I am just missing some good reason for that, but I thought it might be amusing to share:

string[] stringArray = str.Replace("Jan, ", "Jan ").Replace("Feb, ", "Feb ").Replace("Mar, ", "Mar ").Replace("Apr, ", "Apr ").Replace("May, ", "May ").Replace("Jun, ", "Jun ").Replace("Jul, ", "Jul ").Replace("Aug, ", "Aug ").Replace("Sep, ", "Sep ").Replace("Oct, ", "Oct ").Replace("Nov, ", "Nov ").Replace("Dec, ", "Dec ").Split(new[] { ',' });

The top 9+7 things every programmer or architect should know

2011-07-18T08:40:00.000-07:00

Great list of things programmer and architects should know:
http://www.javacodegeeks.com/2011/07/top-97-things-every-programmer-or.html

About learning to estimate, I recommend the following book:
http://www.amazon.com/How-Measure-Anything-Intangibles-Business/dp/0470539399/ref=sr_1_3?ie=UTF8&qid=1311003181&sr=8-3

These are three top things for me:
1. The Boy Scout Rule - Robert C. Martin (Uncle Bob)"You don’t have to make every module perfect before you check it in. You simply have to make it a little bit better than when you checked it out."

To be honest this is not something I have followed throughout my career, and although I certainly try improve code where I can, I never did it per check-in. I do however feel that it is an awesome principle and should be something that is actually part of a code review process. It is all to easy to just say:
"It was like that already"
"that nasty code was there for years, I am not going to touch it."
"It never had any tests"

I work in a corporate environment were applications often last for 4-10 years. If part of the process is always to just make something a little better, everything from deleting unused code to writing a single extra unit test, year after year... it will end up with saving a lot of people a lot of time and money.

This seems obvious, but not always we make it better when checking in. It takes a lot of diligence to do that - and agreement in a team environment on what better actually means.

4. Continuous Learning - Clint ShankThis is a very important topic, we are in a industry that is constantly growing, changing, shifting and as a programmer you need to be learning and improving yourself wherever you can. It's very easy to get into a comfort zone and just rest on your laurels, I did that for a couple years, and I do regret it now.
Things I am trying to do to keep up and would recommend:

Get a Kindle... then buy & read books.
Use Google Reader add the popular blogs and website RSS feeds for your specific field as well as a couple outside your field that interest you.
Start a blog, by putting my code and thoughts out there, I put in more effort knowing that it's going to visible than if I just wrote the code/article for myself. I also force myself to do 1 - 2 posts a week, ensuring that I must always find new content to learn about.
Join an open source community, we generally don't get to do enough "technical" development in our corporate environments.

Related to the first one: keep getting better - nothing will stop you. Build on your strengths, but don't let your weaknesses get in the way.

5. Record your rationale - Timothy HighThis is something that I feel is often neglected. Quite recently a project that I had been involved in for a long time hit the spotlight for all the wrong reasons: customer, user and management dissatisfaction. The first thing to be questioned was not the analysis, requirements, testing, management or expectations, but rather the architecture. Documentation discussing all the decisions, options looked at and reason for options taken would have been valuable. When things go fine, no one will even know about the document, but when things turn bad as they sometimes do having justification and documentation for all the major decisions will be a lifesaver.

That happened a lot in my experience. People will disagree, or some people will not have been involved in some discussions, and then you need to be able to articulate why some decisions were made. Most of the times new arguments and ideas were considered, but without recording your rationale, you may not be able to discuss them, especially in public forums like large meetings. I'd say that one should write the rationale and make sure it's in your head when the subject comes up - and obviously this assumes that you communicate effectively.

Publishing to Blogger from Microsoft OneNote

2011-07-16T11:08:00.000-07:00

I was trying to post something today and was wondering how it can be still so complicated to post and format something to a blog nowadays. One of the integration that I wanted is from OneNote, where I've been keeping many of my notes for quite sometime, to Blogger. And looking it up on the web, I found that it's already a reality and works very well - you may need to go over the formatting to remove some additional spaces, but it reduces the work substantially compared to a regular copy and paste.

For more info, this is the link:
http://www.fanhow.com/knowhow:Use_OneNote_2010_to_Create_Blog_Post_41815125

Design Principles and Design Patterns

2011-07-16T10:50:00.001-07:00

http://www.objectmentor.com/resources/articles/Principles_and_Patterns.pdf

Object Oriented Class Design

Open-Closed Principle (OCP)
- We should write our modules so that they can be extended, without requiring them to be modified.

Liskov Substitution Principle (LSP)

Derived classes should be substitutable for their base classes.
Derives from concept of Design by Contract
In terms of contracts, a derived class is substitutable for its base class if:
- Its preconditions are not stronger than the base class method.
- Its postconditions are no weaker than the base class method.
- (In other words, derived methods should expect no more and provide no less)
There are subtleties: canonical example is the Circle/Ellipse dilemma

Dependency Inversion Principle (DIP)

Strategy of depending upon interfaces or abstract functions and classes, rather than upon concrete functions and classes.
Motivation behind DIP is to prevent you from depending upon volatile modules. The DIP makes the assumption that anything concrete is volatile.
Object creation: abstract factory

Interface Segregation Principle (ISP)

Many client specific interfaces are better than one general purpose interface

Package Architecture

Release Reuse Equivalency Principle (REP)
- One criterion for grouping classes into packages is reuse
Common Closure Principle (CCP)
- Group together classes that we think will change together
Common Reuse Principle (CRP)
- Classes that aren't reused together should not be grouped together
- "Changes to a class that I don't care about will still force a new release of the package, and still cause me to go through the effort of upgrading and revalidating."
Acyclic Dependencies Principle (ADP)
- The dependencies between packages must not form cycles.
- Breaking cycles
  - Creating a new package
  - Make use of DIP and ISP
Stable Dependencies Principle (SDP)
- Depend in the direction of stability
- Stability is related to the amount of work required to make a change
- Should all software be stable? We greatly desire that portions of our software be instable. We want certain modules to be easy to change so that when requirements drift, the design can respond with ease.
Stable Abstractions Principle (SAP)

Stable packages should be abstract packages
The more packages that are hard to change, the less flexible our overall design will be. Highly stable packages at the bottom of the dependency network may be very difficult to change, but according to the OCP they do not have to be difficult to extend!
SAP is just a restatement of the DIP - it states the packages that are the most depended upon (i.e. stable) should also be the most abstract

Hacker's Delight: reversing bits

2011-02-28T20:42:00.000-08:00

From "Hacker's Delight" book, how to reverse bits elegantly:

x = (x & 0x55555555) <<  1 | (x & 0xAAAAAAAA) >>  1;
x = (x & 0x33333333) <<  2 | (x & 0xCCCCCCCC) >>  2;
x = (x & 0x0F0F0F0F) <<  4 | (x & 0xF0F0F0F0) >>  4;
x = (x & 0x00FF00FF) <<  8 | (x & 0xFF00FF00) >>  8;
x = (x & 0x0000FFFF) << 16 | (x & 0xFFFF0000) >> 16;

An slightly more efficient version:

x = (x & 0x55555555) <<  1 | (x & 0xAAAAAAAA) >>  1;
x = (x & 0x33333333) <<  2 | (x & 0xCCCCCCCC) >>  2;
x = (x & 0x0F0F0F0F) <<  4 | (x & 0xF0F0F0F0) >>  4;
x = (x << 24) | ((x & 0xFF00) << 8) | 
      ((x >> 8) & 0xFF00) | (x >> 24);

And this last line gives you byte reversal:

x = (x << 24) | ((x & 0xFF00) << 8) |
      ((x >> 8) & 0xFF00) | (x >> 24);

SEDA architecture

2011-02-20T14:23:00.000-08:00

Not long ago, I worked on system that, in a way, resembled SEDA architecture. Although I had attended SOSP '01 and attended Matt Welsh's presentation on SEDA, I couldn't remember much about it so many years later. At the time, I did not connect the dots, but a Principal Engineer mentioned it and quickly I reminded myself about SEDA. Today I read an article explaining it and also a Matt's restrospective on SEDA that I wanted to share with you:

http://muratbuffalo.blogspot.com/2011/02/seda-architecture-for-well-conditioned.html
http://matt-welsh.blogspot.com/2010/07/retrospective-on-seda.html

Matt's retrospective is interesting as the architecture is something that he stills sees a valid and worth considering for modern system, but the stages is something he would group to reduce latency. I would say that, from my experience, the issue is how to have the proper visibility into each stage and its queues, to understand how the system is behaving. Sometimes one just butt heads due to the lack of understanding of the overall architecture and due to lack of metrics on where your backlog may be to understand your bottleneck.

One great question about this retrospective is how Matt views Actors (as in Scala)? The poster asks whether each actor could be a sort of micro-stage in a SEDA architecture. Unfortunately Matt hasn't replied to this question (yet).

Queueing Theory Books On Line

2011-02-08T07:22:00.001-08:00

Good list of books online on queueing theory:
http://web2.uwindsor.ca/math/hlynka/qonline.html

How to find topmost frequent items from a data stream?

2011-01-25T21:43:00.000-08:00

I've been asked this question in a interview a long time back: if you have a huge amount of data and don't have memory to keep all of them, how do you get the top items, even if you have some error. Answering this precisely is not simple and requiring the interviewee to provide a 100% correct may be too much, but the goal is more about the discussion - and luckily it seems I did well at that time. Anyway, below you can find two good references on the topic, in case you are interested in learning more:

Manku, Motwani - "Approximate Frequency Counts over Data Streams" [pdf]

Data Streams: Algorithms and Applications

First program in Scala: porting binary search add/remove from Java

2011-01-23T20:39:00.000-08:00

Today I worked on my first program in Scala, after reading most of Programming Scala. This is the first time in a very long time that I try to learn a new programming language and I took the approach of first reading the book. After starting writing the code, though, I noticed how important it is to actually code something in the language, as without coding, you can't remember much.

So, what I did first was to port the code to delete nodes from a binary tree available here in Java to Scala. The outcome was almost identical, except for some Scala-specific things, like trying to use Option.

Second, I tried to make is more Scala ("scalafy my code") and, since it's been a couple of weeks since I last read the "Programming Scala", I thought that there wasn't a whole lot I could do for this kind of program. However, after trying to implement pattern matching in some sections, I realized that I could get rid of my if statements (which were long, with a bunch of elses) only with pattern matching. Also, some of the "get" calls on the Options objects ended up being unnecessary with the pattern matching. I thought the result was quite elegant compared to the Java version. Open the Java version in a different tab and compare the code, and let me know what you think. Also, if you know Scala, please let me know what suggestions you would have to make use of more Scala features.

case class Node(value: Int, var right: Option[Node], var left: Option[Node])

object TreeUtils {
 def printInOrder(node: Option[Node]) {
  node match {
   case Some(node) => {
    printInOrder(node.left);
    Console.print(" " + node.value);
    printInOrder(node.right);    
   }
   case _ => ;
  }
 }
 
 def addNode(node: Option[Node], valueToAdd: Int): Option[Node] = {
  node match {
   case None => 
    Option(new Node(valueToAdd, None, None));
   case Some(currentNode) => {
    currentNode.value match {
     case v if v < valueToAdd =>
      currentNode.right = addNode(currentNode.right, valueToAdd);
      node;
     case v if v > valueToAdd =>
      currentNode.left = addNode(currentNode.left , valueToAdd);
      node;
     case _ => node;
    }
   }
  }
 }
 
 def deleteNode(node: Option[Node], valueToDelete: Int): Option[Node] = {
  node match {
   case None => 
    None;
   case Some(currentNode) => {
    currentNode.value match {
     case v if v < valueToDelete =>
      currentNode.right  = deleteNode(currentNode.right, valueToDelete);
      node;
     case v if v > valueToDelete =>
      currentNode.left  = deleteNode(currentNode.left , valueToDelete);
      node;
     case _ => {
      (currentNode.left, currentNode.right) match {
       case (None, None) => None
       case (None, _) => currentNode.right 
       case (_, None) => currentNode.left
       case (Some(leftNode), Some(rightNode)) if rightNode.left == None => {
        rightNode.left = currentNode.left;
        currentNode.right;
       }
       case (Some(leftNode), Some(rightNode)) => {
        var q = rightNode;
        var p = rightNode;
        
        while (p.left.get.left != None)
         p = p.left.get;
        
        q = p.left.get;
        p.left = q.right;
        q.left  = currentNode.left;
        q.right = currentNode.right;
        Some(q);
       }
      }
     }
    }
   }
  }
 }
}

object BinaryTree {
 def main(args: Array[String]) = {
  var root : Option[Node] = None;
  val numbers: Array[Int] = Array(56,86,71,97,82,99,65,36,16,10,28,52,46);
  for (i <- numbers) {
   Console.println("Inserting " + i);
   root = TreeUtils.addNode(root, i);
  }
  Console.print("Tree: ");
  TreeUtils.printInOrder(root);
  Console.println();
  for (i <- numbers) {
   Console.println("Removing " + i);
   root = TreeUtils.deleteNode(root, i);
   Console.print("Tree: ");
   root match {
    case None => Console.println("  ");
    case _ => 
     TreeUtils.printInOrder(root);
     Console.println();
   }
  }
  Console.println("Done");
 }
}

DO NOT BUY on Frys.com

2011-01-07T19:06:00.000-08:00

It's been a while that I don't post, but this deserves a post in capital letter: DO NOT BUY on Frys.com. I placed an order on 11/25/2010, and their system recorded two orders, charged twice, and shipped two packages. When I called them, they told me to refuse the package when it was delivered by USPS. That is what I did on 12/07/2010. It's been a month since I refused the package, and several calls to their customer service number, one email sent through their web site (without response after a week), and I haven't received any refund. Except for the last associate I talked to, they did not seem to care a bit about the problem. Most of them say that I should keep waiting, because it can TAKE MONTHS. One of them said that he would talk to the manager and call me back: HE'S NEVER DONE THAT. Now, after a month of following their directions, they will investigate further without any guarantee whatsoever - yes, it's possible that they will never refund.

I think I've been really spoiled by Amazon.com and other companies that care about their customers and forgot about companies like Frys. Frys clearly doesn't care about their customers, provide a horrible experience, and I will definitely recommend all my friends against buying anything from this company.

After a quick search on Google, this is the first link I get about customers reviews of Frys.com - 950 reviews, 1 star out of 5. It seems I am not the only one:
http://www.resellerratings.com/store/Fry_s_Electronics_Outpost

Do you use or consider subversion as source code repository?

2010-08-12T08:49:00.001-07:00

Check this out in case you use branches:
http://blogs.collab.net/subversion/2008/07/subversion-merg/

Unless you use it only for checking in and out files, probably subversion is not the more robust solution.

(Update: CollabNet seems to have deployed a different blog and broke existing links. Thanks to Paulo Naddeo for pointing that out.)

Synchronized Map in Oracle driver?

2010-06-24T19:53:00.000-07:00

I posted this thread on Oracle's forum after finding out that the top monitor in my application was a "SynchronizedMap" in Oracle driver. It seems that each prepared statement needs to access this map, getting worse as one increases the number of database operations in the system.

StringBuilder memory allocation

2010-06-24T19:39:00.000-07:00

Today I was trying to understand a heap dump for a String that, when saved to a file, had 3 Mb in size. However, its retained dump was over 9 Mb! First, this String was a ThreadLocal variable - a library that we use to marshal this String into XML uses ThreadLocal (a good post on StringBuilders and ThreadLocal here).

A colleague tried to understand why it was 9Mb and found out a couple of things that we were not considering:

UTF16: each character consumes 2 bytes in a String to be represented. So, our 3Mb file would consume a minimum of 6Mb of memory space
StringBuilder allocation: whenever StringBuilder needs to expand its capacity, it doubles the current allocated memory. So, if we had 4.5Mb and needed one byte more, it would allocate an internal buffer of 9Mb. In our case, it was clear that 3 out of 9Mb had never been used for anything

Apache Commons DBCP and initial pool size

2010-06-09T21:25:00.000-07:00

Unless you are using BasicDataSource, DBCP does not provide an initial pool size, which is very important if you want to avoid pool expansion out of sudden. It is particularly important if you are migrating from C3P0, like what I did recently. So, in this case, you will have to code an initializer to do it for you. In my case, I wrote a static method that is invoked through Spring:

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.apache.commons.pool.ObjectPool;

public class DataSourceInitializer {
    private static final Log log = LogFactory.getLog(DataSourceInitializer.class);

    public static void init(ObjectPool pool, int initialSize, String dataSourceName) {
        int count = 0;

        log.info("Initializing connections for data source: " + dataSourceName);

        for (int i = 0; i < initialSize; i++) {
            try {
                pool.addObject();
                count++;
            }
            catch (Exception e) {
                log.warn("Failure to initialize connection.", e);
            }
        }

        log.info("Initialized " + count + " out of " + initialSize
                + " connections for data source: " + dataSourceName);
    }
}

And, in Spring, I call it like this:

<bean id="dataSourceInitializer"
class="org.springframework.beans.factory.config.MethodInvokingFactoryBean"
                depends-on="pollableConnectionFactory">
       <property name="staticMethod"
           value="com.example.DataSourceInitializer.init" />
       <property name="arguments">
           <list>
               <ref bean="objectPool" />
               <ref bean="databasePoolSize" />
               <value>mainDataSource</value>
           </list>
       </property>
</bean>

How to bind to all local addresses

2010-06-01T20:51:00.000-07:00

Whenever we bind to an address, typically we bind to an address and port.

InetSocketAddress local = new InetSocketAddress("localhost", port);
socket.bind(local);

This binds your socket only to "localhost". What if your host has multiple names, such as "localhost" and "myhost.mydomain"? You can still bind to all local addresses by doing the following:

InetSocketAddress local = new InetSocketAddress(port);
socket.bind(local);

Internally, this binds to all localhost addresses by using InetAddress.anyLocalAddress() (which is package private).

Java Thread States

2010-06-01T20:21:00.000-07:00

We had a little bit of debate last week about Java Thread states and I found the explanation for our questions in the Thread.State javadoc. The main question is the difference between BLOCKED and WAITING, but I guess this subtle difference is clear below:

NEW
Thread state for a thread which has not yet started.

RUNNABLE
Thread state for a runnable thread. A thread in the runnable state is executing in the Java virtual machine but it may be waiting for other resources from the operating system such as processor.

BLOCKED
Thread state for a thread blocked waiting for a monitor lock. A thread in the blocked state is waiting for a monitor lock to enter a synchronized block/method or reenter a synchronized block/method after calling Object.wait.

WAITING
Thread state for a waiting thread. A thread is in the waiting state due to calling one of the following methods:

* Object.wait with no timeout
* Thread.join with no timeout
* LockSupport.park

A thread in the waiting state is waiting for another thread to perform a particular action. For example, a thread that has called Object.wait() on an object is waiting for another thread to call Object.notify() or Object.notifyAll() on that object. A thread that has called Thread.join() is waiting for a specified thread to terminate.

TIMED_WAITING
Thread state for a waiting thread with a specified waiting time. A thread is in the timed waiting state due to calling one of the following methods with a specified positive waiting time:
* Thread.sleep
* Object.wait with timeout
* Thread.join with timeout
* LockSupport.parkNanos
* LockSupport.parkUntil

TERMINATED
Thread state for a terminated thread. The thread has completed execution.

Book Review: iReport 3.7

2010-05-29T18:36:00.000-07:00

Packt Publishing contacted me a few weeks ago and sent me a copy of their new iReport 3.7 book, by Shamsuddin Ahammad. Although it took me longer to manage to find the time to read the entire book, I got done this weekend and would like to share my thoughts about it.

What is this book about?

This book is about iReport, a very neat tool to design reports intended to run using Jasper Reports library. Although Jasper Reports is probably the best open source report library, it is very hard to use it unless you have a tool to design your reports. I had the opportunity of using iReport a few years back, and without it, I would not have used Jasper Reports at all. Although Jasper Reports has always been very powerful, I would not have written raw XML to design my reports, so now you may understand why iReport turns out to be relevant and worthy of its own book.

In addition to that, not only Jasper Reports is very powerful, but iReport is as well. I had some troubles a few years back for some more advanced tasks (like being able to use scriptlets for my reports), but even then I was able to figure out how to make it work. Recent versions are years-light better than at the time I used, so I highly recommend it as my open-source option for reports.

This book targets those who want to quickly learn iReports. It is a hands-on book, introducing the basic and necessary concepts to get started, following a tutorial format. Each chapter covers a specific topic that will get the user to quickly learn a new feature and leverage them for their own projects.

What does it cover?

As mentioned before, it covers a specific feature in each of its chapters. These are the covered topics: Report Layout and Formatting, Variables, Parameters, Groups, Subreports, Crosstab Reports, Charting, Images. It also covers how to display reports in Java applications.

Opinion

This book is definitely relevant given the importance of iReport and Jasper Reports in the open source world. It doesn't require any prior knowledge of Jasper, iReport, and only very minimal knowledge of SQL to be able to learn what iReport is about. I think the tutorial format is very interesting and does not bore you even if you read it cover-to-cover, like I did. Also, if you prefer to learn just what matters to you, it leads you to the basic knowledge of iReport very quickly, what is very important nowadays. This is definitely the kind of book I would recommend to anyone that doesn't know anything about iReport.

On the other hand, this is not the only source of information you will need when using JasperReports. The integration of Jasper Reports with your applications is covered primarily through a Swing application and the integration with a web application is very short and doesn't provide details. If I were to review this book before it was published, I would have suggested that the author spent more time on the integration part. I would also remove or shorten the last chapter on Netbeans, as I don't think it's so relevant and does not provide much new information. Given that, if you buy this book, very likely you will need to look up on the web more information on how to actually generate the report files.

Also, if you are using more advanced features, they are not covered in this book. One example is scriptlets. Also, it touches on the concepts only lightly, so if you are expecting to learn theory/concepts, internals of iReport or JasperReports, or even details about iReport (like configuration options, all the properties, all available report elements), this is not the book for you.

In terms of the tutorial format, it provides the database schema and how to create in the appendices, which is definitely handy for those not familiar with MySQL. In my case, it was very simple to create the schemas copying the SQL statements from the ebook version. However, it does not provide data and it would have so simple to have provided some sample data so users do not have to populate the database to be able to follow each chapter's instructions. If you buy the print version, the code is not still available on Packt's website, so you will have to create the schema copying the SQL statements by hand. Being nit-picky, the only Java code that is provided in the book is badly indented/formatted and reviewers may have overlooked that.

Finally, I have to say that I really enjoyed reading this book and learning more about the new version of iReport. I recommend this book to all of you that would like to start learning about iReport 3.7. Please leave your comments below if you had the chance to read this book as well.

Alternative to Wicket Ajax serialization

2009-08-31T14:41:00.000-07:00

In the last post, I talked about an issue I found with Wicket. The JIRA I filed with the Wicket team (WICKET-2437) was resolved as "Won't Fix". The reason is that they do not want to make the compromise of having the user synchronizing the page, what is quite understandable.

Igor (Wicket's author) suggested that I created a shared resource and point all my images to this shared resource. That is done by removing my panels and modifying the img src attribute directly. Of course that was painful as I need to serialize all my parameters to the panels as URL parameters, and then be able to parse on the other end shared resource). I tried this path and it worked beautifully.

I know that Wicket supports multiple content types for a DynamicWebResource, but I wonder if I can make some other parts of my site (which were supposed to be async) truly async making them async. For now I can stick with Wicket, which is pretty good.

Apache Wicket and Ajax: deal-breaker?

2009-08-27T18:10:00.000-07:00

I've been using Wicket for the past months and found something that it's a deal-breaker if you have a web application with lots of Ajax where you rely on asynchronicity. Wicket serializes client and server side requests for the same page. You understood it correctly: serialized. So, if you have multiple panels, all the requests will be serialized. The whole purpose of ajax for asynchronicity is defeated.

So, this is a big disadvantage and, although I've been a big Wicket advocate, I would not consider it unless I find a good and simple workaround.

For more context, see this JIRA I file with the Wicket team:
https://issues.apache.org/jira/browse/WICKET-2437

Update: I found an alternative. See this post.

Wicket + Spring Security: do NOT post info to j_spring_security_check as part of the URL

2009-07-25T16:55:00.000-07:00

This week, when working on the integration of Spring Security with Wicket, I was trying to understand the best approach to create a customized login page. Then I came across this Apache wiki page:

http://cwiki.apache.org/confluence/display/WICKET/Servlet+container+authentication

It suggests that you post the info as a Wicket form and, in the Wicket class, you validate the info and post it to j_spring_security_check. This looks very nice at first, but later I realized a major problem. It posts the username and password as part of the URL. Yes, it posts something like that:

j_spring_security?j_username=rodrigo&j_password=mypassword

What's wrong with that? If you have access log, this is the URL you are accessing, which shows up in the log files. I updated the Wiki page with this info and definitely did not follow this path.

It turned out that, after searching everything I found on the web, the solution was pretty straightforwad. I added a regular form to my LoginPage with action set to j_spring_security_check and do not intercept this request through Wicket. That simple, no validation or check in my Wicket code at all.

If you have any questions about this, don't hesitate to send me an email.

Wicket:: how to output text (like a Servlet)

2009-07-20T11:00:00.000-07:00

I wanted to output regular text, like a servlet, rather than an HTML - which is Wicket's default. This was to be used by a PingPage, to make sure the service is up and running. This is the way you can do that:


public class PingPage extends WebPage {
    public PingPage() {
        getRequestCycle().setRequestTarget(new IRequestTarget() {
            public void detach(RequestCycle requestCycle) {}
            public Object getLock(RequestCycle requestCycle) { return null; }

            public void respond(RequestCycle requestCycle) {
              WebResponse r = (WebResponse)requestCycle.getResponse();
              r.setContentType( "text/plain" );

              PrintStream printStream = new PrintStream(r.getOutputStream());
              printStream.println("healthy");
          }
        });
    }
}

Wicket: link/url relative to the context path

2009-07-16T13:31:00.001-07:00

I was trying to figure that out and it took me long enough to find how to do that in wicket that it may be worth posting how to solve this problem.


ExternalLink logoutLink = new ExternalLink("logout_link", "/j_spring_security_logout");
logoutLink.setContextRelative(true);
add(logoutLink);

I was integrating Spring Security and wanted to add the logout link. There is no need to figure out the context path, as the ExternalLink class has an option to set the link as relative to the context.

You can find more info here: ExternalLink (javadoc)

WeakHashMap is not a cache!

2009-05-13T12:31:00.001-07:00

Good article about WeakHashMap:

http://www.codeinstructions.com/2008/09/weakhashmap-is-not-cache-understanding.html

Wicket, Form and GET method

2009-05-07T09:56:00.000-07:00

If you are using Wicket and has previous web development experience, one of the good things is that, everytime you submit, Wicket takes care of everything. However, URL after form submission is something strange as it leads you to wicket URLs that is depending on the user's session. The question is: how do you make Wicket behave as a plain form that submits using GET method?

First, I set my URL strategy to make it bookmarkable:


        MixedParamUrlCodingStrategy mypageURL = new MixedParamUrlCodingStrategy(
                "/service/test",
                MyPage.class,
                new String[]{"type"}
        );
        mount(mypageURL);

Then, in the page code, I override the form onSubmit() method to do the following:


        form.add(new SubmitLink("update") {
            @Override
            public void onSubmit() {
                PageParameters parameters = new PageParameters();
                parameters.add("period", Integer.toString(getPeriod()));
                parameters.add("unit", getUnit().toString());
                parameters.add("type", getType());
                setResponsePage(getPage().getClass(), parameters);
            }            
        });

So, after the user click on the submit link, it submits it similarly to a GET method and I get a nice URL like:


https://localhost:5443/application/service/test/app/?unit=HOUR&period=6

The result is that this makes URLs much more bookmarkable throughout the session.

Extracting SOAP attachments with Axis

2009-05-05T20:53:00.000-07:00

It was a little hard to find this information today, so I think it would be interesting to share it here. First, what I wanted was to get the attachment for a SOAP response. The response was empty, but the attachment had a png I needed to display in my Wicket application.

It turns out that it is very simple to extract SOAP attachments. All you have to do is to write a handler that will be called during the web service call. Any number of handlers can be set in the BindingProvider. For example, this is my code to set the handlers:


MyServicePort port = mws.getMyServicePort();
BindingProvider bp = (BindingProvider) port;
bp.getRequestContext().put(
           BindingProvider.ENDPOINT_ADDRESS_PROPERTY,
           DEFAULT_WS_URL);
Binding binding = bp.getBinding();
     
// Add the logging handler
List handlerList = binding.getHandlerChain();
if (handlerList == null) {
    handlerList = new ArrayList();
    binding.setHandlerChain(handlerList);
}
     
handlerList.add(handler);

With this code, we only need to write our handler which will handle the message and can do whatever it wants with it - including accessing attachments.


public class SOAPAttachmentHandler
    implements SOAPHandler<SOAPMessageContext> {
   private Collection<Attachment> attachments;
 
   @Override
   public boolean handleFault(SOAPMessageContext context) {
       context.getMessage();
       return true;
   }

   @Override
   public boolean handleMessage(SOAPMessageContext context) {
       attachments = ((SOAPMessageContextImpl)context).
                           getWrappedMessage().getAttachments();
       return true;
   }

   @Override
   public Set<QName> getHeaders() {
       return null;
   }

   @Override
   public void close(MessageContext context) {
       // blank
   }

   public Collection<Attachment> getAttachments() {
       return attachments;
   }

The class above extracts the attachments and store then in a class variable. After invoking the web service, I can access the attachments.

Lack of memory

2009-04-28T22:25:00.000-07:00

A very interesting thing happened a few days ago. After searching for a solution to an error message I was getting when running Apache Tomcat, I came across the following post:

https://issues.apache.org/bugzilla/show_bug.cgi?id=39090

And I found the solution to my problem in the comment #5. By accident, I saw the author of the comment and, to my surprise, I had been the author in 2006! Yes, that's right. I had absolutely no memory of writing this comment. And this solution is mentioned in many other places after I added this comment there.

Actually, I ended up using Tomcat 6 due to the nuisance of having to change a lot of stuff in my project at Amazon, but this finding was funny anyway.

Cygwin: created files (e.g. tar) have shared icon on Windows Vista

2009-04-04T19:36:00.000-07:00

Everytime I create a file in cygwin shell, files end up with the shared folder icon on Windows Vista and it is a hell of pain to remove this icon (unsharing a folder takes a whole lot of time). This happens with files under your Documents folder, not with files created elsewhere.

In order to fix this behavior, you can do the following in a cygwin shell:

export CYGWIN=nontsec

Or even better, edit your .bashrc and add this export to be done everytime you launch a new shell.

Hudson Plugin 2: Adding a Post-Build action (for a Reporter/Publisher)

2009-04-02T20:02:00.000-07:00

This is a follow-up to the Hudson Plugin 1. As told before, you create the plugin skeleton using:

mvn hpi:create

This creates a plugin skeleton through the maven-hpi-plugin. As of this writing, the latest version (1.34) generates a code that uses adeprecated way of doing so, and it is the skeleton of a Builder plugin.

Since we are building a Recorder/Publisher and we will use the recommended way of defining a plugin (through the Extension annotation), we will have to change most of the code. But it is important to understand what is the structure, so that's the value of generating the skeleton. Specially because of Hudson's choice of using Jelly for the plugin portions of the HTML page.

src/main/java
plugin Java code
src/main/resource
jelly files where you specify code for the configuration of plugin (global and project configuration).

Java Code
We need to create at least two classes to start testing our plugin. Since we want to see results quickly, let's do the minimum possible here. Let's remember what we want to do here: to add a new "Post-build action" where we can configure the file pattern that we will use to find files to report the test results.

Subclass of hudson.Recorder.
Descriptor class

For 2. there are many options, but we will create a subclass of BuildStepDescriptor<publisher>. More details below.

Resources

config.jelly: jelly code that will be shown in the "Post-build action" section for our plugin
global.jelly: jelly code that will be shown in the Manage Hudson/Configure System section (this file is not used here since we don't have global configuration so far)
help.html: html code with help text for the "Post-build action". This is shown if you click on the ? on the right of your action.
help-artifact.html: html code with help text for the plugin option that will be entered by the user. In our case, the file pattern for the report files.

Results

Files and Details
1. Since our Descriptor class is an inner class, this is the only Java class so far:

src/main/java/com/sacaluta/DistributedTestRecorder.java

public class DistributedTestRecorder extends Recorder {
public static String DISPLAY_NAME = "Distributed Test Report";

private final String report;

@DataBoundConstructor
public DistributedTestRecorder(String report) {
this.report = report;
}

public String getReport() {
return report;
}

public boolean perform(AbstractBuild build, Launcher launcher,
    BuildListener listener) {
return true;
}

@Extension
public static final class DescriptorImpl extends
    BuildStepDescriptor<publisher> {
@Override
public String getDisplayName() {
    return "Publish " + DistributedTestRecorder.DISPLAY_NAME;
}

public void doCheck(StaplerRequest res, StaplerResponse rsp)
        throws IOException, ServletException {
    new FormFieldValidator.WorkspaceDirectory(res, rsp).process();
}

@Override
public boolean isApplicable(Class arg0) {
    return true;
}
}
}

2. Jelly file with the code for our Post-Build action

src/main/resources/com/sacaluta/config.jelly

<j:jelly xmlns:j="jelly:core" xmlns:st="jelly:stapler" xmlns:d="jelly:define" xmlns:l="/lib/layout"
 xmlns:t="/lib/hudson" xmlns:f="/lib/form" xmlns:bh="/lib/health">
<f:entry title="Distributed Test Report pattern" field="report"
     description="
     This is a file name pattern that can be used to locate the Distributed Test report files
     (for example <b>**/performance/perf*</b>).<br/>
   The path is relative to <a href='ws/'>the module root</a> unless
   you are using Subversion as SCM and have configured multiple modules, in which case it is
   relative to the workspace root.<br/>
   ">
<f:textbox />
</f:entry>
</j:jelly>

3. HTML code for help files

src/main/resources/com/sacaluta/DistributedTestRecorder/help.html
(main help file)
src/main/resources/com/sacaluta/DistributedTestRecorder/help-report.html
(help file for "report" configuration field - see above in the config.jelly file)

<div>
This is the distributed test plugin help to be added later.
</div>

Yes, that's all for now. You should already have a post-build action being displayed when you launch Hudson (remember to use "mvn hpi:run" to run Hudson and do not run "mvn package" before that).

Guice

2009-04-01T20:11:00.000-07:00

An interesting dependency injection framework was coded within Google called Guice (you say "juice"). The following video provides a good introduction to it:

http://crazybob.org/2007/06/introduction-to-guice-video-redux.html

I found the following features particularly interesting, which will probably be taken aboard by Spring in the future:

Provider: you can inject the provider rather than the dependency. That allows the class to instantiate multiple copies of the class, instantiate the copies lazily or conditionally. Also, if you have dependencies that have different scope (like having a request dependency in a session object), you can handle this case much better.
Development stages: this seems to be something for the next version, but it is pretty cool. You can specify beans to be loaded according to the stage (devel/prod), not loading unnecessary beans while you are developing.
Constructor listener: another feature for the next version. In short, to be able to intercept the construction of any of the dependencies to be injected.

For a comparison with Spring, check out the following link:

http://code.google.com/p/google-guice/wiki/SpringComparison

I guess it will be hard to come up with a framework to beat Spring, but it seems that Guice and Google products have much better political acceptance and may have better chances of making its dependency injections officially supported in the JDK and sponsored by the JCP.

Update: I found an article that supports my comment above about Web Beans + Guice where one of the readers commented:

There's been a lot of talk over the past few years that perhaps Interface 21 should push to formally make the Spring Framework a part of the JEE specs -- it seemed like it might be possible with Rod Johnson officially declaring his support for JEE 6... well it looks like "Crazy" Bob Lee and the team behind Guice may have found a back door to get themselves into the party first -- according to a new series of articles about the upcoming Web Beans, the new spec is actually influenced by a combination of Seam and Guice ... I find these articles interesting in that Google has apparently taken the JBoss approach to supporting the JCP -- that is, create an independent product to fill a whole in the JEE specs, and then use the JCP to make that product into a spec itself (take a look at the JPA for a previous example)...

The Grinder Framework: 10 limitations

2009-03-31T22:12:00.000-07:00

The Grinder Framework looked very promising when I found it 2 or 3 weeks ago. I thought I had found the solution to do distributed performance testing, but after understanding how it works and testing it for sometime, I came across some limitations that made me code a client/server protocol and go from there:

GUI-centric: Grinder should have been developed with an API-centric view. And then build the GUI on top of it. Not the reverse. I want to launch and run performance tests in an automated fashion. I admit that this is possible with the experimental Console API, but it was frustrating that this is something still experimental - and apparently without enough focus.
Grinder does not provide a way of distributing the agents. You must distribute them and start them manually. Of course I didn't expect Grinder to provide implementations for all possible agents distributions, but I would expect at least having a way of plugging in a class to distribute the agents or scripts that could do that. That way I could start my test, have my agents distributed (according to my needs), have them started, and then actually kick off the test execution. Some implementations for common cases (like a passwordless ssh connection) could be provided.
Grinder doesn't allow agents to collect additional metrics. For example, I don't want only to collect the total execution time, but some individual operations that are part of the tests (but may not be even called at all). As long as you find a way to wrap this in a Test class, you are fine, but that is not always the case - besides we may want to have some metrics hierarchy. And my use case does not include only times, but possibly collecting counts as well.
Grinder has the summary of statistics for the test run, not all the agents metrics. There may be a way of doing it, but my initial tests showed that The Grinder only stores the summary of the test execution. What about collecting and making available all metrics, so you can do calculations on these metrics? Or if I want to plot a graph based on them doing some correlations specific to my case?
Grinder does not allow you to use heterogeneous agents. I would like to run different agents in the same test execution.
Grinder requires all the agents to have the same number of threads - and this is not specified in the code, so you can't perform more complex tests or have a different behavior depending on the number of threads.
Grinder requires Python. Although Python is pretty cool and can be very useful, do I need to create a Python script even if I have my test ready to run? For example, I'd like to use a JUnit test seamlessly.
Grinder does not provide an API to store the statistics. The only of seeing them is through the console. This is related to item 4, but it goes over that limitation because I need to have access to all metrics and store for further processing and an API for this is needed.
Grinder does not provide a timed test resource. You can work around with the Console API, sleeping after starting the test and then stopping it. This should be part of the standard API.
Grinder does not provider a way of breaking the test depending on a certain condition.

Grinder seems very useful for standard HTTP GET tests, which seems to be the most common use case, but it has serious limitations for more complex distributed systems testing.

Spring 3 and REST

2009-03-31T21:57:00.000-07:00

After listening to Rod Johnson's interview on the JavaPosse, I was trying to catch up with the Spring 3 improvements and I found this blog entry about the upcoming REST support, which seems pretty interesting:

http://blog.springsource.com/2009/03/08/rest-in-spring-3-mvc/

I really liked the workaround for HTML only supporting GET/POST. Other operations are added as a field (e.g. "_method") and Spring has a servlet filter (HiddenHttpMethodFilter) to change to the desired method.

Introduction to REST

2009-03-31T21:43:00.000-07:00

Good introduction to REST:
http://www.infoq.com/articles/rest-introduction

Hudson Plugin 1: Setting up the system

2009-03-28T15:02:00.001-07:00

This is an updated version of the tutorial written by Stephen Connoly to write a Hudson plugin. The plugin I create here was done on Hudson ver. 1.293.

Hudson is a continuous build system that I've been using for the past several months. It's been very useful, but my needs require a customized plugin. Although I have some ideas on how to create a great plugin, let's start with the following goal in mind:

Add a Post-Build Action, which is persisted. Example: default JUnit plugin
Have a trend graph for the build and project results. See an example below.

The only prerequisites to start working on a Hudson plugin are Maven 2 and Java 1.5 (JavaSE) or later. Yes, you don't need to worry about downloading or install Hudson itself, as Maven takes care of downloading the Hudson version you're developing for. You can launch Hudson with your development code using Maven, what speeds up the development greatly.

So, to set up your system, do the following:

Install Java SDK 1.5 or later
Add it to your path and set up JAVA_HOME environment variable
Install the latest version of Maven 2
Add it to your path and set up MAVEN_HOME environment variable
Install your IDE (in this example, we will use Eclipse)

Let's configure Maven to find Hudson. To do it, let's edit Maven's configuration file:

Find Maven's configuration file depending on your environment:
Unix: $HOME/.m2/settings.xml
Windows: $HOME/.m2/settings.xml

Add the following to your file:

<settings>
<profiles>
<profile>
<id>hudson</id>
<activation>
  <activeByDefault />
</activation>
<pluginRepositories>
  <pluginRepository>
    <id>java.net2</id>
    <url>http://download.java.net/maven/2</url>
  </pluginRepository>
</pluginRepositories>
</profile>
</profiles>
<activeProfiles>
<activeProfile>hudson</activeProfile>
</activeProfiles>
<pluginGroups>
<pluginGroup>org.jvnet.hudson.tools</pluginGroup>
</pluginGroups>
</settings>

Time to create the plugin skeleton (with Maven in the path):

mvn hpi:create

Note that groupId is the equivalent of package name and artifactId is the equivalent of project name. We used "com.sacaluta" for groupId and "performance" for package name.

Let's update the Hudson version this plugin will be for. As of this writing, the version that is set up in the pom.xml file (inside the plugin directory) is 1.279. Let's change it to 1.293.

<properties>
<!-- which version of Hudson is this plugin built against? -->
<hudson.version>1.293</hudson.version>
</properties>

And now we use Maven to make a Eclipse project out of it to open in our IDE:

cd  (e.g. cd performance)
mvn -DdownloadSources=true eclipse:eclipse

Finally, let's learn how to run Hudson from the command-line with your plugin code.

mvn hpi:run

Once we are done with the plugin development, we can package and distribute your .hpi plugin. This is the command to do this:

mvn package

DO NOT run "mvn package" while developing the plugin. If you do it and then want to run "mvn hpi:run", make sure to run "mvn clean" before that.

In the next post, we will start creating the plugin classes.

This first part of the plugin creation was heavily inspired in the Hudson Plugin Tutorial.

Every man is two people

2008-11-20T08:08:00.000-08:00

"What is the underlying need, the psychological law behind this common tendency that makes men turn their backs on who they are in their professional lives? Every man is two people, and one hardly knows whether it is in the morning or in the evening that he reverts to his real self."

The Man Without Qualities - Robert Musil

Lao Tzu quote

2008-11-20T06:35:00.000-08:00

“Be content with what you have, rejoice in the way things are. When you realize there is nothing lacking, the whole world belongs to you.” - Lao Tzu

Asynchronous Architectures

2008-11-02T10:18:00.000-08:00

Here are some posts about asynchronous architectures that I've been reading recently and found very insightful, including a presentation by Werner Vogels (Amazon CTO).

Asynchronous Architectures 1
Asynchronous Architectures 2
Asynchronous Architectures 3
Asynchronous Architectures 4

Availability & Consistency (Werner Vogels)

Hotmail and email forwarding

2008-09-26T21:30:00.000-07:00

I always thought hotmail is one of the worst free email services, but I decided to give it a try when I joined Microsoft and was able to register a nice address with the domain live.com that had just been released. Now I am trying to get rid of it using a custom domain along with GMail. I am very happy with the decision so far. And for the first time I am even started using Google Talk, providing a more efficient communication and clean user interface than MSN Messenger. Even the Outlook Connector ended up being a way to move all my email from my live.com account - probably something Microsoft didn't thought about, otherwise they would have blocked in some way.

What just made me mad - and it's not exactly surprising giving Microsoft's records - is that you cannot forward your hotmail/live.com emails. Usually the expected is that they don't have this feature, but they DO. And email is only forwarded if you are forwarding to hotmail accounts. This is just stupid - and another attempt to lock you in. First, you DON'T have POP, you DON'T have IMAP, and by using a proprietary protocol, you only have two options to access your email from a client program: using Outlook with a plug-in (Connector, which doesn't have exactly the best performance) or Windows Live Mail. No other option. I accepted these limitatinos since I am working at the Office group and have been trying to use Outlook. Not being able to forward your emails though just proves the point of everyone that criticizes Microsoft. Had they been offering a good service - and would first implement the very basic feature of checking for the new emails automatically without requiring the user to click "Check Email" all the time - more users would be using their services and no lock-in strategy would be necessary.

Other complaint from my 11-month experience with Live + Outlook is the way junk email is handled. If you have a false positive, when clicking on the "Not Junk" button, you always get a dialog saying "so now you trust all email from this address?". If you click "no", it doesn't move the message back to the Inbox. You MUST add the sender's address to a white list. I don't want to have a white list to do that, but that it learns to recognize the email contents to do a better work at recognizing actual spam.

Vista and System Restore out of control

2008-09-12T12:37:00.000-07:00

I thought very strange that, no matter how much of my data I deleted, I was running out of space constantly these past weeks. Then, I decided to find out how much each directly was using. Due to the lack of "du" in Windows OSs, I thought that Windows Explorer would have similar functionality. I really thought so, because I remember seeing GUIs displaying how much each directory was taking up. Later I would remember that this was KDE or GNOME, not Windows. So, first discovery:

Windows does not ship with any utility to find out how much space is being used on your hard drive.

Having to deal with this limitation, here I go to find out any 3rd party solution rather than right clicking on each of the root-level directories and checking their properties. I found two: diruse.exe (part of Windows 2000 resource kit) and another tool that I can't remember now.

Diruse.exe reports inaccurate values compared to Windows Explorer. It seems it counts the file multiple times for some reason that I didn't care to understand. The graphical tool, which is nice but not free, displayed that I had almost 30Gb that was being used by "unscannable folders" (note: this partition has 120Gb or, in other words, these unscannable folders amount to 25% of it). Go figure what are the unscannable folders. Even being admin, there are some system folders that you don't have access, including one name "System Volume Information".

I gave a few tries to unlock this "System Volume Information", but without being lucky. So I found out what this folder is about: "System Restore" feature - something that I never used and maybe will never user. I followed some instructions on how to disable "System Restore" and clean up this folder and voila: I have 28Gb back.

What surprises me is that:
1. Apparently there is no limit for how much disk this system restore will use.
2. Using Windows cleanup tool, it does not suggest this folder as a way to get disk space back.
3. This folder is locked, so even an admin doesn't find out that it is using lots of disk space. You have to unlock to figure out.
4. Getting to "System Restore" is not intuitive and, once there, it does not say anything about being using disk space. When you disable, there is a warning before disabling, but nothing about the disk space.
5. In the end, what one would do is the same old mantra: reformat and reinstall the system. But, installing from a media that does not have the updates will not help you very much as all the updates will be installed again. And what is the result? System restore points will be logged automatically again, taking up something like 30Gb of disk space.

What's the solution? Unless you understand why that happens, in which case you might get a Vista SP1 or later version that does not require installing new updates, the only solution is getting a larger hard disk or, what is more common, a new computer.

Creative People

2008-09-12T12:25:00.001-07:00

"Aw, c'mon," Freddie said with disbelief. "I know all kinds of creative types who just got big promotions."

"That's because they met an even higher standard than cleverness that makes up for the fact that they are creative," Caroline answered.

"And what's that?"

"They threatened to leave the company."

Excerpt from Reeling in the Yearlings, which is insightful and funny.

Mar Adentro: laugh by crying

2008-08-10T09:58:00.001-07:00

From the movie Mar Adentro (The Sea Inside, The Sea Within):

Why do you smile so much Ramon?

When you can't escape and
depend constantly on others

You learn to cry by laughing

The cost of intelectual property

2008-08-10T09:48:00.000-07:00

Through Slashdot, I learned about an interesting article in this week's New Yorker magazine. It talks about striking the balance between private property in terms of intellectual property, but not lock the economy due to them. You can read the article here.

Carbonite again: now I can't unsubscribe

2008-08-10T09:09:00.000-07:00

Carbonite renews your subscription automatically, unless you go to the site and unsubscribe. It is not clear where I should do, but it seems to be in the "update payment info" link. So far, so good. I had to do that becuase they sent me an email that they will charge me an annual fee in 7 days automatically, and I am pretty sure I will try some alternative solution. And guess what I get if I try to unsubscribe?

Server Error
An error occurred while processing your request.
Click back to return to the previous page and try your request again. If the problem persists, contact customersupport@carbonite.com.

And, if I send an email to the customer support, they say that they will reply within 72 hours!!! Fortunately I found out there is a phone number I can call, but given my experience so far, I will not be disappointed if it does not work.

Randy Pausch quotes

2008-08-01T21:42:00.000-07:00

Being almost 10PM on a Friday, that reminded of something I saw Randy Pausch saying in his famous "Last Lecture". And of course I added a couple of other quotes of his that are also very interesting.

* "Work hard. I got tenure a year early. Junior faculty members used to say to me, 'What's your secret?' I said, 'It's pretty simple: call me any Friday night in my office at ten o'clock and I'll tell you.'"

It is not the things we do in life that we regret on our death bed. It is the things we do not. Find your passion and follow it. And if there is anything that I have learned in life, you will not find that passion in things. And you will not find that passion in money. Because the more things and the more money you have, the more you will just look around and use that as the metric — and there will always be someone with more. Your passion must come from the things that fuel you from the inside. That passion will be grounded in people. It will be grounded in the relationships you have with people and what they think of you when your time comes.

Wait long enough and people will surprise and impress you. He said, when you are pissed off at somebody, and you're angry at them, you just haven't given them enough time. Just give them a little more time — and they'll almost always impress you. And that really stuck with me. I think he's absolutely right on that one.

Spellchecker (Levenshtein distance)

2008-07-27T13:05:00.001-07:00

Have you ever wondered how the spellchecker suggestions are computed? I had no idea, and found out that the editing distance between words is calculated to come up with a list of words that are "close" to the one you are typed. Those with the fewest number of edits are displayed. The algorithm, known as Levenshtein distance, was devised in 1965. It uses dynamic programming, and it is a very clever and simple solution.

Apple Update annoyances

2008-07-25T17:42:00.000-07:00

First Apple not only suggests that you install Safari, but keep it checked. Everytime I have to uncheck this box. But now that was great. They release a new update with a bug in the updater: the description is displayed as "WhichDescription()".

Java benchmark: String, StringBuilder, and StringBuffer

2008-07-21T21:43:00.000-07:00

Some results first comparing String, StringBuilder and StringBuffer, with and without pre-allocation.

100000 times
String: 30786
StringBuilder (no pre-alloc): 5
StringBuilder (pre-alloc): 3
StringBuffer (no pre-alloc): 6
StringBuffer (pre-alloc): 5

The results clearly show how bad String is if you need to change it constantly in comparison to StringBuilder or StringBuffer. Now, let's remove String and compare StringBuilder and StringBuffer further:

1000000 (one million) times
StringBuilder (no pre-alloc): 58
StringBuilder (pre-alloc): 42
StringBuffer (no pre-alloc): 142
StringBuffer (pre-alloc): 122

This benchmark seems to prove to me that StringBuilder, introduced in Java 1.5, is really the best option when running in a single thread. And one can see that the overhead of running StringBuffer, which is synchronized, is actually huge.

Excerpt of my code to run this benchmark (*):

  final int n = 100000;
  final String a = "a";
  
  String s = "";
  Date start = new Date();
  for (int i = 0; i < n; ++i)
  {
   s = s + a;
  }
  Date end = new Date();
  System.out.println("String: " +
                     (end.getTime()-start.getTime()));
   
  StringBuilder sb = new StringBuilder();
  start = new Date();
  for (int i = 0; i < n; ++i)
  {
   sb.append(a);
  }
  end = new Date();
  System.out.println("StringBuilder (no pre-alloc): " +
                     (end.getTime()-start.getTime()));

(*) Based on this post, which does not compare StringBuffer

Immigration and other golden cuffs

2008-07-21T17:24:00.000-07:00

Currently immigration is something quite interesting to be studied, especially in the US with these long queues to get the green card. If you join a company in the US on a non-immigrant visa (like L-1 or H-1B), you usually start your green card process, either right away or after some time. To set the stage, let me explain better about these visas. L-1 implies that you must leave the country almost immediately if you leave the company. H-1B has a little bit more flexibility: you have one month to find another company that sponsors your visa or you will be out of legal status. Back to the green card, depending on the job requirements, on how your case is filed, it may take many many years to get it. And by many many years, I talk about 5-10 years, sometimes even more than that. Although theoretically possible to change jobs under certain circumstances, few do that because there is a chance the process is withdrawn and you lose your place in the line to get the green card. That means that you are stuck with one company, with one type of job (changing the job may jeopardize the whole process). And, even after you get it, you should stay with the company at least one year to avoid being potentially charged with green card fraud.

I never liked being stuck with one company. And, when living in another country, I never liked the idea of living in that country without being sure that I could stay there and your personal investment in the country won't be lost. For this reason, I never really thought of coming to the US, because of the hard work to get a green card (besides the compromise). And, in my case, I could work in Europe since I am a EU citizen, so that makes me reflect more about what to do.

For a big company, the point is not in getting the actually best and brightest minds (although all big companies say that all employees are the best and brightest!). Nevertheless, the point is not to hire genius, but relatively smart people and make the rules and decisions that benefit the average employee. And who would that be? The one that achieves or even exceeds what is expected based on his/her experience, and that stick with the company for a long time. These delays in the immigration process end up serving the best interest of big companies that are sponsoring the green card for the employees: in average, their immigrant employees will be around for a long time. It may not be used on purpose, but that is the fact and it benefits big companies. Who is brave enough to throw away many years of theirs lives, especially with a family that may be well settled in the country? And isn't it the best deal if you can hire an experienced foreigner paying less than what you would pay for an American with equivalent experience? The difference is only the relocation (which is one-time charge) and green card process (again, one-time charge). When outsourcing is not the best option, that is a great deal to keep big companies competitive in internal US market and also globally. These professionals (the average employee) get stuck over the time due to this immigration issues, sponsorships and related rules. But that is not all.

Other common policy to make people stay with the company: you offer deferred compensation over time. That can be in the form of bonus, stocks, or anything as long that means that the employee will only get it if he/she spends many years at the company. And, on a regular basis, based on contribution, you pay more and more of his/her compensation in this way. Who will leave the company and leave that much money on the table? These are known as golden cuffs.

Third, you create a competitive scheme that only the very best get compensated. That is great for the company, but that has different effects on the employees that end up being blind by immigration, deferred compensation, and this competition. These employees become more focused on whatever they need to do, losing track of skills that are generally applicable to other companies, and more and more skilled on whatever is specific to their employer. And, although I don't know if on purpose or not, but I see that people lose track of what is outside the company and their own view of the company is based on what they learn from the company material (what may be influenced by the company ideology). Back to the market, not knowing what is going on in the market is not too bad if your company follows market standards. But that is not often the case and some particular experiences are very specific and not much valued elsewhere. A professional who leaves such a company, being valued by his/her specific and narrowed knowledge with a high salary, may only get a job that pays much less that what he/she makes. Who would do that?

Fourth, you create an environment where people feel good about themselves because they work for that company. More ideology work is necessary here, but it is not too hard. One can offer entertainment and other sorts of offers that seduce the employee family as well. And associate a good image of the company, so nobody will feel bad about being immoral or unethical to work for such an employer. Any bad press must be worked on and reverted, at least internally.

In this environment, what happens is that the least important thing is if you really love what you are doing. And unfortunately the rest does not pay off if you pursue to do what you love and what you believe in. From my experience since university, though, the majority of people does not care much about that. After all, if all jobs are the same, why not to be with those that offers the points outlined before? At least there is some advantage to be at the big company.

That is sad, very sad. Especially if you care about what you do and/or you are an outlier. [And not to mention how that compare to open source software, but that is a topic for another post.]

Java covariant return types

2008-07-21T16:46:00.000-07:00

In my attempt to try to catch up with more recent features in Java and Linux, I found out that a feature introduced in Java 5 that I was not aware of is covariant return types, what is neat. You can override a method and change the return type, what was impossible before Java 5, as long the returned type is a subclass of the return type of the superclass method.

For example,

class SuperClass {
   SuperClass getX() {
      return new SuperClass();
   }
}
class SubClass extends SuperClass {
   SubClass getX() {
      return new SubClass();
}

There is an interesting page here about it.

Stronger schools in the US

2008-07-18T21:33:00.000-07:00

Very good post by my friend Tiago. See the video below:

I think most of people that left their countries realized that this is a serious issue. But when you come from a developing country, that is not news. What surprises me above is to see that rich countries are starting to worry about that. And I am glad that countries like US do that ahead of time, before it is too late for them.

On a related note, today I a sticker on a car window saying something like that: "Send our children to universities, not to war". It is a good start when this kind of mindset starts becoming more common around here.

Greg Kroah Hartman on the Linux Kernel

2008-07-16T20:32:00.000-07:00

It's an interesting video about the Linux kernel. What always amazes is how it attracts contributors that would not happen if it were not open source and were that famous. Like Greg mentions people doing janitorial work or submitting even spelling patches. Although I don't know about the dynamics of the Windows team, it is hard to conceive that there are developers like that contributing to the Windows kernel.

Fixed vs Growth Mind-set

2008-07-13T10:26:00.000-07:00

A friend sent me this interesting NY Times article: If You’re Open to Growth, You Tend to Grow. It starts with one question: "WHY do some people reach their creative potential in business while other equally talented peers don’t?" and shows how the best and the brightest sometimes have a fixed mind-set that prevent them to realize their potential. Looking out for the chances to grow and also learn from past mistakes, what is called a growth mind-set, may make the whole difference in terms of professional (and may I add personal) growth. Recommended reading.

Universidade Brasileira

2008-07-05T08:22:00.001-07:00

Trechos do blog do Reinaldo Azevedo sobre universidades que achei bastante interessantes.

Com exceções, claro, como sempre, a universidade brasileira é um hospício de iletrados.

A academia, no Brasil, é um dos últimos redutos do comunismo no mundo. Há mais comunistas nas universidades estaduais e federais do Brasil do que em Pequim.

-- Reinaldo Azevedo

Brazil: Skilled Workers Wanted

2008-07-02T08:08:00.000-07:00

Read this NYT article about the need of skilled workers in Brazil. Being that a global demand, how to compete with other countries that offer more attractive compensation packages to top talents?

Google Docs and broken page layout

2008-06-30T17:53:00.000-07:00

See the bug below. A document with a horizontal page is not supported by Google Docs page layout.

GOOG->MSFT Exodus?

2008-06-30T17:36:00.000-07:00

This article tells a bit about the experience of people that worked at Google. One comment is particularly interesting to me:

2) Google employ everybody as junior developer, ignoring the existing experience. It is nice to work in Google if it is your first job, really nice, but if you have 12 years of experience with lots of languages, technologies and platforms, at lots of senior positions, you should expect higher position in Google, right?

And this give us an impression that MSFT is better. I disagree. And, although I won't tell about any particular experience that I've been through or have seen, trust me that this is something that happens to MSFT as well. I've been down this road myself. Maybe more with some groups than others, but it happens. The reason is always the same: you experience is irrelevant to our company and we do lots of things that are very specific and require experience with these tools, processes, etc, to be more productive. After 8 months working at MSFT, I agree to some extent, but definitely this is not a valid point in its entirety.

There are a couple of things one should think before joining company that does not value previous experience for the reason that everything is so specific to the company. First, it is how the experience with this company may be valued outside of it and, second, if you are not getting more and more stuck with this company as time goes by, as no one else will pay what you will get there after sometime and some salary raises.

In spite of all that, and far from saying that MSFT is perfect, but definitely it is one of the most organized companies I worked for and I got the chance to know. Probably more than GOOG given the age of both companies.

Losing your Photoshop Elements catalog twice with Carbonite

2008-06-27T21:18:00.000-07:00

Sometime ago I posted an entry about my experience with Photoshop Elements. First, I really like Adobe Photoshop Elements, but I lost my Photos catalog for the second time in less than an year and it is really painful to have to recreate your catalog all over again for over 4,000 photos. The first time, I didn't recover the catalog when recovering from my Carbonite backup when I bought a new laptop. This time, Vista issues after trying to convert a video made me reinstall the computer and although I manually backed up my C:\Users directory before reinstalling, guess what? The catalog was gone again. I thought: well, Carbonite must have it backed up. But no, it does not back up the location where Adobe stores the Photoshop Elements catalog: C:\ProgramData. The option is not even displayed in the context menu. Is there any hope this issue will be fixed in Carbonite later versions?

Is Google making us Stupid?

2008-06-23T17:09:00.000-07:00

This is a fantastic article about how internet is changing the way we read and think.

Caverna

2008-05-31T10:08:00.000-07:00

"Quando digo que as pessoas que estão na caverna somos todos nós é porque damos muito mais atenção às imagens do que àquilo que a realidade é. Estamos lá dentro olhando uma parede, vendo sombras e acreditando que elas são reais."

JOSÉ SARAMAGO, escritor

System-level access to Vista

2008-05-26T08:15:00.000-07:00

This exploit allows an user to gain system-level access without any user or password by copying cmd.exe over to an executable that can be launched without logging in. Very interesting exploit.

Help not wanted

2008-05-19T17:56:00.000-07:00

Help not wanted
Congress is doing its best to lose the global talent war

Apr 10th 2008
From The Economist

And, on a related subject, in Japan

High-Tech Japanese, Running Out of Engineers

May 17th 2008
From The New York Times

Eggs at Steve Ballmer in Hungary

2008-05-19T17:43:00.000-07:00

See here the video. The assailant accuses Microsoft of bribery in business with Hungarian government and asks the taxpayer's money back.

MSN (Live) Messenger blocking YouTube URLs?

2008-05-12T08:11:00.000-07:00

YouTube URLs being blocked by Microsoft IMs? I noticed some troubles with youtube URLs for quite some time, but never tried to investigate what is wrong. This is simply ridiculous.

That reminded me of when I signed up on Zoho.com and Thinkfree.com using my live email, and they always go to the junk mail folder. I also recall reading that some people noticed that particular invite emails from competitors never make into their hotmail inboxes. Although I can't prove anything here, it is a bit strange, isn't it?

US Immigration Laws: scary

2008-05-08T16:32:00.000-07:00

Perfectly Legal Immigrants, Until They Applied for Citizenship.

Spend many years in the US, have a family, house, and a job. Apply then for the US citizenship. If the US government finds something they don't like, be prepared to be deported, what would not happen if you hadn't asked for the citizenship. The article above tells the sad story of a Philippine couple that is to be deported after 25 years as legal immigrants in the US.

Seattle: "[...] He knew what it meant to be lonely."

2008-05-05T17:29:00.000-07:00

Read this Seattle Times' article about the Seattle Freeze, the fear of intimacy of people that live in this area. I wonder if the same happens in other areas in the US or in other countries (especially in Europe) to the same extent as here.

We ride bikes alone; go on walks alone; troll bookstores alone, then go home and read alone.

On the one hand, it's nice to bop in and out of situations knowing people will smile and treat you well. Nice is like bubble gum — it's sugary and pleasant." But if all you ever get is nice, never flirty or risky, she says, that gum loses its flavor pretty quick, and the human experience becomes ultimately less rewarding. Even depressing.

Since arriving from Argentina, he's turned down the volume on his laugh, no longer reaches out to hug friends and has even stopped wearing his favorite loud red pants. Those first lonely years in the Northwest even gave him a bit of a taste for solitude.

[...] [S]he realized that when Seattleites say, "Let's do something sometime," what they really mean is: "Let's never do anything ever."

La Traviata, children and social skills

2008-05-03T08:41:00.000-07:00

[...] A performance of La Traviata has much to enthrall a child: beautiful music, two parties, ball gowns, dancing, pretty scenery, love, straightforward moral issues that can later be discussed. An afternoon at such a performance will be meaningful for some children. In our era, in which information is electronically spoon-fed and young people's social skills are often less fully developed than their computer skills, exposure to moral and emotional situations in a theatrical setting may spark parts of a child's imagination that were previously dormant.

-- Opera 101, Fred Plotkin

Microsoft: "pace is slow"

2008-05-02T11:08:00.000-07:00

Many people wonder how working at Microsoft looks like. It depends on many factors, and particularly on your personal style. I thought I was one of the few outliners when I had some thoughts after completing 6 months working here, but it turns out someone else worked 6 months at the company like 8 years ago and was very talented to put in words many of my thoughts. This is a must read if you are curious about day-to-day life at Microsoft - or if you are interested in what could be my opinion.

Berlin Wall at Microsoft?

2008-04-30T21:25:00.000-07:00

Yes, I never noticed that before. In the building across Office, there is a portion of Berlin Wall that was donated by Daimler-Benz to Microsoft. Something I would never expect to be part of Microsoft Art Collection, and although one could imagine many meanings for having that wall on the campus, even then it is nice to be able to see part of history right there.

See a picture of this portion of the wall:

This picture is from here

Gil Shaham with Berlin Philarmonic

2008-04-30T21:21:00.000-07:00

Definitely worth watching this video not once, but many times.

Reflections about IT

2008-04-30T07:58:00.000-07:00

There is a post on Slashdot about a guy disillusioned with IT. And very interesting comments with different viewpoints. Definitely worth your time if this kind of reflective subject attracts you.

"Do we all hate our jobs?"

2008-04-27T08:14:00.000-07:00

On Safari site I ran into a book name "Four Secrets to Liking Your Work" and decided to read the first pages. I found some interesting paragraphs right on the first page:

Do we all hate our jobs?

Picture a room filled with ten of your coworkers. Odds are that seven of them don't like what they do. Do you know which ones? Are you one of them?

It gets worse. Research suggests that one of those seven coworkers might be actively working at cross purposes to undermine your company. This means that if you are trying to be productive, he or she is working against you, too!

Why bother working at all? We like to believe that it's all about money. We see people endure dread, dissatisfaction, and misery in the name of the paycheck, but Herzberg and Maslow agree that once our monetary needs are met, it ceases to motivate us. And even if we do need the money, it's no substitute for engaging, enjoyable, productive work. Money can't buy back lost time.

Why Scott Berkun left Microsoft (or general reflections about life and one's options)

2008-04-26T09:16:00.001-07:00

The following text is sublime and I found it by accident trying to find the reason the Gentoo founder left Microsoft after only 9 months there. Rarely I found texts that were so similar to my own feelings and impressions.

Why I left Microsoft?, by Scott Berkun

Some quotes from this text:

From day one of kindergarten onward there was always a next step waiting. The choices were easy and safe: which classes, which activities, which universities. But an hour after my college graduation, sitting alone in an empty apartment on Beeler street in Pittsburgh, there were no more choices laid out for me. There was nothing. I confronted my future as a kind of void for the first time and was terrified. I’d never understood that emptiness, despite seeing its effect on older friends and my older brother. Until I was sitting alone surrounded by it, without the defense of a plan or a friend, I had no idea how frightening it was.

My comment: I had exactly the same thought when graduating. Everything was so simple, the next step was always there. And the frightening comes back all every time you start thinking that you may want to leave your current position and change your life. The future is a kind of void. You can create it, but it still scares.

I questioned what I was doing with my time on the planet. Dreams of my college years had been fulfilled, and if I didn’t make big changes soon, I knew I’d be repeating myself. There were other challenges I wanted, and I became terrified of spending my life like a sad, confused bird of prey, circling the same territory over and over again, never understanding why there was nothing new to find. I needed a new situation to jump into and despite what my manager’s and peers said, I knew I couldn’t do that while working in the same place. I had to move on. I was surprised to find that even though I was ten years older, my fears about the big unknowns were just as scary as before. But when I measured my fears about staying, I found they were stronger than those about leaving, so I left.

So I chose to leave Microsoft less for reasons of escaping a particular place or group of people, but more to seek out a new set of circumstances to live in.

My comment: That is exactly the reason that I left some of previous jobs or circumstances I was in and that could be the same reason I may leave Microsoft in the near future. I like to be an explorer and cannot simply accept a given position because it is convenient or because I am at a position many people dream about. The bright side of all of this is that, although you may get disappointed that things didn't work out as expected, you understand the downside (at least partially) of many possibilities in the life while exploring it.

Office 2007 = 12 times more memory than Office 2000?

2008-04-16T21:42:00.000-07:00

Fat, Fatter, Fattest: Microsoft's Kings of Bloat
posted by Thom Holwerda on Tue 15th Apr 2008 20:12 UTC, submitted by Craig Barth
"What Intel giveth, Microsoft taketh away. Such has been the conventional wisdom surrounding the Windows/Intel (aka Wintel) duopoly since the early days of Windows 95. In practical terms, it means that performance advancements on the hardware side are quickly consumed by the ever-increasing complexity of the Windows/Office code base. Case in point: Microsoft Office 2007, which, when deployed on Windows Vista, consumes more than 12 times as much memory and nearly three times as much processing power as the version that graced PCs just seven short years ago, Office 2000. Despite years of real-world experience with both sides of the duopoly, few organizations have taken the time to directly quantify what my colleagues and I at Intel used to call The Great Moore's Law Compensator (TGMLC). In fact, the hard numbers above represent what is perhaps the first-ever attempt to accurately measure the evolution of the Windows/Office platform in terms of real-world hardware system requirements and resource consumption. In this article I hope to further quantify the impact of TGMLC and to track its effects across four distinct generations of Microsoft's desktop computing software stack."

See the full Infoworld article here.

One interesting comment:

You think it's just Office? Its ALL SOFTWARE. The dirty secret that nobody wants to talk about is that it's the OOP software paradigm. It's been a dismal failure and your bechmarks prove it. OOP was supposed to make software development faster and easier. In fact it simply made it bigger and harder to work on. Easily 80% of all clock cycles are consummed diving through ever-deeper heirachy trees full of do-nothing, duplicated getters and setters at every level. Moores Law is the only reason anything still workds. Imagine where we'd be if we didn't spend all those cycles trying vindicate a failed programming paradigm. Back in the 80's people thought true artificial intellegence would be possible when computers achieved 2G of memory. Today my desktop has 4g, plus 4, 3.2-gigahertz processors and more disk space that my employer owned in the entire company in the 80's. But its performace barely matches the TRS-80 I learned basic on. It makes me wish I had done something useful with my life -- like Walmart greeter.

Government is not your daddy

2008-04-14T07:28:00.000-07:00

Today, after arriving in the office, I see some business cards pinned to the wall. With an American flag as background, the cards have the title "Government is not your daddy." and then "Free will. Free market. What' s so hard to understand about thtat?". The cards are to advertise the following blog:

http://notyourdadddy.wordpress.com

And although I couldn't read it yet to know if their texts are good or not, I really liked their creativity about the name and their efforts to make it known.

A 101-year old marathoner

2008-04-13T20:32:00.000-07:00

This news really surprised me. A 101-year old marathoner that crossed the finish line of a marathon, and has an average of 14 smokes and 8 pints of beer a day. Wow! And his advise is good: "no matter how old you are, even if you are only seventy". I will remember him when I decide to stop running after 1 or 2 miles ;-)