Throughout my career, I found myself on different sides of a debate that I currently have with myself: being technical vs. being pragmatic. I believe that the more experience an engineer has and, as long as s/he keeps an open mind to new lessons, s/he will have a broader perspective of the work being performed. As a consequence of this broader perspective, s/he is able to think from different perspectives and tries do the work in a way that takes care of the different concerns that are raised through the process. If one doesn't have or doesn't learn these different perspectives, s/he will keep doing things mostly in the same way. On the other hand, as one comes across different issues and problems and learns these lessons, this person will know what it takes to avoid them and this can be invaluable. In the software development world, this spans from how to be diligent about the theoretical side in order to guarantee correctness for complex problems all the way to the required tests in order to declare that that piece of software is "production ready".
The challenge for this more experience person is how to determine where to draw the line and also be able to release products or services. After all, typically this kind of project is being done with a business purpose. One one hand, if an engineer is too theoretical or too "formal", then too much can be spent on unnecessary formalisms or concerns and not see things from a practical perspective. On the other hand, not doing the minimum required analysis will potentially bite one in the future. And actually, if the outcome of this work is on behalf of a big company (like I did at Amazon and do currently at Microsoft), the question is whether one is actually being responsible if s/he allows to let something like that happen.
Working in a team environment is the other big challenge. Unless there is a generalized culture of spreading the knowledge and engineers are genuinely interested in getting better, reaching a consensus of what to do is impossible. Actually, to my surprise I realized early that companies and groups vary a lot concerning the "getting better" aspect, and definitely assuming that all people will be interested in learning and getting better is just an utopia. Environments will be always heterogeneous, especially if the culture (including but not limited to mechanisms in place like performance appraisal) rewards people mostly for releasing something rather than actually releasing something good.
Also, even if a subset of those involved in the development process sees issues, then the challenge is how to convince others of them. A typical debate is maintainability. First, people have different opinions about how to write maintainable software, but the question is how to convince others that maintainability is important. Unless upper management actually values that, it will be probably a secondary priority. And not only that, but as a typical political game (read Games at Work for more details) is to do something of impact with the goal of standing out, even if there will be mess to be cleaned up later. And software maintainability is a great example of something that will be a problem only in the future.
Another example is how to convince others of potential future issues either that may happen under stress scenarios or simply because the team did not work through the scenarios to guarantee correctness of the solution from a theoretical perspective. Oftentimes such arguments are dismissed if the system behaves well for some functional tests.
At an earlier point of my career I've been more of the pragmatic type that wanted to ship and getting things done, until I started learning more about the issues that caused real problems in the experiences I've been involved in. I developed a real appreciation for doing things responsibly - what could involve getting to the theoretical part and involve experts when one is dealing with something outside his/her area of expertise. Once I got to that point, I started questioning myself if I am going overboard or being that diligent is actually the right thing to do. Is there any to transmit these experiences to others? Or is it better just to let the process move on and, if the concerns were legitimate, let the organization learn in the hard way?
Another aspect of starting developing this appreciation for a thought through process is that it becomes harder to be proud of what one is working on if you're involved in the process with others that have different mindsets. However, being realistic, one can't really expect homogeneous environments as I mentioned above, and to a certain extent that is a challenge that will be part of one's professional career until the end: there will be always someone around you (with or without more power to make decisions) that will require effort on your part to communicate your concerns. And more often than not, unless the person has gone through the experience him/herself, s/he will not get it.
This Slashdot post touches exactly on the same point: "The last few years have become particularly taxing as I struggle to reiterate basic concepts to the same technically illiterate managers and stakeholders who keep turning up in charge. While most are knowledgeable about the industries our software is targeting, they just don't get the mechanics of what we do and never will. After so many years, I'm tired of repeating myself. I need a break."
On the upside, this discussion leads me to think what is the best organization structure and who should be the upper management. As a trend that I see across some companies, currently I believe that engineers with strong theoretical background with the right practical balance are the right people to be on top. They will fundamentally help spread a culture where the right solutions - from an academic point of view - are encouraged. Brad Calder, a former professor at UC San Diego, moved to Microsoft some years back and is a good example of person that I believe should be at the top. Albert Greenberg, with a research background, is currently in Windows Azure too (as mentioned here) and seems to me the right person to be at the top as well. Typical non-technical business executives may see mostly the releases from the business standpoint but will not understand that many times that cheap solution has some serious drawbacks. And, if one is talking about a service (as in cloud computing), just a few serious production issues can harm a brand in a very significant way.