Web Site of Rajiv Pant

My journalist colleagues at Wired.com published some of my comments related to Amazon S3.¹ Wired also posted another article titled Customers Shrug Off S3 Service Failure. I agree with the views of many of the customers expressed in the article. Don MacAskill, CEO of the popular photo hosting site Smugmug, wrote an understanding post about it.

My entire career working for media companies, I’ve held firm the belief that the uptime, reliability, performance, scalability, performance and security of commercial Web sites is of paramount importance. When sites that I’ve been responsible for have had issues, my colleagues and I have given our personal time and energy to resolution. With my teams, I spend considerable time on proactive measures. I’ve had the honor of working closely with and learning from some who do an excellent job running technology operations.

Experience has taught that things can and sometimes do go wrong. Sometimes calculated risks don’t pan out. Sometimes mistakes cause problems. We are human. We should strive for perfection; we can get close to it, but not fully attain it. We should be prepared for such scenarios. When they happen, we should work diligently and expeditiously on resolution and have frequent and honest communications with stakeholders and customers. Such communications during the incident should include:

Update 2010-Jan-24: This checklist is now maintained on the Checklists Wiki Web site at:

www.listswiki.com/wiki/IT_Incident_Reporting

During-Incident Communication Checklist

• Current status
• What is the full impact?
• Estimated time to resolution
• Any recommended workarounds until resolution, if practical
• Assurance that it is being worked on
  • It often helps to mention who all are working on it and what they are doing

The post-incident communications to stakeholders and customers should include:

Post-Incident Communication Checklist

• Summary
• What happened, how and why it happened?
  • Including full description of all impact
  • Do not blame² third-parties or say things like “beyond our control”. A technology leader takes responsibility equally for both insourced and outsourced products and services.³
• How it was resolved
  • If the resolution is temporary or long-term
• Next steps
• Plan for eliminating or minimizing this and similar incidents from happening again
• Thank all those who helped resolve and the customers for their understanding
• Mention the monetary credits you plan to give as per the Service Level Agreement (SLA)
  • Specify any additional ‘make goods’ or returns you plan to make to the customers above and beyond the credits as per SLA, if appropriate.

Stakeholders and customers here refer to internal customers of the technology operations team (e.g. the concerned folks in editorial, marketing, sales, finance, legal and other departments). External communications to the public Internet should be handled in consultation with legal and public relations.

S3′s outage (or any outage) isn’t to be taken lightly, but I have faith Amazon and their customers will learn from it.

Disclaimers:

• As explained in the terms of use of this site, any opinions expressed on my personal Web site do not reflect those of any employer, past or present. My Web site and I in my personal life neither represent nor speak for any corporation.
• I have no affiliation, financial or otherwise with Amazon.com. I happen to be a user of their products and services, some of which I like and some that I don’t.
• Personal Web sites like this are exempt from the performance requirements of corporate Web sites :-) My personal Web site is for expressing, learning and R&D. It also happens to be hosted on Amazon EC2 and S3.

Tweet

2 saves

1. Silicon Alley Insider and ValleyWag have amusing spins on it. :-) [↩]
2. There may be extreme instances, especially when criminal activity or malicious wrongdoing was the cause where it would be appropriate to blame someone. [↩]
3. It is ok to mention service providers, or describing external events for explaining what happened, but don’t do it in a “it was their fault, not ours” tone. The technology leader should factually describe what happened and take responsibility. [↩]

Described here is one way to enable technologists to grow their careers in your organization while still allowing them to focus on the type of work they are best at and enjoy most.

The typical management career growth path does not suit some technical people. These information workers need to grow in their careers (gain greater compensation, responsibilities and influence) without having to become managers of other people. A good way to achieve that goal is to create a technical career growth track in your organization.

The following table shows management seniority positions alongside their equivalent technical seniority positions.

In this model, for example, an architect role is at the same compensation and influence level as a manager role, assuming that the particular manager and architect being compared add similar value to the company. If the organization requires more layers, say, a senior architect would be at the same level as a senior manager.

If the organization prefers consistent titles for levels, the system could name them like this: the fellow role as director & fellow, the senior architect role as senior manager & architect, etc. In the case of a fellow who is at a VP or SVP level, they should always be named VP & fellow or SVP & fellow.

Here is a definition of the fellow role from WikiPedia:¹

Large corporations in research and development-intensive industries ² appoint a small number of senior scientists and engineers as Fellows. Fellow is the most senior rank or title one can achieve on a technical career, though some fellows also hold business titles such as vice president or chief technology officer.

Such a technical career growth plan brings many benefits to your organization.

• It helps retain good technologists who want to grow in their careers, but want to do keep doing the type of work they are best at and enjoy doing: technical work.
• It avoids brilliant technical people from being “pushed” (by themselves or their supervisors trying to “reward” them) into people-management responsibilities.
• It reduces situations of having too many people-managers but not enough people-management positions over time as people get promoted.

Care should be taken to recognize that some technical people do enjoy making the transition to people-management roles and the presence such a technical track should not discourage them. Having an alternate career growth track option is about presenting employees with more than one choice.

A similar system could also be applied to other departments with individual contributors. For example, creative design.

Related Articles on Other Sites

• Fog Creek Professional Ladder by Joel Spolsky
• Construx’s professional development ladder

Tweet

1 save

1. Definition of Fellow at WikiPedia http://en.wikipedia.org/wiki/Fellow and Wikitionary http://en.wiktionary.org/wiki/fellow [↩]
2. IBM or Sun Microsystems in information technology, and Boston Scientific in Medical Devices for example [↩]

I’ve found this to be an effective way of evaluating potential hires compared to just interviewing in a question/answer format: Put them to work for a few hours (or even days/weeks/months as contractors) and see how well they perform.

Having someone do the job as a short-term temporary contractor before hiring them is one of my favorite ways of testing them for the job. However, sometimes this isn’t practical. In those cases, having the candidate work on a short test assignment for a few hours can be very telling.

For example, when interviewing a software programmer, set them up on a computer and give them a programming assignment representative of the type of work they would do on the job. Give them reasonable freedom as you’d give an actual employee. If they need to research references / solutions on the Web, let them. If they want to call a friend for consulting help, that’s fine. If they ask for your or their “coworkers” for help within reason, that’s okay too.

Hiring a journalist? Give them a reporting, writing, and/or editing assignment. Hiring a web site producer? Have them build a mini-web site or edit a copy of an existing site. Hiring a designer, have them design a logo for a business or product. (This will help determine how well they can relate their design skills to the business aspects.)

While this is a simulation, the more realistic you can make it, the better.

For information worker type jobs, coming up with a realistic assignment is easier than one for a management or executive leadership position candidate. Another challenge with this is that a manager and leader’s job requires significantly more interaction with others. That means this simulation will consume a fair amount of your and your team’s time. However, remember that manager and executive leaders are very important hires, so the time spent in finding the best possible hire is totally worth it.

Some factors to consider in your evaluation. How resourceful is the person? Do they communicate with the customers? How do they interact with the customers? Do they get the job done? Do they seek help beyond their own resources? Do they become a drain on others’ productivity? How is the quality of their work? What’s their balance of being a team player and contributing individually? How is their planning? How is their presentation? Do they document their solution? How well do they explain their solution?

I admit this is too time consuming to do for everyone who applies for the job. You do need to filter the list down to between three to five candidates for this working interview. There are a couple of ways you can do this. For contract-to-potentially-hire positions, you can rely on a trusted vendor to do the filtering for you according to your guidelines. For others, you do need to review their resumes, see who has recommended them, do phone interviews and give them tests. Computerized tests are a good way to save you and your team some time during the initial screening, provided you present the test in a way that is not demeaning to the candidate. The time/money invested in setting up an automated online test is worth it. The test needs to be respectfully presented as a part of the standard process and should not feel like an entrance exam or impersonal screening.

So next time you have a position to fill, consider using a working-interview.

Tweet

Save

At interviews, technology executives are often asked about build vs. buy. The question would be better articulated as own vs. rent. See, building and buying in the true sense are often part of the same ideology: owning. For example, Google bought Blogger and YouTube. As a result, Blogger and YouTube programmers (builders) became Google employees. When people ask about companies buying software, they are almost always referring to leasing or renting software.

I consider using open source to be in the own camp, because you can withing reason do as you please with your copy of the open source software and no one can take it away from you. Open source makes owning even better. It enables owning and sharing at the same time, which benefits the community.

Often technology executives answer this question ambiguously. They claim they have no preference towards either owning or renting software. To me, when a person provides such a noncommittal answer, it means they might lack leadership, vision, a clear philosophy, or the courage to give an honest answer to a prospective employer.

I generally prefer owning over renting. This applies not just to software, but to other things in my life like owning a home and owning a car. There are other people who prefer to rent apartments and lease cars. Neither philosophy may be better in general than the other, but one of them is always better depending on the circumstances. Circumstances by themselves don’t make the own vs. rent decision. That’s where people come in. A leader does not manage their company or department at the whim of circumstances. A leader has a style, has preferences and applies them to the situation. Leaders who prefer either one of owning or renting can be equally successful in the same organization and circumstances.

Here I present my viewpoint on why sometimes a philosophy of renting software as much as possible runs companies into trouble. There are numerous examples of vendor solutions in search of problems. A vendor will often wow a team of executives with a product presentation and demo. The company will agree to rent the software at a high cost. Later, the company will find it wasn’t worth the investment, even though they may not admit it or even like to talk about it.

When a company owns a software product core to its business, it can use it towards a competetive advantage. When you own the software product, you also have control over your data. When a vendor hosts and manages your data, in most cases you have practically relinquished control of your data even though your contract may say on paper it is yours.

When you own the products you use, it is generally easier to integrate with other systems. Your timelines are less at the mercy of vendors.

However the future of software may be changing. With web applications becoming popular, even home users may be renting software on a pay-per-use basis. I’ll have more to write on that subject. Stay tuned.

Tweet

Save

Here are some tips for making better use of email at work and in personal life.

• When you send an email to multiple people asking for a response or assigning a task, specify by name who you are asking for a response or action from. Otherwise people may read your email and assume one of the others will act on it.
• Avoid using the bcc feature in a sneaky way to tell someone else (e.g. the main recipient’s boss) only your side of the story without telling the known recipients.
• At a workplace, when you send announcements (i.e. when your goal is for you to disseminate information and not to start a discussion), send the message in a way that recipients do not reply-all.
  • You can do that by either using a mailing list that only authorized people can send mail to, or by putting the actual recipients in bcc: field and using a placeholder address in the to: field. That is a legitimate use of bcc:. You could also request in your message that recipients do not reply-all to this message and instead for example, report issues to an alternate address.
  • Managers sometimes cringe when after they send a positive message to a large number of people, one of the recipients replies-all with a negative message that either diminishes or distracts discussion away from the original positive message. An employee who does reply-all to a positive message with a negative one is being foolish or a jerk (usually both). You should avoid giving such people a pulpit and opportunity to lower others’ morale and/or start a flame war. (When you have such employees who don’t stop doing this after being told, fire them and hope they get employed by your competitor.)
  • If you are the recipient of an announcement, do not do a reply-all, unless you have a relevant, positive message to add that adds value to and strengthens the original message.
  • I am not suggesting that you don’t voice your disagreements, corrections, cautions, constructive criticism and other comments. You should express them, but not via reply-all. Communicate them to the appropriate person(s) only, typically that would be the sender of the announcement or your supervisor. If you find an error in someone’s announcement, give the sender the courtesy of an opportunity to send out a correction by letting them know first.
• Give people reasonable time to respond to your email, even if they have mobile devices (like BlackBerry or other email enabled phones).
  • Realize that some people are overwhelmed by email and you should occasionally reach out to them in person or via phone for certain important matters. Email is not a replacement for all personal communication.
• Avoid checking your email on a mobile device when interacting with other people in person except when absolutely necessary, for example in a meeting. It is rude and it implies you are not focusing on your job in the meeting.
  • If your job requires you to be on alert for certain messages, set up alerts via mail filters that will sound or vibrate the device to inform you of messages that require you to interrupt your current activity.

Tweet

Save

When implementing a content management system or other product, customers often ask for workflows that require the least number of steps required to any given complete task. At first, this seems to make perfect sense; however consider this example of a car:

Before you can get inside your car and start driving you have to perform the following steps:

1. unlock the car door
2. pull the door handle
3. open the door
4. get inside the car
5. close the door

Steps 1, 2, 3, and 5 seem to add unnecessary actions to the workflow. The goal here is to be able to start driving to get to the destination. Over the course multiple car trips over a day, these steps seem to “waste” a lot time. An easier and “better” workflow may be for cars not to have a door at all. Then you’d save the steps of having to open and close the door.

However, with the current level of commonly available technologies, it makes sense for a car to have a door and require these steps before you can start driving.

Extra steps are often required to provide security, maintainability, reuse, reliability, scalability and performance.

Shortcuts aren’t always the best solution. You may save steps and thus cost now with shortcuts, but as a result you may pay much more later in other costs.

As technology advances, some necessary steps can be automated or eliminated. For example, some cars now have keyless entry that eliminates some of these steps. In the future, an advanced version of keyless entry may even open and close the door for you. However, expecting those in a car of today would be impractical.

Similarly in content management and other software extra steps aren’t always a bad thing. A good content management system isn’t one that allows web site producers to complete their tasks in the least number of steps. It is one that enables completion of the task in an optimal number of steps balanced with other factors like reuse, maintainability, flexibility and security.

Tweet

Save

Starting early, not driving recklessly fast

People who have worked with me are familiar with my trait of challenging the team to bring products and solutions to market as soon as possible. I’m a strong proponent for quickness to market and love to deliver sooner than the initially projected timeline. In this article, however, I present a different viewpoint for balance.

In product development, the question often comes up: How can we be quicker and faster to market with our products? We should ask instead: How can we be earlier to market with our products than our competitors? We should also ask: Is it more important to be early, or to deliver good quality and innovation?

For the medium and long term good of your organization and in the best interest of your customers, it is more important to deliver a high quality and innovative product than to deliver it quicker.

In most cases, successful companies are not the ones who are fast or early to deliver products, but those that deliver better products.

Take Google for example. They were a couple of years late to the Web search engine market and were reinventing a product that had already been established by others. Many thought the search engine market was already saturated. Remember some of the early ones like Infoseek, Lycos? Where are they now? Consider Microsoft and Apple: most of their products are not early, but they often succeed. The iPod came years after the early portable digital audio players. MySpace.com came up to dominate online social networking a couple of years after Friendster, Tribe and Orkut were already established.

Even when analyzing products whose success was due their being early to market, we find that early does not imply fast. These projects often started early and were executed at a comfortable, smooth pace.

As the saying goes, when you ask for quick and dirty, you get both. The benefits of speed to market are for the short term. In some cases, it does make sense to go for quick, short-term solutions. In all cases, however, one must give serious thought to whether that’s the correct path to choose considering the medium and long term goals.

People & Teamwork

In projects, working fast is often a recipe for failure, especially after starting late. The overwhelming majority of projects are not like 100 meter races, where speed results in victory. They are like football games, where factors like teamwork have much greater influence on winning.

The greatest factor affecting the success of projects is not speed, not technology, not even process or planning. It is people. Invest your time, energy and resources on your people and they will make your projects succeed more than anything else.

Whether you are a leader, manager or information worker, I recommend learning more about the people factor and practicing better people related activities at work. Here is a quote I like from a book: “People under time pressure don’t work better; they just work faster. In order to work faster, they may have to sacrifice the quality of the product and their own job satisfaction.” — Peopleware: Productive Projects and Teams, 2nd Edition, Tom DeMarco and Timothy Lister

Keep in mind this order of descending significance of factors in projects’ success:

1. People & teamwork
2. Priorities
3. Planning
4. Process & operations
5. Products & technologies
6. Pace & acceleration

Tweet

Save

Nowadays, it is becoming fashionable to belittle technology. I hear people say things like “technologies should not drive business decisions“, “define your requirements without worrying about technology and ask technology [people] to deliver them“. May sound logical at first, but is it? Consider this imaginary conversation between Bill, a technologist and Plato, a Platonist businessperson.

Bill: How do you commute to work?
Plato: I drive.
Bill: Why don’t you fly or teleport?
Plato: What do you mean?
Bill: Why do you use a car that drives on the road, why not a personal flying machine or a teleporter?
Plato: Because they aren’t invented yet.

Defining business needs without consideration of technology is impractical. It is being quixotic and ignoring the current reality. Leave the dreaming for the innovators since they and businesspeople are usually different people.

Think about ten important inventions that changed the world. How many of them were created with a business plan? How do you think the wheel was invented? How was fire discovered? Did someone create this world wide web with a business plan? Many important things were created with a business plan. The point here is not that business is unimportant, but that technology is important on its own merit.

Sound business practices have an important place in this world. Technology and innovation have a place in this world. One is not master to the other.

Tweet

Save

Great businesses are often started by people without a formal business background but with passion, vision and the ability to execute.

Neither Craigslist nor [founder and chairman Craig] Newmark benefits financially from the deal. EBay and Craigslist both were started in 1995 by pocket-protector-type programmers; each became wildly successful in its own way, attracting fervent communities of people who wanted to make a deal of one sort or another. … Over the years, Newmark has turned down numerous offers to sell all or part of Craigslist to outside investors. In a posting on his Web log, Newmark said the sale to EBay was conducted by a former employee to whom he had given some equity “with the idea of establishing checks and balances, mostly on myself.” … Over two months of meetings, EBay assured Newmark and Buckmaster [Craigslist CEO] it would not pursue the deal unless they were happy about it, Buckmaster said. Both men said they felt that EBay CEO Meg Whitman and founder Pierre Omidyar “showed that they were interested in us for all the right reasons,” as Newmark put it on his blog.”Once we knew this equity interest was likely to change hands, we can’t imagine a better outcome than the one we’ve gotten here,” Buckmaster said.

 
Full story in the San Francisco Chronicle:
http://www.sfgate.com/cgi-bin/article.cgi?file=/c/a/2004/08/14/BUG6B87P0F1.DTL&type=printable

Tweet

Save

Victory is not defeating others. Victory is winning others.

The three skills that people in an organization have are leading, managing, and working. Every person has a combination of all three. I refer to a leader as someone who is strongest in leadership, a manager as one who is strongest in management and a worker as one who is strongest in workmanship.

To understand better what leadership is, let us first discuss it in the context of what leadership is not.

Leadership is not equivalent to managing others. A leader manages him/her self while teaching and guiding people to manage themselves and others.

Leadership is different from management in several aspects. While a person can be a good leader as well as a good manager, the two are two different talents. A President & CEO needs to be both a leader and a manager. The Chief Architect can just be a leader. A Project Manager can be just a manager. A programmer can just be a worker. A manager often needs to make others do things, but rarely needs to do those things her/himself. This does not imply that a manager’s role is less significant. The manager does the important work of making the workers successfully do their work.

A manager does not need to be an innovator, but a leader needs to be. A manager follows the rules and makes workers follow the rules. A manager is most effective when things are going according to plan. A leader on the other hand, is equally effective when things are going according to plan or not. A leader rewrites the rules when necessary. A leader guides the manager in coming up with a new plan when needed. A leader often realizes it even before a new plan is needed. A leader is a visionary. A leader imagines great things. The leader formulates a high level plan. The manager creates a detailed plan from that and manages workers to implement using that plan.

To be a leader does not mean being a high and mighty boss. A leader is a servant as much as a commander. A good leader cares about others. A leader must lead others to success. If a leader’s goal was to achieve success only for one’s own self, then I’d call him or her a climber, for one who climbs to success, but not a leader. A leader who does not benefit others serves no purpose in an organization or in society.

Being a leader sometimes requires sacrificing ones own interests for the good of others.

Contrary to some beliefs, a leader should be generally popular and liked. A leader does not do just what the leader thinks is best for others, but what the team decides is best for the team. A leader provides results and enlightenment (as in understanding of those results). If people consistently do not understand what or why the leader is doing, the leader needs to be replaced. A leader realizes that people are intelligent and gives them appropriate credit.

A leader is not merely a teacher. A leader is a student and an instructor.

Leadership is not easy. It is not for everyone. Can anyone become a good leader? Yes. Should everyone become a leader? No. (As an illustration consider the question: Should everyone become a carpenter? No.)

(Note: In his career, the author has been successful and won awards while working at various levels of Programmer, Manager, Regional Director, Vice President and External Consultant. The knowledge presented here is compiled from those experiences and by learning from others. The author does not claim these ideas as original.)

Tweet

Save