Web Site of Rajiv Pant

CAREER-CLEAR version 2.0 2010-Aug-26

CAREER-CLEAR is a system for doing fair, consistent and constructive employee performance evaluations and determining employee rank, title and compensation. It is meant to be used by supervisors to identify areas for improvement for their employees and to guide their career growth.

Employees are scored in a total of 5 categories. Upto 10 points can be earned in each category for a total of upto 50 points. The final score is then multiplied by a factor of 2 to give a standard scale of 0 to 100. Using a normalized 100 point scale allows it to remain consistent (by adjusting the factor) even as companies add/remove categories and items.

The resulting numerical score can be mapped to the employee’s level of seniority or rank for title and/or compensation. For example, a score of 81-100 could map to director/VP levels; 61-80 manager; 41-60 engineer/contributor; 21-40 junior level/apprentice. Since different departments — for example, software engineering and quality assurance testing — may have different pay scales, this score maps directly to rank/title, and those are mapped to salaries corresponding to the departments’ market rates.

The first four categories are described below. The fifth category is defined as discretionary/user-defined. CAREER-CLEAR is designed to be used in the real world, in a diversity of organizations and on a regular basis. The system won’t suceed if it is too rigid. On the other hand, the system must meet its goals of being fair, consistent and constructive for all employees. To accomodate and balance these goals, 20% of the criteria is meant to be user-defined at descretion of the manager within the fair, consistent and constructive guidelines.

It is inspired by systems described to be in use at Microsoft, Construx, FogCreek (Joel on Software) and Conde Nast Digital Technology. The latter was developed by Bobby Chowdhury, Brian Murphy, Janet Kasdan and Rajiv Pant.

The 5 categories are: Caliber, Leadership, Expertise, Role and Discretionary.

Caliber

This section measures the talent of the employee in general (non-technical) areas.

Scoring: Above Average=2, Average=1, Below Average=0. Add the score for each of the heuristics. Max Score=10 points.

1. Ownership – Has identifiable long-term ownership of projects. This is a measure of the criticality, complexity and / or number of projects the employee has ownership in.
2. Responsibility – Is consistently reliable in terms of deliverables and time.
3. Communication – Communicates effectively with peers and other colleagues. Listens to and understands others’ viewpoints, challenges, needs and desires.
4. Consistency – Is approachable, predictable, receptive and consistently applies good judgment in all interpersonal interactions in the work place.
5. Innovative – Innovates and stays abreast of emerging technologies and finds ways to incorporate those technologies into systems.

Leadership

This section evaluates the positive influence the employee has on others.

Scoring: Above Average=2, Average=1, Below Average=0. Add the score for each of the heuristics. Max Score=10 points.

1. Teacher, Coach & Motivator – Mentors others, makes great use of all information sharing tools available and is an active presenter. Rallies the troops and improves morale.
2. Enabler – Empowers and enables others to succeed.
3. Exemplary – Leads by example and goes above and beyond the ‘requirements’.
4. Maturity & Humility – Embraces others’ solutions, even when incompatible with one’s own. Incorporates feedback from others to find the best solutions.
5. Connector – Has familiarity with the ecosystem beyond one’s own projects. Functions as a hub which others are drawn to for a quick answer or a quick redirect towards an answer.

Expertise

This section quantifies the skills and experience of the employee related to the job function.

Scoring: Above Average=2, Average=1, Below Average=0. Add the score for each of the heuristics. Max Score=10 points.

1. Fundamentals – Understands of the core technical concepts aligned with the given job function. This may include data structures & algorithms, testing, networking, etc.
2. Breadth of Expertise – Is a subject matter expert and go-to person for many areas of technology.
3. Pragmatic – Has a demonstrated ability to identify the best solution to balance what’s most theoretically ideal against what might be the most practical due to concerns about security, scalability, time to market pressures and cost.
4. Automator – Consistently works to drive improvement in processes and systems.
5. “Boy/Girl Scout Rule” – Leaves code and systems better off than they found them.

Role

This section enumerates the employee’s role and areas of contribution within the organization and beyond.

Scoring: Above Average=2, Average=1, Below Average=0. Add the score for each of the heuristics. Max Score=10 points.

1. Strategic – Provides sound vision for broad, long-term goals.
2. Tactical – Oversees many projects or activities that move the organization towards strategic goals.
3. Operational – Steers day-to-day processes that achieve the tactical goals.
4. Executional – Implements repetitive tasks that make up the operational processes. A measure of quantity and more importantly, quality of work produced.
5. Industry Recognition – Is recognized externally as a leading technologist through contributions to open source projects, blogging, writing books, participating in technical committees, speaking at conferences, etc.

Discretionary

Please be sure to adhere to the goals of being fair, consistent and constructive for all employees in using this discretionary section. This category is not meant to be used to justify favoritism nor meant to be arbitrary. Good descretion comes from rational, reasonable and relevant criteria. Place items here that are not already covered in other categories and are important to your organization. A good rule of thumb is that you must be able to justify any criteria you apply here.

Scoring: Above Average=2, Average=1, Below Average=0. Add the score for each of the heuristics. Max Score=10 points.

1. discretionary / user-defined item 1
2. discretionary / user-defined item 2
3. discretionary / user-defined item 3
4. discretionary / user-defined item 4
5. discretionary / user-defined item 5

Tweet

Save

Problem to solve:

When a team of engineers is dealing with a real time incident, such as a system outage or troubleshooting a problem, having good communications is critically important. The appropriate communications tool can make a world of difference in dealing with the issue and learning from it afterwards. As important as engineering is, lack of good communications is what often gets tech teams in trouble.

Improvements that can be make:
Enable real-time communication in certain collaborative tasks (described below), reduce unnecessary email traffic and clutter, enable people to to focus better on their tasks, reduce time wasted in bringing each other up to speed

Description and solution:

When multiple people are working together in real-time on a near term collaborative task, such as:

• Troubleshooting
• Build and deployment
• Web application migration
• Upgrade or maintenance
• QA testing

Many companies use a phone conference and/or email to assist in real-time while the collaborative activity above is ongoing.

Since Email is not instantaneous and real-time the way a group chat application is, and since email is not a suitable medium for quick questions, and quick one-line  responses, some companies also use a real-time group chat tool like IRC to enable and facilitate real-time conversation.

Benefits of using IRC or a real-time group chat tool are:

• Tech managers, project managers, crisis managers and new tech people joining the effort can quickly catch up with what has been going on (in any level of detail they want) by reading the IRC history transcript so far. This is a much faster and efficient way than using email or pulling someone away to talk in person asking what has been going on. (If email were to be used instead of IRC, a new person joining in would have missed the previous emails on the topic.)
• When an engineer working on such a collaborative task steps away for a while and comes back, they can quickly catch up on what transpired while they were away by reading the IRC history transcript.
• Email is not cluttered by short back and forth messages with lots of text to read and filter
• The IRC transcript can be used for the incident retrospective (“post-mortem”).
• Unlike a phone-only conference, the IRC transcript can be read and analyzed to learn lessons from this incident. For example:
  • Analyze what problems the team ran into
  • Analyze what worked and what didn’t
  • Analyze how well people collaborated and communicated
  • Timelines of events

I can personally attest to the above benefits. Over the past 15+ years, my development and operations teams in different companies have regularly used IRC to great advantage.

Tools like Wikis and blogs are great for collaboration, documentation and sharing information on projects. An group chat like IRC is an indispensable tool for real-time collaboration.

Tweet

Save

For an executive, having a management team of people who are good at their jobs and work well with each other is one of the most important factors that lead to success together. Observing a number of successful projects, I realized that it is critical that your management team members care for each other, work well together and give to each other. Their sincere collaboration is far more important than their individual strengths.

I began to write this article impressed by how well the management team comprising of my direct reports functioned, collaborating with each other towards shared success. I was pleasantly surprised by how these directors shared responsibilities, how closely they worked with people in each other’s teams and how comfortably they gave credit to each other. When conflicts arose between them, they frankly, respectfully and nicely expressed them to each other, often one-on-one. Every time, they resolved them quickly and came out with a closer professional relationship. They actively and regularly talked to quell any turf battles between each other’s departments before they could form.

They had a wonderful professional relationship. They barely knew each other outside of work, having busy personal lives with their families on most evenings and weekends. I felt that my management team and I were like a work-family, sticking together through good and bad times, always believing that our success comes as a team.

When you manage and organize your company or your department, spend time multiple times a week with your direct reports together so that you all work well with each other towards shared success. In turn, they should ensure that their direct reports care about each other and collaborate. If you have, say five direct reports, make sure that just the six of you get together in a room to work openly and collaboratively at least twice a week (assuming you are in the same geographic location). The forum for this need not be always a staff meeting, it could be a working session on a project.

I was struggling to come up with suitable words to describe this and its importance. While reading the book The Four Obsessions of an Extraordinary Executive: A Leadership Fable by Patrick Lencioni, I found that the first discipline described in the story talks exactly of this and hence is the title of this article. The book is written as a fictional story that teaches leadership lessons. It is easy to read being under 200 pages in large typeface which you can read in one evening. I highly recommend it.

Tweet

Save

There are many good ways to organize your technology department. This article presents one of them. It is written for someone in a CTO, CIO, VP Technology or a consultant role looking for suggestions on organizing or reorganizing your Web technology department. It is best suited for you if your organization has the following characteristics:

• You manage software engineering, implementation and technology operations for 3 or more Web sites.
• Yours is a medium to large size organization with somewhere between 20 to 150 technology staff.
• Internal corporate IT functions such as desktop support, telecommunications services and internal business systems are beyond the scope of this article.

The Venn diagram below presents one model of organizing your department into 3 sub-departments.

Web Technology Department Organization Venn Diagram Illustrating Purposeful Overlap Among Sub-Departments

Many CTOs organize their technology departments as 2 sub-departments: Software Engineering and Technology Operations. Software engineering is the function that is responsible for developing and/or implementing Web application software. Technology Operations is responsible for running, maintaining and supporting the Web applications.

If you operate 1 or 2 Web sites, having these 2 sub-departments is a good approach. For 3 or more Web sites, organizing Software Engineering into Site Engineering and Platform Engineering has some benefits.

Site Engineering is focused on working on the Web sites’ direct projects. Its work includes

• Small and large projects for adding or changing functionality on the Web sites
• Bug fixes on the Web site applications

Platform Engineering is typically smaller than the other two organizations and typically includes functions like:

• Architecture across sites
• Shared applications across sites
• Common libraries across sites
• Research & Development (R&D)

Technology Operations includes functions such as:

• Systems & Applications Administration
• Infrastructure Management
• 24×7 Tech Support
• Builds & Configuration
• Release Management
• Testing & Quality Assurance (QA) *
• Technical Analysis
• Technical Project Management
• Budget Management

* Note: QA can also be a completely independent department or part of another department outside technology.

These three departments have purposeful overlap of responsibilities as illustrated in the Venn diagram above. That helps minimize the chances of the departments becoming silos with walls between them. For success, it is important that your entire department functions as one integrated unit. Some shared goals & responsibilities are required for mutual success.

To make this work, you need 3 directors who head up these departments who work well together, collaborate often and are not sensitive about their turf. They should know that a successful technology manager is not an individual-only contributor, but a great team player with peers. They should have strong goodwill among each other and welcome each other to work directly with their teams. Such a collaborative team is essential.

Tweet

1 save

If you supervise employees or are in a senior position relative to some others, make it a habit to thank employees when they do a good job.

Here are some tips for effective appreciation of employees’ work:

Appreciation is good even when it is an expected part of their duties. If the work is good quality, thank the person.

Don’t trivialize your thanks, however. Don’t thank an employee for just sending a meeting agenda or sending a recap of a meeting unless the agenda or recap they sent is so good that it impresses you. (An exception to this rule would be if in the company most people forget to send meeting recaps and this person did and you are working to encourage this good habit.)

Don’t just send an email saying “thank you” and nothing else. Add at least a sentence or two explaining why you appreciate what the person did.

Don’t always send an email. Mix it up. Sometimes walk up to the employee and thank them in person. If they are in a remote office location, call them by phone to tell them.

When thanking someone by replying to an email they have sent, send a thank you note directly to the person. Don’t do a reply-all unless it is a major above & beyond accomplishment that deserves public congratulations. When you do send a public thank you to an employee with other people cc’d, do take the time to write at least a paragraph or a few bullet points mentioning the benefits of the accomplishment.

Mention by name the person(s) when you thank people. Don’t just thank “the team”, or “all who worked on this project” out of laziness instead of calling by name the people you are thanking, unless you have a good reason for not naming names.

Tweet

1 save

My colleague Jon Stokes who is Senior Editor and co-founder at Ars Technica interviewed me on the topic of IT consumerization, and about the shifting boundary between professional and the personal profiles.

Jon writes:

For this third installment of my series on IT consumerization, I interviewed Rajiv Pant, vice president of technology at CondéNet (the digital arm of Condé Nast Publications). Rajiv is mainly responsible for the company’s online presence, but he’s watching many of the same trends and themes that I’ve outlined in the previous two installments play out across CondéNet.

The full article is at:
http://arstechnica.com/articles/paedia/it-consumerization-enterprise.ars

Tweet

Save

My journalist colleagues at Wired.com published some of my comments related to Amazon S3.¹ Wired also posted another article titled Customers Shrug Off S3 Service Failure. I agree with the views of many of the customers expressed in the article. Don MacAskill, CEO of the popular photo hosting site Smugmug, wrote an understanding post about it.

My entire career working for media companies, I’ve held firm the belief that the uptime, reliability, performance, scalability, performance and security of commercial Web sites is of paramount importance. When sites that I’ve been responsible for have had issues, my colleagues and I have given our personal time and energy to resolution. With my teams, I spend considerable time on proactive measures. I’ve had the honor of working closely with and learning from some who do an excellent job running technology operations.

Experience has taught that things can and sometimes do go wrong. Sometimes calculated risks don’t pan out. Sometimes mistakes cause problems. We are human. We should strive for perfection; we can get close to it, but not fully attain it. We should be prepared for such scenarios. When they happen, we should work diligently and expeditiously on resolution and have frequent and honest communications with stakeholders and customers. Such communications during the incident should include:

Update 2010-Jan-24: This checklist is now maintained on the Checklists Wiki Web site at:

www.listswiki.com/wiki/IT_Incident_Reporting

During-Incident Communication Checklist

• Current status
• What is the full impact?
• Estimated time to resolution
• Any recommended workarounds until resolution, if practical
• Assurance that it is being worked on
  • It often helps to mention who all are working on it and what they are doing

The post-incident communications to stakeholders and customers should include:

Post-Incident Communication Checklist

• Summary
• What happened, how and why it happened?
  • Including full description of all impact
  • Do not blame² third-parties or say things like “beyond our control”. A technology leader takes responsibility equally for both insourced and outsourced products and services.³
• How it was resolved
  • If the resolution is temporary or long-term
• Next steps
• Plan for eliminating or minimizing this and similar incidents from happening again
• Thank all those who helped resolve and the customers for their understanding
• Mention the monetary credits you plan to give as per the Service Level Agreement (SLA)
  • Specify any additional ‘make goods’ or returns you plan to make to the customers above and beyond the credits as per SLA, if appropriate.

Stakeholders and customers here refer to internal customers of the technology operations team (e.g. the concerned folks in editorial, marketing, sales, finance, legal and other departments). External communications to the public Internet should be handled in consultation with legal and public relations.

S3′s outage (or any outage) isn’t to be taken lightly, but I have faith Amazon and their customers will learn from it.

Disclaimers:

• As explained in the terms of use of this site, any opinions expressed on my personal Web site do not reflect those of any employer, past or present. My Web site and I in my personal life neither represent nor speak for any corporation.
• I have no affiliation, financial or otherwise with Amazon.com. I happen to be a user of their products and services, some of which I like and some that I don’t.
• Personal Web sites like this are exempt from the performance requirements of corporate Web sites :-) My personal Web site is for expressing, learning and R&D. It also happens to be hosted on Amazon EC2 and S3.

Tweet

2 saves

1. Silicon Alley Insider and ValleyWag have amusing spins on it. :-) [↩]
2. There may be extreme instances, especially when criminal activity or malicious wrongdoing was the cause where it would be appropriate to blame someone. [↩]
3. It is ok to mention service providers, or describing external events for explaining what happened, but don’t do it in a “it was their fault, not ours” tone. The technology leader should factually describe what happened and take responsibility. [↩]

Described here is one way to enable technologists to grow their careers in your organization while still allowing them to focus on the type of work they are best at and enjoy most.

The typical management career growth path does not suit some technical people. These information workers need to grow in their careers (gain greater compensation, responsibilities and influence) without having to become managers of other people. A good way to achieve that goal is to create a technical career growth track in your organization.

The following table shows management seniority positions alongside their equivalent technical seniority positions.

In this model, for example, an architect role is at the same compensation and influence level as a manager role, assuming that the particular manager and architect being compared add similar value to the company. If the organization requires more layers, say, a senior architect would be at the same level as a senior manager.

If the organization prefers consistent titles for levels, the system could name them like this: the fellow role as director & fellow, the senior architect role as senior manager & architect, etc. In the case of a fellow who is at a VP or SVP level, they should always be named VP & fellow or SVP & fellow.

Here is a definition of the fellow role from WikiPedia:¹

Large corporations in research and development-intensive industries ² appoint a small number of senior scientists and engineers as Fellows. Fellow is the most senior rank or title one can achieve on a technical career, though some fellows also hold business titles such as vice president or chief technology officer.

Such a technical career growth plan brings many benefits to your organization.

• It helps retain good technologists who want to grow in their careers, but want to do keep doing the type of work they are best at and enjoy doing: technical work.
• It avoids brilliant technical people from being “pushed” (by themselves or their supervisors trying to “reward” them) into people-management responsibilities.
• It reduces situations of having too many people-managers but not enough people-management positions over time as people get promoted.

Care should be taken to recognize that some technical people do enjoy making the transition to people-management roles and the presence such a technical track should not discourage them. Having an alternate career growth track option is about presenting employees with more than one choice.

A similar system could also be applied to other departments with individual contributors. For example, creative design.

Related Articles on Other Sites

• Fog Creek Professional Ladder by Joel Spolsky
• Construx’s professional development ladder

Tweet

1 save

1. Definition of Fellow at WikiPedia http://en.wikipedia.org/wiki/Fellow and Wikitionary http://en.wiktionary.org/wiki/fellow [↩]
2. IBM or Sun Microsystems in information technology, and Boston Scientific in Medical Devices for example [↩]

I’ve found this to be an effective way of evaluating potential hires compared to just interviewing in a question/answer format: Put them to work for a few hours (or even days/weeks/months as contractors) and see how well they perform.

Having someone do the job as a short-term temporary contractor before hiring them is one of my favorite ways of testing them for the job. However, sometimes this isn’t practical. In those cases, having the candidate work on a short test assignment for a few hours can be very telling.

For example, when interviewing a software programmer, set them up on a computer and give them a programming assignment representative of the type of work they would do on the job. Give them reasonable freedom as you’d give an actual employee. If they need to research references / solutions on the Web, let them. If they want to call a friend for consulting help, that’s fine. If they ask for your or their “coworkers” for help within reason, that’s okay too.

Hiring a journalist? Give them a reporting, writing, and/or editing assignment. Hiring a web site producer? Have them build a mini-web site or edit a copy of an existing site. Hiring a designer, have them design a logo for a business or product. (This will help determine how well they can relate their design skills to the business aspects.)

While this is a simulation, the more realistic you can make it, the better.

For information worker type jobs, coming up with a realistic assignment is easier than one for a management or executive leadership position candidate. Another challenge with this is that a manager and leader’s job requires significantly more interaction with others. That means this simulation will consume a fair amount of your and your team’s time. However, remember that manager and executive leaders are very important hires, so the time spent in finding the best possible hire is totally worth it.

Some factors to consider in your evaluation. How resourceful is the person? Do they communicate with the customers? How do they interact with the customers? Do they get the job done? Do they seek help beyond their own resources? Do they become a drain on others’ productivity? How is the quality of their work? What’s their balance of being a team player and contributing individually? How is their planning? How is their presentation? Do they document their solution? How well do they explain their solution?

I admit this is too time consuming to do for everyone who applies for the job. You do need to filter the list down to between three to five candidates for this working interview. There are a couple of ways you can do this. For contract-to-potentially-hire positions, you can rely on a trusted vendor to do the filtering for you according to your guidelines. For others, you do need to review their resumes, see who has recommended them, do phone interviews and give them tests. Computerized tests are a good way to save you and your team some time during the initial screening, provided you present the test in a way that is not demeaning to the candidate. The time/money invested in setting up an automated online test is worth it. The test needs to be respectfully presented as a part of the standard process and should not feel like an entrance exam or impersonal screening.

So next time you have a position to fill, consider using a working-interview.

Tweet

Save

At interviews, technology executives are often asked about build vs. buy. The question would be better articulated as own vs. rent. See, building and buying in the true sense are often part of the same ideology: owning. For example, Google bought Blogger and YouTube. As a result, Blogger and YouTube programmers (builders) became Google employees. When people ask about companies buying software, they are almost always referring to leasing or renting software.

I consider using open source to be in the own camp, because you can withing reason do as you please with your copy of the open source software and no one can take it away from you. Open source makes owning even better. It enables owning and sharing at the same time, which benefits the community.

Often technology executives answer this question ambiguously. They claim they have no preference towards either owning or renting software. To me, when a person provides such a noncommittal answer, it means they might lack leadership, vision, a clear philosophy, or the courage to give an honest answer to a prospective employer.

I generally prefer owning over renting. This applies not just to software, but to other things in my life like owning a home and owning a car. There are other people who prefer to rent apartments and lease cars. Neither philosophy may be better in general than the other, but one of them is always better depending on the circumstances. Circumstances by themselves don’t make the own vs. rent decision. That’s where people come in. A leader does not manage their company or department at the whim of circumstances. A leader has a style, has preferences and applies them to the situation. Leaders who prefer either one of owning or renting can be equally successful in the same organization and circumstances.

Here I present my viewpoint on why sometimes a philosophy of renting software as much as possible runs companies into trouble. There are numerous examples of vendor solutions in search of problems. A vendor will often wow a team of executives with a product presentation and demo. The company will agree to rent the software at a high cost. Later, the company will find it wasn’t worth the investment, even though they may not admit it or even like to talk about it.

When a company owns a software product core to its business, it can use it towards a competetive advantage. When you own the software product, you also have control over your data. When a vendor hosts and manages your data, in most cases you have practically relinquished control of your data even though your contract may say on paper it is yours.

When you own the products you use, it is generally easier to integrate with other systems. Your timelines are less at the mercy of vendors.

However the future of software may be changing. With web applications becoming popular, even home users may be renting software on a pay-per-use basis. I’ll have more to write on that subject. Stay tuned.

Tweet

Save