Using Amazon Elastic Block Store (EBS) with an EC2 Instance

Posted on March 18, 2009 by Rajiv Pant

Amazon AWS Logo

One of the differences between Amazon EC2 server instances and normal servers is that the server’s local disk storage state (i.e. changes to data) on EC2 instances does not persist over instance shutdowns and powering on. This was mentioned in my earlier post about hosting my Web site on Amazon EC2 and S3,

Therefore, it is a good idea to store your home directory, Web document root and databases on an Amazon EBS volume, where the data does persist like in a normal networked hard drive. Another benefit of using an Amazon EBS volume as a data disk is that it separates your operating system image from your data. This way, when you upgrade from a server instance with less computing power to one with more computing power, you can reattach your data drive to it for use there.

You can create an EBS volume and attach it to your EC2 server instance using a procedure similar to the following.

First, create an EBS volume.

You can use Elasticfox Firefox Extension for Amazon EC2 to:

create a EBS volume
attach it to your EC2 instance
alias it to a device, In this example, we use /dev/sdh

Then attach the “disk” to your EC2 instance and move your folders to it using a procedure similar to the following commands issued from a bash shell.

# Initialize (format) the EBS drive to prepare it for use # Note: replace /dev/sdh below with the device you used for this EBS drive mkfs.ext3 /dev/sdh # # Create the mount point where the EBS drive will be mounted sudo mkdir /mnt/rj-09031301# Side note: I use a naming convention of rj-YYMMDDNN to assign unique names # to my disk drives, where YYMMDD is the date the drive was put into service # and NN is the serial number of the disk created that day.# # Mount the EBS drive sudo mount -t ext3 /dev/sdh /mnt/rj-09031301 # # Temporarily stop the Apache Web server sudo /etc/init.d/apache2 stop# # Move the current /home folder to a temporary backup # This temporary backup folder can be deleted later sudo mv /home /home.backup# # Symbolic link the home folder on the EBS disk as the /home folder sudo ln -s /mnt/rj-09031301/home /home # # Start the Apache Web server sudo /etc/init.d/apache2 start

Limitations:

One current limitation of EBS volumes is that a particular EBS disk can only be attached to one server instance at a given time. Hopefully, in a near future version upgrade of EC2 and EBS, Amazon will enable an EBS volume to be attached to multiple concurrent server instances. That will enable EBS to be used similar to how SAN or NAS storage is used in a traditional (pre cloud computing era) server environment. That will enable scaling Web (and other) applications without having to copy and synchronize data across multiple EBS instances. Until Amazon adds that feature, you will need to maintain one EBS disk per server and keep their data in sync. One method of making the initial clones is to use the feature that creates a snapshot of an EBS volume onto S3.

Organizing a Digital Technology Department of Medium Size in a Media Company

Posted on March 17, 2009 by Rajiv Pant

There are many good ways to organize your technology department. This article presents some of them. It is written for a CTO or VP Technology leading a medium size department looking for suggestions on organizing or reorganizing your Digital (Web, Mobile) technology department. It is best suited for you if your organization has the following characteristics:

You manage software engineering, implementation and technology operations for 3 or more digital brands.
Yours is a medium size technology department with somewhere between 20 to 100 technology staff.
Internal corporate IT functions such as desktop support, telecommunications services and internal business systems are beyond the scope of this article.

The Venn diagram below presents one model of organizing your department into 3 sub-departments.

Web Technology Department Organization Venn Diagram Illustrating Purposeful Overlap Among Sub-Departments

Some CTOs in smaller companies organize their technology departments as 2 sub-departments: Software Engineering and Technology Operations. Software engineering is the function that is responsible for developing and implementing Web & Mobile application software. Technology Operations is responsible for running, maintaining and supporting the Web applications.

If you operate 1 or 2 digital brands (Web sites), having these 2 sub-departments is a good approach. For 3 or more Web sites, organizing Software Engineering into Site Engineering and Platform Engineering has some benefits.

Site Engineering is focused on working on the Web sites’ direct projects. Its work includes

Small and large projects for adding or changing functionality on the Web sites
Bug fixes on the Web site applications

Platform Engineering is typically smaller than the other two organizations and typically includes functions like:

Architecture across sites
Shared applications across sites
Common libraries across sites
Research & Development (R&D)

Technology Operations includes functions such as:

Systems & Applications Administration
Infrastructure Management
24×7 Tech Support
Builds & Configuration
Release Management
Testing & Quality Assurance (QA)¹
Technical Analysis
Technical Project Management
Budget Management

These three departments have purposeful overlap of responsibilities as illustrated in the Venn diagram above. That helps minimize the chances of the departments becoming silos with walls between them. For success, it is important that your entire department functions as one integrated unit. Some shared goals & responsibilities are required for mutual success.

DevOps² is a set of processes, methods and systems for communication, collaboration and integration between departments for Development (Applications/Software Engineering) and Technology Operations. Its purpose is to facilitate meeting business goals by producing good quality software products and services in a timely fashion. It is where development methodologies (such as agile software development) occur in an organization with separate departments for Development, Technology Operations and Quality Assurance. Development and deployment activities that need deep cross-departmental integration with Technology Support or QA require intimate multi-departmental collaboration.³

llustration showing DevOps as the intersection of Development (Software Engineering), Technology Operations and Quality Assurance (QA)

To make this work, you need 3 directors who head up these departments who work well together, collaborate often and are not sensitive about their turf. They should know that a successful technology manager is not an individual-only contributor, but a great team player with peers. They should have strong goodwill among each other and welcome each other to work directly with their teams. Such a collaborative team is essential.

Article Updated: September 25, 2010

QA can also be set up as an independent department. [↩]
WikiPedia entry on DevOps [↩]
Article: What is DevOps? [↩]

I now use a device called Drobo for storing data at my home network (Product Review)

Posted on February 8, 2009 by Rajiv Pant

drobo I now use a device called Drobo (2nd generation), manufactured by Data Robotics, Inc. as the primary data storage and backup medium at my home location. I have attached it via a USB 2.0 cable to my Apple Airport Extreme wireless network router. The Airport Extreme enables me to share USB 2.0 based storage devices on my home network so they can be simultaneously used by multiple computers. This system of making the same hard disk(s) available to multiple computers in a network is called Network Attached Storage (NAS).

The Drobo replaced my USB 2.0 external hard disk drive manufactured by Western Digital (WD) that was earlier attached to my Airport Extreme. The Drobo has significant advantages over the previous WD drive:

Data protection in the event of one hard drive failure as a result of wear and tear due to use over time
Ability to increase the storage size of the device as the volume of my data grows (more photographs, music, videos, etc.)

A data protection strategy should include both local fault tolerance and remote storage in an offsite location. For off site storage, I keep copies of my data at online locations like Amazon S3, Smugmug, Google Docs, IMAP mail servers and Apple’s Mobile Me service. Since the volume of my data is in terabytes (~ 15 years of emails, photographs, music, videos), recovering large amounts of data from online locations is reserved for extreme situations when local storage is destroyed or corrupted. The Drobo uses technology to protect data in cases of failure (via normal wear and tear) of one of the hard disks inside the Drobo. The benefit I get is similar to the benefit provided by a set of technologies called RAID.

Unlike most RAID devices for home/small-business use, the Drobo allows me to mix and match hard drives of varying capacities. It has 4 bays to insert hard drives. For example, I can have two 1 TB drives today. Next month, I can add another 1.5 TB drive to the 3rd slot. A few months later, I can add a 2 TB drive to the 4th slot. Then when I need more space next year, I can replace one of the 1 TB drives with a 2 TB drive. As I make these changes, the Drobo will automatically recalculate the optimal distribution of my data across all these drives to maximize its storage space and provide data protection. Adding a new drive or replacing a drive with another is done without downtime. The Drobo stays up and running during disk changes and the data on it remains usable by my computers, even while I’m replacing a drive or when it is redistributing data on the new set of drives after a drive is inserted.

Benefits

Save money by buying 1 TB drives today and 2 TB drives when they are cheaper in the future
Save money by buying just hard disks for adding storage instead of buying a drive plus an enclosure and power supply adapter for each drive. This is also energy efficient (fewer power adapters) and good for the Environment (fewer drive enclosures made of plastic and power adapter units purchased)
Save time by letting the Drobo take care of data protection at the local level. Also save time that would have been spent recovering data from remote locations in the event of a local drive failure.
Peace of mind having good data protection at home.

Moving data from the WD external drive to Drobo

I transferred the files by having both drives directly connected to my Apple Macbook Pro: The Drobo to the Firewire 800 port and the WD to the USB 2.0 port.

My MacBook Air‘s Time Machine backups used to be stored on the USB external drive. Since it was attached to an Airport Extreme, the Apple Time Machine backups were stored in a special virtual storage location called a sparsebundle. I just copied the sparsebundle from the WD drive to the Drobo and now my Time Machine backups (including all the Time Machine history of my MacBook Air) are now transferred to my Drobo storage device. Thanks to my Time Machine bundles having been on a sparsebundle, it was easy to transfer them to my new Drobo storage device using a simple copy process using the Mac OS Finder.

Rating:

It is all about Balance

Posted on October 25, 2008 by Rajiv Pant

To be effective at anything in life, balance is required Be it personal life, leadership, management, technical work: balance is essential. You need to be aware of balance at all times.

I plan to write a series of articles about this subject.

Some topics I discuss with friends and colleagues:

standardization vs. innovation & creativity (technology, management)
dedicated resources vs. shared services (management, leadership, technology)
security & privacy vs. convenience (technology, product design)
centralization vs. decentralization & empowerment (leadership, management)

I’d love to hear your topic suggestions and read your viewpoints and experiences. Please do so via the comments section here and in future articles.

Ars Technica Interview: IT consumerization in the enterprise

Posted on July 23, 2008 by Rajiv Pant

My colleague Jon Stokes who is Senior Editor and co-founder at Ars Technica interviewed me on the topic of IT consumerization, and about the shifting boundary between professional and the personal profiles.

Jon writes:

For this third installment of my series on IT consumerization, I interviewed Rajiv Pant, vice president of technology at CondeNet (the digital arm of Conde Nast Publications). Rajiv is mainly responsible for the company’s online presence, but he’s watching many of the same trends and themes that I’ve outlined in the previous two installments play out across CondeNet.

The full article is at:
http://arstechnica.com/articles/paedia/it-consumerization-enterprise.ars

Future of Content Management for News Media for Web sites

Posted on March 16, 2008 by Rajiv Pant

Content on Web sites should be managed using systems that were designed from the ground up for the Web. Traditional content management systems with a legacy of features and workflows used for paper-based print products like newspapers and magazines are unsuitable for Web sites. The future of news media content management for Web sites is in:

simple & quick workflows
blogs & wikis as the main content types for text
social networking & community publishing

simple & quick workflows

Complex editorial workflows make sense for print products (on paper) , where once the edition is done, the content and presentation state is “locked” and sent to the presses. Working with Web content writers and editors over the past decade, I have learned that simple, quick workflows are preferable for Web sites. Many Web site producers who hail from print backgrounds now share the same conclusion that complex content management is a hindrance to successful Web site production.

The concept of an edition of the entire product is not necessary for a content Web site. The atomic unit that can be managed and published together can be a package of articles and multimedia or even just one article. A Web site is a living, dynamic, ever changing collection of content where individual items can be updated whenever required or desired or even automatically based on usage.

To be competitive, content needs to be updated and published quickly. Corrections can be made anytime. Thus for Web sites, the editing and approval process should be streamlined and quick all the way from authoring to posting on the site.

A new concept

The ability of online word processors like Google Docs or WriteWith to enable multiple people to edit a document simultaneously and collaboratively is different paradigm from traditional check-in/check-out access control.

blogs & wikis as the main content types for text

Content management system (CMS) which offers the simplicity of blogs and are extensible via plug-ins to add functionality like WordPress or MovableType, make good foundations of a CMS for a news media Web site.

For revisions, editing history and access control, wiki software works well. WikiPedia and WikiNews, which are powered by the MediaWiki software are two good examples.

The concept of content management systems that combine the agility of blogs and editorial control of wikis is interesting to follow. The term bliki seems to be the leading classification of such products.

In many newsrooms, writers are increasingly using blog posts to publish news articles instead of their enterprise-class content management systems. When asked why, they reply because it is simpler and quicker and they don’t need the overhead of things like complex approvals, advanced version tracking and access controls.

social networking & community publishing

Managing content using a blog or wiki is social networking and community publishing activity. On the readership side, successful social news sites like Digg and Reddit have accelerated the evolution of journalism and readership habits towards the social/community model. The distinction between authors and readers itself is blurring with wikis and comments on blogs.

Social networking features are being added to a variety of Web sites. Going forward, expect to see social networking and community features in content management systems.

Conclusion

Media companies should move to using CMS products that prefer simplicity over complex editorial workflows which were a legacy of writing and editing for print products. A news item, story or blog post should be the same content type. It is likely that blogging products that have proven so successful in empowering talented individuals in competing with large companies will evolve into content management systems with the addition of wiki functionality.

Opinion on the Amazon S3 Outage; Checklist for Dealing with Outages

Posted on February 16, 2008 by Rajiv Pant

My journalist colleagues at Wired.com published some of my comments related to Amazon S3.¹ Wired also posted another article titled Customers Shrug Off S3 Service Failure. I agree with the views of many of the customers expressed in the article. Don MacAskill, CEO of the popular photo hosting site Smugmug, wrote an understanding post about it.

My entire career working for media companies, I’ve held firm the belief that the uptime, reliability, performance, scalability, performance and security of commercial Web sites is of paramount importance. When sites that I’ve been responsible for have had issues, my colleagues and I have given our personal time and energy to resolution. With my teams, I spend considerable time on proactive measures. I’ve had the honor of working closely with and learning from some who do an excellent job running technology operations.

Experience has taught that things can and sometimes do go wrong. Sometimes calculated risks don’t pan out. Sometimes mistakes cause problems. We are human. We should strive for perfection; we can get close to it, but not fully attain it. We should be prepared for such scenarios. When they happen, we should work diligently and expeditiously on resolution and have frequent and honest communications with stakeholders and customers. Such communications during the incident should include:

Update 2010-Jan-24: This checklist is now maintained on the Checklists Wiki Web site at:

www.checklistnow.org/wiki/IT_Incident_Reporting

During-Incident Communication Checklist

Current status
What is the full impact?
Estimated time to resolution
Any recommended workarounds until resolution, if practical
Assurance that it is being worked on
- It often helps to mention who all are working on it and what they are doing

The post-incident communications to stakeholders and customers should include:

Post-Incident Communication Checklist

Summary
What happened, how and why it happened?
- Including full description of all impact
- Do not blame² third-parties or say things like “beyond our control”. A technology leader takes responsibility equally for both insourced and outsourced products and services.³
How it was resolved
- If the resolution is temporary or long-term
Next steps
Plan for eliminating or minimizing this and similar incidents from happening again
Thank all those who helped resolve and the customers for their understanding
Mention the monetary credits you plan to give as per the Service Level Agreement (SLA)
- Specify any additional ‘make goods’ or returns you plan to make to the customers above and beyond the credits as per SLA, if appropriate.

Stakeholders and customers here refer to internal customers of the technology operations team (e.g. the concerned folks in editorial, marketing, sales, finance, legal and other departments). External communications to the public Internet should be handled in consultation with legal and public relations.

S3′s outage (or any outage) isn’t to be taken lightly, but I have faith Amazon and their customers will learn from it.

Disclaimers:

As explained in the terms of use of this site, any opinions expressed on my personal Web site do not reflect those of any employer, past or present. My Web site and I in my personal life neither represent nor speak for any corporation.
I have no affiliation, financial or otherwise with Amazon.com. I happen to be a user of their products and services, some of which I like and some that I don’t.
Personal Web sites like this are exempt from the performance requirements of corporate Web sites :-) My personal Web site is for expressing, learning and R&D. It also happens to be hosted on Amazon EC2 and S3.

Silicon Alley Insider and ValleyWag have amusing spins on it. :-) [↩]
There may be extreme instances, especially when criminal activity or malicious wrongdoing was the cause where it would be appropriate to blame someone. [↩]
It is ok to mention service providers, or describing external events for explaining what happened, but don’t do it in a “it was their fault, not ours” tone. The technology leader should factually describe what happened and take responsibility. [↩]

Social Graphs API: WordPress Plugin: Blogroll Links

Posted on February 10, 2008 by Rajiv Pant

If you already know what the Social Graph API and XFN are, you can skip the background information and go directly to the Blogroll Links plugin for WordPress that is designed to work with these.

Update: 2010-Feb-20: Version 2 of the Blogroll Links plugin for WordPress uses the Shortcode API and so introduces a new code-tag format. The new plugin still supports the old (now deprecated) code-tag format for backwards compatibility. See below for examples.

Social Graph API

Google recently announced the Social Graph API.¹ From Google’s Code site:

With so many websites to join, users must decide where to invest significant time in adding their same connections over and over. For developers, this means it is difficult to build successful web applications that hinge upon a critical mass of users for content and interaction. With the Social Graph API, developers can now utilize public connections their users have already created in other web services. It makes information about public connections between people easily available and useful.
We (Google) currently index the public Web for XHTML Friends Network (XFN), Friend of a Friend (FOAF) markup and other publicly declared connections. By supporting open Web standards for describing connections between people, web sites can add to the social infrastructure of the web.

The Google Code site also has a video introduction to the open social graph:

The Google Code site has some interesting example applications. To see the power of the open social graph, follow these links:

Connections between my URLs on my sites and my pages on social networking sites.
Connections with some of my contacts found via social networks and sites that are friendly to the open social graph.

All I did was enter my home page http://www.rajiv.com/ into these applications and got the results linked to above.

XHTML Friends Network, a component of open social networks

XFN (XHTML Friends Network) is a simple way to represent human relationships using hyperlinks. In recent years, blogs and blogrolls have become the fastest growing area of the Web. XFN enables web authors to indicate their relationship(s) to the people in their blogrolls simply by adding a ‘rel‘ attribute to their <a href> tags, e.g.:

<a href="http://www.rajiv.com/" rel="friend met">Home Page: Rajiv Pant</a>

The above link means that the page at http://www.rajiv.com/ belongs to a friend of the person who who owns the page this link is placed on. The met tag specifies that the two friends have met in real life. The link above would not be placed on a page owned by Rajiv Pant. It would be placed by a friend on their page, for example, on http://www.paradox1x.org/

Here is another example:

<a href="http://photos.rajiv.com/" rel="me">Photo Albums: Rajiv Pant</a>

This link states that the page at the URL http://photos.rajiv.com/ belongs to the same person who owns the page this link is placed on. For example, the above link would be placed on http://www.rajiv.com/ telling the Web that the URLs http://photos.rajiv.com/ and http://www.rajiv.com/belong to the same person.

To find out how to write and use XFN, or to write a program to generate or spider it, visit the XFN Web site.

This Web Site is Now Hosted on Amazon EC2 & S3

Posted on February 4, 2008 by Rajiv Pant

This web site, www.rajiv.com is now hosted on Amazon.com’s Elastic Compute Cloud (EC2) and Simple Storage Service (S3) services. They are part of Amazon Web Services offerings. If you are a technologist, I recommend EC2 and S3. To learn more about them, you can follow the links in this article.

Benefits of hosting a Web site on EC2 & S3

The hosting management is self-service. Anytime you want, you can provision additional servers yourself and immediately. Unlike with most traditional hosting companies, there is not need to contact their staff and have to wait for them to set up your server. On EC2, once you have signed up for an account and set up one server, you can provision (or decommission) additional servers within minutes. Even the initial setup is self-service.
EC2 enables you to increase or decrease capacity within minutes. You can commission one or hundreds of server instances simultaneously. Because this is all controlled with web service APIs, your application can automatically scale itself up and down depending on its needs. Billing is metered by an hour as the unit. This flexibility of EC2 can benefits many use cases:
- If your web sites get seasonal traffic (e.g. a fashion site during shows) or can temporarily get much higher traffic for a period of time (e.g. a news site), EC2′s business model of pay for what you use by the hour, is cost-effective and convenient.
- If yours is the R&D or Skunkworks group at a large or medium size organization or a startup company with limited financial resources, renting servers from EC2 can have many benefits. You don’t have to make a capital investment to get a server farm up and running, nor make long-term financial commitments to rent infrastructure. You can even turn off servers when not in use, greatly saving costs.
  - You don’t pay a cent for servers not running, except for disk storage costs for your files on S3, which is so inexpensive that Jeremy Zawodny calculates is even cheaper than buying reliable storage for personal use at home.
  - My favorite photo hosting web site, Smugmug‘s CEO estimates how much money Smugmug saves about by using S3. The numbers are amazing.
It allows me to use the modern Ubuntu¹ GNU/Linux operating system, Server Edition. Among Ubuntu’s many benefits are its user friendliness and ease of use. Software installations and upgrades are a breeze. That means less time is required to maintain the system while retaining the flexibility and power being a systems administrator gives.
EC2 has lower total cost ownership for me than most hosting providers’ virtual hosting or dedicated server plans. Shared (non virtual server) hosting is still cheaper, but no longer meets my sites’ requirements.²

Potential drawbacks/caution with EC2 & S3

While S3 is persistent storage, EC2 virtual server instances’ storage does not persist across server shutdowns. So if your web site is running a database and storing files on an EC2 instance, you should implement scheduled, automated scripts that regularly back up your database and your files to S3 or other storage.
- Consistent with what I read in some comments online, my EC2 virtual server instance did not lose its file-system state or settings when I rebooted it. So rebooting seems to be safe.³
- This potential drawback is arguably a good thing in some ways. It compels you to implement a good backup and recovery system.
- This also means that after installing all the software on your running Amazon Machine Image (AMI), you should save it by creating a new AMI image of it as explained in the Creating an Image section of the EC2 Getting Started Guide.
  - This is an issue since you may want to do this every time after you update your software, especially with security patches. Until Amazon implements persistent storage for EC2 instances, you could do this monthly. You can script this to be partly or fully automated. Since Amazon’s EC2 instances are quite reliable, this is not a major concern.
An EC2 instance’s IP address and public DNS name persists only while that instance is running. This can be worked around as described under the tech specs section below.

Some articles about Amazon’s hosting infrastructure services:

Forbes.com: Amazon’s Hot New Item: Its Data Center
Wired.com: New From Amazon: Guaranteed Server Uptime

Tech specs of my site:

Hosting Environment & Configuration
- Amazon EC2 default “small” size instance
- Dynamic DNS Management:ZoneEdit.com
  - I use a Ruby script to dynamically update DNS from my EC2 host on ZoneEdit. My script is based on the one written by Scott Patten. It uses the ez-ipupdate program in a custom way for EC2.
  - Update (from q2 2008): I now use Amazon EC2 Elastic IP Addresses instead of dynamic DNS.
LAMP Stack:
- Operating System: Ubuntu 7.10 Server Edition, GNU/Linux⁴
- Web Server: Apache 2
- Database: MySQL 5
- Programming Languages: PHP, Java, Python, Ruby
- PHP Accelerator: Alternative PHP Cache (APC)
  - Instructions for installing APC on Ubuntu
- Content Management System: WordPress

www.ubuntu.com [↩]
I plan to split rajiv.com into separate sites, The India Comedy site will move to comedy.rajiv.com and the SPV Alumni site will move to spv.rajiv.com. The latter two are community sites and will benefit from a community CMS like Drupal. [↩]
However, please be aware of a known issue that on some occasions caused instance termination on reboots. [↩]
I created my AMI virtual machine by building on top of a public Ubuntu AMI by Eric Hammond. [↩]

Management & Technical Career Growth Tracks

Posted on January 31, 2008 by Rajiv Pant

Described here is one way to enable technologists to grow their careers in your organization while still allowing them to focus on the type of work they are best at and enjoy most.

The typical management career growth path does not suit some technical people. These information workers need to grow in their careers (gain greater compensation, responsibilities and influence) without having to become managers of other people. A good way to achieve that goal is to create a technical career growth track in your organization.

The following diagram and table illustrate management positions alongside technical positions of similar levels.

Click on the diagram above to view it as a zooming presentation.

People Management Track	Technical (Engineering) Track	Technical (Project Management) Track	Band	Salary
				From (Typical)	To (Max)
Manages team of people and/or manages work assigned to others	May lead people, but usually does not manage people from HR perspective	May lead people, but usually does not manage people from HR perspective		Please refer to the notes at the bottom of this table.

CTO, Executive Vice Present	Chief Scientist & EVP	EVP [Program]	5.2	300	600
Senior Vice President	Distinguished Fellow & SVP	SVP [Program]	5.1	250	500
Vice President	Fellow & VP	VP [Program]	5.0	200	400

Executive Director	Architect & Executive Director	Executive Program Director	4.2	180	220
Senior Director	Architect & Senior Director	Senior Program Director	4.1	160	200
Director	Architect & Director	Program Director	4.0	140	180

Deputy Director	Architect & Deputy Director	Deputy Program Director	3.2	140	180
Senior Manager	Senior Architect	Senior Program Manager	3.1	120	160
Manager	Architect	Program Manager	3.0	100	140

Lead Technical Business Analyst	Lead Engineer	Lead Project Manager	2.2	100	140
Senior Technical Business Analyst	Senior Engineer	Senior Project Manager	2.1	80	120
Technical Business Analyst	Engineer	Project Manager	2.0	60	100

Apprentice Technical Analyst	Apprentice Engineer	Project Coordinator	1.0	30	60

Notes: In this table, a number like 100 may correspond to a salary of US $100,000/year. However, please note that salaries vary greatly based on industry, the particular company, market conditions and location. Comparing your salary to the To (read: Often The Maximum) end of the range is unrealistic. The From (read: Typical) is not the Minimum. In many markets and companies, the minimum of the range is lower than that. The example salary ranges shown here are based on data from online research (Salary.com, RobertHalf.com, GlassDoor.com, etc.), experience and discussions with people in the industry. The salary range within any particular rank is wide since salary is based not just on rank, but also on factors like performance and specific job roles and requirements. There is significant overlap in salary ranges in nearby ranks to reflect the realities of salary differences among employees based on factors like areas of expertise of the employee, the person’s prior salary history and market/company conditions at the time of hiring. Salaries for VP and above jobs vary widely across industries, organizations and even within the same company, especially since performance based bonuses vary greatly.

This system isn’t meant to be rigid. It is designed to find a good balance with most organizations. That balance, i.e. how may “levels of authority” there are will differ across organizations. The focus of this article is to provide a technical track as an alternative to management tracks, whether there are 3 levels or 13. There are pros and cons of having fewer “bands” or ranks. (As a side note, some organizations like the military¹ require lots of ranks.) Ranks need not signify a strict hierarchy where one can only go from one rank to the one immediately above. The ranks could simply be used as “salary bands” and the levels of “hierarchy of authority” could be fewer.

In this model, for example, an architect role is at the same compensation and influence level as a manager role, assuming that the particular manager and architect being compared add similar value to the company. To accommodate more ranks, a senior architect would be at the same level as a senior manager.

If the organization prefers consistent titles for levels regardless of track, the system could name them like this: vice president & fellow, senior director & architect, etc. In the case of a fellow who is at an SVP level, they could be named SVP & distinguished fellow.

Here is a definition of the fellow role from WikiPedia:²

Large corporations in research and development-intensive industries³ appoint a small number of senior scientists and engineers as Fellows. Fellow is the most senior rank or title one can achieve on a technical career, though some fellows also hold business titles such as vice president or chief technology officer.

Such a technical career growth plan brings many benefits to your organization.

It helps retain good technologists who want to grow in their careers, but want to do keep doing the type of work they are best at and enjoy doing: technical work.
It avoids brilliant technical people from being “pushed” (by themselves or their supervisors trying to “reward” them) into people-management responsibilities.
It reduces situations of having too many people-managers but not enough people-management positions over time as people get promoted.

Care should be taken to recognize that some technical people do enjoy making the transition to people-management roles and the presence such a technical track should not discourage them. Having an alternate career growth track option is about presenting employees with more than one choice.

Similar system are also used to enable non-managerial career paths at editorial and design departments at newspapers, magazines and other newsrooms.

Web Site of Rajiv Pant

Victory is winning people, not defeating others.

Tag Archives: Technology

Organizing a Digital Technology Department of Medium Size in a Media Company