Benefits of Using IRC or Group Chat & Video Conference During Incident Management

When a team of engineers is dealing with a real-time incident, such as a system outage, troubleshooting a problem or dealing with a malicious hacking attack, having excellent communications is critically important. The appropriate communications tool can make a world of difference in dealing with the issue and learning from it afterwards. As important as the engineering work itself is, lack of good communications is what often gets tech teams in trouble.

You should enable real-time communication in certain collaborative tasks. This will reduce unnecessary email traffic and clutter, enable people to to focus better on their tasks,and minimize time wasted in bringing each other up to speed When multiple people are working together in real-time on a near term collaborative task, such as:

  • Crisis Management
  • Troubleshooting
  • Dealing with hacking attacks
  • Build and deployment
  • Web application migration
  • Upgrade or maintenance
  • QA testing

Many companies use a phone conference and/or email to assist in real-time while the collaborative activity above is ongoing. Since Email is not instantaneous and real-time the way a group chat application is, and since email is not a suitable medium for quick questions, and quick one-line responses, smart teams use a real-time group chat tool like IRC (Internet Relay Chat) to enable and facilitate real-time conversation. Benefits of using IRC or a real-time textual group chat tool instead of email are:

  • Tech managers, project managers, crisis managers and new tech people joining the effort can quickly catch up with what has been going on (in any level of detail they want) by reading the IRC history transcript so far. This is a much faster and efficient way than using email or pulling someone away to talk in person asking what has been going on. (If email were to be used instead of IRC, a new person joining in would have missed the previous emails on the topic.)
  • When an engineer working on such a collaborative task steps away for a while and comes back, they can quickly catch up on what transpired while they were away by reading the IRC history transcript.
  • Email is not cluttered by short back and forth messages with lots of text to read and filter
  • The IRC transcript can be used for the post incident retrospective and report (“post-mortem”).
  • Unlike a phone-only conference, the IRC transcript can be read and analyzed to learn lessons from this incident. For example:
    • Analyze what problems the team ran into
    • Analyze what worked and what didn’t
    • Analyze how well people collaborated and communicated
    • Timelines of events

I can personally attest to the above benefits. Over the past 15+ years, my development and operations teams in different companies have regularly used IRC to great advantage. Tools like Wikis and blogs are great for collaboration, documentation and sharing information on projects. An group chat like IRC is an indispensable tool for real-time collaboration.

2013 August Update:

With multi-participant video conferencing becoming commonplace thanks to Google Hangouts, I have updated this post to include video conferencing combined with group text chat.

The rest of this update has moved to its own blog entry titled ‘What I Learned During the Hacking Attacks of August 28, 2013.’

2014 July Update:

I now recommend organizations to consider using Slack. See my 2014 July comment below.

Build and maintain a cohesive leadership team

[amazon-product align=”right”]0787954039[/amazon-product]

For an executive, having a management team of people who are good at their jobs and work well with each other is one of the most important factors that lead to success together. Observing a number of successful projects, I realized that it is critical that your management team members care for each other, work well together and give to each other. Their sincere collaboration is far more important than their individual strengths.

I began to write this article impressed by how well the management team comprising of my direct reports functioned, collaborating with each other towards shared success. I was pleasantly surprised by how these directors shared responsibilities, how closely they worked with people in each other’s teams and how comfortably they gave credit to each other. When conflicts arose between them, they frankly, respectfully and nicely expressed them to each other, often one-on-one. Every time, they resolved them quickly and came out with a closer professional relationship. They actively and regularly talked to quell any turf battles between each other’s departments before they could form.

They had a wonderful professional relationship. They barely knew each other outside of work, having busy personal lives with their families on most evenings and weekends. I felt that my management team and I were like a work-family, sticking together through good and bad times, always believing that our success comes as a team.

When you manage and organize your company or your department, spend time multiple times a week with your direct reports together so that you all work well with each other towards shared success. In turn, they should ensure that their direct reports care about each other and collaborate. If you have, say five direct reports, make sure that just the six of you get together in a room to work openly and collaboratively at least twice a week (assuming you are in the same geographic location). The forum for this need not be always a staff meeting, it could be a working session on a project.

I was struggling to come up with suitable words to describe this and its importance. While reading the book The Four Obsessions of an Extraordinary Executive: A Leadership Fable by Patrick Lencioni, I found that the first discipline described in the story talks exactly of this and hence is the title of this article. The book is written as a fictional story that teaches leadership lessons. It is easy to read being under 200 pages in large typeface which you can read in one evening. I highly recommend it.