Engineering leadership | Blog Post
Deck the “On calls”! 8 Tips for Managing On Call Engineers Over the Holidays
Linzi Nield
Share this post
Remember Christmas morning as a kid? You know, waking up before dawn to stare in wonder at your stack of presents, counting down the hours by the fire until everyone else wakes up? The holidays really aren’t so dissimilar for on call engineers. Only it’s a predawn critical alert that wakes them up, a stack of incidents in PagerDuty they’re staring at, and the fire isn’t in the fireplace — it’s raging in the code.
Being an on call software engineer can be one of the most stressful ways to spend the holidays. Tensions are already high, and familial obligations run aplenty, but with the right planning and mindset, you can make on call duty stress-free and (dare we say?) even a little fun for your team.
Here are some time-tested tips to help your team survive and thrive during on call duty over the holidays.
It all starts with planning
Basic on call hygiene
Keeping your on call developers sane during the holidays all starts with the basics. On call hygiene is a year-round affair that involves building out simple alerting protocols, fair schedules for the entire team, and straightforward escalation rules when things are too hot to handle. It’s true during normal times, and it’s even more true during the holidays. If your team still uses spreadsheets or static schedules to manage incident response, it may be a good time to invest in proper incident management software to keep things moving smoothly during the holidays and beyond.
Consider a December code freeze
It may be tempting to get all your year-end code out the door before heading out for the holidays, but the gratification may not be worth the risk. Consider pushing no major changes into production for the entirety of December — because the best bug-free code is no new code at all.
Spread responsibility across distributed teams
If you have a distributed team, chances are everyone is spread across a few different time zones. This makes it much easier to craft a “follow-the-sun” schedule, where team members in multiple time zones participate in the same local rotation. Even if you have only one office, the holidays (and the pandemic more generally) have dispersed team members across many regions, and your team collectively likely has more daylight hours than the meager few that mother nature gives us in November and December. Use this to your advantage when drawing up shifts to reduce the chance of that dreaded 4am alert if something goes wrong.
Consider creating more, shorter shifts for on call engineers
The typical on call rotation lasts a week. While this makes sense in normal times (“just give me a single big pill to swallow!”), it’s a lot of time to sit on edge during the holidays. We’ve heard of teams breaking up their normal rotations into smaller, more bite-sized segments and distributing these across the holiday season. This means the burden will be meted out a little more evenly, rather than people horse-trading the best and worst shifts and hoping they don’t get stuck with the short straw.
Add a secondary on call engineer
Being on call doesn’t need to mean you can’t also be celebrating with friends and family. Protect your company and your brave on call volunteer with a backup. An eager junior dev might relish the chance to be a first responder during such a critical time! Why not use the holidays to create a growth opportunity? As long as alarms are tuned and the team’s tech lead is on speed-dial if things get hairy, the company will be fine.
Rewarding the hardest shifts
There’s no escaping it — someone has to be on call on December 24th at 8pm. Even if nothing goes wrong, it’s hard for whoever’s on call to fully relax knowing that holiday cheer can come crashing down if a syntax error code suddenly makes itself known. That’s why we recommend giving a little extra time off to your engineers taking the most taxing rotations. Empathy goes a long way when building your on call schedule, so consider giving out a few extra days off in early December or in the new year to people working on Thanksgiving, Christmas, and New Years Eve.
Holiday on call enginee rotations can be fun
Being on call during the holidays, when done right, can actually be a secret weapon to building team morale and camaraderie. Here are some ways you can make on call rotations a little jollier during the most wonderful time of the year.
Share horror stories
Everyone’s got an on call horror story. Before your team adjourns for the holidays, try setting aside some time during a team meeting for your engineers to regale the rest of the team with some. Perhaps even make it into a contest: give away a bottle of nice dessert wine or a Thanksgiving bird for the best tale.
Give the perfect excuse for skipping chores
The rigors of on call work are well documented. This can help your team be a little more, erm, selective about what non-work obligations they take on during their shifts. In a stage whisper, try to jokingly encourage your team to focus their whole attention on their rotations — even if it means delegating cooking or last-minute trips to the market to other family members.
Make a game of it
Tap into your team’s competitive streak by gamifying the acknowledging and resolving process. Who responds quickest? Who handles the most incidents? Create teams and tally the scores. If the prize is enticing enough, like an extra day off, people will resolve issues fast, to everyone’s benefit!
Keep fires in the fireplace
The holidays are always stressful, and if the rest of 2020 has been any indication, this holiday season will have its fair share of quirks. So start planning early to keep those fires in the fireplace and add a little holiday cheer into being on call (la-la-la-la luh-la-la-la).
