Chapter 7. How to Survive in Your Organization
I wanted so badly to write a chapter on how to “manage up” if you are a data scientist. Data scientists are in the theory business. No matter how much data is collected or how many algorithms are written, the work is just numbers on a page and graphs on a screen until someone with resources takes action. And that someone is usually your boss. Managing up is the art of convincing your boss that there is enough value in the research to justify taking an action.
I’ve resolved to write only about the things that I’ve seen work firsthand, and the problem is that I don’t manage up—ever. I’ve looked into every persuasion technique I could find: ways to win friends and influence people, raising my emotional intelligence, improving my ability to tell compelling data stories. But, in my experience, people are just going to do what they’re going to do. I haven’t been able to find a Jedi mind trick that could consistently change that. But I have found a handful of factors that are indicators of a healthy, supportive environment for productive data science research.
You Need a Network
Pyramid-shaped businesses have a definite chain of command and control (Figure 7-1). Direction flows down from your boss, who acts as the gatekeeper for passing the value you create up into other parts of the organization. No matter how good the idea, there will be many who miss its value and potential. Sooner or later, your boss will miss the potential of a significant part of your research. And if, when that happens, you find yourself working in a pyramid, that’s the ballgame.
Figure 7-1. The pyramid-shaped organization
Network-shaped businesses are built on informal connections (Figure 7-2). Teams form, stuff gets done, you move on to the next thing. In the network, you have the freedom to reach out to different groups. If your boss doesn’t see value in your research, it’s acceptable to shop it around to see if someone else does.
Regardless of how solid your research or how well-crafted your data story, without an active network, your long-term future as a productive data scientist in your company is probably pretty grim.
Figure 7-2. The network-shaped organization
You Need A Patron
I’m convinced that when an organization transforms, it isn’t because the people change. It’s because new people rise to prominence. Data science is transformative. The whole goal is to find new, hidden insights. To survive in an organization, the data scientist needs a patron capable of connecting you to people interested in organizational change.
The patron removes organizational barriers that stop you from making progress. She’s influential outside the normal circles in which you run. Hers is the name you drop when, for example, the security guys are dragging their feet approving your data access request. The patron is more than a powerful sponsor. She’s a believer in the cause. She’s willing to act on your behalf without much justification.
Without at least one patron in the organization, you are unlikely to secure the resources and support you need to make meaningful progress.
You Need Partners
Most data science projects follow the path of Gartner’s Hype Cycle (Figure 7-3). Someone important declares the need for a data science project or capability. There’s a flood of excitement and inflated expectations. The project gets underway, the first results are produced, and the organization plummets into disillusionment over the difference between what was imagined and what was produced.
Figure 7-3. The data science project hype cycle
This is when having project partners comes in handy. Partners are the coworkers on the team who have bought in to the mission. Your partners take the work produced so far and help channel it into incremental gains for the business. They help to reset expectations with the rest of the group. They help pivot the work and the project goals in ways that make the two match up.
If you promise the right things, it can be surprisingly easy to get a shot at leading a high-profile data science project. But to build up enough steam to make it pass the trough of disillusionment, you need to have partners willing to help get out and push. When data scientists experience the frustration of their efforts not making an impact, it’s usually because they lack partner support.
It’s a Jungle Out There
I started writing this chapter with the goal of addressing a single, specific organizational problem: influencing your boss. I discovered an opportunity to do something (I think) far more valuable. Instead of prescribing remedies for individual political challenges, I described the basic gear you need in order to survive, and even thrive, over the long haul. With a network, patrons, and partners, you have what you need to deal with the unique political challenges that happen as a result of the experimental nature of data science. As for the specifics of how and when to use each, I’ll leave that to the reader.