index.md 26KB
title: Oil is the New Data
url: https://logicmag.io/nature/oil-is-the-new-data/
hash_url: f3f3c3139f
I remember being nervous when I flew into Atyrau, Kazakhstan. Before boarding the flight, one of the business managers who organized the trip sent me a message with precise instructions on how to navigate the local airport:
Once you land, get into the bus on the right side of the driver. This side opens to the terminal. Pass through immigration, pick up your luggage, and go through Customs. The flight crew will pass out white migration cards. Fill them out, and keep it with your passport. You will need to carry these documents on you at all times once you’ve landed.
Another coworker, who had flown in the night before, warned us not to worry if we found ourselves in jail. Don’t panic if you find yourself in jail. Give me a call and we’ll bail you out. Maybe she was joking.
The flight itself was uncanny. I was flying in from Frankfurt, but it felt a lot like a local American flight to somewhere in the Midwest. The plane was filled with middle-aged American businessmen equipped with black Lenovo laptops and baseball caps. The man next to me wore a cowboy-esque leather jacket over a blue-collared business shirt.
After I landed in Atyrau’s single-gate airport, I located my driver, who was holding a card with my name on it. He swiftly led me into a seven-seater Mercedes van and drove me to my hotel, one of the only hotels in the city. Everyone from the flight also seemed to stay there. The drive was short. The city was overwhelmingly gray. Most of it was visibly poor. The hotel was an oasis of wealth.
Across from the hotel was another one of these oases: a gated community with beige bungalows. This was presumably where the expats who worked for Chevron lived. There was a Burger King and a KFC within walking distance. Everyone spoke a bit of English.
Security was taken extremely seriously. Each time we entered one of Chevron’s offices, our passports were checked, our bags were inspected, and our bodies were patted down. Video cameras were mounted on the ceilings of the hallways and conference rooms. We were instructed to travel only using Chevron’s fleet of taxis, which were wired up with cameras and mics.
All of this — Atyrau’s extreme security measures and the steady flow of American businesspeople — comes from the fact that the city is home to Kazakhstan’s biggest and most important oil extraction project. In 1993, shortly after the fall of the Soviet Union, the newly independent nation opened its borders to foreign investment. Kazakhstan’s state-owned energy company agreed to partner with Chevron in a joint venture to extract oil.
The project was named Tengizchevroil, or TCO for short, and it was granted an exclusive forty-year right to the Tengiz oil field near Atyrau. Tengiz carries roughly 26 billion barrels of oil, making it one of the largest fields in the world. Chevron has poured money into the joint venture with the goal of using new technology to increase oil production at the site. And I, a Microsoft engineer, was sent there to help.
Cloud Wars
Despite the climate crisis that our planet faces, Big Oil is doubling down on fossil fuels. At over 30 billion barrels of crude oil a year, production has never been higher. Now, with the help of tech companies like Microsoft, oil companies are using cutting-edge technology to produce even more.
The collaboration between Big Tech and Big Oil might seem counterintuitive. Culturally, who could be further apart? Moreover, many tech companies portray themselves as leaders in corporate sustainability. They try to out-do each other in their support for green initiatives. But in reality, Big Tech and Big Oil are closely linked, and only getting closer.
The foundation of their partnership is the cloud. Cloud computing, like many of today’s online subscription services, is a way for companies to rent servers, as opposed to purchasing them. (This model is more specifically called the public cloud.) It’s like choosing to rent a movie on iTunes for $2.99 instead of buying the DVD for $14.99. In the old days, a company would have to run its website from a server that it bought and maintained itself. By using the cloud, that same company can outsource its infrastructure needs to a cloud provider.
The market is dominated by Amazon’s cloud computing wing, Amazon Web Services (AWS), which now makes up more than half of all of Amazon’s operating income. AWS has grown fast: in 2014, its revenue was $4.6 billion; in 2019, it is set to surpass $36 billion. So many companies run on AWS that when one of its most popular services went down briefly in 2017, it felt like the entire internet stopped working.
Joining the cloud business late, Google and Microsoft are now playing catch-up. As cloud computing becomes widely adopted, Amazon’s competitors are doing whatever they can to grab market share. Over the past several years, Microsoft has reorganized its internal operations to prioritize its cloud business. It is now spending tens of billions of dollars every year on constructing new data centers around the planet. Meanwhile, Google CEO Sundar Pichai announced that in 2019, the company is putting $13 billion into constructing new offices and data centers in the US alone, the majority of which will go to the latter.
Startups have long been the biggest early adopters of the public cloud. They are an obvious fit: they do not own their own data centers, so the opportunity cost of switching to the public cloud is low. By contrast, it is much harder for large companies that do run their own data centers to make the leap, since it would require selling or retiring those centers.
This helps explain why cloud providers have only captured about 30 percent of the total addressable market. While cloud technology has matured considerably over the past half-decade, big corporations that run their own data centers still dominate the majority of the world’s IT infrastructure. For Amazon, Google, and Microsoft, as well as a few smaller cloud competitors like Oracle and IBM, winning the IT spend of the Fortune 500 is where most of the money in the public cloud market will be made. And among those large companies, Big Oil sits at the top. Out of the biggest ten companies in the world by revenue, six are in the business of oil production. In order words, the success of Big Oil, and the production of fossil fuels, are key to winning the cloud race.
Making Friends
In 2017, Chevron signed a seven-year deal with Microsoft, potentially worth hundreds of millions of dollars, to establish Microsoft as its primary cloud provider. Oil companies like Chevron are the perfect customer for cloud providers. For years, they have been generating enormous amounts of data about their oil wells. Chevron alone has thousands of oil wells around the world, and each well is covered with sensors that generate more than a terabyte of data per day. (A terabyte is 1,000 gigabytes.)
At best, Chevron has only been able to use a fraction of that data. One problem is the scale of computation required. Many servers are needed to perform the complex workloads capable of analyzing all of this data. As a result, computational needs may skyrocket — but then abruptly subside when the analysis is complete. These sharp fluctuations can put significant pressure on a company like Chevron. During spikes, their data centers lack capacity. During troughs, they sit idly.
This is where the promise of the public cloud comes in. Oil companies can solve their computational woes by turning to the cloud’s renting model, which gives them as many servers as they need and allows them to pay only for what they use.
But Big Tech doesn’t just supply the infrastructure that enables oil companies to crunch their data. It also offers many of the analytical tools themselves. Cloud services provided by Microsoft, Amazon, and Google have the ability to process and analyze vast amounts of data. The tech giants are also leaders in artificial intelligence and machine learning (AI/ML), a field focused on teaching computer systems to automatically perform complex tasks by “learning” from data. With AI/ML, oil companies can make better sense of all the data they are collecting, and can discover patterns that may help them make their operations more efficient and less costly.
AI/ML gives Big Oil yet another reason to depend on Big Tech: the level of sophistication often requires delving into the cutting edge of a field that the tech titans dominate. And if sharing their AI/ML expertise means getting a leg up on the competition in the cloud market, tech companies are more than willing to help.
In recent years, Big Tech has aggressively marketed the transformative potential of the public cloud and AI/ML to Big Oil, with great success. In 2017, Microsoft signed its seven-year contract with Chevron; in 2018, it announced major partnerships with oil giants BP and Equinor; and in 2019, it signed a deal with ExxonMobil that Exxon claims is “the industry’s largest [contract] in cloud computing.” Amazon recently opened an AWS office in Houston, the US oil and gas hub, and has been hiring AI/ML experts specifically to work on fossil fuel projects. Google has also developed deep relationships in the industry, partnering with Total, Anadarko Petroleum, and Nine Energy, and appointing Darryl Willis, an oil veteran, to lead Google Cloud’s newly formed Oil, Gas & Energy division. Whatever the tech giants are telling their friends in the fossil fuel industry, it’s working.
Drill Baby Drill
The multi-million-dollar partnership between Microsoft and Chevron was the reason I went to Kazakhstan. Microsoft sent me to Atyrau for a week-long workshop to help the Tengiz oil field adopt our technology. I was there to talk about computer vision, a field of AI/ML that gives computers the ability to understand digital images, but the workshop covered a range of topics in both AI/ML and cloud computing. We held it for a team at TCO tasked with boosting daily oil production from 600,000 barrels to 1 million. They wanted to learn about how Microsoft technology could help them modernize their oil field and increase efficiency.
The workshop took place in a large conference room in one of the TCO office buildings. The building itself wasn’t particularly fancy. The exterior was run-down: it looked like it was last renovated in the 1980s. Aside from the security guards dressed in dark clothing, the interior was mostly white, with bright marble floors. The only bits of color came from the biscuits and pastries that were laid out on tables in front of the conference rooms.
At the workshop, I gave a short technical demonstration about running computer vision at scale on Microsoft’s cloud computing platform. There were about forty people in the audience, predominantly businesspeople. My presentation felt like a marketing technique: the point was to flex Microsoft’s engineering prowess to a technically illiterate business crowd. I made sure to include a lot of engineering jargon: “distributed training,” “offline scoring,” “Docker-compatible.”
On the third day of the workshop, a small group of us convened at TCO headquarters in Atyrau to discuss specific AI/ML scenarios they wanted to implement. The meeting room was much nicer than where the workshop was held. It featured new videoconferencing equipment and plush ergonomic chairs. A half-dozen TCO managers were present. Yet, strangely, none of their technical staff attended. The TCO managers were mostly Americans and, with one exception, all white men. They wore monochrome suits and polished leather shoes. I felt out of place wearing sneakers and an oversized button-down. There was not a single Kazakhstani in the room.
To kick off the meeting, a Microsoft account manager gave a PowerPoint presentation that discussed common problems in the oil and gas industry that could be solved using AI/ML. One of the most complex use-cases involved using AI/ML to improve oil exploration. The traditional way to find a new oil or gas deposit is to perform a seismic survey. This is a technique that sends sound waves into the earth and then analyzes the time it takes for those waves to reflect off of different geological features. Because the data is volumetric and spans hundreds of kilometers at a minute granularity, the data collected from a single seismic survey can run over a petabyte. (A petabyte is a million gigabytes.) The output of this data is a 3D geological map, which geophysicists can study in order to recommend promising locations to build wells.
However, interpreting this map is a long and labor-intensive process. It can take months and involve many geophysicists. To make the process more efficient, computer vision technology can automatically segment different geological features to help geophysicists understand the 3D data and identify where best to drill. It seemed like a perfect example of the partnership I had been sent to Kazakhstan to help forge: a technically sophisticated and computationally intensive undertaking that played to the strengths of Big Tech while advancing a core priority of Big Oil, which was to dig more fossil fuels out of the ground while cutting costs.
Big Oil Is Watching
But the TCO managers also wanted to talk about something else. “We have a lot of workers in the oil fields. It would be nice to know where they are and what they are doing,” one manager said. “If they are doing anything at all.”
This is what our Chevron partners were most keen to discuss: how to better surveil their workers. TCO had thirty or forty thousand workers on site, nearly all local Kazakhstanis. They worked on rotating shifts — twelve-hour days for two weeks at a time — to keep the oil field running around the clock. And the managers wanted to use AI/ML to keep a closer eye on them.
They proposed using AI/ML to analyze the video streams from existing CCTV cameras to monitor workers throughout the oil field. In particular, they wanted to implement computer vision algorithms that could detect suspicious activity and then identify the worker engaging in that activity. (My Microsoft colleagues and I doubted the technical feasibility of this idea.) Enhancing workplace safety would be the reason for building this system, the managers claimed: more specifically, they hoped to see whether workers were drunk on site so that they could dispatch help and prevent them from hurting themselves. But in order to implement this safety measure, an “always-on” algorithmic monitoring system would have to be put in place — one that would also happen to give management a way to see whether workers were slacking off.
The TCO managers also talked about using the data from the GPS trackers that were installed on all of the trucks used to transport equipment to the oil sites. They told us that the workers were not trustworthy. Drivers would purportedly steal equipment to sell in the black market. Using GPS data, the managers wanted to build a machine learning model to identify suspicious driving activity. It’s not a coincidence that minor tweaks to the same model would also allow management to monitor drivers’ productivity: tracking how frequently they took bathroom breaks, for example, or whether they were sticking to the fastest possible routes.
The TCO managers were also interested in Microsoft products that could analyze large quantities of text. “Let’s say we have the ability to mine everyone’s emails,” one executive asked. “What information could we find?”
When I reflect back on this meeting, it was a surreal experience. Everyone present discussed the idea of building a workplace panopticon with complete normalcy. The TCO managers claimed that monitoring workers was necessary for keeping them safe, or to prevent them from stealing. But it wasn’t convincing in the slightest. We knew that they simply wanted a way to discipline their low-wage Kazakhstani workforce. We knew they wanted a way to squeeze as much work as they could from each worker.
I held my tongue and made sure to appear calm and collected. So did my colleagues. Collectively representing Microsoft, we turned a blind eye, and played along perfectly. We sympathized with TCO’s incriminating portrayal of their Kazakhstani workers and the need to uphold the rule of law. We accepted their explanation that increased surveillance would improve worker safety. But truth be told, we didn’t even need the excuses. Microsoft was hungry for their business. We were ready to concede.
Skip the Straw
The topic of worker surveillance took me by surprise. I didn’t sign up for it. I did sign up for helping to accelerate the climate crisis, however — and it was something I had thought about a lot by the time I landed in Atyrau.
When I was first asked to present at the workshop, I was excited. It was good for my career, the technology was fascinating, and I had never been to Kazakhstan. But I hesitated. Did I really want to help Chevron destroy the planet? There were others on my team who could have easily gone in my place. Still, I decided to go. I wanted to learn about the oil industry and the kinds of investments that Big Tech was making. I wanted a front-row seat to the Microsoft-Chevron partnership. I wanted to know what we were up against.
During the workshop, I asked a coworker how she felt about Microsoft working with Big Oil. She responded sympathetically, understanding my concerns about climate change. But she also seemed to feel there was nothing we could do. For her and many other colleagues I’ve spoken to, change has to happen at the top. The problem, of course, is that the top has powerful incentives not to change. Microsoft executives aren’t going to give up on the billions of dollars to be made from Big Oil, especially if it helps them win more of the coveted cloud market.
They are happy to offer employees small ways to live more sustainable lives, however. The company runs various recycling programs, encourages employees to “skip the straw” to reduce plastic consumption, and funds sustainability hackathons. (One hackathon project involved using AI/ML to detect trash in the ocean.) More broadly, Microsoft works hard to present an environmentally friendly public face. Its most ambitious green initiative is its promise to power its energy-hungry data centers with renewable sources. In 2016, Microsoft announced its goal to transition its data centers to 50 percent renewable energy by 2018. Hitting that target one year early, president and chief legal officer Brad Smith announced that the next goal is to surpass 70 percent renewable by 2023. “Time is too short, resources too thin and the impact too large to wait for all the answers to act,” he said.
On the surface, then, Microsoft appears to be committed to fighting climate change. Google has constructed a similar reputation. But in reality, these companies are doing just enough to keep their critics distracted while teaming up with the industry that is at the root of the climate crisis. Why go through the effort of using clean energy to power your data centers when those same data centers are being used by companies like Chevron to produce more oil?
After Empire
At the workshop in Atyrau, a young Kazakhstani data scientist approached me to ask about a project that he was migrating to Microsoft’s cloud platform. He didn’t speak English fluently, but I could tell he was a good engineer. I wasn’t sure if he really needed my help. It seemed like he just wanted to chat with another engineer in a room filled with businesspeople.
Afterwards, he told me a bit about how he ended up working for TCO, and how he wasn’t able to find any other opportunities in the country that could match the offer. He had attended Purdue University to get an undergraduate degree in computer science. But since the Kazakhstan government paid for his tuition, he had to return to the country to work. “It means that I have to work in oil,” he said. “It’s basically the only industry that pays.”
Speaking with him made me realize the extent of oil’s dominance in Kazakhstan. Oil is by far the biggest economic sector, accounting for 63 percent of the country’s total exports. In 2013, TCO made $15 billion in direct payments to the government — an enormous figure, considering that the country’s entire tax revenue that year came to $21 billion. TCO is also a major source of wealth for the region. For years, the venture has invested millions of dollars into building schools, community centers, and fitness centers for the local people.
Kazakhstan’s dependence on oil has only grown over the past decade. In 2016, TCO announced a $36.8 billion expansion to the Tengiz project, tying the country’s economic future even more closely to fossil fuels. To make matters worse, the country’s ability to produce oil relies heavily on multinational oil companies. At the time of its founding, TCO was a fifty-fifty partnership between Chevron and the state-owned KazMunayGas. Since then, ExxonMobil and the Russian oil company LukArco have joined the venture, but only KazMunayGas’s share has been diluted.
While the country would struggle to take advantage of its oil-rich lands without the help of these foreign partners, the partnership is far from a win-win deal. Chevron keeps a tight grip on power, appointing most members of TCO’s upper ranks. The power dynamic was clear at the workshop: lower-level employees were Kazahkstanis while management was almost entirely American. The local economy has also completely aligned itself with the needs of the American-dominated TCO. TCO proudly announced in Q1 of 2019 that it spent over $1 billion on Kazakhstani goods and services, which includes hiring more than forty thousand local workers to work in the oil field. But this makes local businesses highly dependent on TCO. If American oil companies pulled out of the venture or slashed funding, TCO would crumble, and many businesses would lose their biggest (and often only) customer, leaving the economy in shambles.
Big Tech isn’t responsible for Kazakhstan’s reliance on oil. Nor can we blame it for the climate catastrophe that we’re facing. But it is certainly exacerbating both. While Kazahkstan’s economy may benefit in the short run, intensifying the climate disaster will ultimately hurt the country too. Research shows that the region will suffer from increased aridity and more frequent heat waves, which could decrease crop yields and challenge food security.
How can tech help, instead of hurt, the climate? How can tech companies make local economies more resilient rather than more vulnerable? How can we demand climate justice from Microsoft, a company that claims to be a leader in the fight against climate change?
While I was in Atyrau, these very questions were being asked back home. Amazon employees in the US published an open letter calling on their company to reduce its carbon footprint and cancel its many contracts with Big Oil. Sitting in my hotel room not far from one of the largest oil fields in the world, I watched the letter blow up on my social media feeds. The number of Amazon signatures exploded: “3,500 employees challenge Bezos”, “4,200 Amazon workers push for climate action”, “6,000 employees sign an open letter to Bezos.”
I was thrilled. Tech workers like me were taking a stand against our industry’s role in accelerating the climate crisis. They weren’t waiting for change at the top; they were demanding change from below.
Then I thought of the young Kazakhstani engineer. What happens to people like him after we decarbonize? If Chevron and other oil giants cease operations, it would decimate the economy of places like Kazakhstan — places whose dependency on oil has been actively encouraged by those companies, which have in turn profited handsomely from it. Resource extraction is an ancient imperial practice. As tech workers join the movement for climate justice, we must also find ways to undo the legacies of Big Oil’s imperialism, and bring countries like Kazakhstan fairly and safely into a carbon-free future.
But it won’t be easy. When I returned to the US, I learned that Bezos had effectively ignored the demands of over 8,000 of his employees. The open letter was an important first step, but more action will be needed for Amazon to drop its oil partnerships. We have a long fight ahead of us, and the stakes are high. We have, quite literally, a world to win.