The question is whether they are about to do the same for oil -- unlocking billions of barrels of crude trapped in similar rock forms, and thereby upending forecasts about increasing oil scarcity and steeply rising prices.
WHY MALTHUS WAS WRONG
In the short term, prices for commodities are determined by the usual forces of supply and demand. In the medium and long term, however, technology is the main determinant of price and availability.
Fracking and horizontal wells are classic examples of transformational or disruptive technologies which completely alter their industries by upsetting long-held assumptions about the feasibility and cost of production.
New technology has proved a blind spot for market analysts and forecasters. In many instances, forecasts implicitly assume technology remains unchanged or evolves only incrementally, even over years or decades.
Yet past experience suggests disruptive and transformational technologies are more common than this static framework assumes. Given sufficient incentives in the form of high prices or threatened shortages technology has proved extraordinarily resourceful, wringing extra raw materials from the planet.
Failure to account for technical change that has repeatedly tripped up forecasters -- from Thomas Malthus in the 18th century (food shortages) and William Jevons in the 19th (peak coal) to U.S. geologists in the early 1900s (peak domestic oil), M King Hubbert in the 1950s (peak oil), the authors of "Limits to Growth" in the 1970s (worldwide food and energy shortages), and Robert Hirsch in 2005 (peak gas).
In most cases, transformational technologies were already being employed when these gloomy forecasts were made, but the authors failed to appreciate their significance. Jevons, for example, considered the potential for petroleum to displace coal in a chapter on "supposed substitutes" but dismissed it.
In his 1866 bestseller "The Coal Question: An Inquiry Concerning the Progress of the Nation, and of the Probable Exhaustion of our Coal Mines", Jevons was more concerned to knock down suggestions that wind and water power could replace diminishing coal supplies when domestic production peaked.
In retrospect, his dismissal of petroleum is almost comically brief: "Petroleum has of late years become the matter of a most extensive trade, and has even been proposed by American inventors for use in marine steam-engine boilers. It is undoubtedly superior to coal for many purposes, and is capable of replacing it. But what is petroleum but the essence of coal, distilled from it by terrestrial or artificial heat. Its natural supply is far more limited than that of coal."
In a modern example, Hirsch and his co-authors warned about peaking U.S. gas supplies in a 2005 report for the Department of Energy, predicted worse to come when global oil supplies peaked sometime between 2008 and 2014.
Even as Hirsch was writing, hydraulic fracturing and horizontal drilling tech had been around for decades and were starting to be deployed extensively across the Barnett shale in Texas, heralding the start of the shale gas boom (here).
Disruptive technologies may remain at the experimental and development stage for many years, but once they take off and reach commercial deployment the impact is usually rapid. Production of shale gas has risen from virtually nothing in the late 1990s to 23 percent of all U.S. output by 2010.
WHAT ARE WE MISSING?
Any forecast for oil supply-demand balances and prices over the medium-to-long term (five years or more) must start by asking the question: what is the market missing?
Projections of rising demand from China and other emerging markets are well known, as is depletion of existing fields and the failure to find new ones on a scale to replace the ageing super-giants discovered between the 1930s and the 1960s.
The question is whether there are any disruptive technologies that would significantly alter these supply and demand trends, and if so over what timescale?
There are a number of candidates for the role. Technologies to make liquid transport fuels from abundant gas and coal; increase the use of biofuels; or promote the use of electric vehicles. But none is as promising over the short to medium term as using hydraulic fracturing and horizontal drilling to produce crude from tight rock formations.
Technologies to extract oil from tight formations are mature (having been extensively used in the natural gas industry); capital costs are comparatively low; and they utilise existing energy infrastructure and pathways rather than requiring construction of new distribution and processing systems and new vehicles.
WHERE BAKKEN LEADS ...
Crude production in the state of North Dakota has nearly quadrupled to around 110 million barrels a year since 2002 as a result of applying these technologies to the Bakken formation, which covers much of the state as well as parts of neighbouring Montana, South Dakota, Manitoba and Saskatchewan.
Soaring output from Bakken coupled with increased deepwater output from the Gulf of Mexico has pushed up U.S. domestic production 600,000 barrels per day (12 percent) since 2008, the first increase since 1985, confounding predictions domestic output was set on an inexorable downtrend (here&GAS.pdf).
The industry has known about the Bakken formation since the early 1950s. But until the last decade the oil was thought to be essentially unrecoverable. Unlike a conventional resource in which oil or gas accumulates in a discrete, permeable reservoir rock over a relatively small area, oil and gas deposits across the Bakken are distributed over a huge area in formations with low permeability.
The U.S. Geological Survey refers to formations like Bakken as a "continuous-type" oil or natural gas resource. Prior to the application of fracking and horizontal drilling, there was no way to get these extensive but unconcentrated gas and oil resources held in relatively impermeable formations to flow to the foot of traditional vertical wells in sufficient volume to make them economically recoverable.
Continuous-type oil and gas resources are more common than concentrated reservoirs. Geological surveys show extensive formations suitable for shale gas extraction across North and South America, northern Europe, north Africa, India, China and Australia. Many of these regions may have formations suitable for oil production as well, though research has so far concentrated on gas rather than liquids.
If the technologies that have revolutionised gas can be rolled out for oil there is potential for a significant increase in output, much of it located away from traditional producing areas in the Middle East, the North Sea and West Africa.
Chief Executive Peter Voser of Royal Dutch Shell, which has been an oil industry leader in adopting new technology, told the Financial Times newspaper last month that it aimed to start producing from tight oil reserves in North America.
Voser told the Financial Times that having built a large position in North American tight gas, Shell planned to apply the same model to tight oil ("Shell targets North American tight oil", Sep 22).
Tight oil production faces formidable problems. Unlike concentrated conventional oil and gas reservoirs, production from tight rock formations is far more extensive, and its footprint on communities and the environment is consequently much larger.