Some of my thoughts, filtered slightly for public consumption.

AI and the Future of Work

What could generative AI change about work in the next 10 years?

Extrapolating from current capabilities, I think the median outcome is that LLM-esque technology improves over the next decade to the point where it is fast, cheap, and reliably produces human-sounding responses in voice, text and even images to any query, at a quality above the median human specialist unless the query specifically requires novel synthesis of information.

Many AI researchers and adjacent folks think that AI capabilities will go further than this, faster, but I make a more conservative prediction for a few reasons:

  1. AI advances have historically come in sudden leaps followed by "winters" in which little progress is made.
  2. Current models rely on a very simple internal model of inference (in the case of LLMs, predicting tokens in text) which I do not believe is sufficient to perform abstract inference at high level. LLMs have internal state which can reflect e.g. simple mathematical structures, but learning more complex structures appears to require exponential model size with current techniques.
  3. I'm close enough to the social circles of a lot of the people making these predictions to notice that they have frankly terrible judgment and make many obviously bad decisions in their lives.

Accepting this forecast for generative AI capabilities, what would the effects look like? Broadly speaking, generative AIs would exceed most (even specialized) humans at:

  1. Re-formatting information (e.g translation, turning a bunch of facts and quotes into an article, drafting a contract)
  2. Extracting information from text/sound/images (e.g. summarizing text, interpreting contracts, captioning)
  3. Expounding on prompts (e.g. illustration, writing formulaic fiction)

First-Order Effects

This suggests a number of jobs will be largely replaced by AI, with a few human practitioners remaining to produce the highest-quality work, supervise and tweak AI work, and satisfy demand specifically for certified "handmade" goods:

The next rung down are jobs where large portions of the work would be replaced by AI:

Further down are jobs where some portion may be enhanced by AI, but where most uses would likely still require heavy specialist guidance the whole way:

Then there are large swaths of work that AI will not change much, at least without significant advances in robotics:

And some jobs I suspect could be done by AI but will not be:

Second-Order Effects

However, this sort of discipline-by-discipline analysis has several shortcomings. The use of generative AI will not impact all workers in a discipline evenly — in many cases the more junior workers could be entirely replaced, while the more senior workers would find their jobs mostly unchanged. This poses a serious second-order problem for the discipline: where will the senior workers come from if there are no juniors? Conceivably, some disciplines may re-define the roles of junior workers in order to get around this problem. However, it is not clear that it would be in the interest of any particular firm to do so — the costs of paying a wage that's in the same realm as a more useful senior worker would be borne entirely by the firm, while the benefits of more senior workers would be largely external given the short tenures of modern workers. (In fact it is not clear to me that the benefits outweigh the costs for many disciplines even today, but firms continue to essentially subsidize training out of social expectation.)

A shock like generative AI could destroy this system of implicit apprenticeship and require collective action to address. We would have to radically revamp schooling, or subsidize firms taking on this training burden.

Another consideration is how the nature of work itself may change due to generative AI. It is easy to look at the amount of time workers spend transforming comparatively small amounts of information into volumes of reports, articles, emails, etc. and conclude that AI would replace this aspect of the work. And in the micro case that analysis is probably correct — any single worker could save that much time by using generative AI. However, we need to ask why the typical worker is writing all of this prose. Is it simply to make the reading experience more pleasant than a bullet-point list? Certainly the most talented writers can do so, but I do not think the typical TPS report or news article is really more enjoyable to read or easier to understand than the bullet-point summary you would feed into a generative AI to write it. As many people have observed, a presentation at a conference on a whiteboard or with a PowerPoint is often faster and easier to understand than the corresponding paper, despite the lack of complete sentences or prosaic style.

Why then do we insist on such formats? In my view, they serve as a kind of honest signal. It is difficult for me to evaluate the truth of almost anything I read, so I have to rely on signals that suggest how much time and thought was put into it and how well trained the author is. If I read a report written with decent prose, I know that the author is at least reasonably educated and reasonably intelligent and that they have learned the jargon of the discipline. If they are taking the time to do so, they're also more incentivized to do the underlying work correctly (e.g. the research that went into the report, or their own work that the report is about) because they have invested more time that will go to waste if I am able to discern that the report is full of shit. At the same time they're less incentivized to cut corners for speed because the time they spend on writing limits the benefits they could realize.

None of that will be true anymore in a world with generative AI — the worst home inspector in the world could write a report just as passable as any other with only a glance at your house. In such a world, does the report have any value anymore? My view is that it will not. The key question then becomes: will we find new ways to signal ability and effort, and how much time will those take? In the best case, the reports go away and are replaced by more reliable but less costly signals. In the middle case, we find other signals that are not as vulnerable to faking with AI and spend just as much time signaling as before, just in a different form. In the worst case, AI ends up powerful enough to let people pretend to be good at their jobs trivially, but not powerful enough to do the actual job well.

There is some evidence that this is already happening. A recent paper by a pair of MIT Econ grad students found that workers' performance on writing tasks improved on average when they had access to ChatGPT, but this mostly came from the lower performing workers' performance converging towards the higher performing workers'.

If writing becomes less of a signal, then there are several ways professions could evolve:

  1. The outputs remain unchanged, with less time spent on writing but less accurate signals of quality to the consumers, resulting in (potentially) worse service quality but cheaper prices. If and how much worse service gets depends on the other existing signals of quality.
  2. The writing disappears or is radically simplified, with similar consequences as the first case except less work for everyone.
  3. The writing is replaced with other signals, such as more robust ratings systems, or paid research like Consumer Reports. The effect this would have on price and quality depends on the level of effort required by these signals and their reliability.

The optimistic view is that other signals of quality are already sufficient and the writing output of the profession is vestigial, or that alternative signals can and will be easily developed and will require less effort than is currently expended on writing. Note that in both of these cases AI isn't providing value directly (unless it necessary for providing the alternative signal), but rather forcing society out of a sub-optimal equilibrium.

This obviously varies by profession, but generally I am not optimistic that existing signals will be sufficient. As an employer, I've found that employees' writing corresponds well to their performance in other tasks, and is much easier to evaluate during the hiring process. As a consumer, I've found that signals like reviews or rating systems are easily gamed and even if not gamed, often do not translate to quality as well as they do to trendiness or superficially pleasant customer service. Review services paid for by the consumer (similar to Consumer Reports) may exist for some high-value professional services, but are not widespread.

It's much harder to make claims about what sort of signals may emerge. Paid review services are one option with clear precedent, but it is unclear that it can be scaled profitably to reviews of individual service providers outside of very high-value services. Additionally, paid review services may well be subject to the exact same signaling problems as the services they are reviewing.

The pessimistic view is that we won't develop other signals, or if we do they are less reliable and/or require more effort than the current system. In this case, either quality will fall or prices will rise — possibly both. One likely consequence is that we will see consolidation into larger firms, which will be better able to police the quality of their employee's work than consumers are, and can use their brands' reputations to fill the void left by individual service providers' inability to signal.

Conclusions

In summary, while a few professions will be entirely replaced by generative AI and others will make heavy use of it, the long-term effects on many professions depend on how the use of AI changes the behavior and preferences of consumers. Furthermore, some professions may be permanently altered by the mere existence of generative AI without ever using it, and whether this change will be beneficial or detrimental to society depends on factors that have little to do with AI.

Much work is needed to understand the signaling problem that generative AI will create, and how to replace the role of competent writing as a signal. This is potentially a large opportunity, although because the incentives involved are so fraught it may not be possible to do both well and profitably.