Good article, I like the subtask / hazard rate formalization and the concrete prediction about the expected slowdown.
But how do you then picture the arrival of general AI?
I find it hard to reconcile “generalization is poor, generalization from math-y domains to non-math-y domains is bad, and we still need more data” with “coin flip chance we will have human-level AI systems in under five years.”
I don't think it's inconsistent at all to think both that generalisation is poor and that human-level AI could be here soon. After all, human-level humans have 'poor' generalisation in this sense! (How many times did someone train on frontier mathematics and then produce expert biology contributions? Only, if ever, by first retraining on bio.)
These snippets are relevant:
> Increasing adoption of AI in task-integrated contexts, industrial deployment, and even explicit approaches to gathering example data such as ‘hand movement farming’ are the leading indicators to watch for progress in particular domains...
This is because on-task data will drive on-task competence in any context. (Then, sufficient automation to warrant (partial) rollout and exploration in-situ could drive further data collection, a flywheel.)
> we might see an ‘explosion’ in the speed and cost-effectiveness of AI... But... on-task data and compute will remain crucial to broadening the frontier of autonomous capabilities. *Collecting* that data and *manufacturing* that compute look to me like the rate-limiting steps, and therefore the major leading indicators to use in foresight.
> The best case I can make for a much more general explosion is if the speed and cost-effectiveness explosions rapidly accelerate the gathering and digestion of diverse task data — but I think that remains mostly rate-limited in the familiar ways: some domains easy and some more difficult.
So a lot of contexts where experience is legible, repeatable, and logged, will make quick progress by default.
Where experience is less legible or logged, AI will make less progress (and humans remain valuable) because of the lack of accrued existing context. As this becomes clearer, companies will go out of their way to start making experience more legible and more logged.
Where experience is less repeatable, AI is less effective at sample-efficient adaptability than humans. Progress in sample-efficiency of learning is the key thing here, coupled with learning 'taste' for what exploratory activity will reveal the most useful further observations (cf https://www.oliversourbut.net/p/you-cant-skip-exploration). AI eventually has an advantage here where a fleet can more easily pool experience and accrue taste compared with the lossy communication between humans in a team.
Good article, I like the subtask / hazard rate formalization and the concrete prediction about the expected slowdown.
But how do you then picture the arrival of general AI?
I find it hard to reconcile “generalization is poor, generalization from math-y domains to non-math-y domains is bad, and we still need more data” with “coin flip chance we will have human-level AI systems in under five years.”
I don't think it's inconsistent at all to think both that generalisation is poor and that human-level AI could be here soon. After all, human-level humans have 'poor' generalisation in this sense! (How many times did someone train on frontier mathematics and then produce expert biology contributions? Only, if ever, by first retraining on bio.)
These snippets are relevant:
> Increasing adoption of AI in task-integrated contexts, industrial deployment, and even explicit approaches to gathering example data such as ‘hand movement farming’ are the leading indicators to watch for progress in particular domains...
This is because on-task data will drive on-task competence in any context. (Then, sufficient automation to warrant (partial) rollout and exploration in-situ could drive further data collection, a flywheel.)
> we might see an ‘explosion’ in the speed and cost-effectiveness of AI... But... on-task data and compute will remain crucial to broadening the frontier of autonomous capabilities. *Collecting* that data and *manufacturing* that compute look to me like the rate-limiting steps, and therefore the major leading indicators to use in foresight.
> The best case I can make for a much more general explosion is if the speed and cost-effectiveness explosions rapidly accelerate the gathering and digestion of diverse task data — but I think that remains mostly rate-limited in the familiar ways: some domains easy and some more difficult.
So a lot of contexts where experience is legible, repeatable, and logged, will make quick progress by default.
Where experience is less legible or logged, AI will make less progress (and humans remain valuable) because of the lack of accrued existing context. As this becomes clearer, companies will go out of their way to start making experience more legible and more logged.
Where experience is less repeatable, AI is less effective at sample-efficient adaptability than humans. Progress in sample-efficiency of learning is the key thing here, coupled with learning 'taste' for what exploratory activity will reveal the most useful further observations (cf https://www.oliversourbut.net/p/you-cant-skip-exploration). AI eventually has an advantage here where a fleet can more easily pool experience and accrue taste compared with the lossy communication between humans in a team.