DP Superintelligence #9.2

Machine

Mission

Mission #9: Your mission, should you choose to accept it, is to concretely analyze the key claims in the book Superintelligence by Nick Bostrom (the book mentioned in the Elon Musk tweet above). He’s a PhD at Oxford who’s been writing about AI safety along with guys like Eliezer for nearly two decades. The book has detailed arguments and examples about all the topics like possible paths to “superintelligence” (whatever that means), types of “superintelligence”, the control problem, etc.

No need to write “Question: “ - doesn’t seem to have changed your answers.

Don’t have to go sentence by sentence; look at one key claim for each section, usually the one in the first few paragraphs, or one for each paragraph if you feel it’s an important section. For example:

CHAPTER 2 Paths to superintelligence

Machines are currently far inferior to humans in general intelligence. Yet one day (we have suggested) they will be superintelligent. How do we get from here to there? This chapter explores several conceivable technological paths. We look at artificial intelligence, whole brain emulation, biological cognition, and human-machine interfaces, as well as networks and organizations. We evaluate their different degrees of plausibility as pathways to superintelligence. The existence of multiple paths increases the probability that the destination can be reached via at least one of them.

The key claim is “How do we get from here to there? Answer: Artificial intelligence, whole brain emulation, …”

Feedback checklist

Feedback checklist:

1. Could it be that this claim has no any example at all? For example, “civilization is at stake”.

2. Could this claim be false? Remember the “there is no doubting” example.

3. Does this claim say anything about “best” (need to compare against the entire set) or “most” (need to show it’s the majority in the set) or “no” (need to show that nothing in the set matches)?

4. Did you stick to examples that are in the chapter itself? That way you don’t have to search online for too long.

5. Did you use a running example for a technical phrase? There will be lots of new phrases in the book, like “convergent instrumental value” and “orthogonality thesis”. Whenever you see them, you should recall whatever running example you’ve used.

6. If this is an “if-then” claim, did you either get a concrete example or mark it as having no example?

Short names: none; false; best; chapter; running; if-then.

Please refer to the checklist after every claim analysis to ensure you’re not making old mistakes. If you want to add to the checklist based on mistakes found in past feedback, that’s great.

Claim

I am possibly going to enjo the experience much much more than the last few days of fighting to complete the [2] hours. It seems like this is a signal for an panindian pandian to do something else or do it differently.

Without Examples I am nothing!

I am more happy to work on this, “because it seems to add more value to my life and it will be interesting”

Claims Chapter 1

Growth modes

Claim: History, at the largest scale, seems to exhibit a sequence of distinct growth modes, each much more rapid than its predecessor.

Claim: the rise of Homo sapiens from our last common ancestor with the great apes happened swiftly.

Claim: These capabilities let humans develop increasingly efficient productive technol- ogies, making it possible for our ancestors to migrate far away from the rainforest and the savanna.

Claim: Such changes in the rate of growth have important consequences.

Claim: Even the present rate of growth will produce impressive results if maintained for a moderately long time.

Claim: Prospect of continuing on a steady exponential growth path pales in comparison to what would happen if the world were to experience another step change in the rate of growth comparable in magnitude to those associated with the Agricultural Revolution and the Industrial Revolution

Claim: Such a growth rate seems fantastic by current lights

Great expectations

the case for taking seriously the prospect of a machine intelligence revolution need not rely on curve-fitting exercises or extrapolations from past economic growth. As we shall see,

Claim: there are stronger reasons for taking heed.

Claim: The growth pattern has taken to suggest that another growth mode might be possible

Claim: Two decades is a sweet spot for prognosticators of radical change: near enough to be attention-grabbing and relevant, yet far enough to make it possible to sup- pose that a string of breakthroughs, currently only vaguely imaginable, might by then have occurred.

Claim: The main reason why progress has been slower than expected is that the technical difficulties of constructing intelligent machines have proved greater than the pioneers foresaw.

Claim: Sometimes a problem that initially looks hopelessly complicated turns out to have a surprisingly simple ­solution (though the reverse is probably more common).

Claim: The next stop, just a short distance farther along the tracks, is super- human-level machine intelligence.

Seasons of hope and dispair

Claim: The methods that produced successes in the early demonstration systems often proved difficult to extend to a wider variety of problems or to harder problem instances

Claim: To overcome the combinatorial explosion, one needs algorithms that exploit structure in the target domain and take advantage of prior knowledge by using heuristic search, planning, and flexible abstract representations—capabilities that were poorly developed in the early AI systems

Claim: The performance of these early systems also suffered because of poor methods for handling uncertainty, reliance on brittle and ungrounded symbolic representations, data scarcity, and severe hardware limitations on memory capacity and processor speed.

Claim: The newly popular techniques, which included neural networks and genetic algorithms, promised to overcome some of the shortcomings of the GOFAI approach, in particular the “brittleness” that characterized classical AI programs (which typically pro- duced complete nonsense if the programmers made even a single slightly erro- neous assumption).

Claim: Behind the razzle-dazzle of machine learning and creative problem-solving thus lies a set of mathematically well-specified tradeoffs.

Claim: artificial intelligence as a quest to find shortcuts: ways of tractably approxi- mating the Bayesian ideal by sacrificing some optimality or generality while pre- serving enough to get high performance in the actual domains of interest.

Claim: One advantage of relating learning problems from specific domains to the gen- eral problem of Bayesian inference is that new algorithms that make Bayesian inference more efficient will then yield immediate improvements across many different areas.

State of the art

Claim: Artificial intelligence already outperforms human intelligence in many domains.

Claim: In other domains, solutions have turned out to be more complicated than initially expected, and progress slower.

Claim: Common sense and natural language understanding have also turned out to be difficult.

Claim: Chess-playing expertise turned out to be achievable by means of a surprisingly simple algorithm

Claim: Modern speech recognition, based on statistical techniques such as hidden Markov models, has become sufficiently accurate for practical use

Claim: Machine translation remains imperfect but is good enough for many applications.

Claim: Face recognition has improved sufficiently in recent years that it is now used at automated border crossings in Europe and Australia.

Claim: Theorem-proving and equation-solving are by now so well established that they are hardly regarded as AI anymore.

Claim: The US military and intelligence establishments have been leading the way to the large-scale deployment of bomb-disposing robots, surveillance and attack drones, and other unmanned vehicles.

Claim: Intelligent scheduling is a major area of success.

Claim: Now, it must be stressed that the demarcation between artificial intelligence and software in general is not sharp.

Claim: many of them contain components that might also play a role in future artificial general intelligence or be of service in its development

Claim: One high-stakes and extremely competitive environment in which AI systems operate today is the global financial market.

Opinions about the future of machines

Claim: Progress on two major fronts has restored to AI research some of its lost prestige.

Claim: One result of this conservatism has been increased concentration on “weak AI”—the variety devoted to providing aids to human thought—and away from “strong AI”—the variety that attempts to mechanize human-level intelligence.

Claim: Expert opinions about the future of AI vary wildly

Claim: There is disagreement about timescales as well as about what forms AI might eventually take.

Claim: we can get a rough impression from various smaller surveys and infor- mal observations.

Claim: These numbers should be taken with some grains of salt:

Claim: median numbers reported in the expert survey do not have enough probability mass on later arrival dates.

Claim: AI researchers have not had a strong record of being able to predict the rate of advances in their own field or the shape that such advances would take.

Claim: Small sample sizes, selection biases, and—above all—the inherent unreliability of the subjective opinions elicited mean that one should not read too much into these expert surveys and interviews.

Claim: it might perhaps fairly soon thereafter result in superintelligence; and that a wide range of outcomes may have a significant chance of occurring, includ- ing extremely good outcomes and outcomes that are as bad as human extinc- tion.

unable to identify one claim per section important!

Questions to an STM

How to identify claims of importance?

What is the goal here?

People are making so many claims (as listed in black in the book)?

How do you go about it?

do you still have the same laptop! kill me!

Statistics

I am dreaming most of the time! I don’t have a deadline or some focus! I think. I am rarely able to do this. I am thinking about the life in India! This should be painful not boring! And I think it is boring and the very second the clock ticks 58 to 60 mins pandian is out!

Need to finish 10 phrases today period!

Letter to an STM

Thalaiva,

If You were me, how would you be spending your time? On what exactly would you be spending your time? Would you just keep trying to clock hours after hours of pure DP on Concrete Thinking? Would you also work on DS?

Why am I looking at DS?

Based on 80khours, I came to the conclusion of working on DS, because it will give me more money(1.5 times in the US) than an engineer, and I could move to the US (for cryonics and more money than here). Everyone from different backgrounds are able to do DS so it should be easy to move. I personally know many people who have moved to DS without too much difficulty after a masters in TU Delft. The route I envision is to start DS work within this year and move to some big DS company in a few years (2-3years) and do a lot of “critical thinking”and make my way to some EAO like say GiveWell within the next 5-8 years and really start saving large numbers of people.

Should I be working on DP for CT completely instead?

Why I ask this is because I am not sure of the consequence of working on this (DP for concrete Thinking), i.e., I don’t have an example where this makes a “difference” in my life. I don’t know how to compare DS and DP for CT. But I will take your word for it and slog my ass off atleast for the coming 4 weeks (just the beginning)(>4hrs per day average of DP guaranteed). Also, over the last few weeks there has been a dip in my amount of hours clocked, so I SUCK and I don’t want to SUCK. 4.15 hours on a good week and 2.9 hrs/day last week (I start half hour after I have had dinner, I take long breaks scrolling FB etc…, including weekends.). (I count work on DS, and DP for concrete thinking together in the above.)

I need your thoughts on this. I am not sure how this 1 hr per day od DP for CT is helping, as I barely get shit done in 1 hr (5 claims, sometimes 1 claim, as things take time to puzzle out.). I stop before I get in the groove. If DS should not be my focus right now, I am more than willing to stop with course 8/10 (20 more hrs of work) and not get closure and not be able to save face when people ask why I am not done with these courses yet.

I don’t want to do 1 hr if it is the most important thing to focus on. I don’t want to do random phrases from texts and move on to others as it gets hard for me. I want to take a full blown essay (80khours key ideas) or chapter from some book on regression and tear it apart over how many ever days it takes full time. Why? that way I guess I get some work done with Large repetitions. Also I can gather some statistics of work done per hour on similar work and compare over the course of the exercise.

So,

Thalaiva, What would you do if you were me? Let’s go big or go home!

And last but not the least, Can you please do this for me? It will help me big time to compare and improve. Can you take this section on Longtermism (5 small paragraphs) and detail it out for me as if you were submitting it for correction by including the phrases the claims and the examples. I am having several questions as I have shown my take, I would like to see you do it and carry that “attitude” throughout the entire essay. It feels like this is one type of essay vagueness I need to handle. And for most parts I am wondering what depth I should go in etc… as discussed below.

I know you always say send me 200 phrases! Name your price for this, just this one time.

Thank You for everything. Cheers!