Actual essay: DP claims #9

Feedback from last essay

Mr. Agent,

Good job on continuously doing the exercise for 5+ hours every day.

There are (many issues)[1] we haven’t been able to look into yet, so we expect there are other (high-impact areas we haven’t listed)[3].

What made them expect that there are other areas left? If I don’t like plain Parle G biscuits even after trying many times, I won’t say that “I expect there are other tasty, plain Parle G biscuits that we haven’t listed”.

Perhaps they’ve seen an average of 3 out of 10 areas to be promising, so based on that they expect that maybe 3 out of the remaining 10 would also be good. Or maybe not 3, but at least 1 or 2.

If they had tried a hundred areas and found zero promising ones, they would not expect too many promising areas in the future.

********Reply*************

Great! Didn’t think of it like this! I tried to look for evidence related to the hypothetical example you gave, but didn’t find info on how many issues they looked into. Could I have done better?

**********End************

(These areas)[6a] can be (particularly worth pursuing)[7] if you’re (especially motivated by one of them)[8].

Your response:

For [6a] we think of, working in promoting EA as in the above example.

For [8], we think of a personal fit of more than 50%

For [7], we think of an impact of 5300*50%=2650 lives which is better than working a DS job resulting in 530 net people.

But that doesn’t use [8] at all. Why is it particularly worth pursuing if you are “especially motivated”? The “impact” [7] you pointed out would seem to be the same if you had [8] or if you didn’t.

I would expect an example where someone who had this magical [8] went on to have particularly great “impact”. And I suspect that they do not have that example.

Consider this: You can bowl a particularly high bouncer if you’re 6 feet tall. Showing Bhuvaneshwar Kumar bowling a bouncer is not an example. You have to show Courtney Walsh bowling a bouncer that is one foot higher than usual bouncers.

********Reply*************

OK! So it seems that I have missed the comparison of [8] vs not [8]. i.e., what happens when AI safety is not motivating or less motivating.

**********End************

The (most effective careers)[1] are those that address the (most pressing bottlenecks to progress)[2] on (the most pressing global problems)[3].

The word “most” should trigger your spider sense. If you recall our past discussions, you can say “most” only when you’ve actually compared against everybody else in the set and found it to have the highest value.

This sentence uses “most” thrice. They may be right about what they’re saying, but can you verify it? For [3], have you compared global problems to see which one is most “pressing” (A)? For [2], have you compared the bottlenecks to progress in that field and found the one that is most “pressing” (B)? Finally, for [1], have you compared the “effectiveness” of different careers, however they define it, and found the most effective one (C)?

For the claim to be true, when you look at career C, you must independently see that what it is addressing is B and the global problem it is working on is A. Otherwise, that claim is false.

********Reply*************

When we have “most”, the work needed to give one example for that claim becomes n-times (where n is the number of items in the set). For example, if we talk about “most careers” we need to evaluate the “effectiveness” of all of these n careers.

For [1], I can’t compare all careers practically, so what should I do? how big should the set practically be? what do you expect? Is it good enough to show say 5 careers that are “top” according to 80k and go from there?

For [2], I can look at say 3 of the bottlenecks to progress of [3].

For [3], I can probably compare the top 5 80khours set of careers that they think is pressing

Is that ok? or?

SST has been re-activated.

**********End************

After your (PhD)[1], it’s hard to (re-enter academia if you leave)[2]

Claim: After [1], its hard to [2]

Was not able to find an example online for someone who came back to academia and how “hard” it was for him

One way is to look at the rejection rate, if available, for professorship or post-doctoral fellowship for those who went to the industry after PhD. This data may not be publicly available, but professors who are on the hiring committee would know and would be able to tell people like 80k. They could tell you if they expected higher results from those who left academia.

********Reply************* So when I come across such claims, what do you expect from me? I think I searched for atleast an hour regarding this. Should I skip these? or should I make a comment like you have done above (“rejection rate …”)?

**********End************

You can also (support the work of other researchers)[1] in a (complementary role, such as a project manager, executive assistant, fundraiser or operations)[2]. We’ve argued (these roles)[3] are often neglected, and therefore especially high-impact. It’s often useful to have (graduate training in the relevant area)[4] before taking these roles.

Claim: It is good to [1] in [2].

I think it is important to contrast it with something otherwise it is hard for someone to understand if it is good or bad. Agree: to always contrast?

They aren’t claiming that it is “good”. They are claiming that it is “especially high-impact”. They should have a rough cutoff for how many lives or dollars or whatever constitute “low-impact”, “high-impact”, or “especially high-impact”. Then, you won’t need to contrast against anything else.

********Reply*************

  1. I have covered the “especially high impact” in the next claim “[3] is high-impact”. I see your point, the first 2 sentences could have been combined to 1, and I would have only one claim to answer. I usually look at sentences in isolation when I write claims, unless they are connected by words like “these roles”. I will try and be aware of this thing.

  2. But 80khours believes in WORDS like “high-impact”, what do you expect from me when I send you these things for feedback? Is the example I have good enough for now? or?

    Example: As discussed earlier, AI safety is really quite neglected with 100 people working on it with 10m $. Neil Bowerman from 80khours is trying to add people required to fill the “talent gaps”. If Neil is able to add 10 more people and even claim 1% of their total impact that would be 570 lives saved just for his work in a few years. Contrast that to a DS job which saves 400 people over a life time.

**********End************

There is no doubting the (force of the arguments)[1] the problem is a (research challenge worthy of the next generation’s best mathematical talent)[2]. (Human civilization)[3] is at stake.

Claim: There is no doubting [1].

Question: Why is there no doubting [1]?

Empirically false. Plenty of people doubted it before AI safety went mainstream in late 2014. Many still doubt it.

Another digital macher who agrees is Marc Andreessen, a creator of the original Netscape browser and now a Silicon Valley VC. In one of his recent tweet-storms he addressed the subject: “I don’t see any reason to believe there will suddenly be some jump where all of a sudden they are like super-human intelligence,” he tweeted, and, “The singularity AI stuff is all just a massive handwave, ‘At some point magic happens.’ Nobody has any theory for what this is.”

–

********Reply*************

This seems to be another “most effective careers” moment, where the word “most”—and in this case “NO”—is critical. Either I provide, every example of people not doubting [1]. Or I provide 1 example for doubting and I am done. Correct me if I am wrong!

**********End************

Claim: [3] is at stake (because of AI).

Example:

The fate of Gorillas currently depends on the actions of humans. They are currently endangered. Similarly the fate of humanity may come to depend on the actions of machines than our own.

In other words, we have no concrete example. If they said that diseases put human civilization at stake, we can point to the Black Plague, which killed nearly half the people of Western Europe. Or nukes (Japan). Or asteroids (dinosaurs).

********Reply*************

Because they are predicting something of the future, you are saying there is no example!

But we can imagine one concrete way atleast (Russia using an autonomous weapon system resulting in war) where [3] is at stake due to AI. Isn’t that the point though, that we do “concrete thinking” aka, remove the words and think of examples (hypothetical as they may be)? Why do you seem to be against hypothetical examples?

**********End************

Around 1800, (civilization)[4] underwent (one of the most profound shifts in human history: the industrial revolution)[5].

Claim: Around 1800, [4], underwent [5].

Example: Around 1800, inventions such as the steam engine fueled transportation using horses or a boat went on to railroads, steam boats and automobiles.

“most”.

********Reply*************

But how will the example look then?

Example: So I would need to compare the industrial revolution to the only other “popular” revolution—agricultural revolution—and compare it to show which one was “profound”. Correct?

If there are 10 other revolutions, do you expect me within the scope of this document to compare all 10 and identify which one was the “most profound”?

**********End************

(A growing number of experts)[8] believe that (a third revolution will occur during the 21st century, through the invention of machines with intelligence which far surpasses our own)[9]. These range from (Stephen Hawking to Stuart Russell, the author of the best-selling AI textbook, AI: A Modern Approach)[10].

Claim: [8] believe [9].

Example: Stephen Hawking says here that “full development of an AI” will spell the end of the world.

i guess this is not an example!

It’s not. There are polls of AI experts asking when they think “AI” will happen.

********Reply*************

  1. My mistake is that I only gave the example for ONE EXPERT? where as I need to give an example (aka polls) where MORE EXPERTS are involved?

  2. And also my interpretation of “third revolution” as the doom of mankind, was also wrong. Correct? **********End************

how to do I give an example for “could lead to”? I don’t think I have given one above!”

We don’t have concrete examples for “could happen”, because it would then be “did happen”. Should ask them why they are so confident if they haven’t seen any concrete examples.

I have no idea how to answer this claim, how do I give an example that will inform if A then B.

Again, it’s great if there is a concrete example, such as “if you boil water on the stove, it will evaporate” - I saw it myself. But if not, you have to mark the claim as “no concrete example”.

Given our take on (the world’s most pressing problems)[1] and the (most pressing bottlenecks these issues face)[2], we think the following (five broad categories of career)[3] are a good place to (start generating ideas)[4] if (you have the flexibility to consider a new career path)[5].

Claim: Given [1] and [2], it appears that following [3] is a good place for [4].

Question: Is following [3], a good place for [4]?

I am not sure how to give an example “Given [1] and [2]”?

Another sentence: “Given that we want batsmen who are short and have curly hair, here are the batsmen we found: Sachin Tendulkar, …”

So, if you sorted the table of “all careers” using the fields “how pressing is this problem?” and “how pressing is the bottleneck they’re focusing on?”, then the five top careers are …

Giving an example of Jacques Kallis would be wrong because he isn’t short and doesn’t have curly hair. Ricky Ponting would be wrong because he doesn’t have curly hair.

********Reply*********** Thanks da! Very nice examples! **********End************

Given our take on (the world’s most pressing problems)[1] and the (most pressing bottlenecks these issues face)[2], we think the following (five broad categories of career)[3] are a good place to (start generating ideas)[4] if (you have the flexibility to consider a new career path)[5].

Claim: It appears that following [3], is a good place for [4].

Question: Is following [3], a good place for [4], if [5]?

What does 4, even mean? What is the point of it? Should I bother myself or skip these type of seemingly shit sentences. Who cares about generating ideas? What is the point here?

“Generating ideas for new career paths”. Maybe AI is tough for someone because he has no CS skills, but he could switch from political science to a PhD in economics. The list gives concrete examples of careers they could consider. Imagine if they didn’t. You might never even “generate the idea” of a PhD in economics. Or you might go for something like becoming a dentist.

(Mozart)[8] scored a (130 percent on the precocity index)[9] whereas (his current contemporaries)[10] scored (thirty to five-hundred percent)[10a].

Claim: [8] scored [9].

Example: All this probably requires is a citation? agree?

This is already a concrete example, not a claim. A claim would be something like “Mozart isn’t a lot more precocious than modern child pianists”, for which you would give the above example.

I am in India for vacation for one month (3 weeks left). Trying to clock 6 hrs a day. :)

Enjoy! I’m also in India on vacation.

Mission #9: Your mission, should you choose to accept it, is to concretely analyze the key claims in the book Superintelligence by Nick Bostrom (the book mentioned in the Elon Musk tweet above). He’s a PhD at Oxford who’s been writing about AI safety along with guys like Eliezer for nearly two decades. The book has detailed arguments and examples about all the topics like possible paths to “superintelligence” (whatever that means), types of “superintelligence”, the control problem, etc.

No need to write “Question: “ - doesn’t seem to have changed your answers.

Don’t have to go sentence by sentence; look at one key claim for each section, usually the one in the first few paragraphs, or one for each paragraph if you feel it’s an important section. For example:

CHAPTER 2 Paths to superintelligence

Machines are currently far inferior to humans in general intelligence. Yet one day (we have suggested) they will be superintelligent. How do we get from here to there? This chapter explores several conceivable technological paths. We look at artificial intelligence, whole brain emulation, biological cognition, and human-machine interfaces, as well as networks and organizations. We evaluate their different degrees of plausibility as pathways to superintelligence. The existence of multiple paths increases the probability that the destination can be reached via at least one of them.

The key claim is “How do we get from here to there? Answer: Artificial intelligence, whole brain emulation, …”

Feedback checklist:

  1. Could it be that this claim has no/any example at all? For example, “civilization is at stake” or “5, could lead to extremely positive developments”

  2. Could this claim be false? Remember the “there is no doubting” example. (Could there be other examples that can disprove this? Have you thought about it?)

  3. Does this claim say anything about “best” (need to compare against the entire set) or “most” (need to show it’s the majority in the set) or “no” (need to show that nothing in the set matches)?

  4. Did you stick to examples that are in the chapter itself? That way you don’t have to search online for too long.

  5. Did you use a running example for a technical phrase? There will be lots of new phrases in the book, like “convergent instrumental value” and “orthogonality thesis”. Whenever you see them, you should recall whatever running example you’ve used.

  6. If this is an “if-then” claim, did you either get a concrete example or mark it as having no example?

    ********addition*********** or is there an If (“if you are especially motivated by one of them” example) or Given (Given Short black hair, Jacques kallis will not fit there in that claim) in the statement? Have you used them appropriately in the example? **********End************

Short names: none; false; best; chapter; running; if-then.

Please refer to the checklist after every claim analysis to ensure you’re not making old mistakes. If you want to add to the checklist based on mistakes found in past feedback, that’s great.

  1. Could it have examples? “civilization is at stake due to AI” (consider the possibility it has not exaples)

  2. True or false? Yes “There is no doubting the force of the argument” (consider a falsifying example)

  3. best most no comparison? (consider if you need to compare)

  4. From chapter? (consider examples from chapter)

  5. Running? (is it a running example like talking about the same thing)

  6. If-then If Given? (consider the given type statements)

Not sure if something is running or not? What is the definition… I remember the DP example of talking about tiger woods and thats it. in this case what is the