update README

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/addiction.txt

@@ -1,116 +0,0 @@
-July 2010What hard liquor, cigarettes, heroin, and crack have in common is
-that they're all more concentrated forms of less addictive predecessors.
-Most if not all the things we describe as addictive are.  And the
-scary thing is, the process that created them is accelerating.We wouldn't want to stop it.  It's the same process that cures
-diseases: technological progress.  Technological progress means
-making things do more of what we want.  When the thing we want is
-something we want to want, we consider technological progress good.
-If some new technique makes solar cells x% more efficient, that
-seems strictly better.  When progress concentrates something we
-don't want to want—when it transforms opium into heroin—it seems
-bad.  But it's the same process at work.
-[1]No one doubts this process is accelerating, which means increasing
-numbers of things we like will be transformed into things we like
-too much.
-[2]As far as I know there's no word for something we like too much.
-The closest is the colloquial sense of "addictive." That usage has
-become increasingly common during my lifetime.  And it's clear why:
-there are an increasing number of things we need it for.  At the
-extreme end of the spectrum are crack and meth.  Food has been
-transformed by a combination of factory farming and innovations in
-food processing into something with way more immediate bang for the
-buck, and you can see the results in any town in America.  Checkers
-and solitaire have been replaced by World of Warcraft and FarmVille.
-TV has become much more engaging, and even so it can't compete with Facebook.The world is more addictive than it was 40 years ago.   And unless
-the forms of technological progress that produced these things are
-subject to different laws than technological progress in general,
-the world will get more addictive in the next 40 years than it did
-in the last 40.The next 40 years will bring us some wonderful things.  I don't
-mean to imply they're all to be avoided.  Alcohol is a dangerous
-drug, but I'd rather live in a world with wine than one without.
-Most people can coexist with alcohol; but you have to be careful.
-More things we like will mean more things we have to be careful
-about.Most people won't, unfortunately.  Which means that as the world
-becomes more addictive, the two senses in which one can live a
-normal life will be driven ever further apart.  One sense of "normal"
-is statistically normal: what everyone else does.  The other is the
-sense we mean when we talk about the normal operating range of a
-piece of machinery: what works best.These two senses are already quite far apart.  Already someone
-trying to live well would seem eccentrically abstemious in most of
-the US.  That phenomenon is only going to become more pronounced.
-You can probably take it as a rule of thumb from now on that if
-people don't think you're weird, you're living badly.Societies eventually develop antibodies to addictive new things.
-I've seen that happen with cigarettes.  When cigarettes first
-appeared, they spread the way an infectious disease spreads through
-a previously isolated population.  Smoking rapidly became a
-(statistically) normal thing.  There were ashtrays everywhere.  We
-had ashtrays in our house when I was a kid, even though neither of
-my parents smoked.  You had to for guests.As knowledge spread about the dangers of smoking, customs changed.
-In the last 20 years, smoking has been transformed from something
-that seemed totally normal into a rather seedy habit: from something
-movie stars did in publicity shots to something small huddles of
-addicts do outside the doors of office buildings.  A lot of the
-change was due to legislation, of course, but the legislation
-couldn't have happened if customs hadn't already changed.It took a while though—on the order of 100 years.  And unless the
-rate at which social antibodies evolve can increase to match the
-accelerating rate at which technological progress throws off new
-addictions, we'll be increasingly unable to rely on customs to
-protect us.
-[3]
-Unless we want to be canaries in the coal mine
-of each new addiction—the people whose sad example becomes a
-lesson to future generations—we'll have to figure out for ourselves
-what to avoid and how.  It will actually become a reasonable strategy
-(or a more reasonable strategy) to suspect 
-everything new.In fact, even that won't be enough.  We'll have to worry not just
-about new things, but also about existing things becoming more
-addictive.  That's what bit me.  I've avoided most addictions, but
-the Internet got me because it became addictive while I was using
-it.
-[4]Most people I know have problems with Internet addiction.  We're
-all trying to figure out our own customs for getting free of it.
-That's why I don't have an iPhone, for example; the last thing I
-want is for the Internet to follow me out into the world.
-[5]
-My latest trick is taking long hikes.  I used to think running was a
-better form of exercise than hiking because it took less time.  Now
-the slowness of hiking seems an advantage, because the longer I
-spend on the trail, the longer I have to think without interruption.Sounds pretty eccentric, doesn't it?  It always will when you're
-trying to solve problems where there are no customs yet to guide
-you.  Maybe I can't plead Occam's razor; maybe I'm simply eccentric.
-But if I'm right about the acceleration of addictiveness, then this
-kind of lonely squirming to avoid it will increasingly be the fate
-of anyone who wants to get things done.  We'll increasingly be
-defined by what we say no to.
-Notes[1]
-Could you restrict technological progress to areas where you
-wanted it?  Only in a limited way, without becoming a police state.
-And even then your restrictions would have undesirable side effects.
-"Good" and "bad" technological progress aren't sharply differentiated,
-so you'd find you couldn't slow the latter without also slowing the
-former.  And in any case, as Prohibition and the "war on drugs"
-show, bans often do more harm than good.[2]
-Technology has always been accelerating.  By Paleolithic
-standards, technology evolved at a blistering pace in the Neolithic
-period.[3]
-Unless we mass produce social customs.  I suspect the recent
-resurgence of evangelical Christianity in the US is partly a reaction
-to drugs.  In desperation people reach for the sledgehammer; if
-their kids won't listen to them, maybe they'll listen to God.  But
-that solution has broader consequences than just getting kids to
-say no to drugs.  You end up saying no to 
-science as well.
-I worry we may be heading for a future in which only a few people
-plot their own itinerary through no-land, while everyone else books
-a package tour.  Or worse still, has one booked for them by the
-government.[4]
-People commonly use the word "procrastination" to describe
-what they do on the Internet.  It seems to me too mild to describe
-what's happening as merely not-doing-work.  We don't call it
-procrastination when someone gets drunk instead of working.[5]
-Several people have told me they like the iPad because it
-lets them bring the Internet into situations where a laptop would
-be too conspicuous.  In other words, it's a hip flask.  (This is
-true of the iPhone too, of course, but this advantage isn't as
-obvious because it reads as a phone, and everyone's used to those.)Thanks to Sam Altman, Patrick Collison, Jessica Livingston, and
-Robert Morris for reading drafts of this.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/aord.txt

@@ -1,126 +0,0 @@
-October 2015When I talk to a startup that's been operating for more than 8 or
-9 months, the first thing I want to know is almost always the same.
-Assuming their expenses remain constant and their revenue growth
-is what it has been over the last several months, do they make it to
-profitability on the money they have left?  Or to put it more
-dramatically, by default do they live or die?The startling thing is how often the founders themselves don't know.
-Half the founders I talk to don't know whether they're default alive
-or default dead.If you're among that number, Trevor Blackwell has made a handy
-calculator you can use to find out.The reason I want to know first whether a startup is default alive
-or default dead is that the rest of the conversation depends on the
-answer.  If the company is default alive, we can talk about ambitious
-new things they could do.  If it's default dead, we probably need
-to talk about how to save it.  We know the current trajectory ends
-badly.  How can they get off that trajectory?Why do so few founders know whether they're default alive or default
-dead?  Mainly, I think, because they're not used to asking that.
-It's not a question that makes sense to ask early on, any more than
-it makes sense to ask a 3 year old how he plans to support
-himself.  But as the company grows older, the question switches from
-meaningless to critical.  That kind of switch often takes people
-by surprise.I propose the following solution: instead of starting to ask too
-late whether you're default alive or default dead, start asking too
-early.  It's hard to say precisely when the question switches
-polarity.  But it's probably not that dangerous to start worrying
-too early that you're default dead, whereas it's very dangerous to
-start worrying too late.The reason is a phenomenon I wrote about earlier: the
-fatal pinch.
-The fatal pinch is default dead + slow growth + not enough
-time to fix it.  And the way founders end up in it is by not realizing
-that's where they're headed.There is another reason founders don't ask themselves whether they're
-default alive or default dead: they assume it will be easy to raise
-more money.  But that assumption is often false, and worse still, the
-more you depend on it, the falser it becomes.Maybe it will help to separate facts from hopes. Instead of thinking
-of the future with vague optimism, explicitly separate the components.
-Say "We're default dead, but we're counting on investors to save
-us." Maybe as you say that, it will set off the same alarms in your
-head that it does in mine.  And if you set off the alarms sufficiently
-early, you may be able to avoid the fatal pinch.It would be safe to be default dead if you could count on investors
-saving you.  As a rule their interest is a function of
-growth.  If you have steep revenue growth, say over 5x a year, you
-can start to count on investors being interested even if you're not
-profitable.
-[1]
-But investors are so fickle that you can never
-do more than start to count on them.  Sometimes something about your
-business will spook investors even if your growth is great.  So no
-matter how good your growth is, you can never safely treat fundraising
-as more than a plan A. You should always have a plan B as well: you
-should know (as in write down) precisely what you'll need to do to
-survive if you can't raise more money, and precisely when you'll 
-have to switch to plan B if plan A isn't working.In any case, growing fast versus operating cheaply is far from the
-sharp dichotomy many founders assume it to be.  In practice there
-is surprisingly little connection between how much a startup spends
-and how fast it grows.  When a startup grows fast, it's usually
-because the product hits a nerve, in the sense of hitting some big
-need straight on.  When a startup spends a lot, it's usually because
-the product is expensive to develop or sell, or simply because
-they're wasteful.If you're paying attention, you'll be asking at this point not just
-how to avoid the fatal pinch, but how to avoid being default dead.
-That one is easy: don't hire too fast.  Hiring too fast is by far
-the biggest killer of startups that raise money.
-[2]Founders tell themselves they need to hire in order to grow.  But
-most err on the side of overestimating this need rather than
-underestimating it.  Why?  Partly because there's so much work to
-do.  Naive founders think that if they can just hire enough
-people, it will all get done.  Partly because successful startups have
-lots of employees, so it seems like that's what one does in order
-to be successful.  In fact the large staffs of successful startups
-are probably more the effect of growth than the cause.  And
-partly because when founders have slow growth they don't want to
-face what is usually the real reason: the product is not appealing
-enough.Plus founders who've just raised money are often encouraged to
-overhire by the VCs who funded them.  Kill-or-cure strategies are
-optimal for VCs because they're protected by the portfolio effect.
-VCs want to blow you up, in one sense of the phrase or the other.
-But as a founder your incentives are different.  You want above all
-to survive.
-[3]Here's a common way startups die.  They make something moderately
-appealing and have decent initial growth. They raise their first
-round fairly easily, because the founders seem smart and the idea
-sounds plausible. But because the product is only moderately
-appealing, growth is ok but not great.  The founders convince
-themselves that hiring a bunch of people is the way to boost growth.
-Their investors agree.  But (because the product is only moderately
-appealing) the growth never comes.  Now they're rapidly running out
-of runway.  They hope further investment will save them. But because
-they have high expenses and slow growth, they're now unappealing
-to investors. They're unable to raise more, and the company dies.What the company should have done is address the fundamental problem:
-that the product is only moderately appealing.  Hiring people is
-rarely the way to fix that.  More often than not it makes it harder.
-At this early stage, the product needs to evolve more than to be
-"built out," and that's usually easier with fewer people.
-[4]Asking whether you're default alive or default dead may save you
-from this.  Maybe the alarm bells it sets off will counteract the
-forces that push you to overhire.  Instead you'll be compelled to
-seek growth in other ways. For example, by doing
-things that don't scale, or by redesigning the product in the
-way only founders can.
-And for many if not most startups, these paths to growth will be
-the ones that actually work.Airbnb waited 4 months after raising money at the end of Y Combinator
-before they hired their first employee.  In the meantime the founders
-were terribly overworked.  But they were overworked evolving Airbnb
-into the astonishingly successful organism it is now.Notes[1]
-Steep usage growth will also interest investors.  Revenue
-will ultimately be a constant multiple of usage, so x% usage growth
-predicts x% revenue growth.  But in practice investors discount
-merely predicted revenue, so if you're measuring usage you need a
-higher growth rate to impress investors.[2]
-Startups that don't raise money are saved from hiring too
-fast because they can't afford to. But that doesn't mean you should
-avoid raising money in order to avoid this problem, any more than
-that total abstinence is the only way to avoid becoming an alcoholic.[3]
-I would not be surprised if VCs' tendency to push founders
-to overhire is not even in their own interest.  They don't know how
-many of the companies that get killed by overspending might have
-done well if they'd survived.  My guess is a significant number.[4]
-After reading a draft, Sam Altman wrote:"I think you should make the hiring point more strongly.  I think
-it's roughly correct to say that YC's most successful companies
-have never been the fastest to hire, and one of the marks of a great
-founder is being able to resist this urge."Paul Buchheit adds:"A related problem that I see a lot is premature scaling—founders
-take a small business that isn't really working (bad unit economics,
-typically) and then scale it up because they want impressive growth
-numbers. This is similar to over-hiring in that it makes the business
-much harder to fix once it's big, plus they are bleeding cash really
-fast."
-Thanks to Sam Altman, Paul Buchheit, Joe Gebbia, Jessica Livingston,
-and Geoff Ralston for reading drafts of this.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/apple.txt

@@ -1,201 +0,0 @@
-
-
-Want to start a startup?  Get funded by
-Y Combinator.
-
-
-
-
-November 2009I don't think Apple realizes how badly the App Store approval process
-is broken.  Or rather, I don't think they realize how much it matters
-that it's broken.The way Apple runs the App Store has harmed their reputation with
-programmers more than anything else they've ever done. 
-Their reputation with programmers used to be great.
-It used to be the most common complaint you heard
-about Apple was that their fans admired them too uncritically.
-The App Store has changed that.  Now a lot of programmers
-have started to see Apple as evil.How much of the goodwill Apple once had with programmers have they
-lost over the App Store?  A third?  Half?  And that's just so far.
-The App Store is an ongoing karma leak.* * *How did Apple get into this mess?  Their fundamental problem is
-that they don't understand software.They treat iPhone apps the way they treat the music they sell through
-iTunes.  Apple is the channel; they own the user; if you want to
-reach users, you do it on their terms. The record labels agreed,
-reluctantly.  But this model doesn't work for software.  It doesn't
-work for an intermediary to own the user.  The software business
-learned that in the early 1980s, when companies like VisiCorp showed
-that although the words "software" and "publisher" fit together,
-the underlying concepts don't.  Software isn't like music or books.
-It's too complicated for a third party to act as an intermediary
-between developer and user.   And yet that's what Apple is trying
-to be with the App Store: a software publisher.  And a particularly
-overreaching one at that, with fussy tastes and a rigidly enforced
-house style.If software publishing didn't work in 1980, it works even less now
-that software development has evolved from a small number of big
-releases to a constant stream of small ones.  But Apple doesn't
-understand that either.  Their model of product development derives
-from hardware.  They work on something till they think it's finished,
-then they release it.  You have to do that with hardware, but because
-software is so easy to change, its design can benefit from evolution.
-The standard way to develop applications now is to launch fast and
-iterate.  Which means it's a disaster to have long, random delays
-each time you release a new version.Apparently Apple's attitude is that developers should be more careful
-when they submit a new version to the App Store.  They would say
-that.  But powerful as they are, they're not powerful enough to
-turn back the evolution of technology.  Programmers don't use
-launch-fast-and-iterate out of laziness.  They use it because it
-yields the best results.  By obstructing that process, Apple is
-making them do bad work, and programmers hate that as much as Apple
-would.How would Apple like it if when they discovered a serious bug in
-OS X, instead of releasing a software update immediately, they had
-to submit their code to an intermediary who sat on it for a month
-and then rejected it because it contained an icon they didn't like?By breaking software development, Apple gets the opposite of what
-they intended: the version of an app currently available in the App
-Store tends to be an old and buggy one.  One developer told me:
-
-  As a result of their process, the App Store is full of half-baked
-  applications. I make a new version almost every day that I release
-  to beta users. The version on the App Store feels old and crappy.
-  I'm sure that a lot of developers feel this way: One emotion is
-  "I'm not really proud about what's in the App Store", and it's
-  combined with the emotion "Really, it's Apple's fault."
-
-Another wrote:
-
-  I believe that they think their approval process helps users by
-  ensuring quality.  In reality, bugs like ours get through all the
-  time and then it can take 4-8 weeks to get that bug fix approved,
-  leaving users to think that iPhone apps sometimes just don't work.
-  Worse for Apple, these apps work just fine on other platforms
-  that have immediate approval processes.
-
-Actually I suppose Apple has a third misconception: that all the
-complaints about App Store approvals are not a serious problem.
-They must hear developers complaining.  But partners and suppliers
-are always complaining.  It would be a bad sign if they weren't;
-it would mean you were being too easy on them.  Meanwhile the iPhone
-is selling better than ever.  So why do they need to fix anything?They get away with maltreating developers, in the short term, because
-they make such great hardware.  I just bought a new 27" iMac a
-couple days ago.  It's fabulous.  The screen's too shiny, and the
-disk is surprisingly loud, but it's so beautiful that you can't
-make yourself care.So I bought it, but I bought it, for the first time, with misgivings.
-I felt the way I'd feel buying something made in a country with a
-bad human rights record.  That was new.  In the past when I bought
-things from Apple it was an unalloyed pleasure.  Oh boy!  They make
-such great stuff.  This time it felt like a Faustian bargain.  They
-make such great stuff, but they're such assholes.  Do I really want
-to support this company?* * *Should Apple care what people like me think?  What difference does
-it make if they alienate a small minority of their users?There are a couple reasons they should care.  One is that these
-users are the people they want as employees.  If your company seems
-evil, the best programmers won't work for you.  That hurt Microsoft
-a lot starting in the 90s.  Programmers started to feel sheepish
-about working there.  It seemed like selling out.  When people from
-Microsoft were talking to other programmers and they mentioned where
-they worked, there were a lot of self-deprecating jokes about having
-gone over to the dark side.  But the real problem for Microsoft
-wasn't the embarrassment of the people they hired.  It was the
-people they never got.  And you know who got them?  Google and
-Apple.  If Microsoft was the Empire, they were the Rebel Alliance.
-And it's largely because they got more of the best people that
-Google and Apple are doing so much better than Microsoft today.Why are programmers so fussy about their employers' morals?  Partly
-because they can afford to be.  The best programmers can work
-wherever they want.  They don't have to work for a company they
-have qualms about.But the other reason programmers are fussy, I think, is that evil
-begets stupidity.  An organization that wins by exercising power
-starts to lose the ability to win by doing better work.  And it's
-not fun for a smart person to work in a place where the best ideas
-aren't the ones that win.  I think the reason Google embraced "Don't
-be evil" so eagerly was not so much to impress the outside world
-as to inoculate themselves against arrogance.
-[1]That has worked for Google so far.  They've become more
-bureaucratic, but otherwise they seem to have held true to their
-original principles. With Apple that seems less the case.  When you
-look at the famous 
-1984 ad 
-now, it's easier to imagine Apple as the
-dictator on the screen than the woman with the hammer.
-[2]
-In fact, if you read the dictator's speech it sounds uncannily like a
-prophecy of the App Store.
-
-  We have triumphed over the unprincipled dissemination of facts.We have created, for the first time in all history, a garden of
-  pure ideology, where each worker may bloom secure from the pests
-  of contradictory and confusing truths.
-
-The other reason Apple should care what programmers think of them
-is that when you sell a platform, developers make or break you.  If
-anyone should know this, Apple should.  VisiCalc made the Apple II.And programmers build applications for the platforms they use.  Most
-applications—most startups, probably—grow out of personal projects.
-Apple itself did.  Apple made microcomputers because that's what
-Steve Wozniak wanted for himself.  He couldn't have afforded a
-minicomputer. 
-[3]
- Microsoft likewise started out making interpreters
-for little microcomputers because
-Bill Gates and Paul Allen were interested in using them.  It's a
-rare startup that doesn't build something the founders use.The main reason there are so many iPhone apps is that so many programmers
-have iPhones.  They may know, because they read it in an article,
-that Blackberry has such and such market share.  But in practice
-it's as if RIM didn't exist. If they're going to build something,
-they want to be able to use it themselves, and that means building
-an iPhone app.So programmers continue to develop iPhone apps, even though Apple
-continues to maltreat them.  They're like someone stuck in an abusive
-relationship.  They're so attracted to the iPhone that they can't
-leave.  But they're looking for a way out.  One wrote:
-
-  While I did enjoy developing for the iPhone, the control they
-  place on the App Store does not give me the drive to develop
-  applications as I would like. In fact I don't intend to make any
-  more iPhone applications unless absolutely necessary.
-[4]
-
-Can anything break this cycle?  No device I've seen so far could.
-Palm and RIM haven't a hope.  The only credible contender is Android.
-But Android is an orphan; Google doesn't really care about it, not
-the way Apple cares about the iPhone.  Apple cares about the iPhone
-the way Google cares about search.* * *Is the future of handheld devices one locked down by Apple?  It's
-a worrying prospect.  It would be a bummer to have another grim
-monoculture like we had in the 1990s.  In 1995, writing software
-for end users was effectively identical with writing Windows
-applications.  Our horror at that prospect was the single biggest
-thing that drove us to start building web apps.At least we know now what it would take to break Apple's lock.
-You'd have to get iPhones out of programmers' hands.  If programmers
-used some other device for mobile web access, they'd start to develop
-apps for that instead.How could you make a device programmers liked better than the iPhone?
-It's unlikely you could make something better designed.  Apple
-leaves no room there.  So this alternative device probably couldn't
-win on general appeal.  It would have to win by virtue of some
-appeal it had to programmers specifically.One way to appeal to programmers is with software.  If you
-could think of an application programmers had to have, but that
-would be impossible in the circumscribed world of the iPhone, 
-you could presumably get them to switch.That would definitely happen if programmers started to use handhelds
-as development machines—if handhelds displaced laptops the
-way laptops displaced desktops.  You need more control of a development
-machine than Apple will let you have over an iPhone.Could anyone make a device that you'd carry around in your pocket
-like a phone, and yet would also work as a development machine?
-It's hard to imagine what it would look like.  But I've learned
-never to say never about technology.  A phone-sized device that
-would work as a development machine is no more miraculous by present
-standards than the iPhone itself would have seemed by the standards
-of 1995.My current development machine is a MacBook Air, which I use with
-an external monitor and keyboard in my office, and by itself when
-traveling.  If there was a version half the size I'd prefer it.
-That still wouldn't be small enough to carry around everywhere like
-a phone, but we're within a factor of 4 or so.  Surely that gap is
-bridgeable.  In fact, let's make it an
-RFS. Wanted: 
-Woman with hammer.Notes[1]
-When Google adopted "Don't be evil," they were still so small
-that no one would have expected them to be, yet.
-[2]
-The dictator in the 1984 ad isn't Microsoft, incidentally;
-it's IBM.  IBM seemed a lot more frightening in those days, but
-they were friendlier to developers than Apple is now.[3]
-He couldn't even afford a monitor.  That's why the Apple
-I used a TV as a monitor.[4]
-Several people I talked to mentioned how much they liked the
-iPhone SDK.  The problem is not Apple's products but their policies.
-Fortunately policies are software; Apple can change them instantly
-if they want to.  Handy that, isn't it?Thanks to Sam Altman, Trevor Blackwell, Ross Boucher, 
-James Bracy, Gabor Cselle,
-Patrick Collison, Jason Freedman, John Gruber, Joe Hewitt, Jessica Livingston,
-Robert Morris, Teng Siong Ong, Nikhil Pandit, Savraj Singh, and Jared Tame for reading drafts of this.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/avg.txt

@@ -1,375 +0,0 @@
-
-
-Want to start a startup?  Get funded by
-Y Combinator.
-
-
-
-
-April 2001, rev. April 2003(This article is derived from a talk given at the 2001 Franz
-Developer Symposium.)
-In the summer of 1995, my friend Robert Morris and I
-started a startup called 
-Viaweb.  
-Our plan was to write
-software that would let end users build online stores.
-What was novel about this software, at the time, was
-that it ran on our server, using ordinary Web pages
-as the interface.A lot of people could have been having this idea at the
-same time, of course, but as far as I know, Viaweb was
-the first Web-based application.  It seemed such
-a novel idea to us that we named the company after it:
-Viaweb, because our software worked via the Web,
-instead of running on your desktop computer.Another unusual thing about this software was that it
-was written primarily in a programming language called
-Lisp. It was one of the first big end-user
-applications to be written in Lisp, which up till then
-had been used mostly in universities and research labs. [1]The Secret WeaponEric Raymond has written an essay called "How to Become a Hacker,"
-and in it, among other things, he tells would-be hackers what
-languages they should learn.  He suggests starting with Python and
-Java, because they are easy to learn.  The serious hacker will also
-want to learn C, in order to hack Unix, and Perl for system
-administration and cgi scripts.  Finally, the truly serious hacker
-should consider learning Lisp:
-
-  Lisp is worth learning for the profound enlightenment experience
-  you will have when you finally get it; that experience will make
-  you a better programmer for the rest of your days, even if you
-  never actually use Lisp itself a lot.
-
-This is the same argument you tend to hear for learning Latin.  It
-won't get you a job, except perhaps as a classics professor, but
-it will improve your mind, and make you a better writer in languages
-you do want to use, like English.But wait a minute.  This metaphor doesn't stretch that far.  The
-reason Latin won't get you a job is that no one speaks it.  If you
-write in Latin, no one can understand you.  But Lisp is a computer
-language, and computers speak whatever language you, the programmer,
-tell them to.So if Lisp makes you a better programmer, like he says, why wouldn't
-you want to use it? If a painter were offered a brush that would
-make him a better painter, it seems to me that he would want to
-use it in all his paintings, wouldn't he? I'm not trying to make
-fun of Eric Raymond here.  On the whole, his advice is good.  What
-he says about Lisp is pretty much the conventional wisdom.  But
-there is a contradiction in the conventional wisdom:  Lisp will
-make you a better programmer, and yet you won't use it.Why not?  Programming languages are just tools, after all.  If Lisp
-really does yield better programs, you should use it.  And if it
-doesn't, then who needs it?This is not just a theoretical question.  Software is a very
-competitive business, prone to natural monopolies.  A company that
-gets software written faster and better will, all other things
-being equal, put its competitors out of business.  And when you're
-starting a startup, you feel this very keenly.  Startups tend to
-be an all or nothing proposition.  You either get rich, or you get
-nothing.  In a startup, if you bet on the wrong technology, your
-competitors will crush you.Robert and I both knew Lisp well, and we couldn't see any reason
-not to trust our instincts and go with Lisp.  We knew that everyone
-else was writing their software in C++ or Perl.  But we also knew
-that that didn't mean anything.  If you chose technology that way,
-you'd be running Windows.  When you choose technology, you have to
-ignore what other people are doing, and consider only what will
-work the best.This is especially true in a startup.  In a big company, you can
-do what all the other big companies are doing.  But a startup can't
-do what all the other startups do.  I don't think a lot of people
-realize this, even in startups.The average big company grows at about ten percent a year.  So if
-you're running a big company and you do everything the way the
-average big company does it, you can expect to do as well as the
-average big company-- that is, to grow about ten percent a year.The same thing will happen if you're running a startup, of course.
-If you do everything the way the average startup does it, you should
-expect average performance.  The problem here is, average performance
-means that you'll go out of business.  The survival rate for startups
-is way less than fifty percent.  So if you're running a startup,
-you had better be doing something odd.  If not, you're in trouble.Back in 1995, we knew something that I don't think our competitors
-understood, and few understand even now:  when you're writing
-software that only has to run on your own servers, you can use
-any language you want.  When you're writing desktop software,
-there's a strong bias toward writing applications in the same
-language as the operating system.  Ten years ago, writing applications
-meant writing applications in C.  But with Web-based software,
-especially when you have the source code of both the language and
-the operating system, you can use whatever language you want.This new freedom is a double-edged sword, however.  Now that you
-can use any language, you have to think about which one to use.
-Companies that try to pretend nothing has changed risk finding that
-their competitors do not.If you can use any language, which do you use?  We chose Lisp.
-For one thing, it was obvious that rapid development would be
-important in this market.  We were all starting from scratch, so
-a company that could get new features done before its competitors
-would have a big advantage.  We knew Lisp was a really good language
-for writing software quickly, and server-based applications magnify
-the effect of rapid development, because you can release software
-the minute it's done.If other companies didn't want to use Lisp, so much the better.
-It might give us a technological edge, and we needed all the help
-we could get.  When we started Viaweb, we had no experience in
-business.  We didn't know anything about marketing, or hiring
-people, or raising money, or getting customers.  Neither of us had
-ever even had what you would call a real job.  The only thing we
-were good at was writing software.  We hoped that would save us.
-Any advantage we could get in the software department, we would
-take.So you could say that using Lisp was an experiment.  Our hypothesis
-was that if we wrote our software in Lisp, we'd be able to get
-features done faster than our competitors, and also to do things
-in our software that they couldn't do.  And because Lisp was so
-high-level, we wouldn't need a big development team, so our costs
-would be lower.  If this were so, we could offer a better product
-for less money, and still make a profit.  We would end up getting
-all the users, and our competitors would get none, and eventually
-go out of business.  That was what we hoped would happen, anyway.What were the results of this experiment?  Somewhat surprisingly,
-it worked.  We eventually had many competitors, on the order of
-twenty to thirty of them, but none of their software could compete
-with ours.  We had a wysiwyg online store builder that ran on the
-server and yet felt like a desktop application.  Our competitors
-had cgi scripts.  And we were always far ahead of them in features.
-Sometimes, in desperation, competitors would try to introduce
-features that we didn't have.  But with Lisp our development cycle
-was so fast that we could sometimes duplicate a new feature within
-a day or two of a competitor announcing it in a press release.  By
-the time journalists covering the press release got round to calling
-us, we would have the new feature too.It must have seemed to our competitors that we had some kind of
-secret weapon-- that we were decoding their Enigma traffic or
-something.  In fact we did have a secret weapon, but it was simpler
-than they realized.  No one was leaking news of their features to
-us.   We were just able to develop software faster than anyone
-thought possible.When I was about nine I happened to get hold of a copy of The Day
-of the Jackal, by Frederick Forsyth.  The main character is an
-assassin who is hired to kill the president of France.  The assassin
-has to get past the police to get up to an apartment that overlooks
-the president's route.  He walks right by them, dressed up as an
-old man on crutches, and they never suspect him.Our secret weapon was similar.  We wrote our software in a weird
-AI language, with a bizarre syntax full of parentheses.  For years
-it had annoyed me to hear Lisp described that way.  But now it
-worked to our advantage.  In business, there is nothing more valuable
-than a technical advantage your competitors don't understand.  In
-business, as in war, surprise is worth as much as force.And so, I'm a little embarrassed to say, I never said anything
-publicly about Lisp while we were working on Viaweb.  We never
-mentioned it to the press, and if you searched for Lisp on our Web
-site, all you'd find were the titles of two books in my bio.  This
-was no accident.  A startup should give its competitors as little
-information as possible.  If they didn't know what language our
-software was written in, or didn't care, I wanted to keep it that
-way.[2]The people who understood our technology best were the customers.
-They didn't care what language Viaweb was written in either, but
-they noticed that it worked really well.  It let them build great
-looking online stores literally in minutes.  And so, by word of
-mouth mostly, we got more and more users.  By the end of 1996 we
-had about 70 stores online.  At the end of 1997 we had 500.  Six
-months later, when Yahoo bought us, we had 1070 users.  Today, as
-Yahoo Store, this software continues to dominate its market.  It's
-one of the more profitable pieces of Yahoo, and the stores built
-with it are the foundation of Yahoo Shopping.  I left Yahoo in
-1999, so I don't know exactly how many users they have now, but
-the last I heard there were about 20,000.
-The Blub ParadoxWhat's so great about Lisp?  And if Lisp is so great, why doesn't
-everyone use it?  These sound like rhetorical questions, but actually
-they have straightforward answers.  Lisp is so great not because
-of some magic quality visible only to devotees, but because it is
-simply the most powerful language available.  And the reason everyone
-doesn't use it is that programming languages are not merely
-technologies, but habits of mind as well, and nothing changes
-slower.  Of course, both these answers need explaining.I'll begin with a shockingly controversial statement:  programming
-languages vary in power.Few would dispute, at least, that high level languages are more
-powerful than machine language.  Most programmers today would agree
-that you do not, ordinarily, want to program in machine language.
-Instead, you should program in a high-level language, and have a
-compiler translate it into machine language for you.  This idea is
-even built into the hardware now: since the 1980s, instruction sets
-have been designed for compilers rather than human programmers.Everyone knows it's a mistake to write your whole program by hand
-in machine language.  What's less often understood is that there
-is a more general principle here: that if you have a choice of
-several languages, it is, all other things being equal, a mistake
-to program in anything but the most powerful one. [3]There are many exceptions to this rule.  If you're writing a program
-that has to work very closely with a program written in a certain
-language, it might be a good idea to write the new program in the
-same language.  If you're writing a program that only has to do
-something very simple, like number crunching or bit manipulation,
-you may as well use a less abstract language, especially since it
-may be slightly faster.  And if you're writing a short, throwaway
-program, you may be better off just using whatever language has
-the best library functions for the task.  But in general, for
-application software, you want to be using the most powerful
-(reasonably efficient) language you can get, and using anything
-else is a mistake, of exactly the same kind, though possibly in a
-lesser degree, as programming in machine language.You can see that machine language is very low level.  But, at least
-as a kind of social convention, high-level languages are often all
-treated as equivalent.  They're not.  Technically the term "high-level
-language" doesn't mean anything very definite.  There's no dividing
-line with machine languages on one side and all the high-level
-languages on the other.  Languages fall along a continuum [4] of
-abstractness, from the most powerful all the way down to machine
-languages, which themselves vary in power.Consider Cobol.  Cobol is a high-level language, in the sense that
-it gets compiled into machine language.  Would anyone seriously
-argue that Cobol is equivalent in power to, say, Python?  It's
-probably closer to machine language than Python.Or how about Perl 4?  Between Perl 4 and Perl 5, lexical closures
-got added to the language.  Most Perl hackers would agree that Perl
-5 is more powerful than Perl 4.  But once you've admitted that,
-you've admitted that one high level language can be more powerful
-than another.  And it follows inexorably that, except in special
-cases, you ought to use the most powerful you can get.This idea is rarely followed to its conclusion, though.  After a
-certain age, programmers rarely switch languages voluntarily.
-Whatever language people happen to be used to, they tend to consider
-just good enough.Programmers get very attached to their favorite languages, and I
-don't want to hurt anyone's feelings, so to explain this point I'm
-going to use a hypothetical language called Blub.  Blub falls right
-in the middle of the abstractness continuum.  It is not the most
-powerful language, but it is more powerful than Cobol or machine
-language.And in fact, our hypothetical Blub programmer wouldn't use either
-of them.  Of course he wouldn't program in machine language.  That's
-what compilers are for.  And as for Cobol, he doesn't know how
-anyone can get anything done with it.  It doesn't even have x (Blub
-feature of your choice).As long as our hypothetical Blub programmer is looking down the
-power continuum, he knows he's looking down.  Languages less powerful
-than Blub are obviously less powerful, because they're missing some
-feature he's used to.  But when our hypothetical Blub programmer
-looks in the other direction, up the power continuum, he doesn't
-realize he's looking up.  What he sees are merely weird languages.
-He probably considers them about equivalent in power to Blub, but
-with all this other hairy stuff thrown in as well.  Blub is good
-enough for him, because he thinks in Blub.When we switch to the point of view of a programmer using any of
-the languages higher up the power continuum, however, we find that
-he in turn looks down upon Blub.  How can you get anything done in
-Blub? It doesn't even have y.By induction, the only programmers in a position to see all the
-differences in power between the various languages are those who
-understand the most powerful one.  (This is probably what Eric
-Raymond meant about Lisp making you a better programmer.) You can't
-trust the opinions of the others, because of the Blub paradox:
-they're satisfied with whatever language they happen to use, because
-it dictates the way they think about programs.I know this from my own experience, as a high school kid writing
-programs in Basic.  That language didn't even support recursion.
-It's hard to imagine writing programs without using recursion, but
-I didn't miss it at the time.  I thought in Basic.  And I was a
-whiz at it.  Master of all I surveyed.The five languages that Eric Raymond recommends to hackers fall at
-various points on the power continuum.  Where they fall relative
-to one another is a sensitive topic.  What I will say is that I
-think Lisp is at the top.  And to support this claim I'll tell you
-about one of the things I find missing when I look at the other
-four languages.  How can you get anything done in them, I think,
-without macros? [5]Many languages have something called a macro.  But Lisp macros are
-unique.  And believe it or not, what they do is related to the
-parentheses.  The designers of Lisp didn't put all those parentheses
-in the language just to be different.  To the Blub programmer, Lisp
-code looks weird.  But those parentheses are there for a reason.
-They are the outward evidence of a fundamental difference between
-Lisp and other languages.Lisp code is made out of Lisp data objects.  And not in the trivial
-sense that the source files contain characters, and strings are
-one of the data types supported by the language.  Lisp code, after
-it's read by the parser, is made of data structures that you can
-traverse.If you understand how compilers work, what's really going on is
-not so much that Lisp has a strange syntax as that Lisp has no
-syntax.  You write programs in the parse trees that get generated
-within the compiler when other languages are parsed.  But these
-parse trees are fully accessible to your programs.  You can write
-programs that manipulate them.  In Lisp, these programs are called
-macros.  They are programs that write programs.Programs that write programs?  When would you ever want to do that?
-Not very often, if you think in Cobol.  All the time, if you think
-in Lisp.  It would be convenient here if I could give an example
-of a powerful macro, and say there! how about that?  But if I did,
-it would just look like gibberish to someone who didn't know Lisp;
-there isn't room here to explain everything you'd need to know to
-understand what it meant.  In 
-Ansi Common Lisp I tried to move
-things along as fast as I could, and even so I didn't get to macros
-until page 160.But I think I can give a kind of argument that might be convincing.
-The source code of the Viaweb editor was probably about 20-25%
-macros.  Macros are harder to write than ordinary Lisp functions,
-and it's considered to be bad style to use them when they're not
-necessary.  So every macro in that code is there because it has to
-be.  What that means is that at least 20-25% of the code in this
-program is doing things that you can't easily do in any other
-language.  However skeptical the Blub programmer might be about my
-claims for the mysterious powers of Lisp, this ought to make him
-curious.  We weren't writing this code for our own amusement.  We
-were a tiny startup, programming as hard as we could in order to
-put technical barriers between us and our competitors.A suspicious person might begin to wonder if there was some
-correlation here.  A big chunk of our code was doing things that
-are very hard to do in other languages.  The resulting software
-did things our competitors' software couldn't do.  Maybe there was
-some kind of connection.  I encourage you to follow that thread.
-There may be more to that old man hobbling along on his crutches
-than meets the eye.Aikido for StartupsBut I don't expect to convince anyone 
-(over 25) 
-to go out and learn
-Lisp.  The purpose of this article is not to change anyone's mind,
-but to reassure people already interested in using Lisp-- people
-who know that Lisp is a powerful language, but worry because it
-isn't widely used.  In a competitive situation, that's an advantage.
-Lisp's power is multiplied by the fact that your competitors don't
-get it.If you think of using Lisp in a startup, you shouldn't worry that
-it isn't widely understood.  You should hope that it stays that
-way. And it's likely to.  It's the nature of programming languages
-to make most people satisfied with whatever they currently use.
-Computer hardware changes so much faster than personal habits that
-programming practice is usually ten to twenty years behind the
-processor.  At places like MIT they were writing programs in
-high-level languages in the early 1960s, but many companies continued
-to write code in machine language well into the 1980s.  I bet a
-lot of people continued to write machine language until the processor,
-like a bartender eager to close up and go home, finally kicked them
-out by switching to a risc instruction set.Ordinarily technology changes fast.  But programming languages are
-different: programming languages are not just technology, but what
-programmers think in.  They're half technology and half religion.[6]
-And so the median language, meaning whatever language the median
-programmer uses, moves as slow as an iceberg.  Garbage collection,
-introduced by Lisp in about 1960, is now widely considered to be
-a good thing.  Runtime typing, ditto, is growing in popularity.
-Lexical closures, introduced by Lisp in the early 1970s, are now,
-just barely, on the radar screen.  Macros, introduced by Lisp in the
-mid 1960s, are still terra incognita.Obviously, the median language has enormous momentum.  I'm not
-proposing that you can fight this powerful force.  What I'm proposing
-is exactly the opposite: that, like a practitioner of Aikido, you
-can use it against your opponents.If you work for a big company, this may not be easy.  You will have
-a hard time convincing the pointy-haired boss to let you build
-things in Lisp, when he has just read in the paper that some other
-language is poised, like Ada was twenty years ago, to take over
-the world.  But if you work for a startup that doesn't have
-pointy-haired bosses yet, you can, like we did, turn the Blub
-paradox to your advantage:  you can use technology that your
-competitors, glued immovably to the median language, will never be
-able to match.If you ever do find yourself working for a startup, here's a handy
-tip for evaluating competitors.  Read their job listings.  Everything
-else on their site may be stock photos or the prose equivalent,
-but the job listings have to be specific about what they want, or
-they'll get the wrong candidates.During the years we worked on Viaweb I read a lot of job descriptions.
-A new competitor seemed to emerge out of the woodwork every month
-or so.  The first thing I would do, after checking to see if they
-had a live online demo, was look at their job listings.  After a
-couple years of this I could tell which companies to worry about
-and which not to.  The more of an IT flavor the job descriptions
-had, the less dangerous the company was.  The safest kind were the
-ones that wanted Oracle experience.  You never had to worry about
-those.  You were also safe if they said they wanted C++ or Java
-developers.  If they wanted Perl or Python programmers, that would
-be a bit frightening-- that's starting to sound like a company
-where the technical side, at least, is run by real hackers.  If I
-had ever seen a job posting looking for Lisp hackers, I would have
-been really worried.
-Notes[1] Viaweb at first had two parts: the editor, written in Lisp,
-which people used to build their sites, and the ordering system,
-written in C, which handled orders.  The first version was mostly
-Lisp, because the ordering system was small.  Later we added two
-more modules, an image generator written in C, and a back-office
-manager written mostly in Perl.In January 2003, Yahoo released a new version of the editor 
-written in C++ and Perl.  It's hard to say whether the program is no
-longer written in Lisp, though, because to translate this program
-into C++ they literally had to write a Lisp interpreter: the source
-files of all the page-generating templates are still, as far as I
-know,  Lisp code.  (See Greenspun's Tenth Rule.)[2] Robert Morris says that I didn't need to be secretive, because
-even if our competitors had known we were using Lisp, they wouldn't
-have understood why:  "If they were that smart they'd already be
-programming in Lisp."[3] All languages are equally powerful in the sense of being Turing
-equivalent, but that's not the sense of the word programmers care
-about. (No one wants to program a Turing machine.)  The kind of
-power programmers care about may not be formally definable, but
-one way to explain it would be to say that it refers to features
-you could only get in the less powerful language by writing an
-interpreter for the more powerful language in it. If language A
-has an operator for removing spaces from strings and language B
-doesn't, that probably doesn't make A more powerful, because you
-can probably write a subroutine to do it in B.  But if A supports,
-say, recursion, and B doesn't, that's not likely to be something
-you can fix by writing library functions.[4] Note to nerds: or possibly a lattice, narrowing toward the top;
-it's not the shape that matters here but the idea that there is at
-least a partial order.[5] It is a bit misleading to treat macros as a separate feature.
-In practice their usefulness is greatly enhanced by other Lisp
-features like lexical closures and rest parameters.[6] As a result, comparisons of programming languages either take
-the form of religious wars or undergraduate textbooks so determinedly
-neutral that they're really works of anthropology.  People who
-value their peace, or want tenure, avoid the topic.  But the question
-is only half a religious one; there is something there worth
-studying, especially if you want to design new languages.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/before.txt

@@ -1,387 +0,0 @@
-
-
-Want to start a startup?  Get funded by
-Y Combinator.
-
-
-
-
-October 2014(This essay is derived from a guest lecture in Sam Altman's startup class at
-Stanford.  It's intended for college students, but much of it is
-applicable to potential founders at other ages.)One of the advantages of having kids is that when you have to give
-advice, you can ask yourself "what would I tell my own kids?"  My
-kids are little, but I can imagine what I'd tell them about startups
-if they were in college, and that's what I'm going to tell you.Startups are very counterintuitive.  I'm not sure why.  Maybe it's
-just because knowledge about them hasn't permeated our culture yet.
-But whatever the reason, starting a startup is a task where you
-can't always trust your instincts.It's like skiing in that way.  When you first try skiing and you
-want to slow down, your instinct is to lean back.  But if you lean
-back on skis you fly down the hill out of control.  So part of
-learning to ski is learning to suppress that impulse.  Eventually
-you get new habits, but at first it takes a conscious effort.  At
-first there's a list of things you're trying to remember as you
-start down the hill.Startups are as unnatural as skiing, so there's a similar list for
-startups. Here I'm going to give you the first part of it — the things
-to remember if you want to prepare yourself to start a startup.
-CounterintuitiveThe first item on it is the fact I already mentioned: that startups
-are so weird that if you trust your instincts, you'll make a lot
-of mistakes.  If you know nothing more than this, you may at least
-pause before making them.When I was running Y Combinator I used to joke that our function
-was to tell founders things they would ignore.  It's really true.
-Batch after batch, the YC partners warn founders about mistakes
-they're about to make, and the founders ignore them, and then come
-back a year later and say "I wish we'd listened."Why do the founders ignore the partners' advice?  Well, that's the
-thing about counterintuitive ideas: they contradict your intuitions.
-They seem wrong.  So of course your first impulse is to disregard
-them.  And in fact my joking description is not merely the curse
-of Y Combinator but part of its raison d'etre. If founders' instincts
-already gave them the right answers, they wouldn't need us.  You
-only need other people to give you advice that surprises you. That's
-why there are a lot of ski instructors and not many running
-instructors.
-[1]You can, however, trust your instincts about people.  And in fact
-one of the most common mistakes young founders make is not to
-do that enough.  They get involved with people who seem impressive,
-but about whom they feel some misgivings personally.  Later when
-things blow up they say "I knew there was something off about him,
-but I ignored it because he seemed so impressive."If you're thinking about getting involved with someone — as a
-cofounder, an employee, an investor, or an acquirer — and you
-have misgivings about them, trust your gut.  If someone seems
-slippery, or bogus, or a jerk, don't ignore it.This is one case where it pays to be self-indulgent. Work with
-people you genuinely like, and you've known long enough to be sure.
-ExpertiseThe second counterintuitive point is that it's not that important
-to know a lot about startups.  The way to succeed in a startup is
-not to be an expert on startups, but to be an expert on your users
-and the problem you're solving for them.
-Mark Zuckerberg didn't succeed because he was an expert on startups.
-He succeeded despite being a complete noob at startups, because he
-understood his users really well.If you don't know anything about, say, how to raise an angel round,
-don't feel bad on that account.  That sort of thing you can learn
-when you need to, and forget after you've done it.In fact, I worry it's not merely unnecessary to learn in great
-detail about the mechanics of startups, but possibly somewhat
-dangerous.  If I met an undergrad who knew all about convertible
-notes and employee agreements and (God forbid) class FF stock, I
-wouldn't think "here is someone who is way ahead of their peers."
-It would set off alarms.  Because another of the characteristic
-mistakes of young founders is to go through the motions of starting
-a startup.  They make up some plausible-sounding idea, raise money
-at a good valuation, rent a cool office, hire a bunch of people.
-From the outside that seems like what startups do.  But the next
-step after rent a cool office and hire a bunch of people is: gradually
-realize how completely fucked they are, because while imitating all
-the outward forms of a startup they have neglected the one thing
-that's actually essential: making something people want.
-GameWe saw this happen so often that we made up a name for it: playing
-house.  Eventually I realized why it was happening.  The reason
-young founders go through the motions of starting a startup is
-because that's what they've been trained to do for their whole lives
-up to that point.  Think about what you have to do to get into
-college, for example.  Extracurricular activities, check.  Even in
-college classes most of the work is as artificial as running laps.I'm not attacking the educational system for being this way. There
-will always be a certain amount of fakeness in the work you do when
-you're being taught something, and if you measure their performance
-it's inevitable that people will exploit the difference to the point
-where much of what you're measuring is artifacts of the fakeness.I confess I did it myself in college. I found that in a lot of
-classes there might only be 20 or 30 ideas that were the right shape
-to make good exam questions.  The way I studied for exams in these
-classes was not (except incidentally) to master the material taught
-in the class, but to make a list of potential exam questions and
-work out the answers in advance. When I walked into the final, the
-main thing I'd be feeling was curiosity about which of my questions
-would turn up on the exam.  It was like a game.It's not surprising that after being trained for their whole lives
-to play such games, young founders' first impulse on starting a
-startup is to try to figure out the tricks for winning at this new
-game. Since fundraising appears to be the measure of success for
-startups (another classic noob mistake), they always want to know what the
-tricks are for convincing investors.  We tell them the best way to
-convince investors is to make a startup
-that's actually doing well, meaning growing fast, and then simply
-tell investors so.  Then they want to know what the tricks are for
-growing fast.  And we have to tell them the best way to do that is
-simply to make something people want.So many of the conversations YC partners have with young founders
-begin with the founder asking "How do we..." and the partner replying
-"Just..."Why do the founders always make things so complicated?  The reason,
-I realized, is that they're looking for the trick.So this is the third counterintuitive thing to remember about
-startups: starting a startup is where gaming the system stops
-working.  Gaming the system may continue to work if you go to work
-for a big company. Depending on how broken the company is, you can
-succeed by sucking up to the right people, giving the impression
-of productivity, and so on. 
-[2]
-But that doesn't work with startups.
-There is no boss to trick, only users, and all users care about is
-whether your product does what they want. Startups are as impersonal
-as physics.  You have to make something people want, and you prosper
-only to the extent you do.The dangerous thing is, faking does work to some degree on investors.
-If you're super good at sounding like you know what you're talking
-about, you can fool investors for at least one and perhaps even two
-rounds of funding.  But it's not in your interest to.  The company
-is ultimately doomed.  All you're doing is wasting your own time
-riding it down.So stop looking for the trick. There are tricks in startups, as
-there are in any domain, but they are an order of magnitude less
-important than solving the real problem. A founder who knows nothing
-about fundraising but has made something users love will have an
-easier time raising money than one who knows every trick in the
-book but has a flat usage graph. And more importantly, the founder
-who has made something users love is the one who will go on to
-succeed after raising the money.Though in a sense it's bad news in that you're deprived of one of
-your most powerful weapons, I think it's exciting that gaming the
-system stops working when you start a startup.  It's exciting that
-there even exist parts of the world where you win by doing good
-work.  Imagine how depressing the world would be if it were all
-like school and big companies, where you either have to spend a lot
-of time on bullshit things or lose to people who do.
-[3]
-I would
-have been delighted if I'd realized in college that there were parts
-of the real world where gaming the system mattered less than others,
-and a few where it hardly mattered at all.  But there are, and this
-variation is one of the most important things to consider when
-you're thinking about your future.  How do you win in each type of
-work, and what would you like to win by doing?
-[4]
-All-ConsumingThat brings us to our fourth counterintuitive point: startups are
-all-consuming.  If you start a startup, it will take over your life
-to a degree you cannot imagine.  And if your startup succeeds, it
-will take over your life for a long time: for several years at the
-very least, maybe for a decade, maybe for the rest of your working
-life.  So there is a real opportunity cost here.Larry Page may seem to have an enviable life, but there are aspects
-of it that are unenviable.  Basically at 25 he started running as
-fast as he could and it must seem to him that he hasn't stopped to
-catch his breath since.  Every day new shit happens in the Google
-empire that only the CEO can deal with, and he, as CEO, has to deal
-with it.  If he goes on vacation for even a week, a whole week's
-backlog of shit accumulates.  And he has to bear this uncomplainingly,
-partly because as the company's daddy he can never show fear or
-weakness, and partly because billionaires get less than zero sympathy
-if they talk about having difficult lives.  Which has the strange
-side effect that the difficulty of being a successful startup founder
-is concealed from almost everyone except those who've done it.Y Combinator has now funded several companies that can be called
-big successes, and in every single case the founders say the same
-thing.  It never gets any easier.  The nature of the problems change.
-You're worrying about construction delays at your London office
-instead of the broken air conditioner in your studio apartment.
-But the total volume of worry never decreases; if anything it
-increases.Starting a successful startup is similar to having kids in that
-it's like a button you push that changes your life irrevocably.
-And while it's truly wonderful having kids, there are a lot of
-things that are easier to do before you have them than after.  Many
-of which will make you a better parent when you do have kids. And
-since you can delay pushing the button for a while, most people in
-rich countries do.Yet when it comes to startups, a lot of people seem to think they're
-supposed to start them while they're still in college.  Are you
-crazy?  And what are the universities thinking?  They go out of
-their way to ensure their students are well supplied with contraceptives,
-and yet they're setting up entrepreneurship programs and startup
-incubators left and right.To be fair, the universities have their hand forced here.  A lot
-of incoming students are interested in startups.  Universities are,
-at least de facto, expected to prepare them for their careers.  So
-students who want to start startups hope universities can teach
-them about startups.  And whether universities can do this or not,
-there's some pressure to claim they can, lest they lose applicants
-to other universities that do.Can universities teach students about startups?  Yes and no.  They
-can teach students about startups, but as I explained before, this
-is not what you need to know.  What you need to learn about are the
-needs of your own users, and you can't do that until you actually
-start the company.
-[5]
-So starting a startup is intrinsically
-something you can only really learn by doing it.  And it's impossible
-to do that in college, for the reason I just explained: startups
-take over your life.  You can't start a startup for real as a
-student, because if you start a startup for real you're not a student
-anymore. You may be nominally a student for a bit, but you won't even
-be that for long.
-[6]Given this dichotomy, which of the two paths should you take?  Be
-a real student and not start a startup, or start a real startup and
-not be a student?  I can answer that one for you. Do not start a
-startup in college.  How to start a startup is just a subset of a
-bigger problem you're trying to solve: how to have a good life.
-And though starting a startup can be part of a good life for a lot
-of ambitious people, age 20 is not the optimal time to do it.
-Starting a startup is like a brutally fast depth-first search.  Most
-people should still be searching breadth-first at 20.You can do things in your early 20s that you can't do as well before
-or after, like plunge deeply into projects on a whim and travel
-super cheaply with no sense of a deadline.  For unambitious people,
-this sort of thing is the dreaded "failure to launch," but for the
-ambitious ones it can be an incomparably valuable sort of exploration.
-If you start a startup at 20 and you're sufficiently successful,
-you'll never get to do it.
-[7]Mark Zuckerberg will never get to bum around a foreign country.  He
-can do other things most people can't, like charter jets to fly him
-to foreign countries. But success has taken a lot of the serendipity
-out of his life. Facebook is running him as much as he's running
-Facebook. And while it can be very cool to be in the grip of a
-project you consider your life's work, there are advantages to
-serendipity too, especially early in life.  Among other things it
-gives you more options to choose your life's work from.There's not even a tradeoff here. You're not sacrificing anything
-if you forgo starting a startup at 20, because you're more likely
-to succeed if you wait.  In the unlikely case that you're 20 and
-one of your side projects takes off like Facebook did, you'll face
-a choice of running with it or not, and it may be reasonable to run
-with it.  But the usual way startups take off is for the founders
-to make them take off, and it's gratuitously
-stupid to do that at 20.
-TryShould you do it at any age?  I realize I've made startups sound
-pretty hard.  If I haven't, let me try again: starting a startup
-is really hard.  What if it's too hard?  How can you tell if you're
-up to this challenge?The answer is the fifth counterintuitive point: you can't tell. Your
-life so far may have given you some idea what your prospects might
-be if you tried to become a mathematician, or a professional football
-player.  But unless you've had a very strange life you haven't done
-much that was like being a startup founder.
-Starting a startup will change you a lot.  So what you're trying
-to estimate is not just what you are, but what you could grow into,
-and who can do that?For the past 9 years it was my job to predict whether people would
-have what it took to start successful startups.  It was easy to
-tell how smart they were, and most people reading this will be over
-that threshold.  The hard part was predicting how tough and ambitious they would become.  There
-may be no one who has more experience at trying to predict that,
-so I can tell you how much an expert can know about it, and the
-answer is: not much.  I learned to keep a completely open mind about
-which of the startups in each batch would turn out to be the stars.The founders sometimes think they know. Some arrive feeling sure
-they will ace Y Combinator just as they've aced every one of the (few,
-artificial, easy) tests they've faced in life so far.  Others arrive
-wondering how they got in, and hoping YC doesn't discover whatever
-mistake caused it to accept them.  But there is little correlation
-between founders' initial attitudes and how well their companies
-do.I've read that the same is true in the military — that the
-swaggering recruits are no more likely to turn out to be really
-tough than the quiet ones. And probably for the same reason: that
-the tests involved are so different from the ones in their previous
-lives.If you're absolutely terrified of starting a startup, you probably
-shouldn't do it.  But if you're merely unsure whether you're up to
-it, the only way to find out is to try.  Just not now.
-IdeasSo if you want to start a startup one day, what should you do in
-college?  There are only two things you need initially: an idea and
-cofounders.  And the m.o. for getting both is the same.  Which leads
-to our sixth and last counterintuitive point: that the way to get
-startup ideas is not to try to think of startup ideas.I've written a whole essay on this,
-so I won't repeat it all here.  But the short version is that if
-you make a conscious effort to think of startup ideas, the ideas
-you come up with will not merely be bad, but bad and plausible-sounding,
-meaning you'll waste a lot of time on them before realizing they're
-bad.The way to come up with good startup ideas is to take a step back.
-Instead of making a conscious effort to think of startup ideas,
-turn your mind into the type that startup ideas form in without any
-conscious effort.  In fact, so unconsciously that you don't even
-realize at first that they're startup ideas.This is not only possible, it's how Apple, Yahoo, Google, and
-Facebook all got started.  None of these companies were even meant
-to be companies at first.  They were all just side projects.  The
-best startups almost have to start as side projects, because great
-ideas tend to be such outliers that your conscious mind would reject
-them as ideas for companies.Ok, so how do you turn your mind into the type that startup ideas
-form in unconsciously?  (1) Learn a lot about things that matter,
-then (2) work on problems that interest you (3) with people you
-like and respect.  The third part, incidentally, is how you get
-cofounders at the same time as the idea.The first time I wrote that paragraph, instead of "learn a lot about
-things that matter," I wrote "become good at some technology." But
-that prescription, though sufficient, is too narrow.  What was
-special about Brian Chesky and Joe Gebbia was not that they were
-experts in technology.  They were good at design, and perhaps even
-more importantly, they were good at organizing groups and making
-projects happen.  So you don't have to work on technology per se,
-so long as you work on problems demanding enough to stretch you.What kind of problems are those?  That is very hard to answer in
-the general case.  History is full of examples of young people who
-were working on important problems that no
-one else at the time thought were important, and in particular
-that their parents didn't think were important.  On the other hand,
-history is even fuller of examples of parents who thought their
-kids were wasting their time and who were right.  So how do you
-know when you're working on real stuff?
-[8]I know how I know.  Real problems are interesting, and I am
-self-indulgent in the sense that I always want to work on interesting
-things, even if no one else cares about them (in fact, especially
-if no one else cares about them), and find it very hard to make
-myself work on boring things, even if they're supposed to be
-important.My life is full of case after case where I worked on something just
-because it seemed interesting, and it turned out later to be useful
-in some worldly way.  Y
-Combinator itself was something I only did because it seemed
-interesting. So I seem to have some sort of internal compass that
-helps me out.  But I don't know what other people have in their
-heads. Maybe if I think more about this I can come up with heuristics
-for recognizing genuinely interesting problems, but for the moment
-the best I can offer is the hopelessly question-begging advice that
-if you have a taste for genuinely interesting problems, indulging
-it energetically is the best way to prepare yourself for a startup.
-And indeed, probably also the best way to live.
-[9]But although I can't explain in the general case what counts as an
-interesting problem, I can tell you about a large subset of them.
-If you think of technology as something that's spreading like a
-sort of fractal stain, every moving point on the edge represents
-an interesting problem.  So one guaranteed way to turn your mind
-into the type that has good startup ideas is to get yourself to the
-leading edge of some technology — to cause yourself, as Paul
-Buchheit put it, to "live in the future." When you reach that point,
-ideas that will seem to other people uncannily prescient will seem
-obvious to you.  You may not realize they're startup ideas, but
-you'll know they're something that ought to exist.For example, back at Harvard in the mid 90s a fellow grad student
-of my friends Robert and Trevor wrote his own voice over IP software.
-He didn't mean it to be a startup, and he never tried to turn it
-into one.  He just wanted to talk to his girlfriend in Taiwan without
-paying for long distance calls, and since he was an expert on
-networks it seemed obvious to him that the way to do it was turn
-the sound into packets and ship it over the Internet. He never did
-any more with his software than talk to his girlfriend, but this
-is exactly the way the best startups get started.So strangely enough the optimal thing to do in college if you want
-to be a successful startup founder is not some sort of new, vocational
-version of college focused on "entrepreneurship." It's the classic
-version of college as education for its own sake. If you want to
-start a startup after college, what you should do in college is
-learn powerful things.  And if you have genuine intellectual
-curiosity, that's what you'll naturally tend to do if you just
-follow your own inclinations.
-[10]The component of entrepreneurship that really matters is domain
-expertise.  The way to become Larry Page was to become an expert
-on search. And the way to become an expert on search was to be
-driven by genuine curiosity, not some ulterior motive.At its best, starting a startup is merely an ulterior motive for
-curiosity.  And you'll do it best if you introduce the ulterior
-motive toward the end of the process.So here is the ultimate advice for young would-be startup founders,
-boiled down to two words: just learn.
-Notes[1]
-Some founders listen more than others, and this tends to be a
-predictor of success. One of the things I
-remember about the Airbnbs during YC is how intently they listened.[2]
-In fact, this is one of the reasons startups are possible.  If
-big companies weren't plagued by internal inefficiencies, they'd
-be proportionately more effective, leaving less room for startups.[3]
-In a startup you have to spend a lot of time on schleps, but this sort of work is merely
-unglamorous, not bogus.[4]
-What should you do if your true calling is gaming the system?
-Management consulting.[5]
-The company may not be incorporated, but if you start to get
-significant numbers of users, you've started it, whether you realize
-it yet or not.[6]
-It shouldn't be that surprising that colleges can't teach
-students how to be good startup founders, because they can't teach
-them how to be good employees either.The way universities "teach" students how to be employees is to
-hand off the task to companies via internship programs.  But you
-couldn't do the equivalent thing for startups, because by definition
-if the students did well they would never come back.[7]
-Charles Darwin was 22 when he received an invitation to travel
-aboard the HMS Beagle as a naturalist.  It was only because he was
-otherwise unoccupied, to a degree that alarmed his family, that he
-could accept it. And yet if he hadn't we probably would not know
-his name.[8]
-Parents can sometimes be especially conservative in this
-department.  There are some whose definition of important problems
-includes only those on the critical path to med school.[9]
-I did manage to think of a heuristic for detecting whether you
-have a taste for interesting ideas: whether you find known boring
-ideas intolerable.  Could you endure studying literary theory, or
-working in middle management at a large company?[10]
-In fact, if your goal is to start a startup, you can stick
-even more closely to the ideal of a liberal education than past
-generations have. Back when students focused mainly on getting a
-job after college, they thought at least a little about how the
-courses they took might look to an employer.  And perhaps even
-worse, they might shy away from taking a difficult class lest they
-get a low grade, which would harm their all-important GPA.  Good
-news: users don't care what your GPA
-was.  And I've never heard of investors caring either.  Y Combinator
-certainly never asks what classes you took in college or what grades
-you got in them.
-Thanks to Sam Altman, Paul Buchheit, John Collison, Patrick
-Collison, Jessica Livingston, Robert Morris, Geoff Ralston, and
-Fred Wilson for reading drafts of this.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/bias.txt

@@ -1,54 +0,0 @@
-October 2015This will come as a surprise to a lot of people, but in some cases
-it's possible to detect bias in a selection process without knowing
-anything about the applicant pool.  Which is exciting because among
-other things it means third parties can use this technique to detect
-bias whether those doing the selecting want them to or not.You can use this technique whenever (a) you have at least
-a random sample of the applicants that were selected, (b) their
-subsequent performance is measured, and (c) the groups of
-applicants you're comparing have roughly equal distribution of ability.How does it work?  Think about what it means to be biased.  What
-it means for a selection process to be biased against applicants
-of type x is that it's harder for them to make it through.  Which
-means applicants of type x have to be better to get selected than
-applicants not of type x.
-[1]
-Which means applicants of type x
-who do make it through the selection process will outperform other
-successful applicants.  And if the performance of all the successful
-applicants is measured, you'll know if they do.Of course, the test you use to measure performance must be a valid
-one.  And in particular it must not be invalidated by the bias you're
-trying to measure.
-But there are some domains where performance can be measured, and
-in those detecting bias is straightforward. Want to know if the
-selection process was biased against some type of applicant?  Check
-whether they outperform the others.  This is not just a heuristic
-for detecting bias.  It's what bias means.For example, many suspect that venture capital firms are biased
-against female founders. This would be easy to detect: among their
-portfolio companies, do startups with female founders outperform
-those without?  A couple months ago, one VC firm (almost certainly
-unintentionally) published a study showing bias of this type. First
-Round Capital found that among its portfolio companies, startups
-with female founders outperformed
-those without by 63%. 
-[2]The reason I began by saying that this technique would come as a
-surprise to many people is that we so rarely see analyses of this
-type.  I'm sure it will come as a surprise to First Round that they
-performed one. I doubt anyone there realized that by limiting their
-sample to their own portfolio, they were producing a study not of
-startup trends but of their own biases when selecting companies.I predict we'll see this technique used more in the future.  The
-information needed to conduct such studies is increasingly available.
-Data about who applies for things is usually closely guarded by the
-organizations selecting them, but nowadays data about who gets
-selected is often publicly available to anyone who takes the trouble
-to aggregate it.
-Notes[1]
-This technique wouldn't work if the selection process looked
-for different things from different types of applicants—for
-example, if an employer hired men based on their ability but women
-based on their appearance.[2]
-As Paul Buchheit points out, First Round excluded their most 
-successful investment, Uber, from the study.  And while it 
-makes sense to exclude outliers from some types of studies, 
-studies of returns from startup investing, which is all about 
-hitting outliers, are not one of them.
-Thanks to Sam Altman, Jessica Livingston, and Geoff Ralston for reading
-drafts of this.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/boss.txt

@@ -1,218 +0,0 @@
-
-
-Want to start a startup?  Get funded by
-Y Combinator.
-
-
-
-
-March 2008, rev. June 2008Technology tends to separate normal from natural.  Our bodies
-weren't designed to eat the foods that people in rich countries eat, or
-to get so little exercise.  
-There may be a similar problem with the way we work: 
-a normal job may be as bad for us intellectually as white flour
-or sugar is for us physically.I began to suspect this after spending several years working 
-with startup founders.  I've now worked with over 200 of them, and I've
-noticed a definite difference between programmers working on their
-own startups and those working for large organizations.
-I wouldn't say founders seem happier, necessarily;
-starting a startup can be very stressful. Maybe the best way to put
-it is to say that they're happier in the sense that your body is
-happier during a long run than sitting on a sofa eating
-doughnuts.Though they're statistically abnormal, startup founders seem to be
-working in a way that's more natural for humans.I was in Africa last year and saw a lot of animals in the wild that
-I'd only seen in zoos before. It was remarkable how different they
-seemed. Particularly lions. Lions in the wild seem about ten times
-more alive. They're like different animals. I suspect that working
-for oneself feels better to humans in much the same way that living
-in the wild must feel better to a wide-ranging predator like a lion.
-Life in a zoo is easier, but it isn't the life they were designed
-for.
-TreesWhat's so unnatural about working for a big company?  The root of
-the problem is that humans weren't meant to work in such large
-groups.Another thing you notice when you see animals in the wild is that
-each species thrives in groups of a certain size.  A herd of impalas
-might have 100 adults; baboons maybe 20; lions rarely 10.  Humans
-also seem designed to work in groups, and what I've read about
-hunter-gatherers accords with research on organizations and my own
-experience to suggest roughly what the ideal size is: groups of 8
-work well; by 20 they're getting hard to manage; and a group of 50
-is really unwieldy.
-[1]
-Whatever the upper limit is, we are clearly not meant to work in
-groups of several hundred.  And yet—for reasons having more
-to do with technology than human nature—a great many people
-work for companies with hundreds or thousands of employees.Companies know groups that large wouldn't work, so they divide
-themselves into units small enough to work together.  But to
-coordinate these they have to introduce something new: bosses.These smaller groups are always arranged in a tree structure.  Your
-boss is the point where your group attaches to the tree.  But when
-you use this trick for dividing a large group into smaller ones,
-something strange happens that I've never heard anyone mention
-explicitly.  In the group one level up from yours, your boss
-represents your entire group.  A group of 10 managers is not merely
-a group of 10 people working together in the usual way.  It's really
-a group of groups.  Which means for a group of 10 managers to work
-together as if they were simply a group of 10 individuals, the group
-working for each manager would have to work as if they were a single
-person—the workers and manager would each share only one
-person's worth of freedom between them.In practice a group of people are never able to act as if they were
-one person.  But in a large organization divided into groups in
-this way, the pressure is always in that direction.  Each group
-tries its best to work as if it were the small group of individuals
-that humans were designed to work in.  That was the point of creating
-it.  And when you propagate that constraint, the result is that
-each person gets freedom of action in inverse proportion to the
-size of the entire tree.
-[2]Anyone who's worked for a large organization has felt this.  You
-can feel the difference between working for a company with 100
-employees and one with 10,000, even if your group has only 10 people.
-Corn SyrupA group of 10 people within a large organization is a kind of fake
-tribe.  The number of people you interact with is about right.  But
-something is missing: individual initiative.  Tribes of hunter-gatherers
-have much more freedom.  The leaders have a little more power than other
-members of the tribe, but they don't generally tell them what to
-do and when the way a boss can.It's not your boss's fault.  The real problem is that in the group
-above you in the hierarchy, your entire group is one virtual person.
-Your boss is just the way that constraint is imparted to you.So working in a group of 10 people within a large organization feels
-both right and wrong at the same time.   On the surface it feels
-like the kind of group you're meant to work in, but something major
-is missing.  A job at a big company is like high fructose corn
-syrup: it has some of the qualities of things you're meant to like,
-but is disastrously lacking in others.Indeed, food is an excellent metaphor to explain what's wrong with
-the usual sort of job.For example, working for a big company is the default thing to do,
-at least for programmers.  How bad could it be?  Well, food shows
-that pretty clearly.  If you were dropped at a random point in
-America today, nearly all the food around you would be bad for you.
-Humans were not designed to eat white flour, refined sugar, high
-fructose corn syrup, and hydrogenated vegetable oil.  And yet if
-you analyzed the contents of the average grocery store you'd probably
-find these four ingredients accounted for most of the calories.
-"Normal" food is terribly bad for you.  The only people who eat
-what humans were actually designed to eat are a few Birkenstock-wearing
-weirdos in Berkeley.If "normal" food is so bad for us, why is it so common?  There are
-two main reasons. One is that it has more immediate appeal.  You
-may feel lousy an hour after eating that pizza, but eating the first
-couple bites feels great.  The other is economies of scale.
-Producing junk food scales; producing fresh vegetables doesn't.
-Which means (a) junk food can be very cheap, and (b) it's worth
-spending a lot to market it.If people have to choose between something that's cheap, heavily
-marketed, and appealing in the short term, and something that's
-expensive, obscure, and appealing in the long term, which do you
-think most will choose?It's the same with work.  The average MIT graduate wants to work
-at Google or Microsoft, because it's a recognized brand, it's safe,
-and they'll get paid a good salary right away.  It's the job
-equivalent of the pizza they had for lunch.  The drawbacks will
-only become apparent later, and then only in a vague sense of
-malaise.And founders and early employees of startups, meanwhile, are like
-the Birkenstock-wearing weirdos of Berkeley:  though a tiny minority
-of the population, they're the ones living as humans are meant to.
-In an artificial world, only extremists live naturally.
-ProgrammersThe restrictiveness of big company jobs is particularly hard on
-programmers, because the essence of programming is to build new
-things.  Sales people make much the same pitches every day; support
-people answer much the same questions; but once you've written a
-piece of code you don't need to write it again.  So a programmer
-working as programmers are meant to is always making new things.
-And when you're part of an organization whose structure gives each
-person freedom in inverse proportion to the size of the tree, you're
-going to face resistance when you do something new.This seems an inevitable consequence of bigness.  It's true even
-in the smartest companies.  I was talking recently to a founder who
-considered starting a startup right out of college, but went to
-work for Google instead because he thought he'd learn more there.
-He didn't learn as much as he expected.  Programmers learn by doing,
-and most of the things he wanted to do, he couldn't—sometimes
-because the company wouldn't let him, but often because the company's
-code wouldn't let him.  Between the drag of legacy code, the overhead
-of doing development in such a large organization, and the restrictions
-imposed by interfaces owned by other groups, he could only try a
-fraction of the things he would have liked to.  He said he has
-learned much more in his own startup, despite the fact that he has
-to do all the company's errands as well as programming, because at
-least when he's programming he can do whatever he wants.An obstacle downstream propagates upstream.  If you're not allowed
-to implement new ideas, you stop having them.  And vice versa: when
-you can do whatever you want, you have more ideas about what to do.
-So working for yourself makes your brain more powerful in the same
-way a low-restriction exhaust system makes an engine more powerful.Working for yourself doesn't have to mean starting a startup, of
-course.  But a programmer deciding between a regular job at a big
-company and their own startup is probably going to learn more doing
-the startup.You can adjust the amount of freedom you get by scaling the size
-of company you work for.  If you start the company, you'll have the
-most freedom.  If you become one of the first 10 employees you'll
-have almost as much freedom as the founders.  Even a company with
-100 people will feel different from one with 1000.Working for a small company doesn't ensure freedom.  The tree
-structure of large organizations sets an upper bound on freedom,
-not a lower bound.  The head of a small company may still choose
-to be a tyrant.  The point is that a large organization is compelled
-by its structure to be one.
-ConsequencesThat has real consequences for both organizations and individuals.
-One is that companies will inevitably slow down as they grow larger,
-no matter how hard they try to keep their startup mojo.  It's a
-consequence of the tree structure that every large organization is
-forced to adopt.Or rather, a large organization could only avoid slowing down if
-they avoided tree structure.  And since human nature limits the
-size of group that can work together, the only way I can imagine
-for larger groups to avoid tree structure would be to have no
-structure: to have each group actually be independent, and to work
-together the way components of a market economy do.That might be worth exploring.  I suspect there are already some
-highly partitionable businesses that lean this way.  But I don't
-know any technology companies that have done it.There is one thing companies can do short of structuring themselves
-as sponges:  they can stay small.  If I'm right, then it really
-pays to keep a company as small as it can be at every stage.
-Particularly a technology company.  Which means it's doubly important
-to hire the best people.  Mediocre hires hurt you twice: they get
-less done, but they also make you big, because you need more of
-them to solve a given problem.For individuals the upshot is the same: aim small.  It will always
-suck to work for large organizations, and the larger the organization,
-the more it will suck.In an essay I wrote a couple years ago 
-I advised graduating seniors
-to work for a couple years for another company before starting their
-own.  I'd modify that now.  Work for another company if you want
-to, but only for a small one, and if you want to start your own
-startup, go ahead.The reason I suggested college graduates not start startups immediately
-was that I felt most would fail.  And they will.  But ambitious
-programmers are better off doing their own thing and failing than
-going to work at a big company.  Certainly they'll learn more.  They
-might even be better off financially.  A lot of people in their
-early twenties get into debt, because their expenses grow even
-faster than the salary that seemed so high when they left school.
-At least if you start a startup and fail your net worth will be
-zero rather than negative.  
-[3]We've now funded so many different types of founders that we have
-enough data to see patterns, and there seems to be no benefit from
-working for a big company.  The people who've worked for a few years
-do seem better than the ones straight out of college, but only
-because they're that much older.The people who come to us from big companies often seem kind of
-conservative.  It's hard to say how much is because big companies
-made them that way, and how much is the natural conservatism that
-made them work for the big companies in the first place.  But
-certainly a large part of it is learned.  I know because I've seen
-it burn off.Having seen that happen so many times is one of the things that
-convinces me that working for oneself, or at least for a small
-group, is the natural way for programmers to live.  Founders arriving
-at Y Combinator often have the downtrodden air of refugees.  Three
-months later they're transformed: they have so much more 
-confidence
-that they seem as if they've grown several inches taller. 
-[4]
-Strange as this sounds, they seem both more worried and happier at the same
-time.  Which is exactly how I'd describe the way lions seem in the
-wild.Watching employees get transformed into founders makes it clear
-that the difference between the two is due mostly to environment—and
-in particular that the environment in big companies is toxic to
-programmers.   In the first couple weeks of working on their own
-startup they seem to come to life, because finally they're working
-the way people are meant to.Notes[1]
-When I talk about humans being meant or designed to live a
-certain way, I mean by evolution.[2]
-It's not only the leaves who suffer.  The constraint propagates
-up as well as down.  So managers are constrained too; instead of
-just doing things, they have to act through subordinates.[3]
-Do not finance your startup with credit cards.  Financing a
-startup with debt is usually a stupid move, and credit card debt
-stupidest of all.  Credit card debt is a bad idea, period.  It is
-a trap set by evil companies for the desperate and the foolish.[4]
-The founders we fund used to be younger (initially we encouraged
-undergrads to apply), and the first couple times I saw this I used
-to wonder if they were actually getting physically taller.Thanks to Trevor Blackwell, Ross Boucher, Aaron Iba, Abby
-Kirigin, Ivan Kirigin, Jessica Livingston, and Robert Morris for
-reading drafts of this.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/copy.txt

@@ -1,81 +0,0 @@
-July 2006
-When I was in high school I spent a lot of time imitating bad
-writers.  What we studied in English classes was mostly fiction,
-so I assumed that was the highest form of writing.  Mistake number
-one.  The stories that seemed to be most admired were ones in which
-people suffered in complicated ways.  Anything funny or
-gripping was ipso facto suspect, unless it was old enough to be hard to
-understand, like Shakespeare or Chaucer.  Mistake number two.  The
-ideal medium seemed the short story, which I've since learned had
-quite a brief life, roughly coincident with the peak of magazine
-publishing.  But since their size made them perfect for use in
-high school classes, we read a lot of them, which gave us the
-impression the short story was flourishing.  Mistake number three.
-And because they were so short, nothing really had to happen; you
-could just show a randomly truncated slice of life, and that was
-considered advanced.  Mistake number four.  The result was that I
-wrote a lot of stories in which nothing happened except that someone
-was unhappy in a way that seemed deep.For most of college I was a philosophy major.  I was very impressed
-by the papers published in philosophy journals.  They were so
-beautifully typeset, and their tone was just captivating—alternately
-casual and buffer-overflowingly technical.  A fellow would be walking
-along a street and suddenly modality qua modality would spring upon
-him.  I didn't ever quite understand these papers, but I figured
-I'd get around to that later, when I had time to reread them more
-closely.  In the meantime I tried my best to imitate them.  This
-was, I can now see, a doomed undertaking, because they weren't
-really saying anything.  No philosopher ever refuted another, for
-example, because no one said anything definite enough to refute.
-Needless to say, my imitations didn't say anything either.In grad school I was still wasting time imitating the wrong things.
-There was then a fashionable type of program called an expert system,
-at the core of which was something called an inference engine.  I
-looked at what these things did and thought "I could write that in
-a thousand lines of code."  And yet eminent professors were writing
-books about them, and startups were selling them for a year's salary
-a copy.  What an opportunity, I thought; these impressive things
-seem easy to me; I must be pretty sharp.  Wrong.  It was simply a
-fad.  The books the professors wrote about expert systems are now
-ignored.  They were not even on a path to anything interesting.
-And the customers paying so much for them were largely the same
-government agencies that paid thousands for screwdrivers and toilet
-seats.How do you avoid copying the wrong things?  Copy only what you
-genuinely like.  That would have saved me in all three cases.  I
-didn't enjoy the short stories we had to read in English classes;
-I didn't learn anything from philosophy papers; I didn't use expert
-systems myself.  I believed these things were good because they
-were admired.It can be hard to separate the things you like from the things
-you're impressed with.  One trick is to ignore presentation.  Whenever
-I see a painting impressively hung in a museum, I ask myself: how
-much would I pay for this if I found it at a garage sale, dirty and
-frameless, and with no idea who painted it?  If you walk around a
-museum trying this experiment, you'll find you get some truly
-startling results.  Don't ignore this data point just because it's
-an outlier.Another way to figure out what you like is to look at what you enjoy
-as guilty pleasures.  Many things people like, especially if they're
-young and ambitious, they like largely for the feeling of virtue
-in liking them.  99% of people reading Ulysses are thinking
-"I'm reading Ulysses" as they do it. A guilty pleasure is
-at least a pure one.  What do you read when you don't feel up to being
-virtuous?  What kind of book do you read and feel sad that there's
-only half of it left, instead of being impressed that you're half
-way through?  That's what you really like.Even when you find genuinely good things to copy, there's another
-pitfall to be avoided.  Be careful to copy what makes them good,
-rather than their flaws.  It's easy to be drawn into imitating
-flaws, because they're easier to see, and of course easier to copy
-too.  For example, most painters in the eighteenth and nineteenth
-centuries used brownish colors.  They were imitating the great
-painters of the Renaissance, whose paintings by that time were brown
-with dirt.  Those paintings have since been cleaned, revealing
-brilliant colors; their imitators are of course still brown.It was painting, incidentally, that cured me of copying the wrong
-things.  Halfway through grad school I decided I wanted to try being
-a painter, and the art world was so manifestly corrupt that it
-snapped the leash of credulity.  These people made philosophy
-professors seem as scrupulous as mathematicians.  It was so clearly
-a choice of doing good work xor being an insider that I was forced
-to see the distinction.  It's there to some degree in almost every
-field, but I had till then managed to avoid facing it.That was one of the most valuable things I learned from painting:
-you have to figure out for yourself what's 
-good.  You can't trust
-authorities. They'll lie to you on this one.
-
-Comment on this essay.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/corpdev.txt

@@ -1,107 +0,0 @@
-January 2015Corporate Development, aka corp dev, is the group within companies
-that buys other companies. If you're talking to someone from corp
-dev, that's why, whether you realize it yet or not.It's usually a mistake to talk to corp dev unless (a) you want to
-sell your company right now and (b) you're sufficiently likely to
-get an offer at an acceptable price.  In practice that means startups
-should only talk to corp dev when they're either doing really well
-or really badly.  If you're doing really badly, meaning the company
-is about to die, you may as well talk to them, because you have
-nothing to lose. And if you're doing really well, you can safely
-talk to them, because you both know the price will have to be high,
-and if they show the slightest sign of wasting your time, you'll
-be confident enough to tell them to get lost.The danger is to companies in the middle.  Particularly to young
-companies that are growing fast, but haven't been doing it for long
-enough to have grown big yet.  It's usually a mistake for a promising
-company less than a year old even to talk to corp dev.But it's a mistake founders constantly make.  When someone from
-corp dev wants to meet, the founders tell themselves they should
-at least find out what they want.  Besides, they don't want to
-offend Big Company by refusing to meet.Well, I'll tell you what they want.  They want to talk about buying
-you.  That's what the title "corp dev" means.   So before agreeing
-to meet with someone from corp dev, ask yourselves, "Do we want to
-sell the company right now?"  And if the answer is no, tell them
-"Sorry, but we're focusing on growing the company."  They won't be
-offended.  And certainly the founders of Big Company won't be
-offended. If anything they'll think more highly of you.  You'll
-remind them of themselves.  They didn't sell either; that's why
-they're in a position now to buy other companies.
-[1]Most founders who get contacted by corp dev already know what it
-means.  And yet even when they know what corp dev does and know
-they don't want to sell, they take the meeting.  Why do they do it?
-The same mix of denial and wishful thinking that underlies most
-mistakes founders make. It's flattering to talk to someone who wants
-to buy you.  And who knows, maybe their offer will be surprisingly
-high.  You should at least see what it is, right?No.  If they were going to send you an offer immediately by email,
-sure, you might as well open it.  But that is not how conversations
-with corp dev work.  If you get an offer at all, it will be at the
-end of a long and unbelievably distracting process.  And if the
-offer is surprising, it will be surprisingly low.Distractions are the thing you can least afford in a startup.  And
-conversations with corp dev are the worst sort of distraction,
-because as well as consuming your attention they undermine your
-morale.  One of the tricks to surviving a grueling process is not
-to stop and think how tired you are.  Instead you get into a sort
-of flow. 
-[2]
-Imagine what it would do to you if at mile 20 of a
-marathon, someone ran up beside you and said "You must feel really
-tired.  Would you like to stop and take a rest?"  Conversations
-with corp dev are like that but worse, because the suggestion of
-stopping gets combined in your mind with the imaginary high price
-you think they'll offer.And then you're really in trouble.  If they can, corp dev people
-like to turn the tables on you. They like to get you to the point
-where you're trying to convince them to buy instead of them trying
-to convince you to sell.  And surprisingly often they succeed.This is a very slippery slope, greased with some of the most powerful
-forces that can work on founders' minds, and attended by an experienced
-professional whose full time job is to push you down it.Their tactics in pushing you down that slope are usually fairly
-brutal. Corp dev people's whole job is to buy companies, and they
-don't even get to choose which.  The only way their performance is
-measured is by how cheaply they can buy you, and the more ambitious
-ones will stop at nothing to achieve that. For example, they'll
-almost always start with a lowball offer, just to see if you'll
-take it. Even if you don't, a low initial offer will demoralize you
-and make you easier to manipulate.And that is the most innocent of their tactics. Just wait till
-you've agreed on a price and think you have a done deal, and then
-they come back and say their boss has vetoed the deal and won't do
-it for more than half the agreed upon price. Happens all the time.
-If you think investors can behave badly, it's nothing compared to
-what corp dev people can do.  Even corp dev people at companies
-that are otherwise benevolent.I remember once complaining to a
-friend at Google about some nasty trick their corp dev people had
-pulled on a YC startup."What happened to Don't be Evil?" I asked."I don't think corp dev got the memo," he replied.The tactics you encounter in M&A conversations can be like nothing
-you've experienced in the otherwise comparatively 
-upstanding world
-of Silicon Valley.  It's as if a chunk of genetic material from the
-old-fashioned robber baron business world got incorporated into the
-startup world.
-[3]The simplest way to protect yourself is to use the trick that John
-D. Rockefeller, whose grandfather was an alcoholic, used to protect
-himself from becoming one.  He once told a Sunday school class
-
-  Boys, do you know why I never became a drunkard?  Because I never
-  took the first drink.
-
-Do you want to sell your company right now?  Not eventually, right
-now.  If not, just don't take the first meeting.  They won't be
-offended.  And you in turn will be guaranteed to be spared one of
-the worst experiences that can happen to a startup.If you do want to sell, there's another set of 
-techniques
- for doing
-that.  But the biggest mistake founders make in dealing with corp
-dev is not doing a bad job of talking to them when they're ready
-to, but talking to them before they are.  So if you remember only
-the title of this essay, you already know most of what you need to
-know about M&A in the first year.Notes[1]
-I'm not saying you should never sell.  I'm saying you should
-be clear in your own mind about whether you want to sell or not,
-and not be led by manipulation or wishful thinking into trying to
-sell earlier than you otherwise would have.[2]
-In a startup, as in most competitive sports, the task at hand
-almost does this for you; you're too busy to feel tired.  But when
-you lose that protection, e.g. at the final whistle, the fatigue
-hits you like a wave.  To talk to corp dev is to let yourself feel
-it mid-game.[3]
-To be fair, the apparent misdeeds of corp dev people are magnified
-by the fact that they function as the face of a large organization
-that often doesn't know its own mind.  Acquirers can be surprisingly
-indecisive about acquisitions, and their flakiness is indistinguishable
-from dishonesty by the time it filters down to you.Thanks to Marc Andreessen, Jessica Livingston, Geoff
-Ralston, and Qasar Younis for reading drafts of this.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/desres.txt

@@ -1,234 +0,0 @@
-January 2003(This article is derived from a keynote talk at the fall 2002 meeting
-of NEPLS.)Visitors to this country are often surprised to find that
-Americans like to begin a conversation by asking "what do you do?"
-I've never liked this question.  I've rarely had a
-neat answer to it.  But I think I have finally solved the problem.
-Now, when someone asks me what I do, I look them straight
-in the eye and say "I'm designing a 
-new dialect of Lisp."   
-I recommend this answer to anyone who doesn't like being asked what
-they do.  The conversation will turn immediately to other topics.I don't consider myself to be doing research on programming languages.
-I'm just designing one, in the same way that someone might design
-a building or a chair or a new typeface.
-I'm not trying to discover anything new.  I just want
-to make a language that will be good to program in.  In some ways,
-this assumption makes life a lot easier.The difference between design and research seems to be a question
-of new versus good.  Design doesn't have to be new, but it has to  
-be good.  Research doesn't have to be good, but it has to be new.
-I think these two paths converge at the top: the best design
-surpasses its predecessors by using new ideas, and the best research
-solves problems that are not only new, but actually worth solving.
-So ultimately we're aiming for the same destination, just approaching
-it from different directions.What I'm going to talk about today is what your target looks like
-from the back.  What do you do differently when you treat
-programming languages as a design problem instead of a research topic?The biggest difference is that you focus more on the user.
-Design begins by asking, who is this
-for and what do they need from it?  A good architect,
-for example, does not begin by creating a design that he then
-imposes on the users, but by studying the intended users and figuring
-out what they need.Notice I said "what they need," not "what they want."  I don't mean
-to give the impression that working as a designer means working as 
-a sort of short-order cook, making whatever the client tells you
-to.  This varies from field to field in the arts, but
-I don't think there is any field in which the best work is done by
-the people who just make exactly what the customers tell them to.The customer is always right in
-the sense that the measure of good design is how well it works
-for the user.  If you make a novel that bores everyone, or a chair
-that's horribly uncomfortable to sit in, then you've done a bad
-job, period.  It's no defense to say that the novel or the chair  
-is designed according to the most advanced theoretical principles.And yet, making what works for the user doesn't mean simply making
-what the user tells you to.  Users don't know what all the choices
-are, and are often mistaken about what they really want.The answer to the paradox, I think, is that you have to design
-for the user, but you have to design what the user needs, not simply  
-what he says he wants.
-It's much like being a doctor.  You can't just treat a patient's
-symptoms.  When a patient tells you his symptoms, you have to figure
-out what's actually wrong with him, and treat that.This focus on the user is a kind of axiom from which most of the
-practice of good design can be derived, and around which most design
-issues center.If good design must do what the user needs, who is the user?  When
-I say that design must be for users, I don't mean to imply that good 
-design aims at some kind of  
-lowest common denominator.  You can pick any group of users you
-want.  If you're designing a tool, for example, you can design it
-for anyone from beginners to experts, and what's good design
-for one group might be bad for another.  The point
-is, you have to pick some group of users.  I don't think you can
-even talk about good or bad design except with
-reference to some intended user.You're most likely to get good design if the intended users include
-the designer himself.  When you design something
-for a group that doesn't include you, it tends to be for people
-you consider to be less sophisticated than you, not more sophisticated.That's a problem, because looking down on the user, however benevolently,
-seems inevitably to corrupt the designer.
-I suspect that very few housing
-projects in the US were designed by architects who expected to live
-in them.   You can see the same thing
-in programming languages.  C, Lisp, and Smalltalk were created for
-their own designers to use.  Cobol, Ada, and Java, were created   
-for other people to use.If you think you're designing something for idiots, the odds are
-that you're not designing something good, even for idiots.
-Even if you're designing something for the most sophisticated
-users, though, you're still designing for humans.  It's different 
-in research.  In math you
-don't choose abstractions because they're
-easy for humans to understand; you choose whichever make the
-proof shorter.  I think this is true for the sciences generally.
-Scientific ideas are not meant to be ergonomic.Over in the arts, things are very different.  Design is
-all about people.  The human body is a strange
-thing, but when you're designing a chair,
-that's what you're designing for, and there's no way around it.
-All the arts have to pander to the interests and limitations
-of humans.   In painting, for example, all other things being
-equal a painting with people in it will be more interesting than
-one without.  It is not merely an accident of history that
-the great paintings of the Renaissance are all full of people.
-If they hadn't been, painting as a medium wouldn't have the prestige
-that it does.Like it or not, programming languages are also for people,
-and I suspect the human brain is just as lumpy and idiosyncratic
-as the human body.  Some ideas are easy for people to grasp
-and some aren't.  For example, we seem to have a very limited
-capacity for dealing with detail.  It's this fact that makes
-programing languages a good idea in the first place; if we
-could handle the detail, we could just program in machine
-language.Remember, too, that languages are not
-primarily a form for finished programs, but something that
-programs have to be developed in.  Anyone in the arts could
-tell you that you might want different mediums for the
-two situations.  Marble, for example, is a nice, durable
-medium for finished ideas, but a hopelessly inflexible one
-for developing new ideas.A program, like a proof,
-is a pruned version of a tree that in the past has had
-false starts branching off all over it.  So the test of
-a language is not simply how clean the finished program looks
-in it, but how clean the path to the finished program was.
-A design choice that gives you elegant finished programs
-may not give you an elegant design process.  For example, 
-I've written a few macro-defining macros full of nested
-backquotes that look now like little gems, but writing them
-took hours of the ugliest trial and error, and frankly, I'm still
-not entirely sure they're correct.We often act as if the test of a language were how good
-finished programs look in it.
-It seems so convincing when you see the same program
-written in two languages, and one version is much shorter.
-When you approach the problem from the direction of the
-arts, you're less likely to depend on this sort of
-test.  You don't want to end up with a programming
-language like marble.For example, it is a huge win in developing software to
-have an interactive toplevel, what in Lisp is called a
-read-eval-print loop.  And when you have one this has
-real effects on the design of the language.  It would not
-work well for a language where you have to declare
-variables before using them, for example.  When you're
-just typing expressions into the toplevel, you want to be 
-able to set x to some value and then start doing things
-to x.  You don't want to have to declare the type of x
-first.  You may dispute either of the premises, but if
-a language has to have a toplevel to be convenient, and
-mandatory type declarations are incompatible with a
-toplevel, then no language that makes type declarations  
-mandatory could be convenient to program in.In practice, to get good design you have to get close, and stay
-close, to your users.  You have to calibrate your ideas on actual
-users constantly, especially in the beginning.  One of the reasons
-Jane Austen's novels are so good is that she read them out loud to
-her family.  That's why she never sinks into self-indulgently arty
-descriptions of landscapes,
-or pretentious philosophizing.  (The philosophy's there, but it's
-woven into the story instead of being pasted onto it like a label.)
-If you open an average "literary" novel and imagine reading it out loud
-to your friends as something you'd written, you'll feel all too
-keenly what an imposition that kind of thing is upon the reader.In the software world, this idea is known as Worse is Better.
-Actually, there are several ideas mixed together in the concept of
-Worse is Better, which is why people are still arguing about
-whether worse
-is actually better or not.  But one of the main ideas in that
-mix is that if you're building something new, you should get a
-prototype in front of users as soon as possible.The alternative approach might be called the Hail Mary strategy.
-Instead of getting a prototype out quickly and gradually refining
-it, you try to create the complete, finished, product in one long
-touchdown pass.  As far as I know, this is a
-recipe for disaster.  Countless startups destroyed themselves this
-way during the Internet bubble.  I've never heard of a case
-where it worked.What people outside the software world may not realize is that
-Worse is Better is found throughout the arts.
-In drawing, for example, the idea was discovered during the
-Renaissance.  Now almost every drawing teacher will tell you that
-the right way to get an accurate drawing is not to
-work your way slowly around the contour of an object, because errors will
-accumulate and you'll find at the end that the lines don't meet.
-Instead you should draw a few quick lines in roughly the right place,
-and then gradually refine this initial sketch.In most fields, prototypes
-have traditionally been made out of different materials.
-Typefaces to be cut in metal were initially designed  
-with a brush on paper.  Statues to be cast in bronze   
-were modelled in wax.  Patterns to be embroidered on tapestries
-were drawn on paper with ink wash.  Buildings to be
-constructed from stone were tested on a smaller scale in wood.What made oil paint so exciting, when it
-first became popular in the fifteenth century, was that you
-could actually make the finished work from the prototype.
-You could make a preliminary drawing if you wanted to, but you
-weren't held to it; you could work out all the details, and
-even make major changes, as you finished the painting.You can do this in software too.  A prototype doesn't have to
-be just a model; you can refine it into the finished product.
-I think you should always do this when you can.  It lets you
-take advantage of new insights you have along the way.  But
-perhaps even more important, it's good for morale.Morale is key in design.  I'm surprised people
-don't talk more about it.  One of my first
-drawing teachers told me: if you're bored when you're
-drawing something, the drawing will look boring.
-For example, suppose you have to draw a building, and you
-decide to draw each brick individually.  You can do this
-if you want, but if you get bored halfway through and start
-making the bricks mechanically instead of observing each one,   
-the drawing will look worse than if you had merely suggested
-the bricks.Building something by gradually refining a prototype is good
-for morale because it keeps you engaged.  In software, my  
-rule is: always have working code.  If you're writing
-something that you'll be able to test in an hour, then you
-have the prospect of an immediate reward to motivate you.
-The same is true in the arts, and particularly in oil painting.
-Most painters start with a blurry sketch and gradually
-refine it.
-If you work this way, then in principle
-you never have to end the day with something that actually
-looks unfinished.  Indeed, there is even a saying among
-painters: "A painting is never finished, you just stop
-working on it."  This idea will be familiar to anyone who
-has worked on software.Morale is another reason that it's hard to design something
-for an unsophisticated user.   It's hard to stay interested in
-something you don't like yourself.  To make something  
-good, you have to be thinking, "wow, this is really great,"
-not "what a piece of shit; those fools will love it."Design means making things for humans.  But it's not just the
-user who's human.  The designer is human too.Notice all this time I've been talking about "the designer."
-Design usually has to be under the control of a single person to
-be any good.   And yet it seems to be possible for several people
-to collaborate on a research project.  This seems to
-me one of the most interesting differences between research and
-design.There have been famous instances of collaboration in the arts,
-but most of them seem to have been cases of molecular bonding rather
-than nuclear fusion.  In an opera it's common for one person to
-write the libretto and another to write the music.   And during the Renaissance, 
-journeymen from northern
-Europe were often employed to do the landscapes in the
-backgrounds of Italian paintings.  But these aren't true collaborations.
-They're more like examples of Robert Frost's
-"good fences make good neighbors."  You can stick instances
-of good design together, but within each individual project,
-one person has to be in control.I'm not saying that good design requires that one person think
-of everything.  There's nothing more valuable than the advice
-of someone whose judgement you trust.  But after the talking is
-done, the decision about what to do has to rest with one person.Why is it that research can be done by collaborators and  
-design can't?  This is an interesting question.  I don't 
-know the answer.  Perhaps,
-if design and research converge, the best research is also
-good design, and in fact can't be done by collaborators.
-A lot of the most famous scientists seem to have worked alone.
-But I don't know enough to say whether there
-is a pattern here.  It could be simply that many famous scientists
-worked when collaboration was less common.Whatever the story is in the sciences, true collaboration
-seems to be vanishingly rare in the arts.  Design by committee is a
-synonym for bad design.  Why is that so?  Is there some way to
-beat this limitation?I'm inclined to think there isn't-- that good design requires
-a dictator.  One reason is that good design has to   
-be all of a piece.  Design is not just for humans, but
-for individual humans.  If a design represents an idea that  
-fits in one person's head, then the idea will fit in the user's
-head too.Related:

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/diff.txt

@@ -1,73 +0,0 @@
-December 2001 (rev. May 2002)
-
-(This article came about in response to some questions on
-the LL1 mailing list.  It is now
-incorporated in Revenge of the Nerds.)When McCarthy designed Lisp in the late 1950s, it was
-a radical departure from existing languages,
-the most important of which was Fortran.Lisp embodied nine new ideas:
-1. Conditionals.  A conditional is an if-then-else
-construct.  We take these for granted now.  They were 
-invented
-by McCarthy in the course of developing Lisp. 
-(Fortran at that time only had a conditional
-goto, closely based on the branch instruction in the 
-underlying hardware.)  McCarthy, who was on the Algol committee, got
-conditionals into Algol, whence they spread to most other
-languages.2. A function type. In Lisp, functions are first class 
-objects-- they're a data type just like integers, strings,
-etc, and have a literal representation, can be stored in variables,
-can be passed as arguments, and so on.3. Recursion.  Recursion existed as a mathematical concept
-before Lisp of course, but Lisp was the first programming language to support
-it.  (It's arguably implicit in making functions first class
-objects.)4. A new concept of variables.  In Lisp, all variables
-are effectively pointers. Values are what
-have types, not variables, and assigning or binding
-variables means copying pointers, not what they point to.5. Garbage-collection.6. Programs composed of expressions. Lisp programs are 
-trees of expressions, each of which returns a value.  
-(In some Lisps expressions
-can return multiple values.)  This is in contrast to Fortran
-and most succeeding languages, which distinguish between
-expressions and statements.It was natural to have this
-distinction in Fortran because (not surprisingly in a language
-where the input format was punched cards) the language was
-line-oriented.  You could not nest statements.  And
-so while you needed expressions for math to work, there was
-no point in making anything else return a value, because
-there could not be anything waiting for it.This limitation
-went away with the arrival of block-structured languages,
-but by then it was too late. The distinction between
-expressions and statements was entrenched.  It spread from 
-Fortran into Algol and thence to both their descendants.When a language is made entirely of expressions, you can
-compose expressions however you want.  You can say either
-(using Arc syntax)(if foo (= x 1) (= x 2))or(= x (if foo 1 2))7. A symbol type.  Symbols differ from strings in that
-you can test equality by comparing a pointer.8. A notation for code using trees of symbols.9. The whole language always available.  
-There is
-no real distinction between read-time, compile-time, and runtime.
-You can compile or run code while reading, read or run code
-while compiling, and read or compile code at runtime.Running code at read-time lets users reprogram Lisp's syntax;
-running code at compile-time is the basis of macros; compiling
-at runtime is the basis of Lisp's use as an extension
-language in programs like Emacs; and reading at runtime
-enables programs to communicate using s-expressions, an
-idea recently reinvented as XML.
-When Lisp was first invented, all these ideas were far
-removed from ordinary programming practice, which was
-dictated largely by the hardware available in the late 1950s.Over time, the default language, embodied
-in a succession of popular languages, has
-gradually evolved toward Lisp.  1-5 are now widespread.
-6 is starting to appear in the mainstream.
-Python has a form of 7, though there doesn't seem to be
-any syntax for it.  
-8, which (with 9) is what makes Lisp macros
-possible, is so far still unique to Lisp,
-perhaps because (a) it requires those parens, or something 
-just as bad, and (b) if you add that final increment of power, 
-you can no 
-longer claim to have invented a new language, but only
-to have designed a new dialect of Lisp ; -)Though useful to present-day programmers, it's
-strange to describe Lisp in terms of its
-variation from the random expedients other languages
-adopted.  That was not, probably, how McCarthy
-thought of it.  Lisp wasn't designed to fix the mistakes
-in Fortran; it came about more as the byproduct of an
-attempt to axiomatize computation.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/ecw.txt

@@ -1,98 +0,0 @@
-December 2014If the world were static, we could have monotonically increasing
-confidence in our beliefs.  The more (and more varied) experience
-a belief survived, the less likely it would be false.  Most people
-implicitly believe something like this about their opinions.  And
-they're justified in doing so with opinions about things that don't
-change much, like human nature.  But you can't trust your opinions
-in the same way about things that change, which could include
-practically everything else.When experts are wrong, it's often because they're experts on an
-earlier version of the world.Is it possible to avoid that?  Can you protect yourself against
-obsolete beliefs?  To some extent, yes. I spent almost a decade
-investing in early stage startups, and curiously enough protecting
-yourself against obsolete beliefs is exactly what you have to do
-to succeed as a startup investor.  Most really good startup ideas
-look like bad ideas at first, and many of those look bad specifically
-because some change in the world just switched them from bad to
-good.  I spent a lot of time learning to recognize such ideas, and
-the techniques I used may be applicable to ideas in general.The first step is to have an explicit belief in change.  People who
-fall victim to a monotonically increasing confidence in their
-opinions are implicitly concluding the world is static.  If you
-consciously remind yourself it isn't, you start to look for change.Where should one look for it?  Beyond the moderately useful
-generalization that human nature doesn't change much, the unfortunate
-fact is that change is hard to predict.  This is largely a tautology
-but worth remembering all the same: change that matters usually
-comes from an unforeseen quarter.So I don't even try to predict it.  When I get asked in interviews
-to predict the future, I always have to struggle to come up with
-something plausible-sounding on the fly, like a student who hasn't
-prepared for an exam.
-[1]
-But it's not out of laziness that I haven't
-prepared.  It seems to me that beliefs about the future are so
-rarely correct that they usually aren't worth the extra rigidity
-they impose, and that the best strategy is simply to be aggressively
-open-minded.  Instead of trying to point yourself in the right
-direction, admit you have no idea what the right direction is, and
-try instead to be super sensitive to the winds of change.It's ok to have working hypotheses, even though they may constrain
-you a bit, because they also motivate you.  It's exciting to chase
-things and exciting to try to guess answers.  But you have to be
-disciplined about not letting your hypotheses harden into anything
-more.
-[2]I believe this passive m.o. works not just for evaluating new ideas
-but also for having them.  The way to come up with new ideas is not
-to try explicitly to, but to try to solve problems and simply not
-discount weird hunches you have in the process.The winds of change originate in the unconscious minds of domain
-experts.  If you're sufficiently expert in a field, any weird idea
-or apparently irrelevant question that occurs to you is ipso facto
-worth exploring. 
-[3]
- Within Y Combinator, when an idea is described
-as crazy, it's a compliment—in fact, on average probably a
-higher compliment than when an idea is described as good.Startup investors have extraordinary incentives for correcting
-obsolete beliefs.  If they can realize before other investors that
-some apparently unpromising startup isn't, they can make a huge
-amount of money.  But the incentives are more than just financial.
-Investors' opinions are explicitly tested: startups come to them
-and they have to say yes or no, and then, fairly quickly, they learn
-whether they guessed right.  The investors who say no to a Google
-(and there were several) will remember it for the rest of their
-lives.Anyone who must in some sense bet on ideas rather than merely
-commenting on them has similar incentives.  Which means anyone who
-wants such incentives can have them, by turning their comments into
-bets: if you write about a topic in some fairly durable and public
-form, you'll find you worry much more about getting things right
-than most people would in a casual conversation.
-[4]Another trick I've found to protect myself against obsolete beliefs
-is to focus initially on people rather than ideas. Though the nature
-of future discoveries is hard to predict, I've found I can predict
-quite well what sort of people will make them.  Good new ideas come
-from earnest, energetic, independent-minded people.Betting on people over ideas saved me countless times as an investor.
-We thought Airbnb was a bad idea, for example. But we could tell
-the founders were earnest, energetic, and independent-minded.
-(Indeed, almost pathologically so.)  So we suspended disbelief and
-funded them.This too seems a technique that should be generally applicable.
-Surround yourself with the sort of people new ideas come from.  If
-you want to notice quickly when your beliefs become obsolete, you
-can't do better than to be friends with the people whose discoveries
-will make them so.It's hard enough already not to become the prisoner of your own
-expertise, but it will only get harder, because change is accelerating.
-That's not a recent trend; change has been accelerating since the
-paleolithic era.  Ideas beget ideas.  I don't expect that to change.
-But I could be wrong.
-Notes[1]
-My usual trick is to talk about aspects of the present that
-most people haven't noticed yet.[2]
-Especially if they become well enough known that people start
-to identify them with you.  You have to be extra skeptical about
-things you want to believe, and once a hypothesis starts to be
-identified with you, it will almost certainly start to be in that
-category.[3]
-In practice "sufficiently expert" doesn't require one to be
-recognized as an expert—which is a trailing indicator in any
-case.  In many fields a year of focused work plus caring a lot would
-be enough.[4]
-Though they are public and persist indefinitely, comments on
-e.g. forums and places like Twitter seem empirically to work like
-casual conversation.  The threshold may be whether what you write
-has a title.
-Thanks to Sam Altman, Patrick Collison, and Robert Morris
-for reading drafts of this.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/founders.txt

@@ -1,83 +0,0 @@
-
-
-Want to start a startup?  Get funded by
-Y Combinator.
-
-
-
-
-October 2010
-
-(I wrote this for Forbes, who asked me to write something
-about the qualities we look for in founders.  In print they had to cut
-the last item because they didn't have room.)1. DeterminationThis has turned out to be the most important quality in startup
-founders.  We thought when we started Y Combinator that the most
-important quality would be intelligence.  That's the myth in the
-Valley. And certainly you don't want founders to be stupid.  But
-as long as you're over a certain threshold of intelligence, what
-matters most is determination.  You're going to hit a lot of
-obstacles.  You can't be the sort of person who gets demoralized
-easily.Bill Clerico and Rich Aberman of WePay 
-are a good example.  They're
-doing a finance startup, which means endless negotiations with big,
-bureaucratic companies.  When you're starting a startup that depends
-on deals with big companies to exist, it often feels like they're
-trying to ignore you out of existence.  But when Bill Clerico starts
-calling you, you may as well do what he asks, because he is not
-going away.
-2. FlexibilityYou do not however want the sort of determination implied by phrases
-like "don't give up on your dreams."  The world of startups is so
-unpredictable that you need to be able to modify your dreams on the
-fly.  The best metaphor I've found for the combination of determination
-and flexibility you need is a running back.  
-He's determined to get
-downfield, but at any given moment he may need to go sideways or
-even backwards to get there.The current record holder for flexibility may be Daniel Gross of
-Greplin.  He applied to YC with 
-some bad ecommerce idea.  We told
-him we'd fund him if he did something else.  He thought for a second,
-and said ok.  He then went through two more ideas before settling
-on Greplin.  He'd only been working on it for a couple days when
-he presented to investors at Demo Day, but he got a lot of interest.
-He always seems to land on his feet.
-3. ImaginationIntelligence does matter a lot of course.  It seems like the type
-that matters most is imagination.  It's not so important to be able
-to solve predefined problems quickly as to be able to come up with
-surprising new ideas.  In the startup world, most good ideas 
-seem
-bad initially.  If they were obviously good, someone would already
-be doing them.  So you need the kind of intelligence that produces
-ideas with just the right level of craziness.Airbnb is that kind of idea.  
-In fact, when we funded Airbnb, we
-thought it was too crazy.  We couldn't believe large numbers of
-people would want to stay in other people's places.  We funded them
-because we liked the founders so much.  As soon as we heard they'd
-been supporting themselves by selling Obama and McCain branded
-breakfast cereal, they were in.  And it turned out the idea was on
-the right side of crazy after all.
-4. NaughtinessThough the most successful founders are usually good people, they
-tend to have a piratical gleam in their eye.  They're not Goody
-Two-Shoes type good.  Morally, they care about getting the big
-questions right, but not about observing proprieties.  That's why
-I'd use the word naughty rather than evil.  They delight in 
-breaking
-rules, but not rules that matter.  This quality may be redundant
-though; it may be implied by imagination.Sam Altman of Loopt 
-is one of the most successful alumni, so we
-asked him what question we could put on the Y Combinator application
-that would help us discover more people like him.  He said to ask
-about a time when they'd hacked something to their advantage—hacked in the sense of beating the system, not breaking into
-computers.  It has become one of the questions we pay most attention
-to when judging applications.
-5. FriendshipEmpirically it seems to be hard to start a startup with just 
-one
-founder.  Most of the big successes have two or three.  And the
-relationship between the founders has to be strong.  They must
-genuinely like one another, and work well together.  Startups do
-to the relationship between the founders what a dog does to a sock:
-if it can be pulled apart, it will be.Emmett Shear and Justin Kan of Justin.tv 
-are a good example of close
-friends who work well together.  They've known each other since
-second grade.  They can practically read one another's minds.  I'm
-sure they argue, like all founders, but I have never once sensed
-any unresolved tension between them.Thanks to Jessica Livingston and Chris Steiner for reading drafts of this.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/gap.txt

@@ -1,485 +0,0 @@
-May 2004When people care enough about something to do it well, those who
-do it best tend to be far better than everyone else.  There's a
-huge gap between Leonardo and second-rate contemporaries like
-Borgognone.  You see the same gap between Raymond Chandler and the
-average writer of detective novels.  A top-ranked professional chess
-player could play ten thousand games against an ordinary club player
-without losing once.Like chess or painting or writing novels, making money is a very
-specialized skill.   But for some reason we treat this skill
-differently.  No one complains when a few people surpass all the
-rest at playing chess or writing novels, but when a few people make
-more money than the rest, we get editorials saying this is wrong.Why?  The pattern of variation seems no different than for any other
-skill.  What causes people to react so strongly when the skill is
-making money?I think there are three reasons we treat making money as different:
-the misleading model of wealth we learn as children; the disreputable
-way in which, till recently, most fortunes were accumulated; and
-the worry that great variations in income are somehow bad for
-society.  As far as I can tell, the first is mistaken, the second
-outdated, and the third empirically false.  Could it be that, in a
-modern democracy, variation in income is actually a sign of health?The Daddy Model of WealthWhen I was five I thought electricity was created by electric
-sockets.  I didn't realize there were power plants out there
-generating it.  Likewise, it doesn't occur to most kids that wealth
-is something that has to be generated.  It seems to be something
-that flows from parents.Because of the circumstances in which they encounter it, children
-tend to misunderstand wealth.  They confuse it with money.  They
-think that there is a fixed amount of it.  And they think of it as
-something that's distributed by authorities (and so should be
-distributed equally), rather than something that has to be created
-(and might be created unequally).In fact, wealth is not money.  Money is just a convenient way of
-trading one form of wealth for another.  Wealth is the underlying
-stuff—the goods and services we buy.  When you travel to a
-rich or poor country, you don't have to look at people's bank
-accounts to tell which kind you're in.  You can see
-wealth—in buildings and streets, in the clothes and the health
-of the people.Where does wealth come from?  People make it.  This was easier to
-grasp when most people lived on farms, and made many of the things
-they wanted with their own hands.  Then you could see in the house,
-the herds, and the granary the wealth that each family created.  It
-was obvious then too that the wealth of the world was not a fixed
-quantity that had to be shared out, like slices of a pie.  If you
-wanted more wealth, you could make it.This is just as true today, though few of us create wealth directly
-for ourselves (except for a few vestigial domestic tasks).  Mostly
-we create wealth for other people in exchange for money, which we
-then trade for the forms of wealth we want. 
-[1]Because kids are unable to create wealth, whatever they have has
-to be given to them.  And when wealth is something you're given,
-then of course it seems that it should be distributed equally.
-[2]
-As in most families it is.  The kids see to that.  "Unfair," they
-cry, when one sibling gets more than another.In the real world, you can't keep living off your parents.  If you
-want something, you either have to make it, or do something of
-equivalent value for someone else, in order to get them to give you
-enough money to buy it.  In the real world, wealth is (except for
-a few specialists like thieves and speculators) something you have
-to create, not something that's distributed by Daddy.  And since
-the ability and desire to create it vary from person to person,
-it's not made equally.You get paid by doing or making something people want, and those
-who make more money are often simply better at doing what people
-want.  Top actors make a lot more money than B-list actors.  The
-B-list actors might be almost as charismatic, but when people go
-to the theater and look at the list of movies playing, they want
-that extra oomph that the big stars have.Doing what people want is not the only way to get money, of course.
-You could also rob banks, or solicit bribes, or establish a monopoly.
-Such tricks account for some variation in wealth, and indeed for
-some of the biggest individual fortunes, but they are not the root
-cause of variation in income.  The root cause of variation in income,
-as Occam's Razor implies, is the same as the root cause of variation
-in every other human skill.In the United States, the CEO of a large public company makes about
-100 times as much as the average person. 
-[3]
-Basketball players
-make about 128 times as much, and baseball players 72 times as much.
-Editorials quote this kind of statistic with horror.  But I have
-no trouble imagining that one person could be 100 times as productive
-as another.  In ancient Rome the price of slaves varied by
-a factor of 50 depending on their skills. 
-[4]
-And that's without
-considering motivation, or the extra leverage in productivity that
-you can get from modern technology.Editorials about athletes' or CEOs' salaries remind me of early
-Christian writers, arguing from first principles about whether the
-Earth was round, when they could just walk outside and check.
-[5]
-How much someone's work is worth is not a policy question.  It's
-something the market already determines."Are they really worth 100 of us?" editorialists ask.  Depends on
-what you mean by worth.  If you mean worth in the sense of what
-people will pay for their skills, the answer is yes, apparently.A few CEOs' incomes reflect some kind of wrongdoing.  But are there
-not others whose incomes really do reflect the wealth they generate?
-Steve Jobs saved a company that was in a terminal decline.  And not
-merely in the way a turnaround specialist does, by cutting costs;
-he had to decide what Apple's next products should be.  Few others
-could have done it.  And regardless of the case with CEOs, it's
-hard to see how anyone could argue that the salaries of professional
-basketball players don't reflect supply and demand.It may seem unlikely in principle that one individual could really
-generate so much more wealth than another.  The key to this mystery
-is to revisit that question, are they really worth 100 of us?
-Would a basketball team trade one of their players for 100
-random people?  What would Apple's next product look like if you
-replaced Steve Jobs with a committee of 100 random people? 
-[6]
-These
-things don't scale linearly.  Perhaps the CEO or the professional
-athlete has only ten times (whatever that means) the skill and
-determination of an ordinary person.  But it makes all the difference
-that it's concentrated in one individual.When we say that one kind of work is overpaid and another underpaid,
-what are we really saying?  In a free market, prices are determined
-by what buyers want.  People like baseball more than  poetry, so
-baseball players make more than poets.  To say that a certain kind
-of work is underpaid is thus identical with saying that people want
-the wrong things.Well, of course people want the wrong things.  It seems odd to be
-surprised by that.  And it seems even odder to say that it's
-unjust that certain kinds of work are underpaid. 
-[7]
-Then
-you're saying that it's unjust that people want the wrong things.
-It's  lamentable that people prefer reality TV and corndogs to
-Shakespeare and steamed vegetables, but unjust?  That seems like
-saying that blue is heavy, or that up is circular.The appearance of the word "unjust" here is the unmistakable spectral
-signature of the Daddy Model.  Why else would this idea occur in
-this odd context?  Whereas if the speaker were still operating on
-the Daddy Model, and saw wealth as something that flowed from a
-common source and had to be shared out, rather than something
-generated by doing what other people wanted, this is exactly what
-you'd get on noticing that some people made much more than others.When we talk about "unequal distribution of income," we should
-also ask, where does that income come from?
-[8]
-Who made the wealth
-it represents?  Because to the extent that income varies simply
-according to how much wealth people create, the distribution may
-be unequal, but it's hardly unjust.Stealing ItThe second reason we tend to find great disparities of wealth
-alarming is that for most of human history the usual way to accumulate
-a fortune was to steal it: in pastoral societies by cattle raiding;
-in agricultural societies by appropriating others' estates in times
-of war, and taxing them in times of peace.In conflicts, those on the winning side would receive the estates
-confiscated from the losers.  In England in the 1060s, when William
-the Conqueror distributed the estates of the defeated Anglo-Saxon
-nobles to his followers, the conflict was military.  By the 1530s,
-when Henry VIII distributed the estates of the monasteries to his
-followers, it was mostly political. 
-[9]
-But the principle was the
-same.  Indeed, the same principle is at work now in Zimbabwe.In more organized societies, like China, the ruler and his officials
-used taxation instead of confiscation.  But here too we see the
-same principle: the way to get rich was not to create wealth, but
-to serve a ruler powerful enough to appropriate it.This started to change in Europe with the rise of the middle class.
-Now we think of the middle class as people who are neither rich nor
-poor, but originally they were a distinct group.  In a feudal
-society, there are just two classes: a warrior aristocracy, and the
-serfs who work their estates.  The middle class were a new, third
-group who lived in towns and supported themselves by manufacturing
-and trade.Starting in the tenth and eleventh centuries, petty nobles and
-former serfs banded together in towns that gradually became powerful
-enough to ignore the local feudal lords. 
-[10]
-Like serfs, the middle
-class made a living largely by creating wealth.  (In port cities
-like Genoa and Pisa, they also engaged in piracy.) But unlike serfs
-they had an incentive to create a lot of it.  Any wealth a serf
-created belonged to his master.  There was not much point in making
-more than you could hide.  Whereas the independence of the townsmen
-allowed them to keep whatever wealth they created.Once it became possible to get rich by creating wealth, society as
-a whole started to get richer very rapidly.  Nearly everything we
-have was created by the middle class.  Indeed, the other two classes
-have effectively disappeared in industrial societies, and their
-names been given to either end of the middle class.  (In the original
-sense of the word, Bill Gates is middle class.)But it was not till the Industrial Revolution that wealth creation
-definitively replaced corruption as the best way to get rich.  In
-England, at least, corruption only became unfashionable (and in
-fact only started to be called "corruption") when there started to
-be other, faster ways to get rich.Seventeenth-century England was much like the third world today,
-in that government office was a recognized route to wealth.  The
-great fortunes of that time still derived more from what we would
-now call corruption than from commerce. 
-[11]
-By the nineteenth
-century that had changed.  There continued to be bribes, as there
-still are everywhere, but politics had by then been left to men who
-were driven more by vanity than greed.  Technology had made it
-possible to create wealth faster than you could steal it.  The
-prototypical rich man of the nineteenth century was not a courtier
-but an industrialist.With the rise of the middle class, wealth stopped being a zero-sum
-game.  Jobs and Wozniak didn't have to make us poor to make themselves
-rich.  Quite the opposite: they created things that made our lives
-materially richer.  They had to, or we wouldn't have paid for them.But since for most of the world's history the main route to wealth
-was to steal it, we tend to be suspicious of rich people.  Idealistic
-undergraduates find their unconsciously preserved child's model of
-wealth confirmed by eminent writers of the past.  It is a case of
-the mistaken meeting the outdated."Behind every great fortune, there is a crime," Balzac wrote.  Except
-he didn't.  What he actually said was that a great fortune with no
-apparent cause was probably due to a crime well enough executed
-that it had been forgotten.  If we were talking about Europe in
-1000, or most of the third world today, the standard misquotation
-would be spot on.  But Balzac lived in nineteenth-century France,
-where the Industrial Revolution was well advanced.  He knew you
-could make a fortune without stealing it.  After all, he did himself,
-as a popular novelist.
-[12]Only a few countries (by no coincidence, the richest ones) have
-reached this stage.  In most, corruption still has the upper hand.
-In most, the fastest way to get wealth is by stealing it.  And so
-when we see increasing differences in income in a rich country,
-there is a tendency to worry that it's sliding back toward becoming
-another Venezuela.  I think the opposite is happening. I think
-you're seeing a country a full step ahead of Venezuela.The Lever of TechnologyWill technology increase the gap between rich and poor?  It will
-certainly increase the gap between the productive and the unproductive.
-That's the whole point of technology.   With a tractor an energetic
-farmer could plow six times as much land in a day as he could with
-a team of horses.  But only if he mastered a new kind of farming.I've seen the lever of technology grow visibly in my own time.  In
-high school I made money by mowing lawns and scooping ice cream at
-Baskin-Robbins.  This was the only kind of work available at the
-time.  Now high school kids could write software or design web
-sites.  But only some of them will; the rest will still be scooping
-ice cream.I remember very vividly when in 1985 improved technology made it
-possible for me to buy a computer of my own.  Within months I was
-using it to make money as a freelance programmer.  A few years
-before, I couldn't have done this.  A few years before, there was
-no such thing as a freelance programmer.  But Apple created
-wealth, in the form of powerful, inexpensive computers, and programmers
-immediately set to work using it to create more.As this example suggests, the rate at which technology increases
-our productive capacity is probably exponential, rather than linear.
-So we should expect to see ever-increasing variation in individual
-productivity as time goes on.   Will that increase the gap between
-rich and the poor?  Depends which gap you mean.Technology should increase the gap in income, but it seems to
-decrease other gaps.  A hundred years ago, the rich led a different
-kind of life from ordinary people.  They lived in houses
-full of servants, wore elaborately uncomfortable clothes, and
-travelled about in carriages drawn by teams of horses which themselves
-required their own houses and servants.  Now, thanks to technology,
-the rich live more like the average person.Cars are a good example of why.  It's possible to buy expensive,
-handmade cars that cost hundreds of thousands of dollars.  But there
-is not much point.  Companies make more money by building a large
-number of ordinary cars than a small number of expensive ones.  So
-a company making a mass-produced car can afford to spend a lot more
-on its design.  If you buy a custom-made car, something will always
-be breaking.  The only point of buying one now is to advertise that
-you can.Or consider watches.  Fifty years ago, by spending a lot of money
-on a watch you could get better performance.  When watches had
-mechanical movements, expensive watches kept better time.  Not any
-more.  Since the invention of the quartz movement, an ordinary Timex
-is more accurate than a Patek Philippe costing hundreds of thousands
-of dollars.
-[13]
-Indeed, as with expensive cars, if you're determined
-to spend a lot of money on a watch, you have to put up with some
-inconvenience to do it: as well as keeping worse time, mechanical
-watches have to be wound.The only thing technology can't cheapen is brand.  Which is precisely
-why we hear ever more about it.  Brand is the residue left as the
-substantive differences between rich and poor evaporate.  But what
-label you have on your stuff is a much smaller matter than having
-it versus not having it.  In 1900, if you kept a carriage, no one
-asked what year or brand it was.  If you had one, you were rich.
-And if you weren't rich, you took the omnibus or walked.  Now even
-the poorest Americans drive cars, and it is only because we're so
-well trained by advertising that we can even recognize the especially
-expensive ones.
-[14]The same pattern has played out in industry after industry.  If
-there is enough demand for something, technology will make it cheap
-enough to sell in large volumes, and the mass-produced versions
-will be, if not better, at least more convenient.
-[15]
-And there
-is nothing the rich like more than convenience.  The rich people I
-know drive the same cars, wear the same clothes, have the same kind
-of furniture, and eat the same foods as my other friends.  Their
-houses are in different neighborhoods, or if in the same neighborhood
-are different sizes, but within them life is similar.  The houses
-are made using the same construction techniques and contain much
-the same objects.  It's inconvenient to do something expensive and
-custom.The rich spend their time more like everyone else too.  Bertie
-Wooster seems long gone.  Now, most people who are rich enough not
-to work do anyway.  It's not just social pressure that makes them;
-idleness is lonely and demoralizing.Nor do we have the social distinctions there were a hundred years
-ago.   The novels and etiquette manuals of that period read now
-like descriptions of some strange tribal society.  "With respect
-to the continuance of friendships..." hints Mrs. Beeton's Book
-of Household Management (1880), "it may be found necessary, in
-some cases, for a mistress to relinquish, on assuming the responsibility
-of a household, many of those commenced in the earlier part of her
-life." A woman who married a rich man was expected to drop friends
-who didn't.  You'd seem a barbarian if you behaved that way today.
-You'd also have a very boring life.  People still tend to segregate
-themselves somewhat, but much more on the basis of education than
-wealth.
-[16]Materially and socially, technology seems to be decreasing the gap
-between the rich and the poor, not increasing it.  If Lenin walked
-around the offices of a company like Yahoo or Intel or Cisco, he'd
-think communism had won.  Everyone would be wearing the same clothes,
-have the same kind of office (or rather, cubicle) with the same
-furnishings, and address one another by their first names instead
-of by honorifics.  Everything would seem exactly as he'd predicted,
-until he looked at their bank accounts.  Oops.Is it a problem if technology increases that gap?  It doesn't seem
-to be so far.  As it increases the gap in income, it seems to
-decrease most other gaps.Alternative to an AxiomOne often hears a policy criticized on the grounds that it would
-increase the income gap between rich and poor.  As if it were an
-axiom that this would be bad.  It might be true that increased
-variation in income would be bad, but I don't see how we can say
-it's axiomatic.Indeed, it may even be false, in industrial democracies.  In a
-society of serfs and warlords, certainly, variation in income is a
-sign of an underlying problem.  But serfdom is not the only cause
-of variation in income.  A 747 pilot doesn't make 40 times as much
-as a checkout clerk because he is a warlord who somehow holds her
-in thrall.  His skills are simply much more valuable.I'd like to propose an alternative idea: that in a modern society,
-increasing variation in income is a sign of health.  Technology
-seems to increase the variation in productivity at faster than
-linear rates.  If we don't see corresponding variation in income,
-there are three possible explanations: (a) that technical innovation
-has stopped, (b) that the people who would create the most wealth
-aren't doing it, or (c) that they aren't getting paid for it.I think we can safely say that (a) and (b) would be bad.  If you
-disagree, try living for a year using only the resources available
-to the average Frankish nobleman in 800, and report back to us.
-(I'll be generous and not send you back to the stone age.)The only option, if you're going to have an increasingly prosperous
-society without increasing variation in income, seems to be (c),
-that people will create a lot of wealth without being paid for it.
-That Jobs and Wozniak, for example, will cheerfully work 20-hour
-days to produce the Apple computer for a society that allows them,
-after taxes, to keep just enough of their income to match what they
-would have made working 9 to 5 at a big company.Will people create wealth if they can't get paid for it?  Only if
-it's fun.  People will write operating systems for free.  But they
-won't install them, or take support calls, or train customers to
-use them.  And at least 90% of the work that even the highest tech
-companies do is of this second, unedifying kind.All the unfun kinds of wealth creation slow dramatically in a society
-that confiscates private fortunes.  We can confirm this empirically.
-Suppose you hear a strange noise that you think may be due to a
-nearby fan.  You turn the fan off, and the noise stops.  You turn
-the fan back on, and the noise starts again.  Off, quiet.  On,
-noise.  In the absence of other information, it would seem the noise
-is caused by the fan.At various times and places in history, whether you could accumulate
-a fortune by creating wealth has been turned on and off.  Northern
-Italy in 800, off (warlords would steal it).  Northern Italy in
-1100, on.  Central France in 1100, off (still feudal).  England in
-1800, on.  England in 1974, off (98% tax on investment income).
-United States in 1974, on.  We've even had a twin study: West
-Germany, on;  East Germany, off.  In every case, the creation of
-wealth seems to appear and disappear like the noise of a fan as you
-switch on and off the prospect of keeping it.There is some momentum involved.  It probably takes at least a
-generation to turn people into East Germans (luckily for England).
-But if it were merely a fan we were studying, without all the extra
-baggage that comes from the controversial topic of wealth, no one
-would have any doubt that the fan was causing the noise.If you suppress variations in income, whether by stealing private
-fortunes, as feudal rulers used to do, or by taxing them away, as
-some modern governments have done, the result always seems to be
-the same.    Society as a whole ends up poorer.If I had a choice of living in a society where I was materially
-much better off than I am now, but was among the poorest, or in one
-where I was the richest, but much worse off than I am now, I'd take
-the first option.  If I had children, it would arguably be immoral
-not to.  It's absolute poverty you want to avoid, not relative
-poverty.  If, as the evidence so far implies, you have to have one
-or the other in your society, take relative poverty.You need rich people in your society not so much because in spending
-their money they create jobs, but because of what they have to do
-to get rich.  I'm not talking about the trickle-down effect
-here.  I'm not saying that if you let Henry Ford get rich, he'll
-hire you as a waiter at his next party.  I'm saying that he'll make
-you a tractor to replace your horse.Notes[1]
-Part of the reason this subject is so contentious is that some
-of those most vocal on the subject of wealth—university
-students, heirs, professors, politicians, and journalists—have
-the least experience creating it.  (This phenomenon will be familiar
-to anyone who has overheard conversations about sports in a bar.)Students are mostly still on the parental dole, and have not stopped
-to think about where that money comes from.  Heirs will be on the
-parental dole for life.  Professors and politicians live within
-socialist eddies of the economy, at one remove from the creation
-of wealth, and are paid a flat rate regardless of how hard they
-work.  And journalists as part of their professional code segregate
-themselves from the revenue-collecting half of the businesses they
-work for (the ad sales department).  Many of these people never
-come face to face with the fact that the money they receive represents
-wealth—wealth that, except in the case of journalists, someone
-else created earlier.  They live in a world in which income is
-doled out by a central authority according to some abstract notion
-of fairness (or randomly, in the case of heirs), rather than given
-by other people in return for something they wanted, so it may seem
-to them unfair that things don't work the same in the rest of the
-economy.(Some professors do create a great deal of wealth for
-society.  But the money they're paid isn't a quid pro quo.
-It's more in the nature of an investment.)[2]
-When one reads about the origins of the Fabian Society, it
-sounds like something cooked up by the high-minded Edwardian
-child-heroes of Edith Nesbit's The Wouldbegoods.[3]
-According to a study by the Corporate Library, the median total
-compensation, including salary, bonus, stock grants, and the exercise
-of stock options, of S&P 500 CEOs in 2002 was $3.65 million.
-According to Sports Illustrated, the average NBA player's
-salary during the 2002-03 season was $4.54 million, and the average
-major league baseball player's salary at the start of the 2003
-season was $2.56 million.  According to the Bureau of Labor
-Statistics, the mean annual wage in the US in 2002 was $35,560.[4]
-In the early empire the price of an ordinary adult slave seems
-to have been about 2,000 sestertii (e.g. Horace, Sat. ii.7.43).
-A servant girl cost 600 (Martial vi.66), while Columella (iii.3.8)
-says that a skilled vine-dresser was worth 8,000.  A doctor, P.
-Decimus Eros Merula, paid 50,000 sestertii for his freedom (Dessau,
-Inscriptiones 7812).  Seneca (Ep. xxvii.7) reports
-that one Calvisius Sabinus paid 100,000 sestertii apiece for slaves
-learned in the Greek classics.  Pliny (Hist. Nat. vii.39)
-says that the highest price paid for a slave up to his time was
-700,000 sestertii, for the linguist (and presumably teacher) Daphnis,
-but that this had since been exceeded by actors buying their own
-freedom.Classical Athens saw a similar variation in prices.  An ordinary
-laborer was worth about 125 to 150 drachmae.  Xenophon (Mem.
-ii.5) mentions prices ranging from 50 to 6,000 drachmae (for the
-manager of a silver mine).For more on the economics of ancient slavery see:Jones, A. H. M., "Slavery in the Ancient World," Economic History
-Review, 2:9 (1956), 185-199, reprinted in Finley, M. I. (ed.),
-Slavery in Classical Antiquity, Heffer, 1964.[5]
-Eratosthenes (276—195 BC) used shadow lengths in different
-cities to estimate the Earth's circumference.  He was off by only
-about 2%.[6]
-No, and Windows, respectively.[7]
-One of the biggest divergences between the Daddy Model and
-reality is the valuation of hard work.  In the Daddy Model, hard
-work is in itself deserving.  In reality, wealth is measured by
-what one delivers, not how much effort it costs.  If I paint someone's
-house, the owner shouldn't pay me extra for doing it with a toothbrush.It will seem to someone still implicitly operating on the Daddy
-Model that it is unfair when someone works hard and doesn't get
-paid much.  To help clarify the matter, get rid of everyone else
-and put our worker on a desert island, hunting and gathering fruit.
-If he's bad at it he'll work very hard and not end up with much
-food.  Is this unfair?  Who is being unfair to him?[8]
-Part of the reason for the tenacity of the Daddy Model may be
-the dual meaning of "distribution." When economists talk about
-"distribution of income," they mean statistical distribution.  But
-when you use the phrase frequently, you can't help associating it
-with the other sense of the word (as in e.g. "distribution of alms"),
-and thereby subconsciously seeing wealth as something that flows
-from some central tap.  The word "regressive" as applied to tax
-rates has a similar effect, at least on me; how can anything
-regressive be good?[9]
-"From the beginning of the reign Thomas Lord Roos was an assiduous
-courtier of the young Henry VIII and was soon to reap the rewards.
-In 1525 he was made a Knight of the Garter and given the Earldom
-of Rutland.  In the thirties his support of the breach with Rome,
-his zeal in crushing the Pilgrimage of Grace, and his readiness to
-vote the death-penalty in the succession of spectacular treason
-trials that punctuated Henry's erratic matrimonial progress made
-him an obvious candidate for grants of monastic property."Stone, Lawrence, Family and Fortune: Studies in Aristocratic
-Finance in the Sixteenth and Seventeenth Centuries, Oxford
-University Press, 1973, p. 166.[10]
-There is archaeological evidence for large settlements earlier,
-but it's hard to say what was happening in them.Hodges, Richard and David Whitehouse, Mohammed, Charlemagne and
-the Origins of Europe, Cornell University Press, 1983.[11]
-William Cecil and his son Robert were each in turn the most
-powerful minister of the crown, and both used their position to
-amass fortunes among the largest of their times.  Robert in particular
-took bribery to the point of treason.  "As Secretary of State and
-the leading advisor to King James on foreign policy, [he] was a
-special recipient of favour, being offered large bribes by the Dutch
-not to make peace with Spain, and large bribes by Spain to make
-peace." (Stone, op. cit., p. 17.)[12]
-Though Balzac made a lot of money from writing, he was notoriously
-improvident and was troubled by debts all his life.[13]
-A Timex will gain or lose about .5 seconds per day.  The most
-accurate mechanical watch, the Patek Philippe 10 Day Tourbillon,
-is rated at -1.5 to +2 seconds.  Its retail price is about $220,000.[14]
-If asked to choose which was more expensive, a well-preserved
-1989 Lincoln Town Car ten-passenger limousine ($5,000) or a 2004
-Mercedes S600 sedan ($122,000), the average Edwardian might well
-guess wrong.[15]
-To say anything meaningful about income trends, you have to
-talk about real income, or income as measured in what it can buy.
-But the usual way of calculating real income ignores much of the
-growth in wealth over time, because it depends on a consumer price
-index created by bolting end to end a series of numbers that are
-only locally accurate, and that don't include the prices of new
-inventions until they become so common that their prices stabilize.So while we might think it was very much better to live in a world
-with antibiotics or air travel or an electric power grid than
-without, real income statistics calculated in the usual way will
-prove to us that we are only slightly richer for having these things.Another approach would be to ask, if you were going back to the
-year x in a time machine, how much would you have to spend on trade
-goods to make your fortune?  For example, if you were going back
-to 1970 it would certainly be less than $500, because the processing
-power you can get for $500 today would have been worth at least
-$150 million in 1970.  The function goes asymptotic fairly quickly,
-because for times over a hundred years or so you could get all you
-needed in present-day trash.  In 1800 an empty plastic drink bottle
-with a screw top would have seemed a miracle of workmanship.[16]
-Some will say this amounts to the same thing, because the rich
-have better opportunities for education.  That's a valid point.  It
-is still possible, to a degree, to buy your kids' way into top
-colleges by sending them to private schools that in effect hack the
-college admissions process.According to a 2002 report by the National Center for Education
-Statistics, about 1.7% of American kids attend private, non-sectarian
-schools.  At Princeton, 36% of the class of 2007 came from such
-schools.  (Interestingly, the number at Harvard is significantly
-lower, about 28%.)  Obviously this is a huge loophole.  It does at
-least seem to be closing, not widening.Perhaps the designers of admissions processes should take a lesson
-from the example of computer security, and instead of just assuming
-that their system can't be hacked, measure the degree to which it
-is.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/gba.txt

@@ -1,198 +0,0 @@
-April 2004To the popular press, "hacker" means someone who breaks
-into computers.  Among programmers it means a good programmer.
-But the two meanings are connected.  To programmers,
-"hacker" connotes mastery in the most literal sense: someone
-who can make a computer do what he wants—whether the computer
-wants to or not.To add to the confusion, the noun "hack" also has two senses.  It can
-be either a compliment or an insult.  It's called a hack when
-you do something in an ugly way.  But when you do something
-so clever that you somehow beat the system, that's also
-called a hack.  The word is used more often in the former than
-the latter sense, probably because ugly solutions are more
-common than brilliant ones.Believe it or not, the two senses of "hack" are also
-connected.  Ugly and imaginative solutions have something in
-common: they both break the rules.  And there is a gradual
-continuum between rule breaking that's merely ugly (using
-duct tape to attach something to your bike) and rule breaking
-that is brilliantly imaginative (discarding Euclidean space).Hacking predates computers.  When he
-was working on the Manhattan Project, Richard Feynman used to
-amuse himself by breaking into safes containing secret documents.
-This tradition continues today.
-When we were in grad school, a hacker friend of mine who spent too much
-time around MIT had
-his own lock picking kit.
-(He now runs a hedge fund, a not unrelated enterprise.)It is sometimes hard to explain to authorities why one would
-want to do such things.
-Another friend of mine once got in trouble with the government for
-breaking into computers.  This had only recently been declared
-a crime, and the FBI found that their usual investigative
-technique didn't work.  Police investigation apparently begins with
-a motive.  The usual motives are few: drugs, money, sex,
-revenge.  Intellectual curiosity was not one of the motives on
-the FBI's list.  Indeed, the whole concept seemed foreign to
-them.Those in authority tend to be annoyed by hackers'
-general attitude of disobedience.  But that disobedience is
-a byproduct of the qualities that make them good programmers.
-They may laugh at the CEO when he talks in generic corporate
-newspeech, but they also laugh at someone who tells them
-a certain problem can't be solved.
-Suppress one, and you suppress the other.This attitude is sometimes affected.  Sometimes young programmers
-notice the eccentricities of eminent hackers and decide to
-adopt some of their own in order to seem smarter.
-The fake version is not merely
-annoying; the prickly attitude of these posers
-can actually slow the process of innovation.But even factoring in their annoying eccentricities,
-the disobedient attitude of hackers is a net win.  I wish its
-advantages were better understood.For example, I suspect people in Hollywood are
-simply mystified by
-hackers' attitudes toward copyrights.  They are a perennial
-topic of heated discussion on Slashdot.
-But why should people who program computers
-be so concerned about copyrights, of all things?Partly because some companies use mechanisms to prevent
-copying.  Show any hacker a lock and his first thought is
-how to pick it.  But there is a deeper reason that
-hackers are alarmed by measures like copyrights and patents.
-They see increasingly aggressive measures to protect
-"intellectual property"
-as a threat to the intellectual
-freedom they need to do their job.
-And they are right.It is by poking about inside current technology that
-hackers get ideas for the next generation.  No thanks,
-intellectual homeowners may say, we don't need any
-outside help.  But they're wrong.
-The next generation of computer technology has
-often—perhaps more often than not—been developed by outsiders.In 1977 there was no doubt some group within IBM developing
-what they expected to be
-the next generation of business computer.  They were mistaken.
-The next generation of business computer was
-being developed on entirely different lines by two long-haired
-guys called Steve in a garage in Los Altos.  At about the
-same time, the powers that be
-were cooperating to develop the
-official next generation operating system, Multics.
-But two guys who thought Multics excessively complex went off
-and wrote their own.  They gave it a name that
-was a joking reference to Multics: Unix.The latest intellectual property laws impose
-unprecedented restrictions on the sort of poking around that
-leads to new ideas. In the past, a competitor might use patents
-to prevent you from selling a copy of something they
-made, but they couldn't prevent you from
-taking one apart to see how it worked.   The latest
-laws make this a crime.  How are we
-to develop new technology if we can't study current
-technology to figure out how to improve it?Ironically, hackers have brought this on themselves.
-Computers are responsible for the problem.  The control systems
-inside machines used to be physical: gears and levers and cams.
-Increasingly, the brains (and thus the value) of products is
-in software. And by this I mean software in the general sense:
-i.e. data.  A song on an LP is physically stamped into the
-plastic.  A song on an iPod's disk is merely stored on it.Data is by definition easy to copy.  And the Internet
-makes copies easy to distribute.  So it is no wonder
-companies are afraid.  But, as so often happens, fear has
-clouded their judgement.  The government has responded
-with draconian laws to protect intellectual property.
-They probably mean well. But
-they may not realize that such laws will do more harm
-than good.Why are programmers so violently opposed to these laws?
-If I were a legislator, I'd be interested in this
-mystery—for the same reason that, if I were a farmer and suddenly
-heard a lot of squawking coming from my hen house one night,
-I'd want to go out and investigate.  Hackers are not stupid,
-and unanimity is very rare in this world.
-So if they're all squawking,   
-perhaps there is something amiss.Could it be that such laws, though intended to protect America,
-will actually harm it?  Think about it.  There is something
-very American about Feynman breaking into safes during
-the Manhattan Project.  It's hard to imagine the authorities
-having a sense of humor about such things over
-in Germany at that time.  Maybe it's not a coincidence.Hackers are unruly.  That is the essence of hacking.  And it
-is also the essence of Americanness.  It is no accident
-that Silicon Valley
-is in America, and not France, or Germany,
-or England, or Japan. In those countries, people color inside
-the lines.I lived for a while in Florence.  But after I'd been there
-a few months I realized that what I'd been unconsciously hoping
-to find there was back in the place I'd just left.
-The reason Florence is famous is that in 1450, it was New York.
-In 1450 it was filled with the kind of turbulent and ambitious
-people you find now in America.  (So I went back to America.)It is greatly to America's advantage that it is
-a congenial atmosphere for the right sort of unruliness—that
-it is a home not just for the smart, but for smart-alecks.
-And hackers are invariably smart-alecks.  If we had a national
-holiday, it would be April 1st.  It says a great deal about
-our work that we use the same word for a brilliant or a
-horribly cheesy solution.   When we cook one up we're not
-always 100% sure which kind it is.  But as long as it has
-the right sort of wrongness, that's a promising sign.
-It's odd that people
-think of programming as precise and methodical.  Computers
-are precise and methodical.  Hacking is something you do
-with a gleeful laugh.In our world some of the most characteristic solutions
-are not far removed from practical
-jokes.  IBM was no doubt rather surprised by the consequences
-of the licensing deal for DOS, just as the hypothetical
-"adversary" must be when Michael Rabin solves a problem by
-redefining it as one that's easier to solve.Smart-alecks have to develop a keen sense of how much they
-can get away with.  And lately hackers 
-have sensed a change
-in the atmosphere.
-Lately hackerliness seems rather frowned upon.To hackers the recent contraction in civil liberties seems
-especially ominous.  That must also mystify outsiders. 
-Why should we care especially about civil
-liberties?  Why programmers, more than
-dentists or salesmen or landscapers?Let me put the case in terms a government official would appreciate.
-Civil liberties are not just an ornament, or a quaint
-American tradition.  Civil liberties make countries rich.
-If you made a graph of
-GNP per capita vs. civil liberties, you'd notice a definite
-trend.  Could civil liberties really be a cause, rather
-than just an effect?  I think so.  I think a society in which
-people can do and say what they want will also tend to
-be one in which the most efficient solutions win, rather than
-those sponsored by the most influential people.
-Authoritarian countries become corrupt;
-corrupt countries become poor; and poor countries are weak. 
-It seems to me there is
-a Laffer curve for government power, just as for
-tax revenues.  At least, it seems likely enough that it
-would be stupid to try the experiment and find out.  Unlike
-high tax rates, you can't repeal totalitarianism if it
-turns out to be a mistake.This is why hackers worry.  The government spying on people doesn't
-literally make programmers write worse code.  It just leads
-eventually to a world in which bad ideas win.  And because
-this is so important to hackers, they're especially sensitive
-to it.  They can sense totalitarianism approaching from a
-distance, as animals can sense an approaching  
-thunderstorm.It would be ironic if, as hackers fear, recent measures
-intended to protect national security and intellectual property
-turned out to be a missile aimed right at what makes   
-America successful.  But it would not be the first time that
-measures taken in an atmosphere of panic had
-the opposite of the intended effect.There is such a thing as Americanness.
-There's nothing like living abroad to teach you that.   
-And if you want to know whether something will nurture or squash
-this quality, it would be hard to find a better focus
-group than hackers, because they come closest of any group
-I know to embodying it.  Closer, probably,  than
-the men running our government,
-who for all their talk of patriotism
-remind me more of Richelieu or Mazarin
-than Thomas Jefferson or George Washington.When you read what the founding fathers had to say for
-themselves, they sound more like hackers.
-"The spirit of resistance to government,"
-Jefferson wrote, "is so valuable on certain occasions, that I wish
-it always to be kept alive."Imagine an American president saying that today.
-Like the remarks of an outspoken old grandmother, the sayings of
-the founding fathers have embarrassed generations of
-their less confident successors.  They remind us where we come from.
-They remind us that it is the people who break rules that are
-the source of America's wealth and power.Those in a position to impose rules naturally want them to be
-obeyed.  But be careful what you ask for. You might get it.Thanks to Ken Anderson, Trevor Blackwell, Daniel Giffin, 
-Sarah Harlin,  Shiro Kawai, Jessica Livingston, Matz, 
-Jackie McDonough, Robert Morris, Eric Raymond, Guido van Rossum,
-David Weinberger, and
-Steven Wolfram for reading drafts of this essay.
-(The image shows Steves Jobs and Wozniak 
-with a "blue box."
-Photo by Margret Wozniak. Reproduced by permission of Steve
-Wozniak.)

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/gh.txt

@@ -1,434 +0,0 @@
-
-
-Want to start a startup?  Get funded by
-Y Combinator.
-
-
-
-
-July 2004(This essay is derived from a talk at Oscon 2004.)
-A few months ago I finished a new 
-book, 
-and in reviews I keep
-noticing words like "provocative'' and "controversial.'' To say
-nothing of "idiotic.''I didn't mean to make the book controversial.  I was trying to make
-it efficient.  I didn't want to waste people's time telling them
-things they already knew.  It's more efficient just to give them
-the diffs.  But I suppose that's bound to yield an alarming book.EdisonsThere's no controversy about which idea is most controversial:
-the suggestion that variation in wealth might not be as big a
-problem as we think.I didn't say in the book that variation in wealth was in itself a
-good thing.  I said in some situations it might be a sign of good
-things.  A throbbing headache is not a good thing, but it can be
-a sign of a good thing-- for example, that you're recovering
-consciousness after being hit on the head.Variation in wealth can be a sign of variation in productivity.
-(In a society of one, they're identical.) And that
-is almost certainly a good thing: if your society has no variation
-in productivity, it's probably not because everyone is Thomas
-Edison.  It's probably because you have no Thomas Edisons.In a low-tech society you don't see much variation in productivity.
-If you have a tribe of nomads collecting sticks for a fire, how
-much more productive is the best stick gatherer going to be than
-the worst?  A factor of two?  Whereas when you hand people a complex tool
-like a computer, the variation in what they can do with
-it is enormous.That's not a new idea.  Fred Brooks wrote about it in 1974, and
-the study he quoted was published in 1968.  But I think he
-underestimated the variation between programmers.  He wrote about productivity in lines
-of code:  the best programmers can solve a given problem in a tenth
-the time.  But what if the problem isn't given? In programming, as
-in many fields, the hard part isn't solving problems, but deciding
-what problems to solve.  Imagination is hard to measure, but
-in practice it dominates the kind of productivity that's measured
-in lines of code.Productivity varies in any field, but there are few in which it
-varies so much.  The variation between programmers
-is so great that it becomes a difference in kind.  I don't
-think this is something intrinsic to programming, though.  In every field,
-technology magnifies differences in productivity.  I think what's
-happening in programming is just that we have a lot of technological
-leverage.  But in every field the lever is getting longer, so the
-variation we see is something that more and more fields will see
-as time goes on.  And the success of companies, and countries, will
-depend increasingly on how they deal with it.If variation in productivity increases with technology, then the
-contribution of the most productive individuals will not only be
-disproportionately large, but will actually grow with time.  When
-you reach the point where 90% of a group's output is created by 1%
-of its members, you lose big if something (whether Viking raids,
-or central planning) drags their productivity down to the average.If we want to get the most out of them, we need to understand these
-especially productive people.  What motivates them?  What do they
-need to do their jobs?  How do you recognize them? How do you
-get them to come and work for you?  And then of course there's the
-question, how do you become one?More than MoneyI know a handful of super-hackers, so I sat down and thought about
-what they have in common.  Their defining quality is probably that
-they really love to program.  Ordinary programmers write code to pay
-the bills.  Great hackers think of it as something they do for fun,
-and which they're delighted to find people will pay them for.Great programmers are sometimes said to be indifferent to money.
-This isn't quite true.  It is true that all they really care about
-is doing interesting work.  But if you make enough money, you get
-to work on whatever you want, and for that reason hackers are
-attracted by the idea of making really large amounts of money.
-But as long as they still have to show up for work every day, they
-care more about what they do there than how much they get paid for
-it.Economically, this is a fact of the greatest importance, because
-it means you don't have to pay great hackers anything like what
-they're worth.  A great programmer might be ten or a hundred times
-as productive as an ordinary one, but he'll consider himself lucky
-to get paid three times as much.  As I'll explain later, this is
-partly because great hackers don't know how good they are.  But
-it's also because money is not the main thing they want.What do hackers want?  Like all craftsmen, hackers like good tools.
-In fact, that's an understatement.  Good hackers find it unbearable
-to use bad tools.  They'll simply refuse to work on projects with
-the wrong infrastructure.At a startup I once worked for, one of the things pinned up on our
-bulletin board was an ad from IBM.  It was a picture of an AS400,
-and the headline read, I think, "hackers despise
-it.'' [1]When you decide what infrastructure to use for a project, you're
-not just making a technical decision.  You're also making a social
-decision, and this may be the more important of the two.  For
-example, if your company wants to write some software, it might
-seem a prudent choice to write it in Java.  But when you choose a
-language, you're also choosing a community.  The programmers you'll
-be able to hire to work on a Java project won't be as
-smart as the
-ones you could get to work on a project written in Python.
-And the quality of your hackers probably matters more than the
-language you choose.  Though, frankly, the fact that good hackers
-prefer Python to Java should tell you something about the relative
-merits of those languages.Business types prefer the most popular languages because they view
-languages as standards. They don't want to bet the company on
-Betamax.  The thing about languages, though, is that they're not
-just standards.  If you have to move bits over a network, by all
-means use TCP/IP.  But a programming language isn't just a format.
-A programming language is a medium of expression.I've read that Java has just overtaken Cobol as the most popular
-language.  As a standard, you couldn't wish for more.  But as a
-medium of expression, you could do a lot better.  Of all the great
-programmers I can think of, I know of only one who would voluntarily
-program in Java.  And of all the great programmers I can think of
-who don't work for Sun, on Java, I know of zero.Great hackers also generally insist on using open source software.
-Not just because it's better, but because it gives them more control.
-Good hackers insist on control.  This is part of what makes them
-good hackers:  when something's broken, they need to fix it.  You
-want them to feel this way about the software they're writing for
-you.  You shouldn't be surprised when they feel the same way about
-the operating system.A couple years ago a venture capitalist friend told me about a new
-startup he was involved with.  It sounded promising.  But the next
-time I talked to him, he said they'd decided to build their software
-on Windows NT, and had just hired a very experienced NT developer
-to be their chief technical officer.  When I heard this, I thought,
-these guys are doomed.  One, the CTO couldn't be a first rate
-hacker, because to become an eminent NT developer he would have
-had to use NT voluntarily, multiple times, and I couldn't imagine
-a great hacker doing that; and two, even if he was good, he'd have
-a hard time hiring anyone good to work for him if the project had
-to be built on NT. [2]The Final FrontierAfter software, the most important tool to a hacker is probably
-his office.  Big companies think the function of office space is to express
-rank.  But hackers use their offices for more than that: they
-use their office as a place to think in.  And if you're a technology
-company, their thoughts are your product.  So making hackers work
-in a noisy, distracting environment is like having a paint factory
-where the air is full of soot.The cartoon strip Dilbert has a lot to say about cubicles, and with
-good reason.  All the hackers I know despise them.  The mere prospect
-of being interrupted is enough to prevent hackers from working on
-hard problems.  If you want to get real work done in an office with
-cubicles, you have two options: work at home, or come in early or
-late or on a weekend, when no one else is there.  Don't companies
-realize this is a sign that something is broken?  An office
-environment is supposed to be something that helps
-you work, not something you work despite.Companies like Cisco are proud that everyone there has a cubicle,
-even the CEO.  But they're not so advanced as they think; obviously
-they still view office space as a badge of rank.  Note too that
-Cisco is famous for doing very little product development in house.
-They get new technology by buying the startups that created it-- where
-presumably the hackers did have somewhere quiet to work.One big company that understands what hackers need is Microsoft.
-I once saw a recruiting ad for Microsoft with a big picture of a
-door.  Work for us, the premise was, and we'll give you a place to
-work where you can actually get work done.   And you know, Microsoft
-is remarkable among big companies in that they are able to develop
-software in house.  Not well, perhaps, but well enough.If companies want hackers to be productive, they should look at
-what they do at home.  At home, hackers can arrange things themselves
-so they can get the most done.  And when they work at home, hackers
-don't work in noisy, open spaces; they work in rooms with doors.  They
-work in cosy, neighborhoody places with people around and somewhere
-to walk when they need to mull something over, instead of in glass
-boxes set in acres of parking lots.  They have a sofa they can take
-a nap on when they feel tired, instead of sitting in a coma at
-their desk, pretending to work.  There's no crew of people with
-vacuum cleaners that roars through every evening during the prime
-hacking hours.  There are no meetings or, God forbid, corporate
-retreats or team-building exercises.  And when you look at what
-they're doing on that computer, you'll find it reinforces what I
-said earlier about tools.  They may have to use Java and Windows
-at work, but at home, where they can choose for themselves, you're
-more likely to find them using Perl and Linux.Indeed, these statistics about Cobol or Java being the most popular
-language can be misleading.  What we ought to look at, if we want
-to know what tools are best, is what hackers choose when they can
-choose freely-- that is, in projects of their own.  When you ask
-that question, you find that open source operating systems already
-have a dominant market share, and the number one language is probably
-Perl.InterestingAlong with good tools, hackers want interesting projects.  What
-makes a project interesting?  Well, obviously overtly sexy
-applications like stealth planes or special effects software would
-be interesting to work on.  But any application can be interesting
-if it poses novel technical challenges.  So it's hard to predict
-which problems hackers will like, because some become
-interesting only when the people working on them discover a new
-kind of solution.  Before ITA
-(who wrote the software inside Orbitz),
-the people working on airline fare searches probably thought it
-was one of the most boring applications imaginable.  But ITA made
-it interesting by 
-redefining the problem in a more ambitious way.I think the same thing happened at Google.  When Google was founded,
-the conventional wisdom among the so-called portals was that search
-was boring and unimportant.  But the guys at Google didn't think
-search was boring, and that's why they do it so well.This is an area where managers can make a difference.  Like a parent
-saying to a child, I bet you can't clean up your whole room in
-ten minutes, a good manager can sometimes redefine a problem as a
-more interesting one.  Steve Jobs seems to be particularly good at
-this, in part simply by having high standards.  There were a lot
-of small, inexpensive computers before the Mac.  He redefined the
-problem as: make one that's beautiful.  And that probably drove
-the developers harder than any carrot or stick could.They certainly delivered.  When the Mac first appeared, you didn't
-even have to turn it on to know it would be good; you could tell
-from the case.  A few weeks ago I was walking along the street in
-Cambridge, and in someone's trash I saw what appeared to be a Mac
-carrying case.  I looked inside, and there was a Mac SE.  I carried
-it home and plugged it in, and it booted.  The happy Macintosh
-face, and then the finder.  My God, it was so simple.  It was just
-like ... Google.Hackers like to work for people with high standards.  But it's not
-enough just to be exacting.  You have to insist on the right things.
-Which usually means that you have to be a hacker yourself.  I've
-seen occasional articles about how to manage programmers.  Really
-there should be two articles: one about what to do if
-you are yourself a programmer, and one about what to do if you're not.  And the 
-second could probably be condensed into two words:  give up.The problem is not so much the day to day management.  Really good
-hackers are practically self-managing.  The problem is, if you're
-not a hacker, you can't tell who the good hackers are.  A similar
-problem explains why American cars are so ugly.  I call it the
-design paradox.  You might think that you could make your products
-beautiful just by hiring a great designer to design them.  But if
-you yourself don't have good taste, 
-how are you going to recognize
-a good designer?  By definition you can't tell from his portfolio.
-And you can't go by the awards he's won or the jobs he's had,
-because in design, as in most fields, those tend to be driven by
-fashion and schmoozing, with actual ability a distant third.
-There's no way around it:  you can't manage a process intended to
-produce beautiful things without knowing what beautiful is.  American
-cars are ugly because American car companies are run by people with
-bad taste.Many people in this country think of taste as something elusive,
-or even frivolous.  It is neither.  To drive design, a manager must
-be the most demanding user of a company's products.  And if you
-have really good taste, you can, as Steve Jobs does, make satisfying
-you the kind of problem that good people like to work on.Nasty Little ProblemsIt's pretty easy to say what kinds of problems are not interesting:
-those where instead of solving a few big, clear, problems, you have
-to solve a lot of nasty little ones.  One of the worst kinds of
-projects is writing an interface to a piece of software that's
-full of bugs.  Another is when you have to customize
-something for an individual client's complex and ill-defined needs.
-To hackers these kinds of projects are the death of a thousand
-cuts.The distinguishing feature of nasty little problems is that you
-don't learn anything from them.   Writing a compiler is interesting
-because it teaches you what a compiler is.  But writing an interface
-to a buggy piece of software doesn't teach you anything, because the
-bugs are random.  [3] So it's not just fastidiousness that makes good
-hackers avoid nasty little problems.  It's more a question of
-self-preservation.  Working on nasty little problems makes you
-stupid.  Good hackers avoid it for the same reason models avoid
-cheeseburgers.Of course some problems inherently have this character.  And because
-of supply and demand, they pay especially well.  So a company that
-found a way to get great hackers to work on tedious problems would
-be very successful.  How would you do it?One place this happens is in startups.  At our startup we had 
-Robert Morris working as a system administrator.  That's like having the
-Rolling Stones play at a bar mitzvah.  You can't hire that kind of
-talent.  But people will do any amount of drudgery for companies
-of which they're the founders.  [4]Bigger companies solve the problem by partitioning the company.
-They get smart people to work for them by establishing a separate
-R&D department where employees don't have to work directly on
-customers' nasty little problems. [5] In this model, the research
-department functions like a mine. They produce new ideas; maybe
-the rest of the company will be able to use them.You may not have to go to this extreme.  
-Bottom-up programming
-suggests another way to partition the company: have the smart people
-work as toolmakers.  If your company makes software to do x, have
-one group that builds tools for writing software of that type, and
-another that uses these tools to write the applications.  This way
-you might be able to get smart people to write 99% of your code,
-but still keep them almost as insulated from users as they would
-be in a traditional research department.  The toolmakers would have
-users, but they'd only be the company's own developers.  [6]If Microsoft used this approach, their software wouldn't be so full
-of security holes, because the less smart people writing the actual
-applications wouldn't be doing low-level stuff like allocating
-memory.  Instead of writing Word directly in C, they'd be plugging
-together big Lego blocks of Word-language.  (Duplo, I believe, is
-the technical term.)ClumpingAlong with interesting problems, what good hackers like is other
-good hackers.  Great hackers tend to clump together-- sometimes
-spectacularly so, as at Xerox Parc.   So you won't attract good
-hackers in linear proportion to how good an environment you create
-for them.  The tendency to clump means it's more like the square
-of the environment.  So it's winner take all.  At any given time,
-there are only about ten or twenty places where hackers most want to
-work, and if you aren't one of them, you won't just have fewer
-great hackers, you'll have zero.Having great hackers is not, by itself, enough to make a company
-successful.  It works well for Google and ITA, which are two of
-the hot spots right now, but it didn't help Thinking Machines or
-Xerox.  Sun had a good run for a while, but their business model
-is a down elevator.  In that situation, even the best hackers can't
-save you.I think, though, that all other things being equal, a company that
-can attract great hackers will have a huge advantage.  There are
-people who would disagree with this.  When we were making the rounds
-of venture capital firms in the 1990s, several told us that software
-companies didn't win by writing great software, but through brand,
-and dominating channels, and doing the right deals.They really seemed to believe this, and I think I know why.  I
-think what a lot of VCs are looking for, at least unconsciously,
-is the next Microsoft.  And of course if Microsoft is your model,
-you shouldn't be looking for companies that hope to win by writing
-great software.  But VCs are mistaken to look for the next Microsoft,
-because no startup can be the next Microsoft unless some other
-company is prepared to bend over at just the right moment and be
-the next IBM.It's a mistake to use Microsoft as a model, because their whole
-culture derives from that one lucky break.  Microsoft is a bad data
-point.  If you throw them out, you find that good products do tend
-to win in the market.  What VCs should be looking for is the next
-Apple, or the next Google.I think Bill Gates knows this.  What worries him about Google is
-not the power of their brand, but the fact that they have
-better hackers. [7]
-RecognitionSo who are the great hackers?  How do you know when you meet one?
-That turns out to be very hard.  Even hackers can't tell.  I'm
-pretty sure now that my friend Trevor Blackwell is a great hacker.
-You may have read on Slashdot how he made his 
-own Segway.  The
-remarkable thing about this project was that he wrote all the
-software in one day (in Python, incidentally).For Trevor, that's
-par for the course.  But when I first met him, I thought he was a
-complete idiot.  He was standing in Robert Morris's office babbling
-at him about something or other, and I remember standing behind
-him making frantic gestures at Robert to shoo this nut out of his
-office so we could go to lunch.  Robert says he misjudged Trevor
-at first too.  Apparently when Robert first met him, Trevor had
-just begun a new scheme that involved writing down everything about
-every aspect of his life on a stack of index cards, which he carried
-with him everywhere.  He'd also just arrived from Canada, and had
-a strong Canadian accent and a mullet.The problem is compounded by the fact that hackers, despite their
-reputation for social obliviousness, sometimes put a good deal of
-effort into seeming smart.  When I was in grad school I used to
-hang around the MIT AI Lab occasionally. It was kind of intimidating
-at first.  Everyone there spoke so fast.  But after a while I
-learned the trick of speaking fast.  You don't have to think any
-faster; just use twice as many words to say everything.  With this amount of noise in the signal, it's hard to tell good
-hackers when you meet them.  I can't tell, even now.  You also
-can't tell from their resumes.  It seems like the only way to judge
-a hacker is to work with him on something.And this is the reason that high-tech areas 
-only happen around universities.  The active ingredient
-here is not so much the professors as the students.  Startups grow up
-around universities because universities bring together promising young
-people and make them work on the same projects.  The
-smart ones learn who the other smart ones are, and together
-they cook up new projects of their own.Because you can't tell a great hacker except by working with him,
-hackers themselves can't tell how good they are.  This is true to
-a degree in most fields.  I've found that people who
-are great at something are not so much convinced of their own
-greatness as mystified at why everyone else seems so incompetent.
-But it's particularly hard for hackers to know how good they are,
-because it's hard to compare their work.  This is easier in most
-other fields.  In the hundred meters, you know in 10 seconds who's
-fastest.  Even in math there seems to be a general consensus about
-which problems are hard to solve, and what constitutes a good
-solution.  But hacking is like writing.  Who can say which of two
-novels is better?  Certainly not the authors.With hackers, at least, other hackers can tell.  That's because,
-unlike novelists, hackers collaborate on projects.  When you get
-to hit a few difficult problems over the net at someone, you learn
-pretty quickly how hard they hit them back.  But hackers can't
-watch themselves at work.  So if you ask a great hacker how good
-he is, he's almost certain to reply, I don't know.  He's not just
-being modest.  He really doesn't know.And none of us know, except about people we've actually worked
-with.  Which puts us in a weird situation: we don't know who our
-heroes should be.  The hackers who become famous tend to become
-famous by random accidents of PR.  Occasionally I need to give an
-example of a great hacker, and I never know who to use.  The first
-names that come to mind always tend to be people I know personally,
-but it seems lame to use them.  So, I think, maybe I should say
-Richard Stallman, or Linus Torvalds, or Alan Kay, or someone famous
-like that.  But I have no idea if these guys are great hackers.
-I've never worked with them on anything.If there is a Michael Jordan of hacking, no one knows, including
-him.CultivationFinally, the question the hackers have all been wondering about:
-how do you become a great hacker?  I don't know if it's possible
-to make yourself into one.  But it's certainly possible to do things
-that make you stupid, and if you can make yourself stupid, you
-can probably make yourself smart too.The key to being a good hacker may be to work on what you like.
-When I think about the great hackers I know, one thing they have
-in common is the extreme 
-difficulty of making them work 
-on anything they
-don't want to.  I don't know if this is cause or effect; it may be
-both.To do something well you have to love it.  
-So to the extent you
-can preserve hacking as something you love, you're likely to do it
-well.  Try to keep the sense of wonder you had about programming at
-age 14.  If you're worried that your current job is rotting your
-brain, it probably is.The best hackers tend to be smart, of course, but that's true in
-a lot of fields.  Is there some quality that's unique to hackers?
-I asked some friends, and the number one thing they mentioned was
-curiosity.  
-I'd always supposed that all smart people were curious--
-that curiosity was simply the first derivative of knowledge.  But
-apparently hackers are particularly curious, especially about how
-things work.  That makes sense, because programs are in effect
-giant descriptions of how things work.Several friends mentioned hackers' ability to concentrate-- their
-ability, as one put it, to "tune out everything outside their own
-heads.''  I've certainly noticed this.  And I've heard several 
-hackers say that after drinking even half a beer they can't program at
-all.   So maybe hacking does require some special ability to focus.
-Perhaps great hackers can load a large amount of context into their
-head, so that when they look at a line of code, they see not just
-that line but the whole program around it.  John McPhee
-wrote that Bill Bradley's success as a basketball player was due
-partly to his extraordinary peripheral vision.  "Perfect'' eyesight
-means about 47 degrees of vertical peripheral vision.  Bill Bradley
-had 70; he could see the basket when he was looking at the floor.
-Maybe great hackers have some similar inborn ability.  (I cheat by
-using a very dense language, 
-which shrinks the court.)This could explain the disconnect over cubicles.  Maybe the people
-in charge of facilities, not having any concentration to shatter,
-have no idea that working in a cubicle feels to a hacker like having
-one's brain in a blender.  (Whereas Bill, if the rumors of autism
-are true, knows all too well.)One difference I've noticed between great hackers and smart people
-in general is that hackers are more 
-politically incorrect.  To the
-extent there is a secret handshake among good hackers, it's when they
-know one another well enough to express opinions that would get
-them stoned to death by the general public.  And I can see why
-political incorrectness would be a useful quality in programming.
-Programs are very complex and, at least in the hands of good
-programmers, very fluid.  In such situations it's helpful to have
-a habit of questioning assumptions.Can you cultivate these qualities?  I don't know.  But you can at
-least not repress them.  So here is my best shot at a recipe.  If
-it is possible to make yourself into a great hacker, the way to do
-it may be to make the following deal with yourself: you never have
-to work on boring projects (unless your family will starve otherwise),
-and in return, you'll never allow yourself to do a half-assed job.
-All the great hackers I know seem to have made that deal, though
-perhaps none of them had any choice in the matter.Notes
-[1] In fairness, I have to say that IBM makes decent hardware.  I
-wrote this on an IBM laptop.[2] They did turn out to be doomed.  They shut down a few months
-later.[3] I think this is what people mean when they talk
-about the "meaning of life."  On the face of it, this seems an 
-odd idea.  Life isn't an expression; how could it have meaning?
-But it can have a quality that feels a lot like meaning.  In a project
-like a compiler, you have to solve a lot of problems, but the problems
-all fall into a pattern, as in a signal.  Whereas when the problems
-you have to solve are random, they seem like noise.
-[4] Einstein at one point worked designing refrigerators. (He had equity.)[5] It's hard to say exactly what constitutes research in the
-computer world, but as a first approximation, it's software that
-doesn't have users.I don't think it's publication that makes the best hackers want to work
-in research departments.  I think it's mainly not having to have a
-three hour meeting with a product manager about problems integrating
-the Korean version of Word 13.27 with the talking paperclip.[6] Something similar has been happening for a long time in the
-construction industry. When you had a house built a couple hundred
-years ago, the local builders built everything in it.  But increasingly
-what builders do is assemble components designed and manufactured
-by someone else.  This has, like the arrival of desktop publishing,
-given people the freedom to experiment in disastrous ways, but it
-is certainly more efficient.[7] Google is much more dangerous to Microsoft than Netscape was.
-Probably more dangerous than any other company has ever been.  Not
-least because they're determined to fight.  On their job listing
-page, they say that one of their "core values'' is "Don't be evil.''
-From a company selling soybean oil or mining equipment, such a
-statement would merely be eccentric.  But I think all of us in the
-computer world recognize who that is a declaration of war on.Thanks to Jessica Livingston, Robert Morris, and Sarah Harlin
-for reading earlier versions of this talk.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/goodtaste.txt

@@ -1,86 +0,0 @@
-November 2021(This essay is derived from a talk at the Cambridge Union.)When I was a kid, I'd have said there wasn't. My father told me so.
-Some people like some things, and other people like other things,
-and who's to say who's right?It seemed so obvious that there was no such thing as good taste
-that it was only through indirect evidence that I realized my father
-was wrong. And that's what I'm going to give you here: a proof by
-reductio ad absurdum. If we start from the premise that there's no
-such thing as good taste, we end up with conclusions that are
-obviously false, and therefore the premise must be wrong.We'd better start by saying what good taste is. There's a narrow
-sense in which it refers to aesthetic judgements and a broader one
-in which it refers to preferences of any kind. The strongest proof
-would be to show that taste exists in the narrowest sense, so I'm
-going to talk about taste in art. You have better taste than me if
-the art you like is better than the art I like.If there's no such thing as good taste, then there's no such thing
-as good art. Because if there is such a
-thing as good art, it's
-easy to tell which of two people has better taste. Show them a lot
-of works by artists they've never seen before and ask them to
-choose the best, and whoever chooses the better art has better
-taste.So if you want to discard the concept of good taste, you also have
-to discard the concept of good art. And that means you have to
-discard the possibility of people being good at making it. Which
-means there's no way for artists to be good at their jobs. And not
-just visual artists, but anyone who is in any sense an artist. You
-can't have good actors, or novelists, or composers, or dancers
-either. You can have popular novelists, but not good ones.We don't realize how far we'd have to go if we discarded the concept
-of good taste, because we don't even debate the most obvious cases.
-But it doesn't just mean we can't say which of two famous painters
-is better. It means we can't say that any painter is better than a
-randomly chosen eight year old.That was how I realized my father was wrong. I started studying
-painting. And it was just like other kinds of work I'd done: you
-could do it well, or badly, and if you tried hard, you could get
-better at it. And it was obvious that Leonardo and Bellini were
-much better at it than me. That gap between us was not imaginary.
-They were so good. And if they could be good, then art could be
-good, and there was such a thing as good taste after all.Now that I've explained how to show there is such a thing as good
-taste, I should also explain why people think there isn't. There
-are two reasons. One is that there's always so much disagreement
-about taste. Most people's response to art is a tangle of unexamined
-impulses. Is the artist famous? Is the subject attractive? Is this
-the sort of art they're supposed to like? Is it hanging in a famous
-museum, or reproduced in a big, expensive book? In practice most
-people's response to art is dominated by such extraneous factors.And the people who do claim to have good taste are so often mistaken.
-The paintings admired by the so-called experts in one generation
-are often so different from those admired a few generations later.
-It's easy to conclude there's nothing real there at all. It's only
-when you isolate this force, for example by trying to paint and
-comparing your work to Bellini's, that you can see that it does in
-fact exist.The other reason people doubt that art can be good is that there
-doesn't seem to be any room in the art for this goodness. The
-argument goes like this. Imagine several people looking at a work
-of art and judging how good it is. If being good art really is a
-property of objects, it should be in the object somehow. But it
-doesn't seem to be; it seems to be something happening in the heads
-of each of the observers. And if they disagree, how do you choose
-between them?The solution to this puzzle is to realize that the purpose of art
-is to work on its human audience, and humans have a lot in common.
-And to the extent the things an object acts upon respond in the
-same way, that's arguably what it means for the object to have the
-corresponding property. If everything a particle interacts with
-behaves as if the particle had a mass of m, then it has a mass of
-m. So the distinction between "objective" and "subjective" is not
-binary, but a matter of degree, depending on how much the subjects
-have in common. Particles interacting with one another are at one
-pole, but people interacting with art are not all the way at the
-other; their reactions aren't random.Because people's responses to art aren't random, art can be designed
-to operate on people, and be good or bad depending on how effectively
-it does so. Much as a vaccine can be. If someone were talking about
-the ability of a vaccine to confer immunity, it would seem very
-frivolous to object that conferring immunity wasn't really a property
-of vaccines, because acquiring immunity is something that happens
-in the immune system of each individual person. Sure, people's
-immune systems vary, and a vaccine that worked on one might not
-work on another, but that doesn't make it meaningless to talk about
-the effectiveness of a vaccine.The situation with art is messier, of course. You can't measure
-effectiveness by simply taking a vote, as you do with vaccines.
-You have to imagine the responses of subjects with a deep knowledge
-of art, and enough clarity of mind to be able to ignore extraneous
-influences like the fame of the artist. And even then you'd still
-see some disagreement. People do vary, and judging art is hard,
-especially recent art. There is definitely not a total order either
-of works or of people's ability to judge them. But there is equally
-definitely a partial order of both. So while it's not possible to
-have perfect taste, it is possible to have good taste.
-Thanks to the Cambridge Union for inviting me, and to Trevor
-Blackwell, Jessica Livingston, and Robert Morris for reading drafts
-of this.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/hubs.txt

@@ -1,156 +0,0 @@
-
-
-Want to start a startup?  Get funded by
-Y Combinator.
-
-
-
-
-October 2011If you look at a list of US cities sorted by population, the number
-of successful startups per capita varies by orders of magnitude.
-Somehow it's as if most places were sprayed with startupicide.I wondered about this for years.  I could see the average town was
-like a roach motel for startup ambitions: smart, ambitious people
-went in, but no startups came out.  But I was never able to figure
-out exactly what happened inside the motel—exactly what was
-killing all the potential startups.
-[1]A couple weeks ago I finally figured it out. I was framing the
-question wrong.  The problem is not that most towns kill startups.
-It's that death is the default for startups,
-and most towns don't save them.  Instead of thinking of most places
-as being sprayed with startupicide, it's more accurate to think of
-startups as all being poisoned, and a few places being sprayed with
-the antidote.Startups in other places are just doing what startups naturally do:
-fail.  The real question is, what's saving startups in places
-like Silicon Valley?
-[2]EnvironmentI think there are two components to the antidote: being in a place
-where startups are the cool thing to do, and chance meetings with
-people who can help you.  And what drives them both is the number
-of startup people around you.The first component is particularly helpful in the first stage of
-a startup's life, when you go from merely having an interest in
-starting a company to actually doing it.  It's quite a leap to start
-a startup.  It's an unusual thing to do. But in Silicon Valley it
-seems normal.
-[3]In most places, if you start a startup, people treat you as if
-you're unemployed.  People in the Valley aren't automatically
-impressed with you just because you're starting a company, but they
-pay attention.  Anyone who's been here any amount of time knows not
-to default to skepticism, no matter how inexperienced you seem or
-how unpromising your idea sounds at first, because they've all seen
-inexperienced founders with unpromising sounding ideas who a few
-years later were billionaires.Having people around you care about what you're doing is an
-extraordinarily powerful force.  Even the
-most willful people are susceptible to it.  About a year after we
-started Y Combinator I said something to a partner at a well known
-VC firm that gave him the (mistaken) impression I was considering
-starting another startup.  He responded so eagerly that for about
-half a second I found myself considering doing it.In most other cities, the prospect of starting a startup just doesn't
-seem real.  In the Valley it's not only real but fashionable.  That
-no doubt causes a lot of people to start startups who shouldn't.
-But I think that's ok.  Few people are suited to running a startup,
-and it's very hard to predict beforehand which are (as I know all
-too well from being in the business of trying to predict beforehand),
-so lots of people starting startups who shouldn't is probably the
-optimal state of affairs.  As long as you're at a point in your
-life when you can bear the risk of failure, the best way to find
-out if you're suited to running a startup is to try
-it.ChanceThe second component of the antidote is chance meetings with people
-who can help you.  This force works in both phases: both in the
-transition from the desire to start a startup to starting one, and
-the transition from starting a company to succeeding.  The power
-of chance meetings is more variable than people around you caring
-about startups, which is like a sort of background radiation that
-affects everyone equally, but at its strongest it is far stronger.Chance meetings produce miracles to compensate for the disasters
-that characteristically befall startups.  In the Valley, terrible
-things happen to startups all the time, just like they do to startups
-everywhere.  The reason startups are more likely to make it here
-is that great things happen to them too.  In the Valley, lightning
-has a sign bit.For example, you start a site for college students and you decide
-to move to the Valley for the summer to work on it.  And then on a
-random suburban street in Palo Alto you happen to run into Sean
-Parker, who understands the domain really well because he started
-a similar startup himself, and also knows all the investors.  And
-moreover has advanced views, for 2004, on founders retaining control of their companies.You can't say precisely what the miracle will be, or even for sure
-that one will happen.  The best one can say is: if you're in a
-startup hub, unexpected good things will probably happen to you,
-especially if you deserve them.I bet this is true even for startups we fund.  Even with us working
-to make things happen for them on purpose rather than by accident,
-the frequency of helpful chance meetings in the Valley is so high
-that it's still a significant increment on what we can deliver.Chance meetings play a role like the role relaxation plays in having
-ideas.  Most people have had the experience of working hard on some
-problem, not being able to solve it, giving up and going to bed,
-and then thinking of the answer in the shower in the morning.  What
-makes the answer appear is letting your thoughts drift a bit—and thus drift off the wrong
-path you'd been pursuing last night and onto the right one adjacent
-to it.Chance meetings let your acquaintance drift in the same way taking
-a shower lets your thoughts drift. The critical thing in both cases
-is that they drift just the right amount.  The meeting between Larry
-Page and Sergey Brin was a good example.  They let their acquaintance
-drift, but only a little; they were both meeting someone they had
-a lot in common with.For Larry Page the most important component of the antidote was
-Sergey Brin, and vice versa.  The antidote is 
-people.  It's not the
-physical infrastructure of Silicon Valley that makes it work, or
-the weather, or anything like that.  Those helped get it started,
-but now that the reaction is self-sustaining what drives it is the
-people.Many observers have noticed that one of the most distinctive things
-about startup hubs is the degree to which people help one another
-out, with no expectation of getting anything in return.  I'm not
-sure why this is so.  Perhaps it's because startups are less of a
-zero sum game than most types of business; they are rarely killed
-by competitors.  Or perhaps it's because so many startup founders
-have backgrounds in the sciences, where collaboration is encouraged.A large part of YC's function is to accelerate that process.  We're
-a sort of Valley within the Valley, where the density of people
-working on startups and their willingness to help one another are
-both artificially amplified.NumbersBoth components of the antidote—an environment that encourages
-startups, and chance meetings with people who help you—are
-driven by the same underlying cause: the number of startup people
-around you.  To make a startup hub, you need a lot of people
-interested in startups.There are three reasons. The first, obviously, is that if you don't
-have enough density, the chance meetings don't happen.
-[4]
-The second is that different startups need such different things, so
-you need a lot of people to supply each startup with what they need
-most.  Sean Parker was exactly what Facebook needed in 2004.  Another
-startup might have needed a database guy, or someone with connections
-in the movie business.This is one of the reasons we fund such a large number of companies,
-incidentally.  The bigger the community, the greater the chance it
-will contain the person who has that one thing you need most.The third reason you need a lot of people to make a startup hub is
-that once you have enough people interested in the same problem,
-they start to set the social norms.  And it is a particularly
-valuable thing when the atmosphere around you encourages you to do
-something that would otherwise seem too ambitious.  In most places
-the atmosphere pulls you back toward the mean.I flew into the Bay Area a few days ago.  I notice this every time
-I fly over the Valley: somehow you can sense something is going on.  
-Obviously you can sense prosperity in how well kept a
-place looks.  But there are different kinds of prosperity.  Silicon
-Valley doesn't look like Boston, or New York, or LA, or DC.  I tried
-asking myself what word I'd use to describe the feeling the Valley
-radiated, and the word that came to mind was optimism.Notes[1]
-I'm not saying it's impossible to succeed in a city with few
-other startups, just harder.  If you're sufficiently good at
-generating your own morale, you can survive without external
-encouragement.  Wufoo was based in Tampa and they succeeded.  But
-the Wufoos are exceptionally disciplined.[2]
-Incidentally, this phenomenon is not limited to startups.  Most
-unusual ambitions fail, unless the person who has them manages to
-find the right sort of community.[3]
-Starting a company is common, but starting a startup is rare.
-I've talked about the distinction between the two elsewhere, but
-essentially a startup is a new business designed for scale.  Most
-new businesses are service businesses and except in rare cases those
-don't scale.[4]
-As I was writing this, I had a demonstration of the density of
-startup people in the Valley.  Jessica and I bicycled to University
-Ave in Palo Alto to have lunch at the fabulous Oren's Hummus.  As
-we walked in, we met Charlie Cheever sitting near the door.  Selina
-Tobaccowala stopped to say hello on her way out.  Then Josh Wilson
-came in to pick up a take out order.  After lunch we went to get
-frozen yogurt.  On the way we met Rajat Suri.  When we got to the
-yogurt place, we found Dave Shen there, and as we walked out we ran
-into Yuri Sagalov.  We walked with him for a block or so and we ran
-into Muzzammil Zaveri, and then a block later we met Aydin Senkut.
-This is everyday life in Palo Alto.  I wasn't trying to meet people;
-I was just having lunch.  And I'm sure for every startup founder
-or investor I saw that I knew, there were 5 more I didn't.  If Ron
-Conway had been with us he would have met 30 people he knew.Thanks to Sam Altman, Paul Buchheit, Jessica Livingston, and
-Harj Taggar for reading drafts of this.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/iflisp.txt

@@ -1,46 +0,0 @@
-May 2003If Lisp is so great, why don't more people use it?  I was    
-asked this question by a student in the audience at a 
-talk I gave recently.  Not for the first time, either.In languages, as in so many things, there's not much     
-correlation between popularity and quality.  Why does   
-John Grisham (King of Torts sales rank, 44) outsell
-Jane Austen (Pride and Prejudice sales rank, 6191)?
-Would even Grisham claim that it's because he's a better
-writer?Here's the first sentence of Pride and Prejudice:
-
-It is a truth universally acknowledged, that a single man 
-in possession of a good fortune must be in want of a
-wife.
-
-"It is a truth universally acknowledged?"  Long words for
-the first sentence of a love story.Like Jane Austen, Lisp looks hard.  Its syntax, or lack
-of syntax, makes it look completely unlike 
-the languages
-most people are used to.  Before I learned Lisp, I was afraid
-of it too.  I recently came across a notebook from 1983
-in which I'd written:
-
-I suppose I should learn Lisp, but it seems so foreign.
-
-Fortunately, I was 19 at the time and not too resistant to learning
-new things.  I was so ignorant that learning
-almost anything meant learning new things.People frightened by Lisp make up other reasons for not
-using it.  The standard
-excuse, back when C was the default language, was that Lisp
-was too slow.  Now that Lisp dialects are among
-the faster
-languages available, that excuse has gone away.
-Now the standard excuse is openly circular: that other languages
-are more popular.(Beware of such reasoning.  It gets you Windows.)Popularity is always self-perpetuating, but it's especially
-so in programming languages. More libraries
-get written for popular languages, which makes them still
-more popular.  Programs often have to work with existing programs,
-and this is easier if they're written in the same language,
-so languages spread from program to program like a virus.
-And managers prefer popular languages, because they give them 
-more leverage over developers, who can more easily be replaced.Indeed, if programming languages were all more or less equivalent,
-there would be little justification for using any but the most
-popular.  But they aren't all equivalent, not by a long
-shot.  And that's why less popular languages, like Jane Austen's 
-novels, continue to survive at all.  When everyone else is reading 
-the latest John Grisham novel, there will always be a few people 
-reading Jane Austen instead.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/island.txt

@@ -1,55 +0,0 @@
-July 2006I've discovered a handy test for figuring out what you're addicted
-to.  Imagine you were going to spend the weekend at a friend's house
-on a little island off the coast of Maine.  There are no shops on
-the island and you won't be able to leave while you're there.  Also,
-you've never been to this house before, so you can't assume it will
-have more than any house might.What, besides clothes and toiletries, do you make a point of packing?
-That's what you're addicted to.  For example, if you find yourself
-packing a bottle of vodka (just in case), you may want to stop and
-think about that.For me the list is four things: books, earplugs, a notebook, and a
-pen.There are other things I might bring if I thought of it, like music,
-or tea, but I can live without them.  I'm not so addicted to caffeine
-that I wouldn't risk the house not having any tea, just for a
-weekend.Quiet is another matter.  I realize it seems a bit eccentric to
-take earplugs on a trip to an island off the coast of Maine.  If
-anywhere should be quiet, that should.  But what if the person in
-the next room snored?  What if there was a kid playing basketball?
-(Thump, thump, thump... thump.)  Why risk it?  Earplugs are small.Sometimes I can think with noise.  If I already have momentum on
-some project, I can work in noisy places.  I can edit an essay or
-debug code in an airport.  But airports are not so bad: most of the
-noise is whitish.  I couldn't work with the sound of a sitcom coming
-through the wall, or a car in the street playing thump-thump music.And of course there's another kind of thinking, when you're starting
-something new, that requires complete quiet.   You never
-know when this will strike. It's just as well to carry plugs.The notebook and pen are professional equipment, as it were.  Though
-actually there is something druglike about them, in the sense that
-their main purpose is to make me feel better.  I hardly ever go
-back and read stuff I write down in notebooks.  It's just that if
-I can't write things down, worrying about remembering one idea gets
-in the way of having the next.  Pen and paper wick ideas.The best notebooks I've found are made by a company called Miquelrius.
-I use their smallest size, which is about 2.5 x 4 in.
-The secret to writing on such
-narrow pages is to break words only when you run out of space, like
-a Latin inscription.  I use the cheapest plastic Bic ballpoints,
-partly because their gluey ink doesn't seep through pages, and
-partly so I don't worry about losing them.I only started carrying a notebook about three years ago.  Before
-that I used whatever scraps of paper I could find.  But the problem
-with scraps of paper is that they're not ordered.  In a notebook
-you can guess what a scribble means by looking at the pages
-around it.  In the scrap era I was constantly finding notes I'd
-written years before that might say something I needed to remember,
-if I could only figure out what.As for books, I know the house would probably have something to
-read.  On the average trip I bring four books and only read one of
-them, because I find new books to read en route.  Really bringing
-books is insurance.I realize this dependence on books is not entirely good—that what
-I need them for is distraction.  The books I bring on trips are
-often quite virtuous, the sort of stuff that might be assigned
-reading in a college class.  But I know my motives aren't virtuous.
-I bring books because if the world gets boring I need to be able
-to slip into another distilled by some writer.  It's like eating
-jam when you know you should be eating fruit.There is a point where I'll do without books.  I was walking in
-some steep mountains once, and decided I'd rather just think, if I
-was bored, rather than carry a single unnecessary ounce.  It wasn't
-so bad.  I found I could entertain myself by having ideas instead
-of reading other people's.  If you stop eating jam, fruit starts
-to taste better.So maybe I'll try not bringing books on some future trip.  They're
-going to have to pry the plugs out of my cold, dead ears, however.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/know.txt

@@ -1,53 +0,0 @@
-December 2014I've read Villehardouin's chronicle of the Fourth Crusade at least
-two times, maybe three.  And yet if I had to write down everything
-I remember from it, I doubt it would amount to much more than a
-page.  Multiply this times several hundred, and I get an uneasy
-feeling when I look at my bookshelves. What use is it to read all
-these books if I remember so little from them?A few months ago, as I was reading Constance Reid's excellent
-biography of Hilbert, I figured out if not the answer to this
-question, at least something that made me feel better about it.
-She writes:
-
-  Hilbert had no patience with mathematical lectures which filled
-  the students with facts but did not teach them how to frame a
-  problem and solve it. He often used to tell them that "a perfect
-  formulation of a problem is already half its solution."
-
-That has always seemed to me an important point, and I was even
-more convinced of it after hearing it confirmed by Hilbert.But how had I come to believe in this idea in the first place?  A
-combination of my own experience and other things I'd read.  None
-of which I could at that moment remember!  And eventually I'd forget
-that Hilbert had confirmed it too.  But my increased belief in the
-importance of this idea would remain something I'd learned from
-this book, even after I'd forgotten I'd learned it.Reading and experience train your model of the world.  And even if
-you forget the experience or what you read, its effect on your model
-of the world persists.  Your mind is like a compiled program you've
-lost the source of.  It works, but you don't know why.The place to look for what I learned from Villehardouin's chronicle
-is not what I remember from it, but my mental models of the crusades,
-Venice, medieval culture, siege warfare, and so on.  Which doesn't
-mean I couldn't have read more attentively, but at least the harvest
-of reading is not so miserably small as it might seem.This is one of those things that seem obvious in retrospect.  But
-it was a surprise to me and presumably would be to anyone else who
-felt uneasy about (apparently) forgetting so much they'd read.Realizing it does more than make you feel a little better about
-forgetting, though.  There are specific implications.For example, reading and experience are usually "compiled" at the
-time they happen, using the state of your brain at that time.  The
-same book would get compiled differently at different points in
-your life.  Which means it is very much worth reading important
-books multiple times.  I always used to feel some misgivings about
-rereading books.  I unconsciously lumped reading together with work
-like carpentry, where having to do something again is a sign you
-did it wrong the first time.  Whereas now the phrase "already read"
-seems almost ill-formed.Intriguingly, this implication isn't limited to books.  Technology
-will increasingly make it possible to relive our experiences.  When
-people do that today it's usually to enjoy them again (e.g. when
-looking at pictures of a trip) or to find the origin of some bug in
-their compiled code (e.g. when Stephen Fry succeeded in remembering
-the childhood trauma that prevented him from singing).  But as
-technologies for recording and playing back your life improve, it
-may become common for people to relive experiences without any goal
-in mind, simply to learn from them again as one might when rereading
-a book.Eventually we may be able not just to play back experiences but
-also to index and even edit them. So although not knowing how you
-know things may seem part of being human, it may not be.
-Thanks to Sam Altman, Jessica Livingston, and Robert Morris for reading 
-drafts of this.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/langdes.txt

@@ -1,242 +0,0 @@
-May 2001
-
-(These are some notes I made
-for a panel discussion on programming language design
-at MIT on May 10, 2001.)1. Programming Languages Are for People.Programming languages
-are how people talk to computers.  The computer would be just as
-happy speaking any language that was unambiguous.  The reason we
-have high level languages is because people can't deal with
-machine language.  The point of programming
-languages is to prevent our poor frail human brains from being 
-overwhelmed by a mass of detail.Architects know that some kinds of design problems are more personal
-than others.  One of the cleanest, most abstract design problems
-is designing bridges.  There your job is largely a matter of spanning
-a given distance with the least material.  The other end of the
-spectrum is designing chairs.  Chair designers have to spend their
-time thinking about human butts.Software varies in the same way. Designing algorithms for routing
-data through a network is a nice, abstract problem, like designing
-bridges.  Whereas designing programming languages is like designing
-chairs: it's all about dealing with human weaknesses.Most of us hate to acknowledge this.  Designing systems of great
-mathematical elegance sounds a lot more appealing to most of us
-than pandering to human weaknesses.  And there is a role for mathematical
-elegance: some kinds of elegance make programs easier to understand.
-But elegance is not an end in itself.And when I say languages have to be designed to suit human weaknesses,
-I don't mean that languages have to be designed for bad programmers.
-In fact I think you ought to design for the 
-best programmers, but
-even the best programmers have limitations.  I don't think anyone
-would like programming in a language where all the variables were
-the letter x with integer subscripts.2. Design for Yourself and Your Friends.If you look at the history of programming languages, a lot of the best
-ones were languages designed for their own authors to use, and a
-lot of the worst ones were designed for other people to use.When languages are designed for other people, it's always a specific
-group of other people: people not as smart as the language designer.
-So you get a language that talks down to you.  Cobol is the most
-extreme case, but a lot of languages are pervaded by this spirit.It has nothing to do with how abstract the language is.  C is pretty
-low-level, but it was designed for its authors to use, and that's
-why hackers like it.The argument for designing languages for bad programmers is that
-there are more bad programmers than good programmers.  That may be
-so.  But those few good programmers write a disproportionately
-large percentage of the software.I'm interested in the question, how do you design a language that
-the very best hackers will like?  I happen to think this is
-identical to the question, how do you design a good programming
-language?, but even if it isn't, it is at least an interesting
-question.3. Give the Programmer as Much Control as Possible.Many languages
-(especially the ones designed for other people) have the attitude
-of a governess: they try to prevent you from
-doing things that they think aren't good for you.  I like the   
-opposite approach: give the programmer as much
-control as you can.When I first learned Lisp, what I liked most about it was
-that it considered me an equal partner.  In the other languages
-I had learned up till then, there was the language and there was my   
-program, written in the language, and the two were very separate.
-But in Lisp the functions and macros I wrote were just like those
-that made up the language itself.  I could rewrite the language
-if I wanted.  It had the same appeal as open-source software.4. Aim for Brevity.Brevity is underestimated and even scorned.
-But if you look into the hearts of hackers, you'll see that they
-really love it.  How many times have you heard hackers speak fondly
-of how in, say, APL, they could do amazing things with just a couple
-lines of code?  I think anything that really smart people really
-love is worth paying attention to.I think almost anything
-you can do to make programs shorter is good.  There should be lots
-of library functions; anything that can be implicit should be;
-the syntax should be terse to a fault; even the names of things
-should be short.And it's not only programs that should be short.  The manual should
-be thin as well.  A good part of manuals is taken up with clarifications
-and reservations and warnings and special cases.  If you force  
-yourself to shorten the manual, in the best case you do it by fixing
-the things in the language that required so much explanation.5. Admit What Hacking Is.A lot of people wish that hacking was
-mathematics, or at least something like a natural science.  I think
-hacking is more like architecture.  Architecture is
-related to physics, in the sense that architects have to design
-buildings that don't fall down, but the actual goal of architects
-is to make great buildings, not to make discoveries about statics.What hackers like to do is make great programs.
-And I think, at least in our own minds, we have to remember that it's
-an admirable thing to write great programs, even when this work 
-doesn't translate easily into the conventional intellectual
-currency of research papers.  Intellectually, it is just as
-worthwhile to design a language programmers will love as it is to design a
-horrible one that embodies some idea you can publish a paper
-about.1. How to Organize Big Libraries?Libraries are becoming an
-increasingly important component of programming languages.  They're
-also getting bigger, and this can be dangerous.  If it takes longer
-to find the library function that will do what you want than it
-would take to write it yourself, then all that code is doing nothing
-but make your manual thick.  (The Symbolics manuals were a case in 
-point.)  So I think we will have to work on ways to organize
-libraries.  The ideal would be to design them so that the programmer
-could guess what library call would do the right thing.2. Are People Really Scared of Prefix Syntax?This is an open
-problem in the sense that I have wondered about it for years and
-still don't know the answer.  Prefix syntax seems perfectly natural
-to me, except possibly for math.  But it could be that a lot of 
-Lisp's unpopularity is simply due to having an unfamiliar syntax.   
-Whether to do anything about it, if it is true, is another question. 
-
-3. What Do You Need for Server-Based Software?
-
-I think a lot of the most exciting new applications that get written
-in the next twenty years will be Web-based applications, meaning
-programs that sit on the server and talk to you through a Web
-browser.  And to write these kinds of programs we may need some
-new things.One thing we'll need is support for the new way that server-based 
-apps get released.  Instead of having one or two big releases a
-year, like desktop software, server-based apps get released as a
-series of small changes.  You may have as many as five or ten
-releases a day.  And as a rule everyone will always use the latest
-version.You know how you can design programs to be debuggable?
-Well, server-based software likewise has to be designed to be
-changeable.  You have to be able to change it easily, or at least
-to know what is a small change and what is a momentous one.Another thing that might turn out to be useful for server based
-software, surprisingly, is continuations.  In Web-based software
-you can use something like continuation-passing style to get the
-effect of subroutines in the inherently 
-stateless world of a Web
-session.  Maybe it would be worthwhile having actual continuations,
-if it was not too expensive.4. What New Abstractions Are Left to Discover?I'm not sure how
-reasonable a hope this is, but one thing I would really love to    
-do, personally, is discover a new abstraction-- something that would
-make as much of a difference as having first class functions or
-recursion or even keyword parameters.  This may be an impossible
-dream.  These things don't get discovered that often.  But I am always
-looking.1. You Can Use Whatever Language You Want.Writing application
-programs used to mean writing desktop software.  And in desktop
-software there is a big bias toward writing the application in the
-same language as the operating system.  And so ten years ago,
-writing software pretty much meant writing software in C.
-Eventually a tradition evolved:
-application programs must not be written in unusual languages.  
-And this tradition had so long to develop that nontechnical people
-like managers and venture capitalists also learned it.Server-based software blows away this whole model.  With server-based
-software you can use any language you want.  Almost nobody understands
-this yet (especially not managers and venture capitalists).
-A few hackers understand it, and that's why we even hear
-about new, indy languages like Perl and Python.  We're not hearing
-about Perl and Python because people are using them to write Windows
-apps.What this means for us, as people interested in designing programming
-languages, is that there is now potentially an actual audience for
-our work.2. Speed Comes from Profilers.Language designers, or at least
-language implementors, like to write compilers that generate fast
-code.  But I don't think this is what makes languages fast for users.
-Knuth pointed out long ago that speed only matters in a few critical
-bottlenecks.  And anyone who's tried it knows that you can't guess
-where these bottlenecks are.  Profilers are the answer.Language designers are solving the wrong problem.  Users don't need
-benchmarks to run fast.  What they need is a language that can show
-them what parts of their own programs need to be rewritten.  That's
-where speed comes from in practice.  So maybe it would be a net 
-win if language implementors took half the time they would
-have spent doing compiler optimizations and spent it writing a
-good profiler instead.3. You Need an Application to Drive the Design of a Language.This may not be an absolute rule, but it seems like the best languages
-all evolved together with some application they were being used to
-write.  C was written by people who needed it for systems programming.
-Lisp was developed partly to do symbolic differentiation, and
-McCarthy was so eager to get started that he was writing differentiation
-programs even in the first paper on Lisp, in 1960.It's especially good if your application solves some new problem.
-That will tend to drive your language to have new features that   
-programmers need.  I personally am interested in writing
-a language that will be good for writing server-based applications.[During the panel, Guy Steele also made this point, with the
-additional suggestion that the application should not consist of
-writing the compiler for your language, unless your language
-happens to be intended for writing compilers.]4. A Language Has to Be Good for Writing Throwaway Programs.You know what a throwaway program is: something you write quickly for
-some limited task.  I think if you looked around you'd find that  
-a lot of big, serious programs started as throwaway programs.  I
-would not be surprised if most programs started as throwaway
-programs.  And so if you want to make a language that's good for
-writing software in general, it has to be good for writing throwaway
-programs, because that is the larval stage of most software.5. Syntax Is Connected to Semantics.It's traditional to think of
-syntax and semantics as being completely separate.  This will
-sound shocking, but it may be that they aren't.
-I think that what you want in your language may be related
-to how you express it.I was talking recently to Robert Morris, and he pointed out that
-operator overloading is a bigger win in languages with infix
-syntax.  In a language with prefix syntax, any function you define
-is effectively an operator.  If you want to define a plus for a
-new type of number you've made up, you can just define a new function
-to add them.  If you do that in a language with infix syntax,
-there's a big difference in appearance between the use of an
-overloaded operator and a function call.1. New Programming Languages.Back in the 1970s
-it was fashionable to design new programming languages.  Recently
-it hasn't been.  But I think server-based software will make new  
-languages fashionable again.  With server-based software, you can
-use any language you want, so if someone does design a language that
-actually seems better than others that are available, there will be
-people who take a risk and use it.2. Time-Sharing.Richard Kelsey gave this as an idea whose time
-has come again in the last panel, and I completely agree with him.
-My guess (and Microsoft's guess, it seems) is that much computing
-will move from the desktop onto remote servers.  In other words,  
-time-sharing is back.  And I think there will need to be support
-for it at the language level.  For example, I know that Richard
-and Jonathan Rees have done a lot of work implementing process  
-scheduling within Scheme 48.3. Efficiency.Recently it was starting to seem that computers
-were finally fast enough.  More and more we were starting to hear
-about byte code, which implies to me at least that we feel we have
-cycles to spare.  But I don't think we will, with server-based
-software.   Someone is going to have to pay for the servers that
-the software runs on, and the number of users they can support per
-machine will be the divisor of their capital cost.So I think efficiency will matter, at least in computational
-bottlenecks.  It will be especially important to do i/o fast,
-because server-based applications do a lot of i/o.It may turn out that byte code is not a win, in the end.  Sun and
-Microsoft seem to be facing off in a kind of a battle of the byte
-codes at the moment.  But they're doing it because byte code is a
-convenient place to insert themselves into the process, not because
-byte code is in itself a good idea.  It may turn out that this
-whole battleground gets bypassed.  That would be kind of amusing.1. Clients.This is just a guess, but my guess is that
-the winning model for most applications will be purely server-based.
-Designing software that works on the assumption that everyone will 
-have your client is like designing a society on the assumption that
-everyone will just be honest.  It would certainly be convenient, but
-you have to assume it will never happen.I think there will be a proliferation of devices that have some
-kind of Web access, and all you'll be able to assume about them is
-that they can support simple html and forms.  Will you have a
-browser on your cell phone?  Will there be a phone in your palm  
-pilot?  Will your blackberry get a bigger screen? Will you be able
-to browse the Web on your gameboy?  Your watch?  I don't know.  
-And I don't have to know if I bet on
-everything just being on the server.  It's
-just so much more robust to have all the 
-brains on the server.2. Object-Oriented Programming.I realize this is a
-controversial one, but I don't think object-oriented programming
-is such a big deal.  I think it is a fine model for certain kinds
-of applications that need that specific kind of data structure,   
-like window systems, simulations, and cad programs.  But I don't
-see why it ought to be the model for all programming.I think part of the reason people in big companies like object-oriented
-programming is because it yields a lot of what looks like work.
-Something that might naturally be represented as, say, a list of
-integers, can now be represented as a class with all kinds of
-scaffolding and hustle and bustle.Another attraction of
-object-oriented programming is that methods give you some of the
-effect of first class functions.  But this is old news to Lisp
-programmers.  When you have actual first class functions, you can
-just use them in whatever way is appropriate to the task at hand,
-instead of forcing everything into a mold of classes and methods.What this means for language design, I think, is that you shouldn't
-build object-oriented programming in too deeply.  Maybe the
-answer is to offer more general, underlying stuff, and let people design
-whatever object systems they want as libraries.3. Design by Committee.Having your language designed by a committee is a big pitfall,  
-and not just for the reasons everyone knows about.  Everyone
-knows that committees tend to yield lumpy, inconsistent designs.  
-But I think a greater danger is that they won't take risks.
-When one person is in charge he can take risks
-that a committee would never agree on.Is it necessary to take risks to design a good language though?
-Many people might suspect
-that language design is something where you should stick fairly
-close to the conventional wisdom.  I bet this isn't true.
-In everything else people do, reward is proportionate to risk.
-Why should language design be any different?

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/laundry.txt

@@ -1,487 +0,0 @@
-October 2004
-As E. B. White said, "good writing is rewriting."  I didn't
-realize this when I was in school.  In writing, as in math and 
-science, they only show you the finished product.
-You don't see all the false starts.  This gives students a
-misleading view of how things get made.Part of the reason it happens is that writers don't want   
-people to see their mistakes.  But I'm willing to let people
-see an early draft if it will show how much you have
-to rewrite to beat an essay into shape.Below is the oldest version I can find of
-The Age of the Essay   
-(probably the second or third day), with
-text that ultimately survived in 
-red and text that later
-got deleted in gray.
-There seem to be several categories of cuts: things I got wrong,
-things that seem like bragging, flames,
-digressions, stretches of awkward prose, and unnecessary words.I discarded more from the beginning.  That's
-not surprising; it takes a while to hit your stride.  There
-are more digressions at the start, because I'm not sure where
-I'm heading.The amount of cutting is about average.  I probably write
-three to four words for every one that appears in the final
-version of an essay.(Before anyone gets mad at me for opinions expressed here, remember
-that anything you see here that's not in the final version is obviously
-something I chose not to publish, often because I disagree
-with it.)
-Recently a friend said that what he liked about
-my essays was that they weren't written the way
-we'd been taught to write essays in school.  You
-remember: topic sentence, introductory paragraph,
-supporting paragraphs, conclusion.  It hadn't
-occurred to me till then that those horrible things
-we had to write in school were even connected to
-what I was doing now.  But sure enough, I thought,
-they did call them "essays," didn't they?Well, they're not.  Those things you have to write
-in school are not only not essays, they're one of the
-most pointless of all the pointless hoops you have
-to jump through in school.  And I worry that they
-not only teach students the wrong things about writing,
-but put them off writing entirely.So I'm going to give the other side of the story: what
-an essay really is, and how you write one.  Or at least,
-how I write one.  Students be forewarned: if you actually write
-the kind of essay I describe, you'll probably get bad
-grades.  But knowing how it's really done should
-at least help you to understand the feeling of futility
-you have when you're writing the things they tell you to.
-The most obvious difference between real essays and
-the things one has to write in school is that real
-essays are not exclusively about English literature.
-It's a fine thing for schools to
-
-teach students how to
-write.  But for some bizarre reason (actually, a very specific bizarre
-reason that I'll explain in a moment),
-
-the teaching of
-writing has gotten mixed together with the study
-of literature.  And so all over the country, students are
-writing not about how a baseball team with a small budget 
-might compete with the Yankees, or the role of color in
-fashion, or what constitutes a good dessert, but about
-symbolism in Dickens.With obvious 
-results.  Only a few people really
-
-care about
-symbolism in Dickens.  The teacher doesn't.
-The students don't.  Most of the people who've had to write PhD
-disserations about Dickens don't.  And certainly
-
-Dickens himself would be more interested in an essay
-about color or baseball.How did things get this way?  To answer that we have to go back
-almost a thousand years.  Between about 500 and 1000, life was
-not very good in Europe.  The term "dark ages" is presently
-out of fashion as too judgemental (the period wasn't dark; 
-it was just different), but if this label didn't already
-exist, it would seem an inspired metaphor.  What little
-original thought there was took place in lulls between
-constant wars and had something of the character of
-the thoughts of parents with a new baby.
-The most amusing thing written during this
-period, Liudprand of Cremona's Embassy to Constantinople, is,
-I suspect, mostly inadvertantly so.Around 1000 Europe began to catch its breath.
-And once they
-had the luxury of curiosity, one of the first things they discovered
-was what we call "the classics."
-Imagine if we were visited  
-by aliens.  If they could even get here they'd presumably know a
-few things we don't.  Immediately Alien Studies would become
-the most dynamic field of scholarship: instead of painstakingly
-discovering things for ourselves, we could simply suck up
-everything they'd discovered.  So it was in Europe in 1200.
-When classical texts began to circulate in Europe, they contained
-not just new answers, but new questions.  (If anyone proved
-a theorem in christian Europe before 1200, for example, there
-is no record of it.)For a couple centuries, some of the most important work
-being done was intellectual archaelogy.  Those were also
-the centuries during which schools were first established.
-And since reading ancient texts was the essence of what
-scholars did then, it became the basis of the curriculum.By 1700, someone who wanted to learn about
-physics didn't need to start by mastering Greek in order to read Aristotle.  But schools
-change slower than scholarship: the study of
-ancient texts
-had such prestige that it remained the backbone of 
-education
-until the late 19th century.  By then it was merely a tradition.
-It did serve some purposes: reading a foreign language was difficult,
-and thus taught discipline, or at least, kept students busy;
-it introduced students to
-cultures quite different from their own; and its very uselessness
-made it function (like white gloves) as a social bulwark.
-But it certainly wasn't
-true, and hadn't been true for centuries, that students were
-serving apprenticeships in the hottest area of scholarship.Classical scholarship had also changed.  In the early era, philology
-actually mattered.  The texts that filtered into Europe were
-all corrupted to some degree by the errors of translators and
-copyists.  Scholars had to figure out what Aristotle said
-before they could figure out what he meant.  But by the modern
-era such questions were answered as well as they were ever
-going to be.  And so the study of ancient texts became less
-about ancientness and more about texts.The time was then ripe for the question: if the study of
-ancient texts is a valid field for scholarship, why not modern
-texts?  The answer, of course, is that the raison d'etre
-of classical scholarship was a kind of intellectual archaelogy that
-does not need to be done in the case of contemporary authors.
-But for obvious reasons no one wanted to give that answer.
-The archaeological work being mostly done, it implied that
-the people studying the classics were, if not wasting their
-time, at least working on problems of minor importance.And so began the study of modern literature. There was some
-initial resistance, but it didn't last long.
-The limiting
-reagent in the growth of university departments is what
-parents will let undergraduates study.  If parents will let
-their children major in x, the rest follows straightforwardly.
-There will be jobs teaching x, and professors to fill them.
-The professors will establish scholarly journals and publish
-one another's papers.  Universities with x departments will
-subscribe to the journals.  Graduate students who want jobs
-as professors of x will write dissertations about it.  It may
-take a good long while for the more prestigious universities
-to cave in and establish departments in cheesier xes,  but
-at the other end of the scale there are so many universities
-competing to attract students that the mere establishment of
-a discipline requires little more than the desire to do it.High schools imitate universities.
-And so once university
-English departments were established in the late nineteenth century,
-the 'riting component of the 3 Rs 
-was morphed into English.
-With the bizarre consequence that high school students now
-had to write about English literature-- to write, without
-even realizing it, imitations of whatever
-English professors had been publishing in their journals a
-few decades before.   It's no wonder if this seems to the
-student a pointless exercise, because we're now three steps
-removed from real work: the students are imitating English
-professors, who are imitating classical scholars, who are
-merely the inheritors of a tradition growing out of what
-was, 700 years ago, fascinating and urgently needed work.Perhaps high schools should drop English and just teach writing.
-The valuable part of English classes is learning to write, and
-that could be taught better by itself.  Students learn better
-when they're interested in what they're doing, and it's hard
-to imagine a topic less interesting than symbolism in Dickens.
-Most of the people who write about that sort of thing professionally
-are not really interested in it.  (Though indeed, it's been a
-while since they were writing about symbolism; now they're
-writing about gender.)I have no illusions about how eagerly this suggestion will  
-be adopted.  Public schools probably couldn't stop teaching
-English even if they wanted to; they're probably required to by
-law.  But here's a related suggestion that goes with the grain
-instead of against it: that universities establish a
-writing major.  Many of the students who now major in English
-would major in writing if they could, and most would
-be better off.It will be argued that it is a good thing for students to be
-exposed to their literary heritage.  Certainly.  But is that
-more important than that they learn to write well?  And are
-English classes even the place to do it?  After all,
-the average public high school student gets zero exposure to  
-his artistic heritage.  No disaster results.
-The people who are interested in art learn about it for
-themselves, and those who aren't don't.  I find that American
-adults are no better or worse informed about literature than
-art, despite the fact that they spent years studying literature
-in high school and no time at all studying art.  Which presumably
-means that what they're taught in school is rounding error 
-compared to what they pick up on their own.Indeed, English classes may even be harmful.  In my case they
-were effectively aversion therapy.  Want to make someone dislike
-a book?  Force him to read it and write an essay about it.
-And make the topic so intellectually bogus that you
-could not, if asked, explain why one ought to write about it.
-I love to read more than anything, but by the end of high school
-I never read the books we were assigned.  I was so disgusted with
-what we were doing that it became a point of honor
-with me to write nonsense at least as good at the other students'
-without having more than glanced over the book to learn the names
-of the characters and a few random events in it.I hoped this might be fixed in college, but I found the same
-problem there.  It was not the teachers.  It was English.   
-We were supposed to read novels and write essays about them.
-About what, and why?  That no one seemed to be able to explain.
-Eventually by trial and error I found that what the teacher  
-wanted us to do was pretend that the story had really taken
-place, and to analyze based on what the characters said and did (the
-subtler clues, the better) what their motives must have been.
-One got extra credit for motives having to do with class,
-as I suspect one must now for those involving gender and  
-sexuality.  I learned how to churn out such stuff well enough
-to get an A, but I never took another English class.And the books we did these disgusting things to, like those
-we mishandled in high school, I find still have black marks
-against them in my mind.  The one saving grace was that   
-English courses tend to favor pompous, dull writers like
-Henry James, who deserve black marks against their names anyway.
-One of the principles the IRS uses in deciding whether to
-allow deductions is that, if something is fun, it isn't work.
-Fields that are intellectually unsure of themselves rely on
-a similar principle.  Reading P.G. Wodehouse or Evelyn Waugh or
-Raymond Chandler is too obviously pleasing to seem like
-serious work, as reading Shakespeare would have been before 
-English evolved enough to make it an effort to understand him. [sh]
-And so good writers (just you wait and see who's still in
-print in 300 years) are less likely to have readers turned   
-against them by clumsy, self-appointed tour guides.
-The other big difference between a real essay and the 
-things
-they make you write in school is that a real essay doesn't 
-take a position and then defend it.  That principle,
-like the idea that we ought to be writing about literature,   
-turns out to be another intellectual hangover of long
-forgotten origins.  It's often mistakenly believed that
-medieval universities were mostly seminaries.  In fact they
-were more law schools.  And at least in our tradition
-lawyers are advocates: they are
-trained to be able to
-take
-either side of an argument and make as good a case for it  
-as they can. Whether or not this is a good idea (in the case of prosecutors,
-it probably isn't), it tended to pervade
-the atmosphere of
-early universities.  After the lecture the most common form
-of discussion was the disputation.  This idea
-is at least
-nominally preserved in our present-day thesis defense-- indeed,
-in the very word thesis.  Most people treat the words 
-thesis
-and dissertation as interchangeable, but originally, at least,
-a thesis was a position one took and the dissertation was
-the argument by which one defended it.I'm not complaining that we blur these two words together.
-As far as I'm concerned, the sooner we lose the original
-sense of the word thesis, the better.  For many, perhaps most,  
-graduate students, it is stuffing a square peg into a round
-hole to try to recast one's work as a single thesis.  And
-as for the disputation, that seems clearly a net lose.
-Arguing two sides of a case may be a necessary evil in a
-legal dispute, but it's not the best way to get at the truth,
-as I think lawyers would be the first to admit.
-And yet this principle is built into the very structure of  
-the essays
-they teach you to write in high school.  The topic
-sentence is your thesis, chosen in advance, the supporting 
-paragraphs the blows you strike in the conflict, and the
-conclusion--- uh, what it the conclusion?  I was never sure  
-about that in high school.  If your thesis was well expressed,
-what need was there to restate it?  In theory it seemed that
-the conclusion of a really good essay ought not to need to   
-say any more than QED.
-But when you understand the origins
-of this sort of "essay", you can see where the
-conclusion comes from.  It's the concluding remarks to the 
-jury.
-What other alternative is there?  To answer that
-we have to
-reach back into history again, though this time not so far.
-To Michel de Montaigne, inventor of the essay.
-He was
-doing something quite different from what a
-lawyer does,
-and
-the difference is embodied in the name.  Essayer is the French
-verb meaning "to try" (the cousin of our word assay),
-
-and an "essai" is an effort.
-An essay is something you
-write in order
-to figure something out.Figure out what?  You don't know yet.  And so you can't begin with a
-thesis, because you don't have one, and may never have 
-one.  An essay doesn't begin with a statement, but with a  
-question.  In a real essay, you don't take a position and
-defend it.  You see a door that's ajar, and you open it and
-walk in to see what's inside.If all you want to do is figure things out, why do you need
-to write anything, though?  Why not just sit and think?  Well,
-there precisely is Montaigne's great discovery.  Expressing
-ideas helps to form them.  Indeed, helps is far too weak a
-word.  90%
-of what ends up in my essays was stuff
-I only
-thought of when I sat down to write them.  That's why I
-write them.So there's another difference between essays and
-the things
-you have to write in school.   In school
-
-you are, in theory,
-explaining yourself to someone else.  In the best case---if
-you're really organized---you're just writing it down.
-In a real essay you're writing for yourself.  You're
-thinking out loud.But not quite.  Just as inviting people over forces you to
-clean up your apartment, writing something that you know
-
-other people will read forces you to think well.  So it
-does matter to have an audience.  The things I've written
-just for myself are no good.  Indeed, they're bad in
-a particular way:
-they tend to peter out.  When I run into
-difficulties, I notice that I
-tend to conclude with a few vague
-questions and then drift off to get a cup of tea.This seems a common problem.
-It's practically the standard
-ending in blog entries--- with the addition of a "heh" or an 
-emoticon, prompted by the all too accurate sense that
-something is missing.And indeed, a lot of
-published essays peter out in this
-same way.
-Particularly the sort written by the staff writers of newsmagazines.  Outside writers tend to supply
-editorials of the defend-a-position variety, which
-make a beeline toward a rousing (and
-foreordained) conclusion.   But the staff writers feel
-obliged to write something more
-balanced, which in
-practice ends up meaning blurry.
-Since they're
-writing for a popular magazine, they start with the
-most radioactively controversial questions, from which
-(because they're writing for a popular magazine)
-they then proceed to recoil from
-in terror.
-Gay marriage, for or
-against?  This group says one thing.  That group says
-another.  One thing is certain: the question is a
-complex one.  (But don't get mad at us.  We didn't
-draw any conclusions.)Questions aren't enough.  An essay has to come up with answers.
-They don't always, of course.  Sometimes you start with a  
-promising question and get nowhere.  But those you don't
-publish.  Those are like experiments that get inconclusive
-results.   Something you publish ought to tell the reader  
-something he didn't already know.
-But what you tell him doesn't matter, so long as   
-it's interesting.  I'm sometimes accused of meandering.
-In defend-a-position writing that would be a flaw.
-There you're not concerned with truth.  You already
-know where you're going, and you want to go straight there,
-blustering through obstacles, and hand-waving
-your way across swampy ground.  But that's not what
-you're trying to do in an essay.  An essay is supposed to
-be a search for truth.  It would be suspicious if it didn't
-meander.The Meander is a river in Asia Minor (aka
-Turkey).
-As you might expect, it winds all over the place.
-But does it
-do this out of frivolity?   Quite the opposite.
-Like all rivers, it's rigorously following the laws of physics.
-The path it has discovered,
-winding as it is, represents
-the most economical route to the sea.The river's algorithm is simple.  At each step, flow down.
-For the essayist this translates to: flow interesting.
-Of all the places to go next, choose
-whichever seems
-most interesting.I'm pushing this metaphor a bit.  An essayist
-can't have
-quite as little foresight as a river.  In fact what you do
-(or what I do) is somewhere between a river and a roman
-road-builder.  I have a general idea of the direction
-I want to go in, and
-I choose the next topic with that in mind.  This essay is
-about writing, so I do occasionally yank it back in that
-direction, but it is not all the sort of essay I
-thought I was going to write about writing.Note too that hill-climbing (which is what this algorithm is
-called) can get you in trouble.
-Sometimes, just
-like a river,
-you
-run up against a blank wall.  What
-I do then is just 
-what the river does: backtrack.
-At one point in this essay
-I found that after following a certain thread I ran out
-of ideas.  I had to go back n
-paragraphs and start over
-in another direction.  For illustrative purposes I've left
-the abandoned branch as a footnote.
-Err on the side of the river.  An essay is not a reference
-work.  It's not something you read looking for a specific
-answer, and feel cheated if you don't find it.  I'd much
-rather read an essay that went off in an unexpected but
-interesting direction than one that plodded dutifully along
-a prescribed course.So what's interesting?  For me, interesting means surprise.
-Design, as Matz
-has said, should follow the principle of
-least surprise.
-A button that looks like it will make a
-machine stop should make it stop, not speed up.  Essays
-should do the opposite.  Essays should aim for maximum
-surprise.I was afraid of flying for a long time and could only travel
-vicariously.  When friends came back from faraway places,
-it wasn't just out of politeness that I asked them about
-their trip.
-I really wanted to know.  And I found that
-the best way to get information out of them was to ask
-what surprised them.  How was the place different from what
-they expected?  This is an extremely useful question.
-You can ask it of even
-the most unobservant people, and it will
-extract information they didn't even know they were
-recording. Indeed, you can ask it in real time.  Now when I go somewhere
-new, I make a note of what surprises me about it.  Sometimes I
-even make a conscious effort to visualize the place beforehand,
-so I'll have a detailed image to diff with reality.
-Surprises are facts
-you didn't already 
-know.
-But they're
-more than that.  They're facts
-that contradict things you
-thought you knew.  And so they're the most valuable sort of
-fact you can get.  They're like a food that's not merely
-healthy, but counteracts the unhealthy effects of things
-you've already eaten.
-How do you find surprises?  Well, therein lies half
-the work of essay writing.  (The other half is expressing
-yourself well.)   You can at least
-use yourself as a
-proxy for the reader.  You should only write about things
-you've thought about a lot.  And anything you come across
-that surprises you, who've thought about the topic a lot,
-will probably surprise most readers.For example, in a recent essay I pointed out that because
-you can only judge computer programmers by working with
-them, no one knows in programming who the heroes should
-be.
-I
-certainly
-didn't realize this when I started writing
-the 
-essay, and even now I find it kind of weird.  That's
-what you're looking for.So if you want to write essays, you need two ingredients:
-you need
-a few topics that you think about a lot, and you
-need some ability to ferret out the unexpected.What should you think about?  My guess is that it
-doesn't matter.  Almost everything is
-interesting if you get deeply
-enough into it.  The one possible exception
-are
-things
-like working in fast food, which
-have deliberately had all
-the variation sucked out of them.
-In retrospect, was there
-anything interesting about working in Baskin-Robbins?
-Well, it was interesting to notice
-how important color was
-to the customers.  Kids a certain age would point into
-the case and say that they wanted yellow.  Did they want
-French Vanilla or Lemon?  They would just look at you
-blankly.  They wanted yellow.  And then there was the
-mystery of why the perennial favorite Pralines n' Cream
-was so appealing. I'm inclined now to
-think it was the salt.
-And the mystery of why Passion Fruit tasted so disgusting.
-People would order it because of the name, and were always
-disappointed.  It should have been called In-sink-erator
-Fruit.
-And there was
-the difference in the way fathers and
-mothers bought ice cream for their kids.
-Fathers tended to
-adopt the attitude of
-benevolent kings bestowing largesse,
-and mothers that of
-harried bureaucrats,
-giving in to
-pressure against their better judgement.
-So, yes, there does seem to be material, even in
-fast food.What about the other half, ferreting out the unexpected?
-That may require some natural ability.  I've noticed for
-a long time that I'm pathologically observant.  ....[That was as far as I'd gotten at the time.]Notes[sh] In Shakespeare's own time, serious writing meant theological
-discourses, not the bawdy plays acted over on the other  
-side of the river among the bear gardens and whorehouses.The other extreme, the work that seems formidable from the moment
-it's created (indeed, is deliberately intended to be)
-is represented by Milton.  Like the Aeneid, Paradise Lost is a
-rock imitating a butterfly that happened to get fossilized.
-Even Samuel Johnson seems to have balked at this, on the one  
-hand paying Milton the compliment of an extensive biography,
-and on the other writing of Paradise Lost that "none who read it
-ever wished it longer."

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/love.txt

@@ -1,376 +0,0 @@
-
-
-Want to start a startup?  Get funded by
-Y Combinator.
-
-
-
-
-January 2006To do something well you have to like it.   That idea is not exactly
-novel.  We've got it down to four words: "Do what you love."  But
-it's not enough just to tell people that.  Doing what you love is
-complicated.The very idea is foreign to what most of us learn as kids.  When I
-was a kid, it seemed as if work and fun were opposites by definition.
-Life had two states: some of the time adults were making you do
-things, and that was called work; the rest of the time you could
-do what you wanted, and that was called playing.  Occasionally the
-things adults made you do were fun, just as, occasionally, playing
-wasn't—for example, if you fell and hurt yourself.  But except
-for these few anomalous cases, work was pretty much defined as
-not-fun.And it did not seem to be an accident. School, it was implied, was
-tedious because it was preparation for grownup work.The world then was divided into two groups, grownups and kids.
-Grownups, like some kind of cursed race, had to work.  Kids didn't,
-but they did have to go to school, which was a dilute version of
-work meant to prepare us for the real thing.  Much as we disliked
-school, the grownups all agreed that grownup work was worse, and
-that we had it easy.Teachers in particular all seemed to believe implicitly that work
-was not fun.  Which is not surprising: work wasn't fun for most of
-them.  Why did we have to memorize state capitals instead of playing
-dodgeball?  For the same reason they had to watch over a bunch of
-kids instead of lying on a beach.  You couldn't just do what you
-wanted.I'm not saying we should let little kids do whatever they want.
-They may have to be made to work on certain things.  But if we make
-kids work on dull stuff, it might be wise to tell them that tediousness
-is not the defining quality of work, and indeed that the reason
-they have to work on dull stuff now is so they can work on more
-interesting stuff later.
-[1]Once, when I was about 9 or 10, my father told me I could be whatever
-I wanted when I grew up, so long as I enjoyed it.  I remember that
-precisely because it seemed so anomalous.  It was like being told
-to use dry water.  Whatever I thought he meant, I didn't think he
-meant work could literally be fun—fun like playing.  It
-took me years to grasp that.JobsBy high school, the prospect of an actual job was on the horizon.
-Adults would sometimes come to speak to us about their work, or we
-would go to see them at work.  It was always understood that they
-enjoyed what they did.  In retrospect I think one may have: the
-private jet pilot.  But I don't think the bank manager really did.The main reason they all acted as if they enjoyed their work was
-presumably the upper-middle class convention that you're supposed
-to.  It would not merely be bad for your career to say that you
-despised your job, but a social faux-pas.Why is it conventional to pretend to like what you do?  The first
-sentence of this essay explains that.  If you have to like something
-to do it well, then the most successful people will all like what
-they do.  That's where the upper-middle class tradition comes from.
-Just as houses all over America are full of 
-chairs
-that are, without
-the owners even knowing it, nth-degree imitations of chairs designed
-250 years ago for French kings, conventional attitudes about work
-are, without the owners even knowing it, nth-degree imitations of
-the attitudes of people who've done great things.What a recipe for alienation.  By the time they reach an age to
-think about what they'd like to do, most kids have been thoroughly
-misled about the idea of loving one's work.  School has trained
-them to regard work as an unpleasant duty.  Having a job is said
-to be even more onerous than schoolwork.  And yet all the adults
-claim to like what they do.  You can't blame kids for thinking "I
-am not like these people; I am not suited to this world."Actually they've been told three lies: the stuff they've been taught
-to regard as work in school is not real work; grownup work is not
-(necessarily) worse than schoolwork; and many of the adults around
-them are lying when they say they like what they do.The most dangerous liars can be the kids' own parents.  If you take
-a boring job to give your family a high standard of living, as so
-many people do, you risk infecting your kids with the idea that
-work is boring. 
-[2]
-Maybe it would be better for kids in this one
-case if parents were not so unselfish.  A parent who set an example
-of loving their work might help their kids more than an expensive
-house.
-[3]It was not till I was in college that the idea of work finally broke
-free from the idea of making a living.  Then the important question
-became not how to make money, but what to work on.  Ideally these
-coincided, but some spectacular boundary cases (like Einstein in
-the patent office) proved they weren't identical.The definition of work was now to make some original contribution
-to the world, and in the process not to starve.  But after the habit
-of so many years my idea of work still included a large component
-of pain.  Work still seemed to require discipline, because only
-hard problems yielded grand results, and hard problems couldn't
-literally be fun.   Surely one had to force oneself to work on them.If you think something's supposed to hurt, you're less likely to
-notice if you're doing it wrong.  That about sums up my experience
-of graduate school.BoundsHow much are you supposed to like what you do?  Unless you
-know that, you don't know when to stop searching. And if, like most
-people, you underestimate it, you'll tend to stop searching too
-early.  You'll end up doing something chosen for you by your parents,
-or the desire to make money, or prestige—or sheer inertia.Here's an upper bound: Do what you love doesn't mean, do what you
-would like to do most this second.  Even Einstein probably
-had moments when he wanted to have a cup of coffee, but told himself
-he ought to finish what he was working on first.It used to perplex me when I read about people who liked what they
-did so much that there was nothing they'd rather do.  There didn't
-seem to be any sort of work I liked that much.  If I had a
-choice of (a) spending the next hour working on something or (b)
-be teleported to Rome and spend the next hour wandering about, was
-there any sort of work I'd prefer?  Honestly, no.But the fact is, almost anyone would rather, at any given moment,
-float about in the Carribbean, or have sex, or eat some delicious
-food, than work on hard problems.  The rule about doing what you
-love assumes a certain length of time.  It doesn't mean, do what
-will make you happiest this second, but what will make you happiest
-over some longer period, like a week or a month.Unproductive pleasures pall eventually.  After a while you get tired
-of lying on the beach.  If you want to stay happy, you have to do
-something.As a lower bound, you have to like your work more than any unproductive
-pleasure.  You have to like what you do enough that the concept of
-"spare time" seems mistaken.  Which is not to say you have to spend
-all your time working.  You can only work so much before you get
-tired and start to screw up.  Then you want to do something else—even something mindless.  But you don't regard this time as the
-prize and the time you spend working as the pain you endure to earn
-it.I put the lower bound there for practical reasons.  If your work
-is not your favorite thing to do, you'll have terrible problems
-with procrastination.  You'll have to force yourself to work,  and
-when you resort to that the results are distinctly inferior.To be happy I think you have to be doing something you not only
-enjoy, but admire.  You have to be able to say, at the end, wow,
-that's pretty cool.  This doesn't mean you have to make something.
-If you learn how to hang glide, or to speak a foreign language
-fluently, that will be enough to make you say, for a while at least,
-wow, that's pretty cool.  What there has to be is a test.So one thing that falls just short of the standard, I think, is
-reading books.  Except for some books in math and the hard sciences,
-there's no test of how well you've read a book, and that's why
-merely reading books doesn't quite feel like work.  You have to do
-something with what you've read to feel productive.I think the best test is one Gino Lee taught me: to try to do things
-that would make your friends say wow.  But it probably wouldn't
-start to work properly till about age 22, because most people haven't
-had a big enough sample to pick friends from before then.SirensWhat you should not do, I think, is worry about the opinion of
-anyone beyond your friends.  You shouldn't worry about prestige.
-Prestige is the opinion of the rest of the world.  When you can ask
-the opinions of people whose judgement you respect, what does it
-add to consider the opinions of people you don't even know? 
-[4]This is easy advice to give.  It's hard to follow, especially when
-you're young.  
-[5]
-Prestige is like a powerful magnet that warps
-even your beliefs about what you enjoy.  It causes you to work not
-on what you like, but what you'd like to like.That's what leads people to try to write novels, for example.  They
-like reading novels.  They notice that people who write them win
-Nobel prizes.  What could be more wonderful, they think, than to
-be a novelist?  But liking the idea of being a novelist is not
-enough; you have to like the actual work of novel-writing if you're
-going to be good at it; you have to like making up elaborate lies.Prestige is just fossilized inspiration.  If you do anything well
-enough, you'll make it prestigious.  Plenty of things we now
-consider prestigious were anything but at first.  Jazz comes to
-mind—though almost any established art form would do.   So just
-do what you like, and let prestige take care of itself.Prestige is especially dangerous to the ambitious.  If you want to
-make ambitious people waste their time on errands, the way to do
-it is to bait the hook with prestige.  That's the recipe for getting
-people to give talks, write forewords, serve on committees, be
-department heads, and so on.  It might be a good rule simply to
-avoid any prestigious task. If it didn't suck, they wouldn't have
-had to make it prestigious.Similarly, if you admire two kinds of work equally, but one is more
-prestigious, you should probably choose the other.  Your opinions
-about what's admirable are always going to be slightly influenced
-by prestige, so if the two seem equal to you, you probably have
-more genuine admiration for the less prestigious one.The other big force leading people astray is money.  Money by itself
-is not that dangerous.  When something pays well but is regarded
-with contempt, like telemarketing, or prostitution, or personal
-injury litigation, ambitious people aren't tempted by it.  That
-kind of work ends up being done by people who are "just trying to
-make a living."  (Tip: avoid any field whose practitioners say
-this.)  The danger is when money is combined with prestige, as in,
-say, corporate law, or medicine.  A comparatively safe and prosperous
-career with some automatic baseline prestige is dangerously tempting
-to someone young, who hasn't thought much about what they really
-like.The test of whether people love what they do is whether they'd do
-it even if they weren't paid for it—even if they had to work at
-another job to make a living.  How many corporate lawyers would do
-their current work if they had to do it for free, in their spare
-time, and take day jobs as waiters to support themselves?This test is especially helpful in deciding between different kinds
-of academic work, because fields vary greatly in this respect.  Most
-good mathematicians would work on math even if there were no jobs
-as math professors, whereas in the departments at the other end of
-the spectrum, the availability of teaching jobs is the driver:
-people would rather be English professors than work in ad agencies,
-and publishing papers is the way you compete for such jobs.  Math
-would happen without math departments, but it is the existence of
-English majors, and therefore jobs teaching them, that calls into
-being all those thousands of dreary papers about gender and identity
-in the novels of Conrad.  No one does 
-that 
-kind of thing for fun.The advice of parents will tend to err on the side of money.  It
-seems safe to say there are more undergrads who want to be novelists
-and whose parents want them to be doctors than who want to be doctors
-and whose parents want them to be novelists.  The kids think their
-parents are "materialistic." Not necessarily.  All parents tend to
-be more conservative for their kids than they would for themselves,
-simply because, as parents, they share risks more than rewards.  If
-your eight year old son decides to climb a tall tree, or your teenage
-daughter decides to date the local bad boy, you won't get a share
-in the excitement, but if your son falls, or your daughter gets
-pregnant, you'll have to deal with the consequences.DisciplineWith such powerful forces leading us astray, it's not surprising
-we find it so hard to discover what we like to work on.  Most people
-are doomed in childhood by accepting the axiom that work = pain.
-Those who escape this are nearly all lured onto the rocks by prestige
-or money.  How many even discover something they love to work on?
-A few hundred thousand, perhaps, out of billions.It's hard to find work you love; it must be, if so few do.  So don't
-underestimate this task.  And don't feel bad if you haven't succeeded
-yet.  In fact, if you admit to yourself that you're discontented,
-you're a step ahead of most people, who are still in denial.  If
-you're surrounded by colleagues who claim to enjoy work that you
-find contemptible, odds are they're lying to themselves.  Not
-necessarily, but probably.Although doing great work takes less discipline than people think—because the way to do great work is to find something you like so
-much that you don't have to force yourself to do it—finding
-work you love does usually require discipline.   Some people are
-lucky enough to know what they want to do when they're 12, and just
-glide along as if they were on railroad tracks.  But this seems the
-exception.  More often people who do great things have careers with
-the trajectory of a ping-pong ball.  They go to school to study A,
-drop out and get a job doing B, and then become famous for C after
-taking it up on the side.Sometimes jumping from one sort of work to another is a sign of
-energy, and sometimes it's a sign of laziness.  Are you dropping
-out, or boldly carving a new path?  You often can't tell yourself.
-Plenty of people who will later do great things seem to be disappointments
-early on, when they're trying to find their niche.Is there some test you can use to keep yourself honest?  One is to
-try to do a good job at whatever you're doing, even if you don't
-like it.  Then at least you'll know you're not using dissatisfaction
-as an excuse for being lazy.  Perhaps more importantly, you'll get
-into the habit of doing things well.Another test you can use is: always produce.  For example, if you
-have a day job you don't take seriously because you plan to be a
-novelist, are you producing?  Are you writing pages of fiction,
-however bad?  As long as you're producing, you'll know you're not
-merely using the hazy vision of the grand novel you plan to write
-one day as an opiate.  The view of it will be obstructed by the all
-too palpably flawed one you're actually writing."Always produce" is also a heuristic for finding the work you love.
-If you subject yourself to that constraint, it will automatically
-push you away from things you think you're supposed to work on,
-toward things you actually like.  "Always produce" will discover
-your life's work the way water, with the aid of gravity, finds the
-hole in your roof.Of course, figuring out what you like to work on doesn't mean you
-get to work on it.  That's a separate question.  And if you're
-ambitious you have to keep them separate: you have to make a conscious
-effort to keep your ideas about what you want from being contaminated
-by what seems possible. 
-[6]It's painful to keep them apart, because it's painful to observe
-the gap between them. So most people pre-emptively lower their
-expectations.  For example, if you asked random people on the street
-if they'd like to be able to draw like Leonardo, you'd find most
-would say something like "Oh, I can't draw."  This is more a statement
-of intention than fact; it means, I'm not going to try.  Because
-the fact is, if you took a random person off the street and somehow
-got them to work as hard as they possibly could at drawing for the
-next twenty years, they'd get surprisingly far.  But it would require
-a great moral effort; it would mean staring failure in the eye every
-day for years.  And so to protect themselves people say "I can't."Another related line you often hear is that not everyone can do
-work they love—that someone has to do the unpleasant jobs.  Really?
-How do you make them?  In the US the only mechanism for forcing
-people to do unpleasant jobs is the draft, and that hasn't been
-invoked for over 30 years.  All we can do is encourage people to
-do unpleasant work, with money and prestige.If there's something people still won't do, it seems as if society
-just has to make do without.  That's what happened with domestic
-servants.  For millennia that was the canonical example of a job
-"someone had to do."  And yet in the mid twentieth century servants
-practically disappeared in rich countries, and the rich have just
-had to do without.So while there may be some things someone has to do, there's a good
-chance anyone saying that about any particular job is mistaken.
-Most unpleasant jobs would either get automated or go undone if no
-one were willing to do them.Two RoutesThere's another sense of "not everyone can do work they love"
-that's all too true, however.  One has to make a living, and it's
-hard to get paid for doing work you love.  There are two routes to
-that destination:
-
-  The organic route: as you become more eminent, gradually to
-  increase the parts of your job that you like at the expense of
-  those you don't.The two-job route: to work at things you don't like to get money
-  to work on things you do.
-
-The organic route is more common.  It happens naturally to anyone
-who does good work.  A young architect has to take whatever work
-he can get, but if he does well he'll gradually be in a position
-to pick and choose among projects.  The disadvantage of this route
-is that it's slow and uncertain.  Even tenure is not real freedom.The two-job route has several variants depending on how long you
-work for money at a time.  At one extreme is the "day job," where
-you work regular hours at one job to make money, and work on what
-you love in your spare time.  At the other extreme you work at
-something till you make enough not to 
-have to work for money again.The two-job route is less common than the organic route, because
-it requires a deliberate choice.  It's also more dangerous.  Life
-tends to get more expensive as you get older, so it's easy to get
-sucked into working longer than you expected at the money job.
-Worse still, anything you work on changes you.  If you work too
-long on tedious stuff, it will rot your brain.  And the best paying
-jobs are most dangerous, because they require your full attention.The advantage of the two-job route is that it lets you jump over
-obstacles.  The landscape of possible jobs isn't flat; there are
-walls of varying heights between different kinds of work. 
-[7]
-The trick of maximizing the parts of your job that you like can get you
-from architecture to product design, but not, probably, to music.
-If you make money doing one thing and then work on another, you
-have more freedom of choice.Which route should you take?  That depends on how sure you are of
-what you want to do, how good you are at taking orders, how much
-risk you can stand, and the odds that anyone will pay (in your
-lifetime) for what you want to do.  If you're sure of the general
-area you want to work in and it's something people are likely to
-pay you for, then you should probably take the organic route.  But
-if you don't know what you want to work on, or don't like to take
-orders, you may want to take the two-job route, if you can stand
-the risk.Don't decide too soon.  Kids who know early what they want to do
-seem impressive, as if they got the answer to some math question
-before the other kids.  They have an answer, certainly, but odds
-are it's wrong.A friend of mine who is a quite successful doctor complains constantly
-about her job.  When people applying to medical school ask her for
-advice, she wants to shake them and yell "Don't do it!"  (But she
-never does.) How did she get into this fix?  In high school she
-already wanted to be a doctor.  And she is so ambitious and determined
-that she overcame every obstacle along the way—including,
-unfortunately, not liking it.Now she has a life chosen for her by a high-school kid.When you're young, you're given the impression that you'll get
-enough information to make each choice before you need to make it.
-But this is certainly not so with work.  When you're deciding what
-to do, you have to operate on ridiculously incomplete information.
-Even in college you get little idea what various types of work are
-like.  At best you may have a couple internships, but not all jobs
-offer internships, and those that do don't teach you much more about
-the work than being a batboy teaches you about playing baseball.In the design of lives, as in the design of most other things, you
-get better results if you use flexible media.  So unless you're
-fairly sure what you want to do, your best bet may be to choose a
-type of work that could turn into either an organic or two-job
-career.  That was probably part of the reason I chose computers.
-You can be a professor, or make a lot of money, or morph it into
-any number of other kinds of work.It's also wise, early on, to seek jobs that let you do many different
-things, so you can learn faster what various kinds of work are like.
-Conversely, the extreme version of the two-job route is dangerous
-because it teaches you so little about what you like.  If you work
-hard at being a bond trader for ten years, thinking that you'll
-quit and write novels when you have enough money, what happens when
-you quit and then discover that you don't actually like writing
-novels?Most people would say, I'd take that problem.  Give me a million
-dollars and I'll figure out what to do.  But it's harder than it
-looks.  Constraints give your life shape.  Remove them and most
-people have no idea what to do: look at what happens to those who
-win lotteries or inherit money.  Much as everyone thinks they want
-financial security, the happiest people are not those who have it,
-but those who like what they do.  So a plan that promises freedom
-at the expense of knowing what to do with it may not be as good as
-it seems.Whichever route you take, expect a struggle.  Finding work you love
-is very difficult.  Most people fail.  Even if you succeed, it's
-rare to be free to work on what you want till your thirties or
-forties.  But if you have the destination in sight you'll be more
-likely to arrive at it.  If you know you can love work, you're in
-the home stretch, and if you know what work you love, you're
-practically there.Notes[1]
-Currently we do the opposite: when we make kids do boring work,
-like arithmetic drills, instead of admitting frankly that it's
-boring, we try to disguise it with superficial decorations.[2]
-One father told me about a related phenomenon: he found himself
-concealing from his family how much he liked his work.  When he
-wanted to go to work on a saturday, he found it easier to say that
-it was because he "had to" for some reason, rather than admitting
-he preferred to work than stay home with them.[3]
-Something similar happens with suburbs.  Parents move to suburbs
-to raise their kids in a safe environment, but suburbs are so dull
-and artificial that by the time they're fifteen the kids are convinced
-the whole world is boring.[4]
-I'm not saying friends should be the only audience for your
-work.  The more people you can help, the better.  But friends should
-be your compass.[5]
-Donald Hall said young would-be poets were mistaken to be so
-obsessed with being published.  But you can imagine what it would
-do for a 24 year old to get a poem published in The New Yorker.
-Now to people he meets at parties he's a real poet.  Actually he's
-no better or worse than he was before, but to a clueless audience
-like that, the approval of an official authority makes all the
-difference.   So it's a harder problem than Hall realizes.  The
-reason the young care so much about prestige is that the people
-they want to impress are not very discerning.[6]
-This is isomorphic to the principle that you should prevent
-your beliefs about how things are from being contaminated by how
-you wish they were.  Most people let them mix pretty promiscuously.
-The continuing popularity of religion is the most visible index of
-that.[7]
-A more accurate metaphor would be to say that the graph of jobs
-is not very well connected.Thanks to Trevor Blackwell, Dan Friedman, Sarah Harlin,
-Jessica Livingston, Jackie McDonough, Robert Morris, Peter Norvig, 
-David Sloo, and Aaron Swartz
-for reading drafts of this.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/mod.txt

@@ -1,54 +0,0 @@
-December 2019There are two distinct ways to be politically moderate: on purpose
-and by accident. Intentional moderates are trimmers, deliberately
-choosing a position mid-way between the extremes of right and left.
-Accidental moderates end up in the middle, on average, because they
-make up their own minds about each question, and the far right and
-far left are roughly equally wrong.You can distinguish intentional from accidental moderates by the
-distribution of their opinions. If the far left opinion on some
-matter is 0 and the far right opinion 100, an intentional moderate's
-opinion on every question will be near 50. Whereas an accidental
-moderate's opinions will be scattered over a broad range, but will,
-like those of the intentional moderate, average to about 50.Intentional moderates are similar to those on the far left and the
-far right in that their opinions are, in a sense, not their own.
-The defining quality of an ideologue, whether on the left or the
-right, is to acquire one's opinions in bulk. You don't get to pick
-and choose. Your opinions about taxation can be predicted from your
-opinions about sex. And although intentional moderates
-might seem to be the opposite of ideologues, their beliefs (though
-in their case the word "positions" might be more accurate) are also
-acquired in bulk. If the median opinion shifts to the right or left,
-the intentional moderate must shift with it. Otherwise they stop
-being moderate.Accidental moderates, on the other hand, not only choose their own
-answers, but choose their own questions. They may not care at all
-about questions that the left and right both think are terribly
-important. So you can only even measure the politics of an accidental
-moderate from the intersection of the questions they care about and
-those the left and right care about, and this can
-sometimes be vanishingly small.It is not merely a manipulative rhetorical trick to say "if you're
-not with us, you're against us," but often simply false.Moderates are sometimes derided as cowards, particularly by 
-the extreme left. But while it may be accurate to call intentional
-moderates cowards, openly being an accidental moderate requires the
-most courage of all, because you get attacked from both right and
-left, and you don't have the comfort of being an orthodox member
-of a large group to sustain you.Nearly all the most impressive people I know are accidental moderates.
-If I knew a lot of professional athletes, or people in the entertainment
-business, that might be different. Being on the far left or far
-right doesn't affect how fast you run or how well you sing. But
-someone who works with ideas has to be independent-minded to do it
-well.Or more precisely, you have to be independent-minded about the ideas
-you work with. You could be mindlessly doctrinaire in your politics
-and still be a good mathematician. In the 20th century, a lot of
-very smart people were Marxists — just no one who was smart about
-the subjects Marxism involves. But if the ideas you use in your
-work intersect with the politics of your time, you have two choices:
-be an accidental moderate, or be mediocre.Notes[1] It's possible in theory for one side to be entirely right and
-the other to be entirely wrong. Indeed, ideologues must always
-believe this is the case. But historically it rarely has been.[2] For some reason the far right tend to ignore moderates rather
-than despise them as backsliders. I'm not sure why. Perhaps it
-means that the far right is less ideological than the far left. Or
-perhaps that they are more confident, or more resigned, or simply
-more disorganized. I just don't know.[3] Having heretical opinions doesn't mean you have to express
-them openly. It may be
-easier to have them if you don't.
-Thanks to Austen Allred, Trevor Blackwell, Patrick Collison, Jessica Livingston,
-Amjad Masad, Ryan Petersen, and Harj Taggar for reading drafts of this.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/newideas.txt

@@ -1,114 +0,0 @@
-May 2021There's one kind of opinion I'd be very afraid to express publicly.
-If someone I knew to be both a domain expert and a reasonable person
-proposed an idea that sounded preposterous, I'd be very reluctant
-to say "That will never work."Anyone who has studied the history of ideas, and especially the
-history of science, knows that's how big things start. Someone
-proposes an idea that sounds crazy, most people dismiss it, then
-it gradually takes over the world.Most implausible-sounding ideas are in fact bad and could be safely
-dismissed. But not when they're proposed by reasonable domain
-experts. If the person proposing the idea is reasonable, then they
-know how implausible it sounds. And yet they're proposing it anyway.
-That suggests they know something you don't. And if they have deep
-domain expertise, that's probably the source of it.
-[1]Such ideas are not merely unsafe to dismiss, but disproportionately
-likely to be interesting. When the average person proposes an
-implausible-sounding idea, its implausibility is evidence of their
-incompetence. But when a reasonable domain expert does it, the
-situation is reversed. There's something like an efficient market
-here: on average the ideas that seem craziest will, if correct,
-have the biggest effect. So if you can eliminate the theory that
-the person proposing an implausible-sounding idea is incompetent,
-its implausibility switches from evidence that it's boring to
-evidence that it's exciting.
-[2]Such ideas are not guaranteed to work. But they don't have to be.
-They just have to be sufficiently good bets — to have sufficiently
-high expected value. And I think on average they do. I think if you
-bet on the entire set of implausible-sounding ideas proposed by
-reasonable domain experts, you'd end up net ahead.The reason is that everyone is too conservative. The word "paradigm"
-is overused, but this is a case where it's warranted. Everyone is
-too much in the grip of the current paradigm. Even the people who
-have the new ideas undervalue them initially. Which means that
-before they reach the stage of proposing them publicly, they've
-already subjected them to an excessively strict filter.
-[3]The wise response to such an idea is not to make statements, but
-to ask questions, because there's a real mystery here. Why has this
-smart and reasonable person proposed an idea that seems so wrong?
-Are they mistaken, or are you? One of you has to be. If you're the
-one who's mistaken, that would be good to know, because it means
-there's a hole in your model of the world. But even if they're
-mistaken, it should be interesting to learn why. A trap that an
-expert falls into is one you have to worry about too.This all seems pretty obvious. And yet there are clearly a lot of
-people who don't share my fear of dismissing new ideas. Why do they
-do it? Why risk looking like a jerk now and a fool later, instead
-of just reserving judgement?One reason they do it is envy. If you propose a radical new idea
-and it succeeds, your reputation (and perhaps also your wealth)
-will increase proportionally. Some people would be envious if that
-happened, and this potential envy propagates back into a conviction
-that you must be wrong.Another reason people dismiss new ideas is that it's an easy way
-to seem sophisticated. When a new idea first emerges, it usually
-seems pretty feeble. It's a mere hatchling. Received wisdom is a
-full-grown eagle by comparison. So it's easy to launch a devastating
-attack on a new idea, and anyone who does will seem clever to those
-who don't understand this asymmetry.This phenomenon is exacerbated by the difference between how those
-working on new ideas and those attacking them are rewarded. The
-rewards for working on new ideas are weighted by the value of the
-outcome. So it's worth working on something that only has a 10%
-chance of succeeding if it would make things more than 10x better.
-Whereas the rewards for attacking new ideas are roughly constant;
-such attacks seem roughly equally clever regardless of the target.People will also attack new ideas when they have a vested interest
-in the old ones. It's not surprising, for example, that some of
-Darwin's harshest critics were churchmen. People build whole careers
-on some ideas. When someone claims they're false or obsolete, they
-feel threatened.The lowest form of dismissal is mere factionalism: to automatically
-dismiss any idea associated with the opposing faction. The lowest
-form of all is to dismiss an idea because of who proposed it.But the main thing that leads reasonable people to dismiss new ideas
-is the same thing that holds people back from proposing them: the
-sheer pervasiveness of the current paradigm. It doesn't just affect
-the way we think; it is the Lego blocks we build thoughts out of.
-Popping out of the current paradigm is something only a few people
-can do. And even they usually have to suppress their intuitions at
-first, like a pilot flying through cloud who has to trust his
-instruments over his sense of balance.
-[4]Paradigms don't just define our present thinking. They also vacuum
-up the trail of crumbs that led to them, making our standards for
-new ideas impossibly high. The current paradigm seems so perfect
-to us, its offspring, that we imagine it must have been accepted
-completely as soon as it was discovered — that whatever the church thought
-of the heliocentric model, astronomers must have been convinced as
-soon as Copernicus proposed it. Far, in fact, from it. Copernicus
-published the heliocentric model in 1532, but it wasn't till the
-mid seventeenth century that the balance of scientific opinion
-shifted in its favor.
-[5]Few understand how feeble new ideas look when they first appear.
-So if you want to have new ideas yourself, one of the most valuable
-things you can do is to learn what they look like when they're born.
-Read about how new ideas happened, and try to get yourself into the
-heads of people at the time. How did things look to them, when the
-new idea was only half-finished, and even the person who had it was
-only half-convinced it was right?But you don't have to stop at history. You can observe big new ideas
-being born all around you right now. Just look for a reasonable
-domain expert proposing something that sounds wrong.If you're nice, as well as wise, you won't merely resist attacking
-such people, but encourage them. Having new ideas is a lonely
-business. Only those who've tried it know how lonely. These people
-need your help. And if you help them, you'll probably learn something
-in the process.Notes[1]
-This domain expertise could be in another field. Indeed,
-such crossovers tend to be particularly promising.[2]
-I'm not claiming this principle extends much beyond math,
-engineering, and the hard sciences. In politics, for example,
-crazy-sounding ideas generally are as bad as they sound. Though
-arguably this is not an exception, because the people who propose
-them are not in fact domain experts; politicians are domain experts
-in political tactics, like how to get elected and how to get
-legislation passed, but not in the world that policy acts upon.
-Perhaps no one could be.[3]
-This sense of "paradigm" was defined by Thomas Kuhn in his
-Structure of Scientific Revolutions, but I also recommend his
-Copernican Revolution, where you can see him at work developing the
-idea.[4]
-This is one reason people with a touch of Asperger's may have
-an advantage in discovering new ideas. They're always flying on
-instruments.[5]
-Hall, Rupert. From Galileo to Newton. Collins, 1963. This
-book is particularly good at getting into contemporaries' heads.Thanks to Trevor Blackwell, Patrick Collison, Suhail Doshi, Daniel
-Gackle, Jessica Livingston, and Robert Morris for reading drafts of this.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/nft.txt

@@ -1,26 +0,0 @@
-May 2021Noora Health, a nonprofit I've 
-supported for years, just launched
-a new NFT. It has a dramatic name, Save Thousands of Lives,
-because that's what the proceeds will do.Noora has been saving lives for 7 years. They run programs in
-hospitals in South Asia to teach new mothers how to take care of
-their babies once they get home. They're in 165 hospitals now. And
-because they know the numbers before and after they start at a new
-hospital, they can measure the impact they have. It is massive.
-For every 1000 live births, they save 9 babies.This number comes from a study
-of 133,733 families at 28 different
-hospitals that Noora conducted in collaboration with the Better
-Birth team at Ariadne Labs, a joint center for health systems
-innovation at Brigham and Women’s Hospital and Harvard T.H. Chan
-School of Public Health.Noora is so effective that even if you measure their costs in the
-most conservative way, by dividing their entire budget by the number
-of lives saved, the cost of saving a life is the lowest I've seen.
-$1,235.For this NFT, they're going to issue a public report tracking how
-this specific tranche of money is spent, and estimating the number
-of lives saved as a result.NFTs are a new territory, and this way of using them is especially
-new, but I'm excited about its potential. And I'm excited to see
-what happens with this particular auction, because unlike an NFT
-representing something that has already happened,
-this NFT gets better as the price gets higher.The reserve price was about $2.5 million, because that's what it
-takes for the name to be accurate: that's what it costs to save
-2000 lives. But the higher the price of this NFT goes, the more
-lives will be saved. What a sentence to be able to write.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/philosophy.txt

@@ -1,429 +0,0 @@
-September 2007In high school I decided I was going to study philosophy in college.
-I had several motives, some more honorable than others.  One of the
-less honorable was to shock people.  College was regarded as job
-training where I grew up, so studying philosophy seemed an impressively
-impractical thing to do.  Sort of like slashing holes in your clothes
-or putting a safety pin through your ear, which were other forms
-of impressive impracticality then just coming into fashion.But I had some more honest motives as well.  I thought studying
-philosophy would be a shortcut straight to wisdom.  All the people
-majoring in other things would just end up with a bunch of domain
-knowledge.  I would be learning what was really what.I'd tried to read a few philosophy books.  Not recent ones; you
-wouldn't find those in our high school library.  But I tried to
-read Plato and Aristotle.  I doubt I believed I understood them,
-but they sounded like they were talking about something important.
-I assumed I'd learn what in college.The summer before senior year I took some college classes.  I learned
-a lot in the calculus class, but I didn't learn much in Philosophy
-101.  And yet my plan to study philosophy remained intact.  It was
-my fault I hadn't learned anything.  I hadn't read the books we
-were assigned carefully enough.  I'd give Berkeley's Principles
-of Human Knowledge another shot in college.  Anything so admired
-and so difficult to read must have something in it, if one could
-only figure out what.Twenty-six years later, I still don't understand Berkeley.  I have
-a nice edition of his collected works.  Will I ever read it?  Seems
-unlikely.The difference between then and now is that now I understand why
-Berkeley is probably not worth trying to understand.  I think I see
-now what went wrong with philosophy, and how we might fix it.WordsI did end up being a philosophy major for most of college.  It
-didn't work out as I'd hoped.  I didn't learn any magical truths
-compared to which everything else was mere domain knowledge.  But
-I do at least know now why I didn't.  Philosophy doesn't really
-have a subject matter in the way math or history or most other
-university subjects do.  There is no core of knowledge one must
-master.  The closest you come to that is a knowledge of what various
-individual philosophers have said about different topics over the
-years.  Few were sufficiently correct that people have forgotten
-who discovered what they discovered.Formal logic has some subject matter. I took several classes in
-logic.  I don't know if I learned anything from them.
-[1]
-It does seem to me very important to be able to flip ideas around in
-one's head: to see when two ideas don't fully cover the space of
-possibilities, or when one idea is the same as another but with a
-couple things changed.  But did studying logic teach me the importance
-of thinking this way, or make me any better at it?  I don't know.There are things I know I learned from studying philosophy.  The
-most dramatic I learned immediately, in the first semester of
-freshman year, in a class taught by Sydney Shoemaker.  I learned
-that I don't exist.  I am (and you are) a collection of cells that
-lurches around driven by various forces, and calls itself I.  But
-there's no central, indivisible thing that your identity goes with.
-You could conceivably lose half your brain and live.  Which means
-your brain could conceivably be split into two halves and each
-transplanted into different bodies.  Imagine waking up after such
-an operation.  You have to imagine being two people.The real lesson here is that the concepts we use in everyday life
-are fuzzy, and break down if pushed too hard.  Even a concept as
-dear to us as I.  It took me a while to grasp this, but when I
-did it was fairly sudden, like someone in the nineteenth century
-grasping evolution and realizing the story of creation they'd been
-told as a child was all wrong. 
-[2]
-Outside of math there's a limit
-to how far you can push words; in fact, it would not be a bad
-definition of math to call it the study of terms that have precise
-meanings.  Everyday words are inherently imprecise.  They work well
-enough in everyday life that you don't notice.  Words seem to work,
-just as Newtonian physics seems to.  But you can always make them
-break if you push them far enough.I would say that this has been, unfortunately for philosophy, the
-central fact of philosophy.  Most philosophical debates are not
-merely afflicted by but driven by confusions over words.  Do we
-have free will?  Depends what you mean by "free." Do abstract ideas
-exist?  Depends what you mean by "exist."Wittgenstein is popularly credited with the idea that most philosophical
-controversies are due to confusions over language.  I'm not sure
-how much credit to give him.  I suspect a lot of people realized
-this, but reacted simply by not studying philosophy, rather than
-becoming philosophy professors.How did things get this way?  Can something people have spent
-thousands of years studying really be a waste of time?  Those are
-interesting questions.  In fact, some of the most interesting
-questions you can ask about philosophy.  The most valuable way to
-approach the current philosophical tradition may be neither to get
-lost in pointless speculations like Berkeley, nor to shut them down
-like Wittgenstein, but to study it as an example of reason gone
-wrong.HistoryWestern philosophy really begins with Socrates, Plato, and Aristotle.
-What we know of their predecessors comes from fragments and references
-in later works; their doctrines could be described as speculative
-cosmology that occasionally strays into analysis.  Presumably they
-were driven by whatever makes people in every other society invent
-cosmologies.
-[3]With Socrates, Plato, and particularly Aristotle, this tradition
-turned a corner.  There started to be a lot more analysis.  I suspect
-Plato and Aristotle were encouraged in this by progress in math.
-Mathematicians had by then shown that you could figure things out
-in a much more conclusive way than by making up fine sounding stories
-about them.  
-[4]People talk so much about abstractions now that we don't realize
-what a leap it must have been when they first started to.  It was
-presumably many thousands of years between when people first started
-describing things as hot or cold and when someone asked "what is
-heat?"  No doubt it was a very gradual process.  We don't know if
-Plato or Aristotle were the first to ask any of the questions they
-did.  But their works are the oldest we have that do this on a large
-scale, and there is a freshness (not to say naivete) about them
-that suggests some of the questions they asked were new to them,
-at least.Aristotle in particular reminds me of the phenomenon that happens
-when people discover something new, and are so excited by it that
-they race through a huge percentage of the newly discovered territory
-in one lifetime.  If so, that's evidence of how new this kind of
-thinking was. 
-[5]This is all to explain how Plato and Aristotle can be very impressive
-and yet naive and mistaken.  It was impressive even to ask the
-questions they did.  That doesn't mean they always came up with
-good answers.  It's not considered insulting to say that ancient
-Greek mathematicians were naive in some respects, or at least lacked
-some concepts that would have made their lives easier.  So I hope
-people will not be too offended if I propose that ancient philosophers
-were similarly naive.  In particular, they don't seem to have fully
-grasped what I earlier called the central fact of philosophy: that
-words break if you push them too far."Much to the surprise of the builders of the first digital computers,"
-Rod Brooks wrote, "programs written for them usually did not work."
-[6]
-Something similar happened when people first started trying
-to talk about abstractions.  Much to their surprise, they didn't
-arrive at answers they agreed upon.  In fact, they rarely seemed
-to arrive at answers at all.They were in effect arguing about artifacts induced by sampling at
-too low a resolution.The proof of how useless some of their answers turned out to be is
-how little effect they have.  No one after reading Aristotle's
-Metaphysics does anything differently as a result.
-[7]Surely I'm not claiming that ideas have to have practical applications
-to be interesting?  No, they may not have to.  Hardy's boast that
-number theory had no use whatsoever wouldn't disqualify it.  But
-he turned out to be mistaken.  In fact, it's suspiciously hard to
-find a field of math that truly has no practical use.  And Aristotle's
-explanation of the ultimate goal of philosophy in Book A of the
-Metaphysics implies that philosophy should be useful too.Theoretical KnowledgeAristotle's goal was to find the most general of general principles.
-The examples he gives are convincing: an ordinary worker builds
-things a certain way out of habit; a master craftsman can do more
-because he grasps the underlying principles.  The trend is clear:
-the more general the knowledge, the more admirable it is.  But then
-he makes a mistake—possibly the most important mistake in the
-history of philosophy.  He has noticed that theoretical knowledge
-is often acquired for its own sake, out of curiosity, rather than
-for any practical need.  So he proposes there are two kinds of
-theoretical knowledge: some that's useful in practical matters and
-some that isn't.  Since people interested in the latter are interested
-in it for its own sake, it must be more noble.  So he sets as his
-goal in the Metaphysics the exploration of knowledge that has no
-practical use.  Which means no alarms go off when he takes on grand
-but vaguely understood questions and ends up getting lost in a sea
-of words.His mistake was to confuse motive and result.  Certainly, people
-who want a deep understanding of something are often driven by
-curiosity rather than any practical need.  But that doesn't mean
-what they end up learning is useless.  It's very valuable in practice
-to have a deep understanding of what you're doing; even if you're
-never called on to solve advanced problems, you can see shortcuts
-in the solution of simple ones, and your knowledge won't break down
-in edge cases, as it would if you were relying on formulas you
-didn't understand.  Knowledge is power.  That's what makes theoretical
-knowledge prestigious.  It's also what causes smart people to be
-curious about certain things and not others; our DNA is not so
-disinterested as we might think.So while ideas don't have to have immediate practical applications
-to be interesting, the kinds of things we find interesting will
-surprisingly often turn out to have practical applications.The reason Aristotle didn't get anywhere in the Metaphysics was
-partly that he set off with contradictory aims: to explore the most
-abstract ideas, guided by the assumption that they were useless.
-He was like an explorer looking for a territory to the north of
-him, starting with the assumption that it was located to the south.And since his work became the map used by generations of future
-explorers, he sent them off in the wrong direction as well. 
-[8]
-Perhaps worst of all, he protected them from both the criticism of
-outsiders and the promptings of their own inner compass by establishing
-the principle that the most noble sort of theoretical knowledge had
-to be useless.The Metaphysics is mostly a failed experiment.  A few ideas from
-it turned out to be worth keeping; the bulk of it has had no effect
-at all.  The Metaphysics is among the least read of all famous
-books.  It's not hard to understand the way Newton's Principia
-is, but the way a garbled message is.Arguably it's an interesting failed experiment.  But unfortunately
-that was not the conclusion Aristotle's successors derived from
-works like the Metaphysics. 
-[9]
-Soon after, the western world
-fell on intellectual hard times.  Instead of version 1s to be
-superseded, the works of Plato and Aristotle became revered texts
-to be mastered and discussed.  And so things remained for a shockingly
-long time.  It was not till around 1600 (in Europe, where the center
-of gravity had shifted by then) that one found people confident
-enough to treat Aristotle's work as a catalog of mistakes.  And
-even then they rarely said so outright.If it seems surprising that the gap was so long, consider how little
-progress there was in math between Hellenistic times and the
-Renaissance.In the intervening years an unfortunate idea took hold:  that it
-was not only acceptable to produce works like the Metaphysics,
-but that it was a particularly prestigious line of work, done by a
-class of people called philosophers.  No one thought to go back and
-debug Aristotle's motivating argument.  And so instead of correcting
-the problem Aristotle discovered by falling into it—that you can
-easily get lost if you talk too loosely about very abstract ideas—they 
-continued to fall into it.The SingularityCuriously, however, the works they produced continued to attract
-new readers.  Traditional philosophy occupies a kind of singularity
-in this respect.  If you write in an unclear way about big ideas,
-you produce something that seems tantalizingly attractive to
-inexperienced but intellectually ambitious students.  Till one knows
-better, it's hard to distinguish something that's hard to understand
-because the writer was unclear in his own mind from something like
-a mathematical proof that's hard to understand because the ideas
-it represents are hard to understand.  To someone who hasn't learned
-the difference, traditional philosophy seems extremely attractive:
-as hard (and therefore impressive) as math, yet broader in scope.
-That was what lured me in as a high school student.This singularity is even more singular in having its own defense
-built in.  When things are hard to understand, people who suspect
-they're nonsense generally keep quiet.  There's no way to prove a
-text is meaningless.  The closest you can get is to show that the
-official judges of some class of texts can't distinguish them from
-placebos. 
-[10]And so instead of denouncing philosophy, most people who suspected
-it was a waste of time just studied other things.  That alone is
-fairly damning evidence, considering philosophy's claims.  It's
-supposed to be about the ultimate truths. Surely all smart people
-would be interested in it, if it delivered on that promise.Because philosophy's flaws turned away the sort of people who might
-have corrected them, they tended to be self-perpetuating.  Bertrand
-Russell wrote in a letter in 1912:
-
-  Hitherto the people attracted to philosophy have been mostly those
-  who loved the big generalizations, which were all wrong, so that
-  few people with exact minds have taken up the subject.
-[11]
-
-His response was to launch Wittgenstein at it, with dramatic results.I think Wittgenstein deserves to be famous not for the discovery
-that most previous philosophy was a waste of time, which judging
-from the circumstantial evidence must have been made by every smart
-person who studied a little philosophy and declined to pursue it
-further, but for how he acted in response.
-[12]
-Instead of quietly
-switching to another field, he made a fuss, from inside.  He was
-Gorbachev.The field of philosophy is still shaken from the fright Wittgenstein
-gave it. 
-[13]
-Later in life he spent a lot of time talking about
-how words worked.  Since that seems to be allowed, that's what a
-lot of philosophers do now.  Meanwhile, sensing a vacuum in the
-metaphysical speculation department, the people who used to do
-literary criticism have been edging Kantward, under new names like
-"literary theory," "critical theory," and when they're feeling
-ambitious, plain "theory."  The writing is the familiar word salad:
-
-  Gender is not like some of the other grammatical modes which
-  express precisely a mode of conception without any reality that
-  corresponds to the conceptual mode, and consequently do not express
-  precisely something in reality by which the intellect could be
-  moved to conceive a thing the way it does, even where that motive
-  is not something in the thing as such.
-  [14]
-
-The singularity I've described is not going away.  There's a market
-for writing that sounds impressive and can't be disproven. There
-will always be both supply and demand.  So if one group abandons
-this territory, there will always be others ready to occupy it.A ProposalWe may be able to do better.  Here's an intriguing possibility.
-Perhaps we should do what Aristotle meant to do, instead of what
-he did.  The goal he announces in the Metaphysics seems one worth
-pursuing: to discover the most general truths.  That sounds good.
-But instead of trying to discover them because they're useless,
-let's try to discover them because they're useful.I propose we try again, but that we use that heretofore despised
-criterion, applicability, as a guide to keep us from wondering
-off into a swamp of abstractions.  Instead of trying to answer the
-question:
-
-  What are the most general truths?
-
-let's try to answer the question
-
-  Of all the useful things we can say, which are the most general?
-
-The test of utility I propose is whether we cause people who read
-what we've written to do anything differently afterward.  Knowing
-we have to give definite (if implicit) advice will keep us from
-straying beyond the resolution of the words we're using.The goal is the same as Aristotle's; we just approach it from a
-different direction.As an example of a useful, general idea, consider that of the
-controlled experiment.  There's an idea that has turned out to be
-widely applicable.  Some might say it's part of science, but it's
-not part of any specific science; it's literally meta-physics (in
-our sense of "meta").   The idea of evolution is another. It turns
-out to have quite broad applications—for example, in genetic
-algorithms and even product design.  Frankfurt's distinction between
-lying and bullshitting seems a promising recent example.
-[15]These seem to me what philosophy should look like: quite general
-observations that would cause someone who understood them to do
-something differently.Such observations will necessarily be about things that are imprecisely
-defined.  Once you start using words with precise meanings, you're
-doing math.  So starting from utility won't entirely solve the
-problem I described above—it won't flush out the metaphysical
-singularity.  But it should help.  It gives people with good
-intentions a new roadmap into abstraction.  And they may thereby
-produce things that make the writing of the people with bad intentions
-look bad by comparison.One drawback of this approach is that it won't produce the sort of
-writing that gets you tenure.  And not just because it's not currently
-the fashion.  In order to get tenure in any field you must not
-arrive at conclusions that members of tenure committees can disagree
-with.  In practice there are two kinds of solutions to this problem.
-In math and the sciences, you can prove what you're saying, or at
-any rate adjust your conclusions so you're not claiming anything
-false ("6 of 8 subjects had lower blood pressure after the treatment").
-In the humanities you can either avoid drawing any definite conclusions
-(e.g. conclude that an issue is a complex one), or draw conclusions
-so narrow that no one cares enough to disagree with you.The kind of philosophy I'm advocating won't be able to take either
-of these routes.  At best you'll be able to achieve the essayist's
-standard of proof, not the mathematician's or the experimentalist's.
-And yet you won't be able to meet the usefulness test without
-implying definite and fairly broadly applicable conclusions.  Worse
-still, the usefulness test will tend to produce results that annoy
-people: there's no use in telling people things they already believe,
-and people are often upset to be told things they don't.Here's the exciting thing, though.  Anyone can do this.  Getting
-to general plus useful by starting with useful and cranking up the
-generality may be unsuitable for junior professors trying to get
-tenure, but it's better for everyone else, including professors who
-already have it.  This side of the mountain is a nice gradual slope.
-You can start by writing things that are useful but very specific,
-and then gradually make them more general.  Joe's has good burritos.
-What makes a good burrito?  What makes good food?  What makes
-anything good?  You can take as long as you want.  You don't have
-to get all the way to the top of the mountain.  You don't have to
-tell anyone you're doing philosophy.If it seems like a daunting task to do philosophy, here's an
-encouraging thought.  The field is a lot younger than it seems.
-Though the first philosophers in the western tradition lived about
-2500 years ago, it would be misleading to say the field is 2500
-years old, because for most of that time the leading practitioners
-weren't doing much more than writing commentaries on Plato or
-Aristotle while watching over their shoulders for the next invading
-army.  In the times when they weren't, philosophy was hopelessly
-intermingled with religion.  It didn't shake itself free till a
-couple hundred years ago, and even then was afflicted by the
-structural problems I've described above.  If I say this, some will
-say it's a ridiculously overbroad and uncharitable generalization,
-and others will say it's old news, but here goes: judging from their
-works, most philosophers up to the present have been wasting their
-time.  So in a sense the field is still at the first step. 
-[16]That sounds a preposterous claim to make.  It won't seem so
-preposterous in 10,000 years.  Civilization always seems old, because
-it's always the oldest it's ever been.  The only way to say whether
-something is really old or not is by looking at structural evidence,
-and structurally philosophy is young; it's still reeling from the
-unexpected breakdown of words.Philosophy is as young now as math was in 1500.  There is a lot
-more to discover.Notes
-[1]
-In practice formal logic is not much use, because despite
-some progress in the last 150 years we're still only able to formalize
-a small percentage of statements.  We may never do that much better,
-for the same reason 1980s-style "knowledge representation" could
-never have worked; many statements may have no representation more
-concise than a huge, analog brain state.[2]
-It was harder for Darwin's contemporaries to grasp this than
-we can easily imagine.  The story of creation in the Bible is not
-just a Judeo-Christian concept; it's roughly what everyone must
-have believed since before people were people.  The hard part of
-grasping evolution was to realize that species weren't, as they
-seem to be, unchanging, but had instead evolved from different,
-simpler organisms over unimaginably long periods of time.Now we don't have to make that leap.  No one in an industrialized
-country encounters the idea of evolution for the first time as an
-adult.  Everyone's taught about it as a child, either as truth or
-heresy.[3]
-Greek philosophers before Plato wrote in verse.  This must
-have affected what they said.  If you try to write about the nature
-of the world in verse, it inevitably turns into incantation.  Prose
-lets you be more precise, and more tentative.[4]
-Philosophy is like math's
-ne'er-do-well brother.  It was born when Plato and Aristotle looked
-at the works of their predecessors and said in effect "why can't
-you be more like your brother?"  Russell was still saying the same
-thing 2300 years later.Math is the precise half of the most abstract ideas, and philosophy
-the imprecise half.  It's probably inevitable that philosophy will
-suffer by comparison, because there's no lower bound to its precision.
-Bad math is merely boring, whereas bad philosophy is nonsense.  And
-yet there are some good ideas in the imprecise half.[5]
-Aristotle's best work was in logic and zoology, both of which
-he can  be said to have invented.  But the most dramatic departure
-from his predecessors was a new, much more analytical style of
-thinking.  He was arguably the first scientist.[6]
-Brooks, Rodney, Programming in Common Lisp, Wiley, 1985, p.
-94.[7]
-Some would say we depend on Aristotle more than we realize,
-because his ideas were one of the ingredients in our common culture.
-Certainly a lot of the words we use have a connection with Aristotle,
-but it seems a bit much to suggest that we wouldn't have the concept
-of the essence of something or the distinction between matter and
-form if Aristotle hadn't written about them.One way to see how much we really depend on Aristotle would be to
-diff European culture with Chinese: what ideas did European culture
-have in 1800 that Chinese culture didn't, in virtue of Aristotle's
-contribution?[8]
-The meaning of the word "philosophy" has changed over time.
-In ancient times it covered a broad range of topics, comparable in
-scope to our "scholarship" (though without the methodological
-implications).  Even as late as Newton's time it included what we
-now call "science."  But core of the subject today is still what
-seemed to Aristotle the core: the attempt to discover the most
-general truths.Aristotle didn't call this "metaphysics."  That name got assigned
-to it because the books we now call the Metaphysics came after
-(meta = after) the Physics in the standard edition of Aristotle's
-works compiled by Andronicus of Rhodes three centuries later.  What
-we call "metaphysics" Aristotle called "first philosophy."[9]
-Some of Aristotle's immediate successors may have realized
-this, but it's hard to say because most of their works are lost.[10]
-Sokal, Alan, "Transgressing the Boundaries: Toward a Transformative
-Hermeneutics of Quantum Gravity," Social Text 46/47, pp. 217-252.Abstract-sounding nonsense seems to be most attractive when it's
-aligned with some axe the audience already has to grind.  If this
-is so we should find it's most popular with groups that are (or
-feel) weak.  The powerful don't need its reassurance.[11]
-Letter to Ottoline Morrell, December 1912.  Quoted in:Monk, Ray, Ludwig Wittgenstein: The Duty of Genius, Penguin, 1991,
-p. 75.[12]
-A preliminary result, that all metaphysics between Aristotle
-and 1783 had been a waste of time, is due to I. Kant.[13]
-Wittgenstein asserted a sort of mastery to which the inhabitants
-of early 20th century Cambridge seem to have been peculiarly
-vulnerable—perhaps partly because so many had been raised religious
-and then stopped believing, so had a vacant space in their heads
-for someone to tell them what to do (others chose Marx or Cardinal
-Newman), and partly because a quiet, earnest place like Cambridge
-in that era had no natural immunity to messianic figures, just as
-European politics then had no natural immunity to dictators.[14]
-This is actually from the Ordinatio of Duns Scotus (ca.
-1300), with "number" replaced by "gender."  Plus ca change.Wolter, Allan (trans), Duns Scotus: Philosophical Writings, Nelson,
-1963, p. 92.[15]
-Frankfurt, Harry, On Bullshit,  Princeton University Press,
-2005.[16]
-Some introductions to philosophy now take the line that
-philosophy is worth studying as a process rather than for any
-particular truths you'll learn.  The philosophers whose works they
-cover would be rolling in their graves at that.  They hoped they
-were doing more than serving as examples of how to argue: they hoped
-they were getting results.  Most were wrong, but it doesn't seem
-an impossible hope.This argument seems to me like someone in 1500 looking at the lack
-of results achieved by alchemy and saying its value was as a process.
-No, they were going about it wrong.  It turns out it is possible
-to transmute lead into gold (though not economically at current
-energy prices), but the route to that knowledge was to
-backtrack and try another approach.Thanks to Trevor Blackwell, Paul Buchheit, Jessica Livingston, 
-Robert Morris, Mark Nitzberg, and Peter Norvig for reading drafts of this.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/popular.txt

@@ -1,602 +0,0 @@
-May 2001(This article was written as a kind of business plan for a
-new language.
-So it is missing (because it takes for granted) the most important
-feature of a good programming language: very powerful abstractions.)A friend of mine once told an eminent operating systems
-expert that he wanted to design a really good
-programming language.  The expert told him that it would be a
-waste of time, that programming languages don't become popular
-or unpopular based on their merits, and so no matter how
-good his language was, no one would use it.  At least, that
-was what had happened to the language he had designed.What does make a language popular?  Do popular
-languages deserve their popularity?  Is it worth trying to
-define a good programming language?  How would you do it?I think the answers to these questions can be found by looking 
-at hackers, and learning what they want.  Programming
-languages are for hackers, and a programming language
-is good as a programming language (rather than, say, an
-exercise in denotational semantics or compiler design)
-if and only if hackers like it.1 The Mechanics of PopularityIt's true, certainly, that most people don't choose programming
-languages simply based on their merits.  Most programmers are told
-what language to use by someone else.  And yet I think the effect
-of such external factors on the popularity of programming languages
-is not as great as it's sometimes thought to be. I think a bigger
-problem is that a hacker's idea of a good programming language is
-not the same as most language designers'.Between the two, the hacker's opinion is the one that matters.
-Programming languages are not theorems. They're tools, designed
-for people, and they have to be designed to suit human strengths
-and weaknesses as much as shoes have to be designed for human feet.
-If a shoe pinches when you put it on, it's a bad shoe, however
-elegant it may be as a piece of sculpture.It may be that the majority of programmers can't tell a good language
-from a bad one. But that's no different with any other tool. It
-doesn't mean that it's a waste of time to try designing a good
-language. Expert hackers 
-can tell a good language when they see
-one, and they'll use it. Expert hackers are a tiny minority,
-admittedly, but that tiny minority write all the good software,
-and their influence is such that the rest of the programmers will
-tend to use whatever language they use. Often, indeed, it is not
-merely influence but command: often the expert hackers are the very
-people who, as their bosses or faculty advisors, tell the other
-programmers what language to use.The opinion of expert hackers is not the only force that determines
-the relative popularity of programming languages — legacy software
-(Cobol) and hype (Ada, Java) also play a role — but I think it is
-the most powerful force over the long term. Given an initial critical
-mass and enough time, a programming language probably becomes about
-as popular as it deserves to be. And popularity further separates
-good languages from bad ones, because feedback from real live users
-always leads to improvements. Look at how much any popular language
-has changed during its life. Perl and Fortran are extreme cases,
-but even Lisp has changed a lot. Lisp 1.5 didn't have macros, for
-example; these evolved later, after hackers at MIT had spent a
-couple years using Lisp to write real programs. [1]So whether or not a language has to be good to be popular, I think
-a language has to be popular to be good. And it has to stay popular
-to stay good. The state of the art in programming languages doesn't
-stand still. And yet the Lisps we have today are still pretty much
-what they had at MIT in the mid-1980s, because that's the last time
-Lisp had a sufficiently large and demanding user base.Of course, hackers have to know about a language before they can
-use it. How are they to hear? From other hackers. But there has to
-be some initial group of hackers using the language for others even
-to hear about it. I wonder how large this group has to be; how many
-users make a critical mass? Off the top of my head, I'd say twenty.
-If a language had twenty separate users, meaning twenty users who
-decided on their own to use it, I'd consider it to be real.Getting there can't be easy. I would not be surprised if it is
-harder to get from zero to twenty than from twenty to a thousand.
-The best way to get those initial twenty users is probably to use
-a trojan horse: to give people an application they want, which
-happens to be written in the new language.2 External FactorsLet's start by acknowledging one external factor that does affect
-the popularity of a programming language. To become popular, a
-programming language has to be the scripting language of a popular
-system. Fortran and Cobol were the scripting languages of early
-IBM mainframes. C was the scripting language of Unix, and so, later,
-was Perl. Tcl is the scripting language of Tk. Java and Javascript
-are intended to be the scripting languages of web browsers.Lisp is not a massively popular language because it is not the
-scripting language of a massively popular system. What popularity
-it retains dates back to the 1960s and 1970s, when it was the
-scripting language of MIT. A lot of the great programmers of the
-day were associated with MIT at some point. And in the early 1970s,
-before C, MIT's dialect of Lisp, called MacLisp, was one of the
-only programming languages a serious hacker would want to use.Today Lisp is the scripting language of two moderately popular
-systems, Emacs and Autocad, and for that reason I suspect that most
-of the Lisp programming done today is done in Emacs Lisp or AutoLisp.Programming languages don't exist in isolation. To hack is a
-transitive verb — hackers are usually hacking something — and in
-practice languages are judged relative to whatever they're used to
-hack. So if you want to design a popular language, you either have
-to supply more than a language, or you have to design your language
-to replace the scripting language of some existing system.Common Lisp is unpopular partly because it's an orphan. It did
-originally come with a system to hack: the Lisp Machine. But Lisp
-Machines (along with parallel computers) were steamrollered by the
-increasing power of general purpose processors in the 1980s. Common
-Lisp might have remained popular if it had been a good scripting
-language for Unix. It is, alas, an atrociously bad one.One way to describe this situation is to say that a language isn't
-judged on its own merits. Another view is that a programming language
-really isn't a programming language unless it's also the scripting
-language of something. This only seems unfair if it comes as a
-surprise. I think it's no more unfair than expecting a programming
-language to have, say, an implementation. It's just part of what
-a programming language is.A programming language does need a good implementation, of course,
-and this must be free. Companies will pay for software, but individual
-hackers won't, and it's the hackers you need to attract.A language also needs to have a book about it. The book should be
-thin, well-written, and full of good examples. K&R is the ideal
-here. At the moment I'd almost say that a language has to have a
-book published by O'Reilly. That's becoming the test of mattering
-to hackers.There should be online documentation as well. In fact, the book
-can start as online documentation. But I don't think that physical
-books are outmoded yet. Their format is convenient, and the de
-facto censorship imposed by publishers is a useful if imperfect
-filter. Bookstores are one of the most important places for learning
-about new languages.3 BrevityGiven that you can supply the three things any language needs — a
-free implementation, a book, and something to hack — how do you
-make a language that hackers will like?One thing hackers like is brevity. Hackers are lazy, in the same
-way that mathematicians and modernist architects are lazy: they
-hate anything extraneous. It would not be far from the truth to
-say that a hacker about to write a program decides what language
-to use, at least subconsciously, based on the total number of
-characters he'll have to type. If this isn't precisely how hackers
-think, a language designer would do well to act as if it were.It is a mistake to try to baby the user with long-winded expressions
-that are meant to resemble English. Cobol is notorious for this
-flaw. A hacker would consider being asked to writeadd x to y giving zinstead ofz = x+yas something between an insult to his intelligence and a sin against
-God.It has sometimes been said that Lisp should use first and rest
-instead of car and cdr, because it would make programs easier to
-read. Maybe for the first couple hours. But a hacker can learn
-quickly enough that car means the first element of a list and cdr
-means the rest. Using first and rest means 50% more typing. And
-they are also different lengths, meaning that the arguments won't
-line up when they're called, as car and cdr often are, in successive
-lines. I've found that it matters a lot how code lines up on the
-page. I can barely read Lisp code when it is set in a variable-width
-font, and friends say this is true for other languages too.Brevity is one place where strongly typed languages lose. All other
-things being equal, no one wants to begin a program with a bunch
-of declarations. Anything that can be implicit, should be.The individual tokens should be short as well. Perl and Common Lisp
-occupy opposite poles on this question. Perl programs can be almost
-cryptically dense, while the names of built-in Common Lisp operators
-are comically long. The designers of Common Lisp probably expected
-users to have text editors that would type these long names for
-them. But the cost of a long name is not just the cost of typing
-it. There is also the cost of reading it, and the cost of the space
-it takes up on your screen.4 HackabilityThere is one thing more important than brevity to a hacker: being
-able to do what you want. In the history of programming languages
-a surprising amount of effort has gone into preventing programmers
-from doing things considered to be improper. This is a dangerously
-presumptuous plan. How can the language designer know what the
-programmer is going to need to do? I think language designers would
-do better to consider their target user to be a genius who will
-need to do things they never anticipated, rather than a bumbler
-who needs to be protected from himself. The bumbler will shoot
-himself in the foot anyway. You may save him from referring to
-variables in another package, but you can't save him from writing
-a badly designed program to solve the wrong problem, and taking
-forever to do it.Good programmers often want to do dangerous and unsavory things.
-By unsavory I mean things that go behind whatever semantic facade
-the language is trying to present: getting hold of the internal
-representation of some high-level abstraction, for example. Hackers
-like to hack, and hacking means getting inside things and second
-guessing the original designer.Let yourself be second guessed. When you make any tool, people use
-it in ways you didn't intend, and this is especially true of a
-highly articulated tool like a programming language. Many a hacker
-will want to tweak your semantic model in a way that you never
-imagined. I say, let them; give the programmer access to as much
-internal stuff as you can without endangering runtime systems like
-the garbage collector.In Common Lisp I have often wanted to iterate through the fields
-of a struct — to comb out references to a deleted object, for example,
-or find fields that are uninitialized. I know the structs are just
-vectors underneath. And yet I can't write a general purpose function
-that I can call on any struct. I can only access the fields by
-name, because that's what a struct is supposed to mean.A hacker may only want to subvert the intended model of things once
-or twice in a big program. But what a difference it makes to be
-able to. And it may be more than a question of just solving a
-problem. There is a kind of pleasure here too. Hackers share the
-surgeon's secret pleasure in poking about in gross innards, the
-teenager's secret pleasure in popping zits. [2] For boys, at least,
-certain kinds of horrors are fascinating. Maxim magazine publishes
-an annual volume of photographs, containing a mix of pin-ups and
-grisly accidents. They know their audience.Historically, Lisp has been good at letting hackers have their way.
-The political correctness of Common Lisp is an aberration. Early
-Lisps let you get your hands on everything. A good deal of that
-spirit is, fortunately, preserved in macros. What a wonderful thing,
-to be able to make arbitrary transformations on the source code.Classic macros are a real hacker's tool — simple, powerful, and
-dangerous. It's so easy to understand what they do: you call a
-function on the macro's arguments, and whatever it returns gets
-inserted in place of the macro call. Hygienic macros embody the
-opposite principle. They try to protect you from understanding what
-they're doing. I have never heard hygienic macros explained in one
-sentence. And they are a classic example of the dangers of deciding
-what programmers are allowed to want. Hygienic macros are intended
-to protect me from variable capture, among other things, but variable
-capture is exactly what I want in some macros.A really good language should be both clean and dirty: cleanly
-designed, with a small core of well understood and highly orthogonal
-operators, but dirty in the sense that it lets hackers have their
-way with it. C is like this. So were the early Lisps. A real hacker's
-language will always have a slightly raffish character.A good programming language should have features that make the kind
-of people who use the phrase "software engineering" shake their
-heads disapprovingly. At the other end of the continuum are languages
-like Ada and Pascal, models of propriety that are good for teaching
-and not much else.5 Throwaway ProgramsTo be attractive to hackers, a language must be good for writing
-the kinds of programs they want to write. And that means, perhaps
-surprisingly, that it has to be good for writing throwaway programs.A throwaway program is a program you write quickly for some limited
-task: a program to automate some system administration task, or
-generate test data for a simulation, or convert data from one format
-to another. The surprising thing about throwaway programs is that,
-like the "temporary" buildings built at so many American universities
-during World War II, they often don't get thrown away. Many evolve
-into real programs, with real features and real users.I have a hunch that the best big programs begin life this way,
-rather than being designed big from the start, like the Hoover Dam.
-It's terrifying to build something big from scratch. When people
-take on a project that's too big, they become overwhelmed. The
-project either gets bogged down, or the result is sterile and
-wooden: a shopping mall rather than a real downtown, Brasilia rather
-than Rome, Ada rather than C.Another way to get a big program is to start with a throwaway
-program and keep improving it. This approach is less daunting, and
-the design of the program benefits from evolution. I think, if one
-looked, that this would turn out to be the way most big programs
-were developed. And those that did evolve this way are probably
-still written in whatever language they were first written in,
-because it's rare for a program to be ported, except for political
-reasons. And so, paradoxically, if you want to make a language that
-is used for big systems, you have to make it good for writing
-throwaway programs, because that's where big systems come from.Perl is a striking example of this idea. It was not only designed
-for writing throwaway programs, but was pretty much a throwaway
-program itself. Perl began life as a collection of utilities for
-generating reports, and only evolved into a programming language
-as the throwaway programs people wrote in it grew larger. It was
-not until Perl 5 (if then) that the language was suitable for
-writing serious programs, and yet it was already massively popular.What makes a language good for throwaway programs? To start with,
-it must be readily available. A throwaway program is something that
-you expect to write in an hour. So the language probably must
-already be installed on the computer you're using. It can't be
-something you have to install before you use it. It has to be there.
-C was there because it came with the operating system. Perl was
-there because it was originally a tool for system administrators,
-and yours had already installed it.Being available means more than being installed, though. An
-interactive language, with a command-line interface, is more
-available than one that you have to compile and run separately. A
-popular programming language should be interactive, and start up
-fast.Another thing you want in a throwaway program is brevity. Brevity
-is always attractive to hackers, and never more so than in a program
-they expect to turn out in an hour.6 LibrariesOf course the ultimate in brevity is to have the program already
-written for you, and merely to call it. And this brings us to what
-I think will be an increasingly important feature of programming
-languages: library functions. Perl wins because it has large
-libraries for manipulating strings. This class of library functions
-are especially important for throwaway programs, which are often
-originally written for converting or extracting data.  Many Perl
-programs probably begin as just a couple library calls stuck
-together.I think a lot of the advances that happen in programming languages
-in the next fifty years will have to do with library functions. I
-think future programming languages will have libraries that are as
-carefully designed as the core language. Programming language design
-will not be about whether to make your language strongly or weakly
-typed, or object oriented, or functional, or whatever, but about
-how to design great libraries. The kind of language designers who
-like to think about how to design type systems may shudder at this.
-It's almost like writing applications! Too bad. Languages are for
-programmers, and libraries are what programmers need.It's hard to design good libraries. It's not simply a matter of
-writing a lot of code. Once the libraries get too big, it can
-sometimes take longer to find the function you need than to write
-the code yourself. Libraries need to be designed using a small set
-of orthogonal operators, just like the core language. It ought to
-be possible for the programmer to guess what library call will do
-what he needs.Libraries are one place Common Lisp falls short. There are only
-rudimentary libraries for manipulating strings, and almost none
-for talking to the operating system. For historical reasons, Common
-Lisp tries to pretend that the OS doesn't exist. And because you
-can't talk to the OS, you're unlikely to be able to write a serious
-program using only the built-in operators in Common Lisp. You have
-to use some implementation-specific hacks as well, and in practice
-these tend not to give you everything you want. Hackers would think
-a lot more highly of Lisp if Common Lisp had powerful string
-libraries and good OS support.7 SyntaxCould a language with Lisp's syntax, or more precisely, lack of
-syntax, ever become popular? I don't know the answer to this
-question. I do think that syntax is not the main reason Lisp isn't
-currently popular. Common Lisp has worse problems than unfamiliar
-syntax. I know several programmers who are comfortable with prefix
-syntax and yet use Perl by default, because it has powerful string
-libraries and can talk to the os.There are two possible problems with prefix notation: that it is
-unfamiliar to programmers, and that it is not dense enough. The
-conventional wisdom in the Lisp world is that the first problem is
-the real one. I'm not so sure. Yes, prefix notation makes ordinary
-programmers panic. But I don't think ordinary programmers' opinions
-matter. Languages become popular or unpopular based on what expert
-hackers think of them, and I think expert hackers might be able to
-deal with prefix notation. Perl syntax can be pretty incomprehensible,
-but that has not stood in the way of Perl's popularity. If anything
-it may have helped foster a Perl cult.A more serious problem is the diffuseness of prefix notation. For
-expert hackers, that really is a problem. No one wants to write
-(aref a x y) when they could write a[x,y].In this particular case there is a way to finesse our way out of
-the problem. If we treat data structures as if they were functions
-on indexes, we could write (a x y) instead, which is even shorter
-than the Perl form. Similar tricks may shorten other types of
-expressions.We can get rid of (or make optional) a lot of parentheses by making
-indentation significant. That's how programmers read code anyway:
-when indentation says one thing and delimiters say another, we go
-by the indentation. Treating indentation as significant would
-eliminate this common source of bugs as well as making programs
-shorter.Sometimes infix syntax is easier to read. This is especially true
-for math expressions. I've used Lisp my whole programming life and
-I still don't find prefix math expressions natural. And yet it is
-convenient, especially when you're generating code, to have operators
-that take any number of arguments. So if we do have infix syntax,
-it should probably be implemented as some kind of read-macro.I don't think we should be religiously opposed to introducing syntax
-into Lisp, as long as it translates in a well-understood way into
-underlying s-expressions. There is already a good deal of syntax
-in Lisp. It's not necessarily bad to introduce more, as long as no
-one is forced to use it. In Common Lisp, some delimiters are reserved
-for the language, suggesting that at least some of the designers
-intended to have more syntax in the future.One of the most egregiously unlispy pieces of syntax in Common Lisp
-occurs in format strings; format is a language in its own right,
-and that language is not Lisp. If there were a plan for introducing
-more syntax into Lisp, format specifiers might be able to be included
-in it. It would be a good thing if macros could generate format
-specifiers the way they generate any other kind of code.An eminent Lisp hacker told me that his copy of CLTL falls open to
-the section format. Mine too. This probably indicates room for
-improvement. It may also mean that programs do a lot of I/O.8 EfficiencyA good language, as everyone knows, should generate fast code. But
-in practice I don't think fast code comes primarily from things
-you do in the design of the language. As Knuth pointed out long
-ago, speed only matters in certain critical bottlenecks.  And as
-many programmers have observed since, one is very often mistaken
-about where these bottlenecks are.So, in practice, the way to get fast code is to have a very good
-profiler, rather than by, say, making the language strongly typed.
-You don't need to know the type of every argument in every call in
-the program. You do need to be able to declare the types of arguments
-in the bottlenecks. And even more, you need to be able to find out
-where the bottlenecks are.One complaint people have had with Lisp is that it's hard to tell
-what's expensive. This might be true. It might also be inevitable,
-if you want to have a very abstract language. And in any case I
-think good profiling would go a long way toward fixing the problem:
-you'd soon learn what was expensive.Part of the problem here is social. Language designers like to
-write fast compilers. That's how they measure their skill. They
-think of the profiler as an add-on, at best. But in practice a good
-profiler may do more to improve the speed of actual programs written
-in the language than a compiler that generates fast code. Here,
-again, language designers are somewhat out of touch with their
-users. They do a really good job of solving slightly the wrong
-problem.It might be a good idea to have an active profiler — to push
-performance data to the programmer instead of waiting for him to
-come asking for it. For example, the editor could display bottlenecks
-in red when the programmer edits the source code. Another approach
-would be to somehow represent what's happening in running programs.
-This would be an especially big win in server-based applications,
-where you have lots of running programs to look at. An active
-profiler could show graphically what's happening in memory as a
-program's running, or even make sounds that tell what's happening.Sound is a good cue to problems. In one place I worked, we had a
-big board of dials showing what was happening to our web servers.
-The hands were moved by little servomotors that made a slight noise
-when they turned. I couldn't see the board from my desk, but I
-found that I could tell immediately, by the sound, when there was
-a problem with a server.It might even be possible to write a profiler that would automatically
-detect inefficient algorithms. I would not be surprised if certain
-patterns of memory access turned out to be sure signs of bad
-algorithms. If there were a little guy running around inside the
-computer executing our programs, he would probably have as long
-and plaintive a tale to tell about his job as a federal government
-employee. I often have a feeling that I'm sending the processor on
-a lot of wild goose chases, but I've never had a good way to look
-at what it's doing.A number of Lisps now compile into byte code, which is then executed
-by an interpreter. This is usually done to make the implementation
-easier to port, but it could be a useful language feature. It might
-be a good idea to make the byte code an official part of the
-language, and to allow programmers to use inline byte code in
-bottlenecks. Then such optimizations would be portable too.The nature of speed, as perceived by the end-user, may be changing.
-With the rise of server-based applications, more and more programs
-may turn out to be i/o-bound. It will be worth making i/o fast.
-The language can help with straightforward measures like simple,
-fast, formatted output functions, and also with deep structural
-changes like caching and persistent objects.Users are interested in response time. But another kind of efficiency
-will be increasingly important: the number of simultaneous users
-you can support per processor. Many of the interesting applications
-written in the near future will be server-based, and the number of
-users per server is the critical question for anyone hosting such
-applications. In the capital cost of a business offering a server-based
-application, this is the divisor.For years, efficiency hasn't mattered much in most end-user
-applications. Developers have been able to assume that each user
-would have an increasingly powerful processor sitting on their
-desk. And by Parkinson's Law, software has expanded to use the
-resources available. That will change with server-based applications.
-In that world, the hardware and software will be supplied together.
-For companies that offer server-based applications, it will make
-a very big difference to the bottom line how many users they can
-support per server.In some applications, the processor will be the limiting factor,
-and execution speed will be the most important thing to optimize.
-But often memory will be the limit; the number of simultaneous
-users will be determined by the amount of memory you need for each
-user's data. The language can help here too. Good support for
-threads will enable all the users to share a single heap. It may
-also help to have persistent objects and/or language level support
-for lazy loading.9 TimeThe last ingredient a popular language needs is time. No one wants
-to write programs in a language that might go away, as so many
-programming languages do. So most hackers will tend to wait until
-a language has been around for a couple years before even considering
-using it.Inventors of wonderful new things are often surprised to discover
-this, but you need time to get any message through to people. A
-friend of mine rarely does anything the first time someone asks
-him. He knows that people sometimes ask for things that they turn
-out not to want. To avoid wasting his time, he waits till the third
-or fourth time he's asked to do something; by then, whoever's asking
-him may be fairly annoyed, but at least they probably really do
-want whatever they're asking for.Most people have learned to do a similar sort of filtering on new
-things they hear about. They don't even start paying attention
-until they've heard about something ten times. They're perfectly
-justified: the majority of hot new whatevers do turn out to be a
-waste of time, and eventually go away. By delaying learning VRML,
-I avoided having to learn it at all.So anyone who invents something new has to expect to keep repeating
-their message for years before people will start to get it. We
-wrote what was, as far as I know, the first web-server based
-application, and it took us years to get it through to people that
-it didn't have to be downloaded. It wasn't that they were stupid.
-They just had us tuned out.The good news is, simple repetition solves the problem. All you
-have to do is keep telling your story, and eventually people will
-start to hear. It's not when people notice you're there that they
-pay attention; it's when they notice you're still there.It's just as well that it usually takes a while to gain momentum.
-Most technologies evolve a good deal even after they're first
-launched — programming languages especially. Nothing could be better,
-for a new techology, than a few years of being used only by a small
-number of early adopters. Early adopters are sophisticated and
-demanding, and quickly flush out whatever flaws remain in your
-technology. When you only have a few users you can be in close
-contact with all of them. And early adopters are forgiving when
-you improve your system, even if this causes some breakage.There are two ways new technology gets introduced: the organic
-growth method, and the big bang method. The organic growth method
-is exemplified by the classic seat-of-the-pants underfunded garage
-startup. A couple guys, working in obscurity, develop some new
-technology. They launch it with no marketing and initially have
-only a few (fanatically devoted) users. They continue to improve
-the technology, and meanwhile their user base grows by word of
-mouth. Before they know it, they're big.The other approach, the big bang method, is exemplified by the
-VC-backed, heavily marketed startup. They rush to develop a product,
-launch it with great publicity, and immediately (they hope) have
-a large user base.Generally, the garage guys envy the big bang guys. The big bang
-guys are smooth and confident and respected by the VCs. They can
-afford the best of everything, and the PR campaign surrounding the
-launch has the side effect of making them celebrities. The organic
-growth guys, sitting in their garage, feel poor and unloved. And
-yet I think they are often mistaken to feel sorry for themselves.
-Organic growth seems to yield better technology and richer founders
-than the big bang method. If you look at the dominant technologies
-today, you'll find that most of them grew organically.This pattern doesn't only apply to companies. You see it in sponsored
-research too. Multics and Common Lisp were big-bang projects, and
-Unix and MacLisp were organic growth projects.10 Redesign"The best writing is rewriting," wrote E. B. White.  Every good
-writer knows this, and it's true for software too. The most important
-part of design is redesign. Programming languages, especially,
-don't get redesigned enough.To write good software you must simultaneously keep two opposing
-ideas in your head. You need the young hacker's naive faith in
-his abilities, and at the same time the veteran's skepticism. You
-have to be able to think 
-how hard can it be? with one half of
-your brain while thinking 
-it will never work with the other.The trick is to realize that there's no real contradiction here.
-You want to be optimistic and skeptical about two different things.
-You have to be optimistic about the possibility of solving the
-problem, but skeptical about the value of whatever solution you've
-got so far.People who do good work often think that whatever they're working
-on is no good. Others see what they've done and are full of wonder,
-but the creator is full of worry. This pattern is no coincidence:
-it is the worry that made the work good.If you can keep hope and worry balanced, they will drive a project
-forward the same way your two legs drive a bicycle forward. In the
-first phase of the two-cycle innovation engine, you work furiously
-on some problem, inspired by your confidence that you'll be able
-to solve it. In the second phase, you look at what you've done in
-the cold light of morning, and see all its flaws very clearly. But
-as long as your critical spirit doesn't outweigh your hope, you'll
-be able to look at your admittedly incomplete system, and think,
-how hard can it be to get the rest of the way?, thereby continuing
-the cycle.It's tricky to keep the two forces balanced. In young hackers,
-optimism predominates. They produce something, are convinced it's
-great, and never improve it. In old hackers, skepticism predominates,
-and they won't even dare to take on ambitious projects.Anything you can do to keep the redesign cycle going is good. Prose
-can be rewritten over and over until you're happy with it. But
-software, as a rule, doesn't get redesigned enough. Prose has
-readers, but software has users. If a writer rewrites an essay,
-people who read the old version are unlikely to complain that their
-thoughts have been broken by some newly introduced incompatibility.Users are a double-edged sword. They can help you improve your
-language, but they can also deter you from improving it. So choose
-your users carefully, and be slow to grow their number. Having
-users is like optimization: the wise course is to delay it. Also,
-as a general rule, you can at any given time get away with changing
-more than you think. Introducing change is like pulling off a
-bandage: the pain is a memory almost as soon as you feel it.Everyone knows that it's not a good idea to have a language designed
-by a committee. Committees yield bad design. But I think the worst
-danger of committees is that they interfere with redesign. It is
-so much work to introduce changes that no one wants to bother.
-Whatever a committee decides tends to stay that way, even if most
-of the members don't like it.Even a committee of two gets in the way of redesign. This happens
-particularly in the interfaces between pieces of software written
-by two different people. To change the interface both have to agree
-to change it at once. And so interfaces tend not to change at all,
-which is a problem because they tend to be one of the most ad hoc
-parts of any system.One solution here might be to design systems so that interfaces
-are horizontal instead of vertical — so that modules are always
-vertically stacked strata of abstraction. Then the interface will
-tend to be owned by one of them. The lower of two levels will either
-be a language in which the upper is written, in which case the
-lower level will own the interface, or it will be a slave, in which
-case the interface can be dictated by the upper level.11 LispWhat all this implies is that there is hope for a new Lisp.  There
-is hope for any language that gives hackers what they want, including
-Lisp. I think we may have made a mistake in thinking that hackers
-are turned off by Lisp's strangeness. This comforting illusion may
-have prevented us from seeing the real problem with Lisp, or at
-least Common Lisp, which is that it sucks for doing what hackers
-want to do. A hacker's language needs powerful libraries and
-something to hack. Common Lisp has neither. A hacker's language is
-terse and hackable. Common Lisp is not.The good news is, it's not Lisp that sucks, but Common Lisp. If we
-can develop a new Lisp that is a real hacker's language, I think
-hackers will use it. They will use whatever language does the job.
-All we have to do is make sure this new Lisp does some important
-job better than other languages.History offers some encouragement. Over time, successive new
-programming languages have taken more and more features from Lisp.
-There is no longer much left to copy before the language you've
-made is Lisp. The latest hot language, Python, is a watered-down
-Lisp with infix syntax and no macros. A new Lisp would be a natural
-step in this progression.I sometimes think that it would be a good marketing trick to call
-it an improved version of Python. That sounds hipper than Lisp. To
-many people, Lisp is a slow AI language with a lot of parentheses.
-Fritz Kunze's official biography carefully avoids mentioning the
-L-word.  But my guess is that we shouldn't be afraid to call the
-new Lisp Lisp. Lisp still has a lot of latent respect among the
-very best hackers — the ones who took 6.001 and understood it, for
-example. And those are the users you need to win.In "How to Become a Hacker," Eric Raymond describes Lisp as something
-like Latin or Greek — a language you should learn as an intellectual
-exercise, even though you won't actually use it:
-
-  Lisp is worth learning for the profound enlightenment experience
-  you will have when you finally get it; that experience will make
-  you a better programmer for the rest of your days, even if you
-  never actually use Lisp itself a lot.
-
-If I didn't know Lisp, reading this would set me asking questions.
-A language that would make me a better programmer, if it means
-anything at all, means a language that would be better for programming.
-And that is in fact the implication of what Eric is saying.As long as that idea is still floating around, I think hackers will
-be receptive enough to a new Lisp, even if it is called Lisp. But
-this Lisp must be a hacker's language, like the classic Lisps of
-the 1970s. It must be terse, simple, and hackable. And it must have
-powerful libraries for doing what hackers want to do now.In the matter of libraries I think there is room to beat languages
-like Perl and Python at their own game. A lot of the new applications
-that will need to be written in the coming years will be 
-server-based
-applications. There's no reason a new Lisp shouldn't have string
-libraries as good as Perl, and if this new Lisp also had powerful
-libraries for server-based applications, it could be very popular.
-Real hackers won't turn up their noses at a new tool that will let
-them solve hard problems with a few library calls. Remember, hackers
-are lazy.It could be an even bigger win to have core language support for
-server-based applications. For example, explicit support for programs
-with multiple users, or data ownership at the level of type tags.Server-based applications also give us the answer to the question
-of what this new Lisp will be used to hack. It would not hurt to
-make Lisp better as a scripting language for Unix. (It would be
-hard to make it worse.) But I think there are areas where existing
-languages would be easier to beat. I think it might be better to
-follow the model of Tcl, and supply the Lisp together with a complete
-system for supporting server-based applications. Lisp is a natural
-fit for server-based applications. Lexical closures provide a way
-to get the effect of subroutines when the ui is just a series of
-web pages. S-expressions map nicely onto html, and macros are good
-at generating it. There need to be better tools for writing
-server-based applications, and there needs to be a new Lisp, and
-the two would work very well together.12 The Dream LanguageBy way of summary, let's try describing the hacker's dream language.
-The dream language is 
-beautiful, clean, and terse. It has an
-interactive toplevel that starts up fast. You can write programs
-to solve common problems with very little code.  Nearly all the
-code in any program you write is code that's specific to your
-application. Everything else has been done for you.The syntax of the language is brief to a fault. You never have to
-type an unnecessary character, or even to use the shift key much.Using big abstractions you can write the first version of a program
-very quickly. Later, when you want to optimize, there's a really
-good profiler that tells you where to focus your attention. You
-can make inner loops blindingly fast, even writing inline byte code
-if you need to.There are lots of good examples to learn from, and the language is
-intuitive enough that you can learn how to use it from examples in
-a couple minutes. You don't need to look in the manual much. The
-manual is thin, and has few warnings and qualifications.The language has a small core, and powerful, highly orthogonal
-libraries that are as carefully designed as the core language. The
-libraries all work well together; everything in the language fits
-together like the parts in a fine camera. Nothing is deprecated,
-or retained for compatibility. The source code of all the libraries
-is readily available. It's easy to talk to the operating system
-and to applications written in other languages.The language is built in layers. The higher-level abstractions are
-built in a very transparent way out of lower-level abstractions,
-which you can get hold of if you want.Nothing is hidden from you that doesn't absolutely have to be. The
-language offers abstractions only as a way of saving you work,
-rather than as a way of telling you what to do. In fact, the language
-encourages you to be an equal participant in its design. You can
-change everything about it, including even its syntax, and anything
-you write has, as much as possible, the same status as what comes
-predefined.Notes[1]  Macros very close to the modern idea were proposed by Timothy
-Hart in 1964, two years after Lisp 1.5 was released. What was
-missing, initially, were ways to avoid variable capture and multiple
-evaluation; Hart's examples are subject to both.[2]  In When the Air Hits Your Brain, neurosurgeon Frank Vertosick
-recounts a conversation in which his chief resident, Gary, talks
-about the difference between surgeons and internists ("fleas"):
-
-  Gary and I ordered a large pizza and found an open booth. The
-  chief lit a cigarette. "Look at those goddamn fleas, jabbering
-  about some disease they'll see once in their lifetimes. That's
-  the trouble with fleas, they only like the bizarre stuff. They
-  hate their bread and butter cases. That's the difference between
-  us and the fucking fleas. See, we love big juicy lumbar disc
-  herniations, but they hate hypertension...."
-
-It's hard to think of a lumbar disc herniation as juicy (except
-literally). And yet I think I know what they mean. I've often had
-a juicy bug to track down. Someone who's not a programmer would
-find it hard to imagine that there could be pleasure in a bug.
-Surely it's better if everything just works. In one way, it is.
-And yet there is undeniably a grim satisfaction in hunting down
-certain sorts of bugs.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/pow.txt

@@ -1,10 +0,0 @@
-January 2017People who are powerful but uncharismatic will tend to be disliked.
-Their power makes them a target for criticism that they don't have
-the charisma to disarm. That was Hillary Clinton's problem. It also
-tends to be a problem for any CEO who is more of a builder than a
-schmoozer. And yet the builder-type CEO is (like Hillary) probably
-the best person for the job.I don't think there is any solution to this problem. It's human
-nature. The best we can do is to recognize that it's happening, and
-to understand that being a magnet for criticism is sometimes a sign
-not that someone is the wrong person for a job, but that they're
-the right one.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/rootsoflisp.txt

@@ -1,41 +0,0 @@
-May 2001
-
-(I wrote this article to help myself understand exactly
-what McCarthy discovered.  You don't need to know this stuff
-to program in Lisp, but it should be helpful to 
-anyone who wants to
-understand the essence of Lisp — both in the sense of its
-origins and its semantic core.  The fact that it has such a core
-is one of Lisp's distinguishing features, and the reason why,
-unlike other languages, Lisp has dialects.)In 1960, John 
-McCarthy published a remarkable paper in
-which he did for programming something like what Euclid did for
-geometry. He showed how, given a handful of simple
-operators and a notation for functions, you can
-build a whole programming language.
-He called this language Lisp, for "List Processing,"
-because one of his key ideas was to use a simple
-data structure called a list for both
-code and data.It's worth understanding what McCarthy discovered, not
-just as a landmark in the history of computers, but as
-a model for what programming is tending to become in
-our own time.  It seems to me that there have been
-two really clean, consistent models of programming so
-far: the C model and the Lisp model.
-These two seem points of high ground, with swampy lowlands
-between them.  As computers have grown more powerful,
-the new languages being developed have been moving
-steadily toward the Lisp model.  A popular recipe
-for new programming languages in the past 20 years 
-has been to take the C model of computing and add to
-it, piecemeal, parts taken from the Lisp model,
-like runtime typing and garbage collection.In this article I'm going to try to explain in the
-simplest possible terms what McCarthy discovered.
-The point is not just to learn about an interesting
-theoretical result someone figured out forty years ago,
-but to show where languages are heading.
-The unusual thing about Lisp — in fact, the defining
-quality of Lisp — is that it can be written in
-itself.  To understand what McCarthy meant by this,
-we're going to retrace his steps, with his mathematical
-notation translated into running Common Lisp code.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/rss.txt

@@ -1,3 +0,0 @@
-Aaron Swartz created a scraped
-feed
-of the essays page.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/siliconvalley.txt

@@ -1,292 +0,0 @@
-May 2006(This essay is derived from a keynote at Xtech.)Could you reproduce Silicon Valley elsewhere, or is there something
-unique about it?It wouldn't be surprising if it were hard to reproduce in other
-countries, because you couldn't reproduce it in most of the US
-either.  What does it take to make a silicon valley even here?What it takes is the right people.  If you could get the right ten
-thousand people to move from Silicon Valley to Buffalo, Buffalo
-would become Silicon Valley.  
-[1]That's a striking departure from the past.  Up till a couple decades
-ago, geography was destiny for cities.  All great cities were located
-on waterways, because cities made money by trade, and water was the
-only economical way to ship.Now you could make a great city anywhere, if you could get the right
-people to move there.  So the question of how to make a silicon
-valley becomes: who are the right people, and how do you get them
-to move?Two TypesI think you only need two kinds of people to create a technology
-hub: rich people and nerds.  They're the limiting reagents in the
-reaction that produces startups, because they're the only ones
-present when startups get started.  Everyone else will move.Observation bears this out: within the US, towns have become startup
-hubs if and only if they have both rich people and nerds.  Few
-startups happen in Miami, for example, because although it's full
-of rich people, it has few nerds.  It's not the kind of place nerds
-like.Whereas Pittsburgh has the opposite problem: plenty of nerds, but
-no rich people.  The top US Computer Science departments are said
-to be MIT, Stanford, Berkeley, and Carnegie-Mellon.  MIT yielded
-Route 128.  Stanford and Berkeley yielded Silicon Valley.  But
-Carnegie-Mellon?  The record skips at that point.  Lower down the
-list, the University of Washington yielded a high-tech community
-in Seattle, and the University of Texas at Austin yielded one in
-Austin.  But what happened in Pittsburgh?  And in Ithaca, home of
-Cornell, which is also high on the list?I grew up in Pittsburgh and went to college at Cornell, so I can
-answer for both.  The weather is terrible,  particularly in winter,
-and there's no interesting old city to make up for it, as there is
-in Boston.  Rich people don't want to live in Pittsburgh or Ithaca.
-So while there are plenty of hackers who could start startups,
-there's no one to invest in them.Not BureaucratsDo you really need the rich people?  Wouldn't it work to have the
-government invest in the nerds?  No, it would not.  Startup investors
-are a distinct type of rich people.  They tend to have a lot of
-experience themselves in the technology business.  This (a) helps
-them pick the right startups, and (b) means they can supply advice
-and connections as well as money.  And the fact that they have a
-personal stake in the outcome makes them really pay attention.Bureaucrats by their nature are the exact opposite sort of people
-from startup investors. The idea of them making startup investments
-is comic.  It would be like mathematicians running Vogue-- or
-perhaps more accurately, Vogue editors running a math journal.
-[2]Though indeed, most things bureaucrats do, they do badly.   We just
-don't notice usually, because they only have to compete against
-other bureaucrats.  But as startup investors they'd have to compete
-against pros with a great deal more experience and motivation.Even corporations that have in-house VC groups generally forbid
-them to make their own investment decisions.  Most are only allowed
-to invest in deals where some reputable private VC firm is willing
-to act as lead investor.Not BuildingsIf you go to see Silicon Valley, what you'll see are buildings.
-But it's the people that make it Silicon Valley, not the buildings.
-I read occasionally about attempts to set up "technology
-parks" in other places, as if the active ingredient of Silicon
-Valley were the office space.  An article about Sophia Antipolis
-bragged that companies there included Cisco, Compaq, IBM, NCR, and
-Nortel.  Don't the French realize these aren't startups?Building office buildings for technology companies won't get you a
-silicon valley, because the key stage in the life of a startup
-happens before they want that kind of space.  The key stage is when
-they're three guys operating out of an apartment.  Wherever the
-startup is when it gets funded, it will stay.  The defining quality
-of Silicon Valley is not that Intel or Apple or Google have offices
-there, but that they were started there.So if you want to reproduce Silicon Valley, what you need to reproduce
-is those two or three founders sitting around a kitchen table
-deciding to start a company.  And to reproduce that you need those
-people.UniversitiesThe exciting thing is, all you need are the people.  If you could
-attract a critical mass of nerds and investors to live somewhere,
-you could reproduce Silicon Valley.  And both groups are highly
-mobile.  They'll go where life is good.  So what makes a place good
-to them?What nerds like is other nerds.  Smart people will go wherever other
-smart people are.  And in particular, to great universities.  In
-theory there could be other ways to attract them, but so far
-universities seem to be indispensable.  Within the US, there are
-no technology hubs without first-rate universities-- or at least,
-first-rate computer science departments.So if you want to make a silicon valley, you not only need a
-university, but one of the top handful in the world.  It has to be
-good enough to act as a magnet, drawing the best people from thousands
-of miles away.  And that means it has to stand up to existing magnets
-like MIT and Stanford.This sounds hard.  Actually it might be easy.  My professor friends,
-when they're deciding where they'd like to work, consider one thing
-above all: the quality of the other faculty.  What attracts professors
-is good colleagues.  So if you managed to recruit, en masse, a
-significant number of the best young researchers, you could create
-a first-rate university from nothing overnight.  And you could do
-that for surprisingly little.  If you paid 200 people hiring bonuses
-of $3 million apiece, you could put together a faculty that would
-bear comparison with any in the world.  And from that point the
-chain reaction would be self-sustaining.  So whatever it costs to
-establish a mediocre university, for an additional half billion or
-so you could have a great one.  
-[3]PersonalityHowever, merely creating a new university would not be enough to
-start a silicon valley. The university is just the seed.  It has
-to be planted in the right soil, or it won't germinate.  Plant it
-in the wrong place, and you just create Carnegie-Mellon.To spawn startups, your university has to be in a town that has
-attractions other than the university.  It has to be a place where
-investors want to live, and students want to stay after they graduate.The two like much the same things, because most startup investors
-are nerds themselves.  So what do nerds look for in a town?  Their
-tastes aren't completely different from other people's, because a
-lot of the towns they like most in the US are also big tourist
-destinations: San Francisco, Boston, Seattle.   But their tastes
-can't be quite mainstream either, because they dislike other big
-tourist destinations, like New York, Los Angeles, and Las Vegas.There has been a lot written lately about the "creative class." The
-thesis seems to be that as wealth derives increasingly from ideas,
-cities will prosper only if they attract those who have them.  That
-is certainly true; in fact it was the basis of Amsterdam's prosperity
-400 years ago.A lot of nerd tastes they share with the creative class in general.
-For example, they like well-preserved old neighborhoods instead of
-cookie-cutter suburbs, and locally-owned shops and restaurants
-instead of national chains.  Like the rest of the creative class,
-they want to live somewhere with personality.What exactly is personality?  I think it's the feeling that each
-building is the work of a distinct group of people.  A town with
-personality is one that doesn't feel mass-produced.  So if you want
-to make a startup hub-- or any town to attract the "creative class"--
-you probably have to ban large development projects.
-When a large tract has been developed by a single organization, you
-can always tell. 
-[4]Most towns with personality are old, but they don't have to be.
-Old towns have two advantages: they're denser, because they were
-laid out before cars, and they're more varied, because they were
-built one building at a time.  You could have both now.  Just have
-building codes that ensure density, and ban large scale developments.A corollary is that you have to keep out the biggest developer of
-all: the government.  A government that asks "How can we build a
-silicon valley?" has probably ensured failure by the way they framed
-the question.  You don't build a silicon valley; you let one grow.NerdsIf you want to attract nerds, you need more than a town with
-personality.  You need a town with the right personality.  Nerds
-are a distinct subset of the creative class, with different tastes
-from the rest.  You can see this most clearly in New York, which
-attracts a lot of creative people, but few nerds. 
-[5]What nerds like is the kind of town where people walk around smiling.
-This excludes LA, where no one walks at all, and also New York,
-where people walk, but not smiling. When I was in grad school in
-Boston, a friend came to visit from New York.  On the subway back
-from the airport she asked "Why is everyone smiling?"  I looked and
-they weren't smiling.  They just looked like they were compared to
-the facial expressions she was used to.If you've lived in New York, you know where these facial expressions
-come from.  It's the kind of place where your mind may be excited,
-but your body knows it's having a bad time.  People don't so much
-enjoy living there as endure it for the sake of the excitement.
-And if you like certain kinds of excitement, New York is incomparable.
-It's a hub of glamour, a magnet for all the shorter half-life
-isotopes of style and fame.Nerds don't care about glamour, so to them the appeal of New York
-is a mystery.  People who like New York will pay a fortune for a
-small, dark, noisy apartment in order to live in a town where the
-cool people are really cool.  A nerd looks at that deal and sees
-only: pay a fortune for a small, dark, noisy apartment.Nerds will pay a premium to live in a town where the smart people
-are really smart, but you don't have to pay as much for that.  It's
-supply and demand: glamour is popular, so you have to pay a lot for
-it.Most nerds like quieter pleasures.  They like cafes instead of
-clubs; used bookshops instead of fashionable clothing shops; hiking
-instead of dancing; sunlight instead of tall buildings.  A nerd's
-idea of paradise is Berkeley or Boulder.YouthIt's the young nerds who start startups, so it's those specifically
-the city has to appeal to.  The startup hubs in the US are all
-young-feeling towns.  This doesn't mean they have to be new.
-Cambridge has the oldest town plan in America, but it feels young
-because it's full of students.What you can't have, if you want to create a silicon valley, is a
-large, existing population of stodgy people.  It would be a waste
-of time to try to reverse the fortunes of a declining industrial town
-like Detroit or Philadelphia by trying to encourage startups.  Those
-places have too much momentum in the wrong direction.  You're better
-off starting with a blank slate in the form of a small town.  Or
-better still, if there's a town young people already flock to, that
-one.The Bay Area was a magnet for the young and optimistic for decades
-before it was associated with technology.  It was a place people
-went in search of something new.  And so it became synonymous with
-California nuttiness.  There's still a lot of that there.  If you
-wanted to start a new fad-- a new way to focus one's "energy," for
-example, or a new category of things not to eat-- the Bay Area would
-be the place to do it.  But a place that tolerates oddness in the
-search for the new is exactly what you want in a startup hub, because
-economically that's what startups are.  Most good startup ideas
-seem a little crazy; if they were obviously good ideas, someone
-would have done them already.(How many people are going to want computers in their houses?
-What, another search engine?)That's the connection between technology and liberalism.  Without
-exception the high-tech cities in the US are also the most liberal.
-But it's not because liberals are smarter that this is so.  It's
-because liberal cities tolerate odd ideas, and smart people by
-definition have odd ideas.Conversely, a town that gets praised for being "solid" or representing
-"traditional values" may be a fine place to live, but it's never
-going to succeed as a startup hub.  The 2004 presidential election,
-though a disaster in other respects, conveniently supplied us with
-a county-by-county 
-map of such places.  
-[6]To attract the young, a town must have an intact center.  In most
-American cities the center has been abandoned, and the growth, if
-any, is in the suburbs.  Most American cities have been turned
-inside out. But none of the startup hubs has: not San Francisco,
-or Boston, or Seattle.  They all have intact centers.
-[7]
-My guess is that no city with a dead center could be turned into a
-startup hub.  Young people don't want to live in the suburbs.Within the US, the two cities I think could most easily be turned
-into new silicon valleys are Boulder and Portland.  Both have the
-kind of effervescent feel that attracts the young.  They're each
-only a great university short of becoming a silicon valley, if they
-wanted to.TimeA great university near an attractive town.  Is that all it takes?
-That was all it took to make the original Silicon Valley.  Silicon
-Valley traces its origins to William Shockley, one of the inventors
-of the transistor.  He did the research that won him the Nobel Prize
-at Bell Labs, but when he started his own company in 1956 he moved
-to Palo Alto to do it.   At the time that was an odd thing to do.
-Why did he?  Because he had grown up there and remembered how nice
-it was.  Now Palo Alto is suburbia, but then it was a charming
-college town-- a charming college town with perfect weather and San
-Francisco only an hour away.The companies that rule Silicon Valley now are all descended in
-various ways from Shockley Semiconductor.  Shockley was a difficult
-man, and in 1957 his top people-- "the traitorous eight"-- left to
-start a new company, Fairchild Semiconductor.  Among them were
-Gordon Moore and Robert Noyce, who went on to found Intel, and
-Eugene Kleiner, who founded the VC firm Kleiner Perkins.  Forty-two
-years later, Kleiner Perkins funded Google, and the partner responsible
-for the deal was John Doerr, who came to Silicon Valley in 1974 to
-work for Intel.So although a lot of the newest companies in Silicon Valley don't
-make anything out of silicon, there always seem to be multiple links
-back to Shockley.  There's a lesson here: startups beget startups.
-People who work for startups start their own.  People who get rich
-from startups fund new ones.  I suspect this kind of organic growth
-is the only way to produce a startup hub, because it's the only way
-to grow the expertise you need.That has two important implications.  The first is that you need
-time to grow a silicon valley.  The university you could create in
-a couple years, but the startup community around it has to grow
-organically.   The cycle time is limited by the time it takes a
-company to succeed, which probably averages about five years.The other implication of the organic growth hypothesis is that you
-can't be somewhat of a startup hub.  You either have a self-sustaining
-chain reaction, or not.  Observation confirms this too: cities
-either have a startup scene, or they don't.  There is no middle
-ground.  Chicago has the third largest metropolitan area in America.
-As source of startups it's negligible compared to Seattle, number 15.The good news is that the initial seed can be quite small.  Shockley
-Semiconductor, though itself not very successful, was big enough.
-It brought a critical mass of experts in an important new technology
-together in a place they liked enough to stay.CompetingOf course, a would-be silicon valley faces an obstacle the original
-one didn't: it has to compete with Silicon Valley.  Can that be
-done?  Probably.One of Silicon Valley's biggest advantages is its venture capital
-firms.  This was not a factor in Shockley's day, because VC funds
-didn't exist.  In fact, Shockley Semiconductor and Fairchild
-Semiconductor were not startups at all in our sense.  They were
-subsidiaries-- of Beckman Instruments and Fairchild Camera and
-Instrument respectively.  Those companies were apparently willing
-to establish subsidiaries wherever the experts wanted to live.Venture investors, however, prefer to fund startups within an hour's
-drive.  For one, they're more likely to notice startups nearby.
-But when they do notice startups in other towns they prefer them
-to move.  They don't want to have to travel to attend board meetings,
-and in any case the odds of succeeding are higher in a startup hub.The centralizing effect of venture firms is a double one: they cause
-startups to form around them, and those draw in more startups through
-acquisitions.  And although the first may be weakening because it's
-now so cheap to start some startups, the second seems as strong as ever.
-Three of the most admired
-"Web 2.0" companies were started outside the usual startup hubs,
-but two of them have already been reeled in through acquisitions.Such centralizing forces make it harder for new silicon valleys to
-get started.  But by no means impossible.  Ultimately power rests
-with the founders.  A startup with the best people will beat one
-with funding from famous VCs, and a startup that was sufficiently
-successful would never have to move.  So a town that
-could exert enough pull over the right people could resist and
-perhaps even surpass Silicon Valley.For all its power, Silicon Valley has a great weakness: the paradise
-Shockley found in 1956 is now one giant parking lot.  San Francisco
-and Berkeley are great, but they're forty miles away.  Silicon
-Valley proper is soul-crushing suburban sprawl.  It
-has fabulous weather, which makes it significantly better than the
-soul-crushing sprawl of most other American cities.  But a competitor
-that managed to avoid sprawl would have real leverage.  All a city
-needs is to be the kind of place the next traitorous eight look at
-and say "I want to stay here," and that would be enough to get the
-chain reaction started.Notes[1]
-It's interesting to consider how low this number could be
-made.  I suspect five hundred would be enough, even if they could
-bring no assets with them.  Probably just thirty, if I could pick them, 
-would be enough to turn Buffalo into a significant startup hub.[2]
-Bureaucrats manage to allocate research funding moderately
-well, but only because (like an in-house VC fund) they outsource
-most of the work of selection.  A professor at a famous university
-who is highly regarded by his peers will get funding, pretty much
-regardless of the proposal.  That wouldn't work for startups, whose
-founders aren't sponsored by organizations, and are often unknowns.[3]
-You'd have to do it all at once, or at least a whole department
-at a time, because people would be more likely to come if they
-knew their friends were.  And you should probably start from scratch,
-rather than trying to upgrade an existing university, or much energy
-would be lost in friction.[4]
-Hypothesis: Any plan in which multiple independent buildings
-are gutted or demolished to be "redeveloped" as a single project
-is a net loss of personality for the city, with the exception of
-the conversion of buildings not previously public, like warehouses.[5]
-A few startups get started in New York, but less
-than a tenth as many per capita as in Boston, and mostly
-in less nerdy fields like finance and media.[6]
-Some blue counties are false positives (reflecting the
-remaining power of Democractic party machines), but there are no
-false negatives.  You can safely write off all the red counties.[7]
-Some "urban renewal" experts took a shot at destroying Boston's
-in the 1960s, leaving the area around city hall a bleak wasteland,
-but most neighborhoods successfully resisted them.Thanks to Chris Anderson, Trevor Blackwell, Marc Hedlund,
-Jessica Livingston, Robert Morris, Greg Mcadoo, Fred Wilson,
-and Stephen Wolfram for
-reading drafts of this, and to Ed Dumbill for inviting me to speak.(The second part of this talk became Why Startups
-Condense in America.)

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/startuplessons.txt

@@ -1,395 +0,0 @@
-April 2006(This essay is derived from a talk at the 2006 
-Startup School.)The startups we've funded so far are pretty quick, but they seem
-quicker to learn some lessons than others.  I think it's because
-some things about startups are kind of counterintuitive.We've now 
-invested 
-in enough companies that I've learned a trick
-for determining which points are the counterintuitive ones:
-they're the ones I have to keep repeating.So I'm going to number these points, and maybe with future startups
-I'll be able to pull off a form of Huffman coding. I'll make them
-all read this, and then instead of nagging them in detail, I'll
-just be able to say: number four!
-1. Release Early.The thing I probably repeat most is this recipe for a startup: get
-a version 1 out fast, then improve it based on users' reactions.By "release early" I don't mean you should release something full
-of bugs, but that you should release something minimal.  Users hate
-bugs, but they don't seem to mind a minimal version 1, if there's
-more coming soon.There are several reasons it pays to get version 1 done fast.  One
-is that this is simply the right way to write software, whether for
-a startup or not.  I've been repeating that since 1993, and I haven't seen much since to
-contradict it.  I've seen a lot of startups die because they were
-too slow to release stuff, and none because they were too quick.
-[1]One of the things that will surprise you if you build something
-popular is that you won't know your users.  Reddit now has almost half a million
-unique visitors a month.  Who are all those people?  They have no
-idea.  No web startup does.  And since you don't know your users,
-it's dangerous to guess what they'll like.  Better to release
-something and let them tell you.Wufoo took this to heart and released
-their form-builder before the underlying database.  You can't even
-drive the thing yet, but 83,000 people came to sit in the driver's
-seat and hold the steering wheel.  And Wufoo got valuable feedback
-from it: Linux users complained they used too much Flash, so they
-rewrote their software not to.  If they'd waited to release everything
-at once, they wouldn't have discovered this problem till it was
-more deeply wired in.Even if you had no users, it would still be important to release
-quickly, because for a startup the initial release acts as a shakedown
-cruise.  If anything major is broken-- if the idea's no good,
-for example, or the founders hate one another-- the stress of getting
-that first version out will expose it.  And if you have such problems
-you want to find them early.Perhaps the most important reason to release early, though, is that
-it makes you work harder.  When you're working on something that
-isn't released, problems are intriguing.  In something that's out
-there, problems are alarming.  There is a lot more urgency once you
-release.  And I think that's precisely why people put it off.  They
-know they'll have to work a lot harder once they do. 
-[2]
-2. Keep Pumping Out Features.Of course, "release early" has a second component, without which
-it would be bad advice.  If you're going to start with something
-that doesn't do much, you better improve it fast.What I find myself repeating is "pump out features."  And this rule
-isn't just for the initial stages.  This is something all startups
-should do for as long as they want to be considered startups.I don't mean, of course, that you should make your application ever
-more complex.  By "feature" I mean one unit of hacking-- one quantum
-of making users' lives better.As with exercise, improvements beget improvements.  If you run every
-day, you'll probably feel like running tomorrow.  But if you skip
-running for a couple weeks, it will be an effort to drag yourself
-out.  So it is with hacking: the more ideas you implement, the more
-ideas you'll have.  You should make your system better at least in
-some small way every day or two.This is not just a good way to get development done; it is also a
-form of marketing.  Users love a site that's constantly improving.
-In fact, users expect a site to improve.  Imagine if you visited a
-site that seemed very good, and then returned two months later and
-not one thing had changed.  Wouldn't it start to seem lame? 
-[3]They'll like you even better when you improve in response to their
-comments, because customers are used to companies ignoring them.
-If you're the rare exception-- a company that actually listens--
-you'll generate fanatical loyalty.  You won't need to advertise,
-because your users will do it for you.This seems obvious too, so why do I have to keep repeating it?  I
-think the problem here is that people get used to how things are.
-Once a product gets past the stage where it has glaring flaws, you
-start to get used to it, and gradually whatever features it happens
-to have become its identity.  For example, I doubt many people at
-Yahoo (or Google for that matter) realized how much better web mail
-could be till Paul Buchheit showed them.I think the solution is to assume that anything you've made is far
-short of what it could be.  Force yourself, as a sort of intellectual
-exercise, to keep thinking of improvements.  Ok, sure, what you
-have is perfect.  But if you had to change something, what would
-it be?If your product seems finished, there are two possible explanations:
-(a) it is finished, or (b) you lack imagination.  Experience suggests
-(b) is a thousand times more likely.
-3. Make Users Happy.Improving constantly is an instance of a more general rule: make
-users happy.  One thing all startups have in common is that they
-can't force anyone to do anything.  They can't force anyone to use
-their software, and they can't force anyone to do deals with them.
-A startup has to sing for its supper.  That's why the successful
-ones make great things.  They have to, or die.When you're running a startup you feel like a little bit of debris
-blown about by powerful winds.  The most powerful wind is users.
-They can either catch you and loft you up into the sky, as they did
-with Google, or leave you flat on the pavement, as they do with
-most startups.  Users are a fickle wind, but more powerful than any
-other.  If they take you up, no competitor can keep you down.As a little piece of debris, the rational thing for you to do is
-not to lie flat, but to curl yourself into a shape the wind will
-catch.I like the wind metaphor because it reminds you how impersonal the
-stream of traffic is.  The vast majority of people who visit your
-site will be casual visitors.  It's them you have to design your
-site for.  The people who really care will find what they want by
-themselves.The median visitor will arrive with their finger poised on the Back
-button.  Think about your own experience: most links you
-follow lead to something lame.  Anyone who has used the web for
-more than a couple weeks has been trained to click on Back after
-following a link.  So your site has to say "Wait!  Don't click on
-Back.  This site isn't lame.  Look at this, for example."There are two things you have to do to make people pause.  The most
-important is to explain, as concisely as possible, what the hell
-your site is about.  How often have you visited a site that seemed
-to assume you already knew what they did?  For example, the corporate
-site that says the
-company makes
-
-  enterprise content management solutions for business that enable
-  organizations to unify people, content and processes to minimize
-  business risk, accelerate time-to-value and sustain lower total
-  cost of ownership.
-
-An established company may get away with such an opaque description,
-but no startup can.  A startup
-should be able to explain in one or two sentences exactly what it
-does. 
-[4]
-And not just to users.  You need this for everyone:
-investors, acquirers, partners, reporters, potential employees, and
-even current employees.  You probably shouldn't even start a company
-to do something that can't be described compellingly in one or two
-sentences.The other thing I repeat is to give people everything you've got,
-right away.  If you have something impressive, try to put it on the
-front page, because that's the only one most visitors will see.
-Though indeed there's a paradox here: the more you push the good
-stuff toward the front, the more likely visitors are to explore
-further. 
-[5]In the best case these two suggestions get combined: you tell
-visitors what your site is about by showing them.  One of the
-standard pieces of advice in fiction writing is "show, don't tell."
-Don't say that a character's angry; have him grind his teeth, or
-break his pencil in half.  Nothing will explain what your site does
-so well as using it.The industry term here is "conversion."  The job of your site is
-to convert casual visitors into users-- whatever your definition
-of a user is.  You can measure this in your growth rate.  Either
-your site is catching on, or it isn't, and you must know which.  If
-you have decent growth, you'll win in the end, no matter how obscure
-you are now.  And if you don't, you need to fix something.
-4. Fear the Right Things.Another thing I find myself saying a lot is "don't worry."  Actually,
-it's more often "don't worry about this; worry about that instead."
-Startups are right to be paranoid, but they sometimes fear the wrong
-things.Most visible disasters are not so alarming as they seem.  Disasters
-are normal in a startup: a founder quits, you discover a patent
-that covers what you're doing, your servers keep crashing, you run
-into an insoluble technical problem, you have to change your name,
-a deal falls through-- these are all par for the course.  They won't
-kill you unless you let them.Nor will most competitors.  A lot of startups worry "what if Google
-builds something like us?"  Actually big companies are not the ones
-you have to worry about-- not even Google.  The people at Google
-are smart, but no smarter than you; they're not as motivated, because
-Google is not going to go out of business if this one product fails;
-and even at Google they have a lot of bureaucracy to slow them down.What you should fear, as a startup, is not the established players,
-but other startups you don't know exist yet.  They're way more
-dangerous than Google because, like you, they're cornered animals.Looking just at existing competitors can give you a false sense of
-security.  You should compete against what someone else could be
-doing, not just what you can see people doing.  A corollary is that
-you shouldn't relax just because you have no visible competitors
-yet.  No matter what your idea, there's someone else out there
-working on the same thing.That's the downside of it being easier to start a startup: more people
-are doing it.  But I disagree with Caterina Fake when she says that
-makes this a bad time to start a startup.  More people are starting
-startups, but not as many more as could.  Most college graduates
-still think they have to get a job.  The average person can't ignore
-something that's been beaten into their head since they were three
-just because serving web pages recently got a lot cheaper.And in any case, competitors are not the biggest threat.  Way more
-startups hose themselves than get crushed by competitors.  There
-are a lot of ways to do it, but the three main ones are internal
-disputes, inertia, and ignoring users.  Each is, by itself, enough
-to kill you.  But if I had to pick the worst, it would be ignoring
-users.  If you want a recipe for a startup that's going to die,
-here it is: a couple of founders who have some great idea they know
-everyone is going to love, and that's what they're going to build,
-no matter what.Almost everyone's initial plan is broken.  If companies stuck to
-their initial plans, Microsoft would be selling programming languages,
-and Apple would be selling printed circuit boards.  In both cases
-their customers told them what their business should be-- and they
-were smart enough to listen.As Richard Feynman said, the imagination of nature is greater than
-the imagination of man.  You'll find more interesting things by
-looking at the world than you could ever produce just by thinking.
-This principle is very powerful.  It's why the best abstract painting
-still falls short of Leonardo, for example.  And it applies to
-startups too.  No idea for a product could ever be so clever as the
-ones you can discover by smashing a beam of prototypes into a beam
-of users.
-5. Commitment Is a Self-Fulfilling Prophecy.I now have enough experience with startups to be able to say what
-the most important quality is in a startup founder, and it's not
-what you might think.  The most important quality in a startup
-founder is determination.  Not intelligence-- determination.This is a little depressing.  I'd like to believe Viaweb succeeded
-because we were smart, not merely determined.  A lot of people in
-the startup world want to believe that.  Not just founders, but
-investors too.  They like the idea of inhabiting a world ruled by
-intelligence.  And you can tell they really believe this, because
-it affects their investment decisions.Time after time VCs invest in startups founded by eminent professors.
-This may work in biotech, where a lot of startups simply commercialize
-existing research, but in software you want to invest in students,
-not professors.  Microsoft, Yahoo, and Google were all founded by
-people who dropped out of school to do it.  What students lack in
-experience they more than make up in dedication.Of course, if you want to get rich, it's not enough merely to be
-determined.  You have to be smart too, right?  I'd like to think
-so, but I've had an experience that convinced me otherwise: I spent
-several years living in New York.You can lose quite a lot in the brains department and it won't kill
-you.  But lose even a little bit in the commitment department, and
-that will kill you very rapidly.Running a startup is like walking on your hands: it's possible, but
-it requires extraordinary effort.  If an ordinary employee were
-asked to do the things a startup founder has to, he'd be very
-indignant.  Imagine if you were hired at some big company, and in
-addition to writing software ten times faster than you'd ever had
-to before, they expected you to answer support calls, administer
-the servers, design the web site, cold-call customers, find the
-company office space, and go out and get everyone lunch.And to do all this not in the calm, womb-like atmosphere of a big
-company, but against a backdrop of constant disasters.  That's the
-part that really demands determination.  In a startup, there's
-always some disaster happening.  So if you're the least bit inclined
-to find an excuse to quit, there's always one right there.But if you lack commitment, chances are it will have been hurting
-you long before you actually quit.  Everyone who deals with startups
-knows how important commitment is, so if they sense you're ambivalent,
-they won't give you much attention.  If you lack commitment, you'll
-just find that for some mysterious reason good things happen to
-your competitors but not to you.  If you lack commitment, it will
-seem to you that you're unlucky.Whereas if you're determined to stick around, people will pay
-attention to you, because odds are they'll have to deal with you
-later.  You're a local, not just a tourist, so everyone has to come
-to terms with you.At Y Combinator we sometimes mistakenly fund teams who have the
-attitude that they're going to give this startup thing a shot for
-three months, and if something great happens, they'll stick with
-it-- "something great" meaning either that someone wants to buy
-them or invest millions of dollars in them.  But if this is your
-attitude, "something great" is very unlikely to happen to you,
-because both acquirers and investors judge you by your level of
-commitment.If an acquirer thinks you're going to stick around no matter what,
-they'll be more likely to buy you, because if they don't and you
-stick around, you'll probably grow, your price will go up, and
-they'll be left wishing they'd bought you earlier.  Ditto for
-investors.  What really motivates investors, even big VCs, is not
-the hope of good returns, but the fear of missing out. 
-[6]
-So if
-you make it clear you're going to succeed no matter what, and the only
-reason you need them is to make it happen a little faster, you're
-much more likely to get money.You can't fake this.  The only way to convince everyone that you're
-ready to fight to the death is actually to be ready to.You have to be the right kind of determined, though.  I carefully
-chose the word determined rather than stubborn, because stubbornness
-is a disastrous quality in a startup.  You have to be determined,
-but flexible, like a running back.  A successful running back doesn't
-just put his head down and try to run through people.  He improvises:
-if someone appears in front of him, he runs around them; if someone
-tries to grab him, he spins out of their grip; he'll even run in
-the wrong direction briefly if that will help.  The one thing he'll
-never do is stand still. 
-[7]
-6. There Is Always Room.I was talking recently to a startup founder about whether it might
-be good to add a social component to their software.  He said he
-didn't think so, because the whole social thing was tapped out.
-Really?  So in a hundred years the only social networking sites
-will be the Facebook, MySpace, Flickr, and Del.icio.us?  Not likely.There is always room for new stuff.  At every point in history,
-even the darkest bits of the dark ages, people were discovering
-things that made everyone say "why didn't anyone think of that
-before?"  We know this continued to be true up till 2004, when the
-Facebook was founded-- though strictly speaking someone else did
-think of that.The reason we don't see the opportunities all around us is that we
-adjust to however things are, and assume that's how things have to
-be.  For example, it would seem crazy to most people to try to make
-a better search engine than Google.  Surely that field, at least,
-is tapped out.  Really?  In a hundred years-- or even twenty-- are
-people still going to search for information using something like
-the current Google?  Even Google probably doesn't think that.In particular, I don't think there's any limit to the number of
-startups.  Sometimes you hear people saying "All these guys starting
-startups now are going to be disappointed. How many little startups
-are Google and Yahoo going to buy, after all?" That sounds cleverly
-skeptical, but I can prove it's mistaken.  No one proposes that
-there's some limit to the number of people who can be employed in
-an economy consisting of big, slow-moving companies with a couple
-thousand people each.  Why should there be any limit to the number
-who could be employed by small, fast-moving companies with ten each?
-It seems to me the only limit would be the number of people who
-want to work that hard.The limit on the number of startups is not the number that can get
-acquired by Google and Yahoo-- though it seems even that should
-be unlimited, if the startups were actually worth buying-- but the
-amount of wealth that can be created.  And I don't think there's
-any limit on that, except cosmological ones.So for all practical purposes, there is no limit to the number of
-startups.  Startups make wealth, which means they make things people
-want, and if there's a limit on the number of things people want,
-we are nowhere near it.  I still don't even have a flying car.
-7. Don't Get Your Hopes Up.This is another one I've been repeating since long before Y Combinator.
-It was practically the corporate motto at Viaweb.Startup founders are naturally optimistic.  They wouldn't do it
-otherwise.  But you should treat your optimism the way you'd treat
-the core of a nuclear reactor: as a source of power that's also
-very dangerous.  You have to build a shield around it, or it will
-fry you.The shielding of a reactor is not uniform; the reactor would be
-useless if it were.  It's pierced in a few places to let pipes in.
-An optimism shield has to be pierced too.  I think the place to
-draw the line is between what you expect of yourself, and what you
-expect of other people.  It's ok to be optimistic about what you
-can do, but assume the worst about machines and other people.This is particularly necessary in a startup, because you tend to
-be pushing the limits of whatever you're doing.  So things don't
-happen in the smooth, predictable way they do in the rest of the
-world.  Things change suddenly, and usually for the worse.Shielding your optimism is nowhere more important than with deals.
-If your startup is doing a deal, just assume it's not going to
-happen.  The VCs who say they're going to invest in you aren't.
-The company that says they're going to buy you isn't.  The big
-customer who wants to use your system in their whole company won't.
-Then if things work out you can be pleasantly surprised.The reason I warn startups not to get their hopes up is not to save
-them from being disappointed when things fall through.  It's
-for a more practical reason: to prevent them from leaning their
-company against something that's going to fall over, taking them
-with it.For example, if someone says they want to invest in you, there's a
-natural tendency to stop looking for other investors.  That's why
-people proposing deals seem so positive: they want you to
-stop looking.  And you want to stop too, because doing deals is a
-pain.  Raising money, in particular, is a huge time sink.  So you
-have to consciously force yourself to keep looking.Even if you ultimately do the first deal, it will be to your advantage
-to have kept looking, because you'll get better terms.  Deals are
-dynamic; unless you're negotiating with someone unusually honest,
-there's not a single point where you shake hands and the deal's
-done. There are usually a lot of subsidiary questions to be cleared
-up after the handshake, and if the other side senses weakness-- if
-they sense you need this deal-- they will be very tempted to screw
-you in the details.VCs and corp dev guys are professional negotiators.  They're trained
-to take advantage of weakness. 
-[8]
-So while they're often nice
-guys, they just can't help it.  And as pros they do this more than
-you.  So don't even try to bluff them.  The only way a startup can
-have any leverage in a deal is genuinely not to need it.  And if
-you don't believe in a deal, you'll be less likely to depend on it.So I want to plant a hypnotic suggestion in your heads: when you
-hear someone say the words "we want to invest in you" or "we want
-to acquire you," I want the following phrase to appear automatically
-in your head: don't get your hopes up.  Just continue running
-your company as if this deal didn't exist.  Nothing is more likely
-to make it close.The way to succeed in a startup is to focus on the goal of getting
-lots of users, and keep walking swiftly toward it while investors
-and acquirers scurry alongside trying to wave money in your face.
-Speed, not MoneyThe way I've described it, starting a startup sounds pretty stressful.
-It is.  When I talk to the founders of the companies we've funded,
-they all say the same thing: I knew it would be hard, but I didn't
-realize it would be this hard.So why do it?  It would be worth enduring a lot of pain and stress
-to do something grand or heroic, but just to make money?  Is making
-money really that important?No, not really.  It seems ridiculous to me when people take business
-too seriously.  I regard making money as a boring errand to be got
-out of the way as soon as possible.  There is nothing grand or
-heroic about starting a startup per se.So why do I spend so much time thinking about startups?  I'll tell
-you why.  Economically, a startup is best seen not as a way to get
-rich, but as a way to work faster.  You have to make a living, and
-a startup is a way to get that done quickly, instead of letting it
-drag on through your whole life.
-[9]We take it for granted most of the time, but human life is fairly
-miraculous.  It is also palpably short.  You're given this marvellous
-thing, and then poof, it's taken away.  You can see why people
-invent gods to explain it.  But even to people who don't believe
-in gods, life commands respect.  There are times in most of our
-lives when the days go by in a blur, and almost everyone has a
-sense, when this happens, of wasting something precious.  As Ben
-Franklin said, if you love life, don't waste time, because time is
-what life is made of.So no, there's nothing particularly grand about making money.  That's
-not what makes startups worth the trouble.  What's important about
-startups is the speed.  By compressing the dull but necessary task
-of making a living into the smallest possible time, you show respect
-for life, and there is something grand about that.Notes[1]
-Startups can die from releasing something full of bugs, and not
-fixing them fast enough, but I don't know of any that died from
-releasing something stable but minimal very early, then promptly
-improving it.[2]
-I know this is why I haven't released Arc.  The moment I do,
-I'll have people nagging me for features.[3]
-A web site is different from a book or movie or desktop application
-in this respect.  Users judge a site not as a single snapshot, but
-as an animation with multiple frames.  Of the two, I'd say the rate of
-improvement is more important to users than where you currently
-are.[4]
-It should not always tell this to users, however.  For example,
-MySpace is basically a replacement mall for mallrats.  But it was
-wiser for them, initially, to pretend that the site was about bands.[5]
-Similarly, don't make users register to try your site.  Maybe
-what you have is so valuable that visitors should gladly register
-to get at it.  But they've been trained to expect the opposite.
-Most of the things they've tried on the web have sucked-- and
-probably especially those that made them register.[6]
-VCs have rational reasons for behaving this way. They don't
-make their money (if they make money) off their median investments.
-In a typical fund, half the companies fail, most of the rest generate
-mediocre returns, and one or two "make the fund" by succeeding
-spectacularly.  So if they miss just a few of the most promising
-opportunities, it could hose the whole fund.[7]
-The attitude of a running back doesn't translate to soccer.
-Though it looks great when a forward dribbles past multiple defenders,
-a player who persists in trying such things will do worse in the
-long term than one who passes.[8]
-The reason Y Combinator never negotiates valuations
-is that we're not professional negotiators, and don't want to turn
-into them.[9]
-There are two ways to do 
-work you love: (a) to make money, then work
-on what you love, or (b) to get a job where you get paid to work on
-stuff you love.  In practice the first phases of both
-consist mostly of unedifying schleps, and in (b) the second phase is less
-secure.Thanks to Sam Altman, Trevor Blackwell, Beau Hartshorne, Jessica 
-Livingston, and Robert Morris for reading drafts of this.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/submarine.txt

@@ -1,217 +0,0 @@
-April 2005"Suits make a corporate comeback," says the New
-York Times.  Why does this sound familiar?  Maybe because
-the suit was also back in February,
-
-September
-2004, June
-2004, March
-2004, September
-2003, 
-
-November
-2002, 
-April 2002,
-and February
-2002.
-
-Why do the media keep running stories saying suits are back?  Because
-PR firms tell 
-them to.  One of the most surprising things I discovered
-during my brief business career was the existence of the PR industry,
-lurking like a huge, quiet submarine beneath the news.  Of the
-stories you read in traditional media that aren't about politics,
-crimes, or disasters, more than half probably come from PR firms.I know because I spent years hunting such "press hits."  Our startup spent
-its entire marketing budget on PR: at a time when we were assembling
-our own computers to save money, we were paying a PR firm $16,000
-a month.  And they were worth it.  PR is the news equivalent of
-search engine optimization; instead of buying ads, which readers
-ignore, you get yourself inserted directly into the stories.  [1]Our PR firm
-was one of the best in the business.  In 18 months, they got press
-hits in over 60 different publications.  
-And we weren't the only ones they did great things for.  
-In 1997 I got a call from another
-startup founder considering hiring them to promote his company.  I
-told him they were PR gods, worth every penny of their outrageous   
-fees.  But I remember thinking his company's name was odd.
-Why call an auction site "eBay"?
-SymbiosisPR is not dishonest.  Not quite.  In fact, the reason the best PR
-firms are so effective is precisely that they aren't dishonest.
-They give reporters genuinely valuable information.  A good PR firm
-won't bug reporters just because the client tells them to; they've
-worked hard to build their credibility with reporters, and they
-don't want to destroy it by feeding them mere propaganda.If anyone is dishonest, it's the reporters.  The main reason PR  
-firms exist is that reporters are lazy.  Or, to put it more nicely,
-overworked.  Really they ought to be out there digging up stories
-for themselves.  But it's so tempting to sit in their offices and
-let PR firms bring the stories to them.  After all, they know good
-PR firms won't lie to them.A good flatterer doesn't lie, but tells his victim selective truths
-(what a nice color your eyes are). Good PR firms use the same
-strategy: they give reporters stories that are true, but whose truth
-favors their clients.For example, our PR firm often pitched stories about how the Web  
-let small merchants compete with big ones.  This was perfectly true.
-But the reason reporters ended up writing stories about this
-particular truth, rather than some other one, was that small merchants
-were our target market, and we were paying the piper.Different publications vary greatly in their reliance on PR firms.
-At the bottom of the heap are the trade press, who make most of
-their money from advertising and would give the magazines away for
-free if advertisers would let them.  [2] The average
-trade publication is a  bunch of ads, glued together by just enough
-articles to make it look like a magazine.  They're so desperate for
-"content" that some will print your press releases almost verbatim,
-if you take the trouble to write them to read like articles.At the other extreme are publications like the New York Times
-and the Wall Street Journal.  Their reporters do go out and
-find their own stories, at least some of the time.  They'll listen 
-to PR firms, but briefly and skeptically.  We managed to get press   
-hits in almost every publication we wanted, but we never managed 
-to crack the print edition of the Times.  [3]The weak point of the top reporters is not laziness, but vanity.
-You don't pitch stories to them.  You have to approach them as if
-you were a specimen under their all-seeing microscope, and make it
-seem as if the story you want them to run is something they thought 
-of themselves.Our greatest PR coup was a two-part one.  We estimated, based on
-some fairly informal math, that there were about 5000 stores on the
-Web.  We got one paper to print this number, which seemed neutral   
-enough.  But once this "fact" was out there in print, we could quote
-it to other publications, and claim that with 1000 users we had 20%
-of the online store market.This was roughly true.  We really did have the biggest share of the
-online store market, and 5000 was our best guess at its size.  But
-the way the story appeared in the press sounded a lot more definite.Reporters like definitive statements.  For example, many of the
-stories about Jeremy Jaynes's conviction say that he was one of the
-10 worst spammers.  This "fact" originated in Spamhaus's ROKSO list,
-which I think even Spamhaus would admit is a rough guess at the top
-spammers.  The first stories about Jaynes cited this source, but
-now it's simply repeated as if it were part of the indictment.   
-[4]All you can say with certainty about Jaynes is that he was a fairly
-big spammer.  But reporters don't want to print vague stuff like
-"fairly big."  They want statements with punch, like "top ten." And
-PR firms give them what they want.
-Wearing suits, we're told, will make us 
-3.6
-percent more productive.BuzzWhere the work of PR firms really does get deliberately misleading is in
-the generation of "buzz."  They usually feed the same story to    
-several different publications at once.  And when readers see similar
-stories in multiple places, they think there is some important trend
-afoot.  Which is exactly what they're supposed to think.When Windows 95 was launched, people waited outside stores
-at midnight to buy the first copies.  None of them would have been
-there without PR firms, who generated such a buzz in
-the news media that it became self-reinforcing, like a nuclear chain
-reaction.I doubt PR firms realize it yet, but the Web makes it possible to  
-track them at work.  If you search for the obvious phrases, you
-turn up several efforts over the years to place stories about the  
-return of the suit.  For example, the Reuters article 
-
-that got picked up by USA
-Today in September 2004.  "The suit is back," it begins.Trend articles like this are almost always the work of
-PR firms.  Once you know how to read them, it's straightforward to
-figure out who the client is.  With trend stories, PR firms usually
-line up one or more "experts" to talk about the industry generally. 
-In this case we get three: the NPD Group, the creative director of
-GQ, and a research director at Smith Barney.  [5] When
-you get to the end of the experts, look for the client. And bingo, 
-there it is: The Men's Wearhouse.Not surprising, considering The Men's Wearhouse was at that moment 
-running ads saying "The Suit is Back."  Talk about a successful
-press hit-- a wire service article whose first sentence is your own
-ad copy.The secret to finding other press hits from a given pitch
-is to realize that they all started from the same document back at
-the PR firm.  Search for a few key phrases and the names of the
-clients and the experts, and you'll turn up other variants of this 
-story.Casual
-fridays are out and dress codes are in writes Diane E. Lewis
-in The Boston Globe.  In a remarkable coincidence, Ms. Lewis's
-industry contacts also include the creative director of GQ.Ripped jeans and T-shirts are out, writes Mary Kathleen Flynn in
-US News & World Report.  And she too knows the 
-creative director of GQ.Men's suits
-are back writes Nicole Ford in Sexbuzz.Com ("the ultimate men's
-entertainment magazine").Dressing
-down loses appeal as men suit up at the office writes Tenisha
-Mercer of The Detroit News.
-Now that so many news articles are online, I suspect you could find
-a similar pattern for most trend stories placed by PR firms.  I
-propose we call this new sport "PR diving," and I'm sure there are
-far more striking examples out there than this clump of five stories.OnlineAfter spending years chasing them, it's now second nature
-to me to recognize press hits for what they are.  But before we
-hired a PR firm I had no idea where articles in the mainstream media
-came from.  I could tell a lot of them were crap, but I didn't
-realize why.Remember the exercises in critical reading you did in school, where
-you had to look at a piece of writing and step back and ask whether
-the author was telling the whole truth?  If you really want to be
-a critical reader, it turns out you have to step back one step
-further, and ask not just whether the author is telling the truth,
-but why he's writing about this subject at all.Online, the answer tends to be a lot simpler.  Most people who
-publish online write what they write for the simple reason that
-they want to.  You
-can't see the fingerprints of PR firms all over the articles, as
-you can in so many print publications-- which is one of the reasons,
-though they may not consciously realize it, that readers trust
-bloggers more than Business Week.I was talking recently to a friend who works for a
-big newspaper.  He thought the print media were in serious trouble,
-and that they were still mostly in denial about it.  "They think
-the decline is cyclic," he said.  "Actually it's structural."In other words, the readers are leaving, and they're not coming
-back.
-Why? I think the main reason is that the writing online is more honest.
-Imagine how incongruous the New York Times article about
-suits would sound if you read it in a blog:
-   The urge to look corporate-- sleek, commanding,
-  prudent, yet with just a touch of hubris on your well-cut sleeve--
-  is an unexpected development in a time of business disgrace.
-   
-The problem
-with this article is not just that it originated in a PR firm.
-The whole tone is bogus.  This is the tone of someone writing down
-to their audience.Whatever its flaws, the writing you find online
-is authentic.  It's not mystery meat cooked up
-out of scraps of pitch letters and press releases, and pressed into 
-molds of zippy
-journalese.  It's people writing what they think.I didn't realize, till there was an alternative, just how artificial
-most of the writing in the mainstream media was.  I'm not saying
-I used to believe what I read in Time and Newsweek.  Since high
-school, at least, I've thought of magazines like that more as
-guides to what ordinary people were being
-told to think than as  
-sources of information.  But I didn't realize till the last  
-few years that writing for publication didn't have to mean writing
-that way.  I didn't realize you could write as candidly and
-informally as you would if you were writing to a friend.Readers aren't the only ones who've noticed the
-change.  The PR industry has too.
-A hilarious article
-on the site of the PR Society of America gets to the heart of the   
-matter:
-   Bloggers are sensitive about becoming mouthpieces
-  for other organizations and companies, which is the reason they
-  began blogging in the first place.  
-PR people fear bloggers for the same reason readers
-like them.  And that means there may be a struggle ahead.  As
-this new kind of writing draws readers away from traditional media, we
-should be prepared for whatever PR mutates into to compensate.  
-When I think   
-how hard PR firms work to score press hits in the traditional   
-media, I can't imagine they'll work any less hard to feed stories
-to bloggers, if they can figure out how.
-Notes[1] PR has at least   
-one beneficial feature: it favors small companies.  If PR didn't  
-work, the only alternative would be to advertise, and only big
-companies can afford that.[2] Advertisers pay 
-less for ads in free publications, because they assume readers 
-ignore something they get for free.  This is why so many trade
-publications nominally have a cover price and yet give away free
-subscriptions with such abandon.[3] Different sections
-of the Times vary so much in their standards that they're
-practically different papers.  Whoever fed the style section reporter
-this story about suits coming back would have been sent packing by
-the regular news reporters.[4] The most striking
-example I know of this type is the "fact" that the Internet worm   
-of 1988 infected 6000 computers. I was there when it was cooked up,
-and this was the recipe: someone guessed that there were about
-60,000 computers attached to the Internet, and that the worm might
-have infected ten percent of them.Actually no one knows how many computers the worm infected, because
-the remedy was to reboot them, and this destroyed all traces.  But
-people like numbers.  And so this one is now replicated
-all over the Internet, like a little worm of its own.[5] Not all were
-necessarily supplied by the PR firm. Reporters sometimes call a few
-additional sources on their own, like someone adding a few fresh 
-vegetables to a can of soup.
-Thanks to Ingrid Basset, Trevor Blackwell, Sarah Harlin, Jessica 
-Livingston, Jackie McDonough, Robert Morris, and Aaron Swartz (who
-also found the PRSA article) for reading drafts of this.Correction: Earlier versions used a recent
-Business Week article mentioning del.icio.us as an example
-of a press hit, but Joshua Schachter tells me 
-it was spontaneous.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/superangels.txt

@@ -1,302 +0,0 @@
-
-
-Want to start a startup?  Get funded by
-Y Combinator.
-
-
-
-
-October 2010After barely changing at all for decades, the startup funding
-business is now in what could, at least by comparison, be called
-turmoil.  At Y Combinator we've seen dramatic changes in the funding
-environment for startups.  Fortunately one of them is much higher
-valuations.The trends we've been seeing are probably not YC-specific.  I wish
-I could say they were, but the main cause is probably just that we
-see trends first—partly because the startups we fund are very
-plugged into the Valley and are quick to take advantage of anything
-new, and partly because we fund so many that we have enough data
-points to see patterns clearly.What we're seeing now, everyone's probably going to be seeing in
-the next couple years.  So I'm going to explain what we're seeing,
-and what that will mean for you if you try to raise money.Super-AngelsLet me start by describing what the world of startup funding used
-to look like.  There used to be two sharply differentiated types
-of investors: angels and venture capitalists.  Angels are individual
-rich people who invest small amounts of their own money, while VCs
-are employees of funds that invest large amounts of other people's.For decades there were just those two types of investors, but now
-a third type has appeared halfway between them: the so-called
-super-angels. 
-[1]
-  And VCs have been provoked by their arrival
-into making a lot of angel-style investments themselves.  So the
-previously sharp line between angels and VCs has become hopelessly
-blurred.There used to be a no man's land between angels and VCs.  Angels
-would invest $20k to $50k apiece, and VCs usually a million or more.
-So an angel round meant a collection of angel investments that
-combined to maybe $200k, and a VC round meant a series A round in
-which a single VC fund (or occasionally two) invested $1-5 million.The no man's land between angels and VCs was a very inconvenient
-one for startups, because it coincided with the amount many wanted
-to raise.  Most startups coming out of Demo Day wanted to raise
-around $400k.  But it was a pain to stitch together that much out
-of angel investments, and most VCs weren't interested in investments
-so small.  That's the fundamental reason the super-angels have
-appeared.  They're responding to the market.The arrival of a new type of investor is big news for startups,
-because there used to be only two and they rarely competed with one
-another.  Super-angels compete with both angels and VCs.  That's
-going to change the rules about how to raise money.  I don't know
-yet what the new rules will be, but it looks like most of the changes
-will be for the better.A super-angel has some of the qualities of an angel, and some of
-the qualities of a VC.  They're usually individuals, like angels.
-In fact many of the current super-angels were initially angels of
-the classic type.  But like VCs, they invest other people's money.
-This allows them to invest larger amounts than angels:  a typical
-super-angel investment is currently about $100k.  They make investment
-decisions quickly, like angels.  And they make a lot more investments
-per partner than VCs—up to 10 times as many.The fact that super-angels invest other people's money makes them
-doubly alarming to VCs. They don't just compete for startups; they
-also compete for investors.  What super-angels really are is a new
-form of fast-moving, lightweight VC fund.   And those of us in the
-technology world know what usually happens when something comes
-along that can be described in terms like that.  Usually it's the
-replacement.Will it be?  As of now, few of the startups that take money from
-super-angels are ruling out taking VC money.  They're just postponing
-it.  But that's still a problem for VCs.  Some of the startups that
-postpone raising VC money may do so well on the angel money they
-raise that they never bother to raise more.  And those who do raise
-VC rounds will be able to get higher valuations when they do.  If
-the best startups get 10x higher valuations when they raise series
-A rounds, that would cut VCs' returns from winners at least tenfold.
-[2]So I think VC funds are seriously threatened by the super-angels.
-But one thing that may save them to some extent is the uneven
-distribution of startup outcomes: practically all the returns are
-concentrated in a few big successes.  The expected value of a startup
-is the percentage chance it's Google.  So to the extent that winning
-is a matter of absolute returns, the super-angels could win practically
-all the battles for individual startups and yet lose the war, if
-they merely failed to get those few big winners.  And there's a
-chance that could happen, because the top VC funds have better
-brands, and can also do more for their portfolio companies.  
-[3]Because super-angels make more investments per partner, they have
-less partner per investment.  They can't pay as much attention to
-you as a VC on your board could.  How much is that extra attention
-worth?  It will vary enormously from one partner to another.  There's
-no consensus yet in the general case.  So for now this is something
-startups are deciding individually.Till now, VCs' claims about how much value they added were sort of
-like the government's.  Maybe they made you feel better, but you
-had no choice in the matter, if you needed money on the scale only
-VCs could supply.  Now that VCs have competitors, that's going to
-put a market price on the help they offer.  The interesting thing
-is, no one knows yet what it will be.Do startups that want to get really big need the sort of advice and
-connections only the top VCs can supply?  Or would super-angel money
-do just as well?  The VCs will say you need them, and the super-angels
-will say you don't.  But the truth is, no one knows yet, not even
-the VCs and super-angels themselves.   All the super-angels know
-is that their new model seems promising enough to be worth trying,
-and all the VCs know is that it seems promising enough to worry
-about.RoundsWhatever the outcome, the conflict between VCs and super-angels is
-good news for founders.  And not just for the obvious reason that
-more competition for deals means better terms.  The whole shape of
-deals is changing.One of the biggest differences between angels and VCs is the amount
-of your company they want.  VCs want a lot.  In a series A round
-they want a third of your company, if they can get it.  They don't
-care much how much they pay for it, but they want a lot because the
-number of series A investments they can do is so small.  In a
-traditional series A investment, at least one partner from the VC
-fund takes a seat on your board.  
-[4]
- Since board seats last about
-5 years and each partner can't handle more than about 10 at once,
-that means a VC fund can only do about 2 series A deals per partner
-per year. And that means they need to get as much of the company
-as they can in each one.  You'd have to be a very promising startup
-indeed to get a VC to use up one of his 10 board seats for only a
-few percent of you.Since angels generally don't take board seats, they don't have this
-constraint.  They're happy to buy only a few percent of you.  And
-although the super-angels are in most respects mini VC funds, they've
-retained this critical property of angels.  They don't take board
-seats, so they don't need a big percentage of your company.Though that means you'll get correspondingly less attention from
-them, it's good news in other respects.  Founders never really liked
-giving up as much equity as VCs wanted.  It was a lot of the company
-to give up in one shot.  Most founders doing series A deals would
-prefer to take half as much money for half as much stock, and then
-see what valuation they could get for the second half of the stock
-after using the first half of the money to increase its value.  But
-VCs never offered that option.Now startups have another alternative.  Now it's easy to raise angel
-rounds about half the size of series A rounds.  Many of the startups
-we fund are taking this route, and I predict that will be true of
-startups in general.A typical big angel round might be $600k on a convertible note with
-a valuation cap of $4 million premoney.  Meaning that when the note
-converts into stock (in a later round, or upon acquisition), the
-investors in that round will get .6 / 4.6, or 13% of the company.
-That's a lot less than the 30 to 40% of the company you usually
-give up in a series A round if you do it so early.  
-[5]But the advantage of these medium-sized rounds is not just that
-they cause less dilution.  You also lose less control.  After an
-angel round, the founders almost always still have control of the
-company, whereas after a series A round they often don't.  The
-traditional board structure after a series A round is two founders,
-two VCs, and a (supposedly) neutral fifth person.  Plus series A
-terms usually give the investors a veto over various kinds of
-important decisions, including selling the company.  Founders usually
-have a lot of de facto control after a series A, as long as things
-are going well.  But that's not the same as just being able to do
-what you want, like you could before.A third and quite significant advantage of angel rounds is that
-they're less stressful to raise.  Raising a traditional series A
-round has in the past taken weeks, if not months.  When a VC firm
-can only do 2 deals per partner per year, they're careful about
-which they do.  To get a traditional series A round you have to go
-through a series of meetings, culminating in a full partner meeting
-where the firm as a whole says yes or no.  That's the really scary
-part for founders: not just that series A rounds take so long, but
-at the end of this long process the VCs might still say no.  The
-chance of getting rejected after the full partner meeting averages
-about 25%.  At some firms it's over 50%.Fortunately for founders, VCs have been getting a lot faster.
-Nowadays Valley VCs are more likely to take 2 weeks than 2 months.
-But they're still not as fast as angels and super-angels, the most
-decisive of whom sometimes decide in hours.Raising an angel round is not only quicker, but you get feedback
-as it progresses.  An angel round is not an all or nothing thing
-like a series A.  It's composed of multiple investors with varying
-degrees of seriousness, ranging from the upstanding ones who commit
-unequivocally to the jerks who give you lines like "come back to
-me to fill out the round." You usually start collecting money from
-the most committed investors and work your way out toward the
-ambivalent ones, whose interest increases as the round fills up.But at each point you know how you're doing.  If investors turn
-cold you may have to raise less, but when investors in an angel
-round turn cold the process at least degrades gracefully, instead
-of blowing up in your face and leaving you with nothing, as happens
-if you get rejected by a VC fund after a full partner meeting.
-Whereas if investors seem hot, you can not only close the round
-faster, but now that convertible notes are becoming the norm,
-actually raise the price to reflect demand.ValuationHowever, the VCs have a weapon they can use against the super-angels,
-and they have started to use it.   VCs have started making angel-sized
-investments too.  The term "angel round" doesn't mean that all the
-investors in it are angels; it just describes the structure of the
-round.  Increasingly the participants include VCs making investments
-of a hundred thousand or two.  And when VCs invest in angel rounds
-they can do things that super-angels don't like.  VCs are quite
-valuation-insensitive in angel rounds—partly because they are
-in general, and partly because they don't care that much about the
-returns on angel rounds, which they still view mostly as a way to
-recruit startups for series A rounds later.  So VCs who invest in
-angel rounds can blow up the valuations for angels and super-angels
-who invest in them. 
-[6]Some super-angels seem to care about valuations.  Several turned
-down YC-funded startups after Demo Day because their valuations
-were too high.  This was not a problem for the startups; by definition
-a high valuation means enough investors were willing to accept it.
-But it was mysterious to me that the super-angels would quibble
-about valuations.  Did they not understand that the big returns
-come from a few big successes, and that it therefore mattered far
-more which startups you picked than how much you paid for them?After thinking about it for a while and observing certain other
-signs, I have a theory that explains why the super-angels may be
-smarter than they seem.  It would make sense for super-angels to
-want low valuations if they're hoping to invest in startups that
-get bought early.  If you're hoping to hit the next Google, you
-shouldn't care if the valuation is 20 million.  But if you're looking
-for companies that are going to get bought for 30 million, you care.
-If you invest at 20 and the company gets bought for 30, you only
-get 1.5x.  You might as well buy Apple.So if some of the super-angels were looking for companies that could
-get acquired quickly, that would explain why they'd care about
-valuations.  But why would they be looking for those?   Because
-depending on the meaning of "quickly," it could actually be very
-profitable.  A company that gets acquired for 30 million is a failure
-to a VC, but it could be a 10x return for an angel, and moreover,
-a quick 10x return.  Rate of return is what matters in
-investing—not the multiple you get, but the multiple per year.
-If a super-angel gets 10x in one year, that's a higher rate of
-return than a VC could ever hope to get from a company that took 6
-years to go public.  To get the same rate of return, the VC would
-have to get a multiple of 10^6—one million x.  Even Google
-didn't come close to that.So I think at least some super-angels are looking for companies
-that will get bought.  That's the only rational explanation for
-focusing on getting the right valuations, instead of the right
-companies.  And if so they'll be different to deal with than VCs.
-They'll be tougher on valuations, but more accommodating if you want
-to sell early.PrognosisWho will win, the super-angels or the VCs?  I think the answer to
-that is, some of each.  They'll each become more like one another.
-The super-angels will start to invest larger amounts, and the VCs
-will gradually figure out ways to make more, smaller investments
-faster.  A decade from now the players will be hard to tell apart,
-and there will probably be survivors from each group.What does that mean for founders?  One thing it means is that the
-high valuations startups are presently getting may not last forever.
-To the extent that valuations are being driven up by price-insensitive
-VCs, they'll fall again if VCs become more like super-angels and
-start to become more miserly about valuations.  Fortunately if this
-does happen it will take years.The short term forecast is more competition between investors, which
-is good news for you.  The super-angels will try to undermine the
-VCs by acting faster, and the VCs will try to undermine the
-super-angels by driving up valuations.  Which for founders will
-result in the perfect combination: funding rounds that close fast,
-with high valuations.But remember that to get that combination, your startup will have
-to appeal to both super-angels and VCs.  If you don't seem like you
-have the potential to go public, you won't be able to use VCs to
-drive up the valuation of an angel round.There is a danger of having VCs in an angel round: the so-called
-signalling risk.  If VCs are only doing it in the hope of investing
-more later, what happens if they don't?  That's a signal to everyone
-else that they think you're lame.How much should you worry about that?  The seriousness of signalling
-risk depends on how far along you are.  If by the next time you
-need to raise money, you have graphs showing rising revenue or
-traffic month after month, you don't have to worry about any signals
-your existing investors are sending.  Your results will speak for
-themselves.  
-[7]Whereas if the next time you need to raise money you won't yet have
-concrete results, you may need to think more about the message your
-investors might send if they don't invest more.  I'm not sure yet
-how much you have to worry, because this whole phenomenon of VCs
-doing angel investments is so new. But my instincts tell me you
-don't have to worry much.  Signalling risk smells like one of those
-things founders worry about that's not a real problem.  As a rule,
-the only thing that can kill a good startup is the startup itself.
-Startups hurt themselves way more often than competitors hurt them,
-for example.  I suspect signalling risk is in this category too.One thing YC-funded startups have been doing to mitigate the risk
-of taking money from VCs in angel rounds is not to take too much
-from any one VC.  Maybe that will help, if you have the luxury of
-turning down money.Fortunately, more and more startups will.  After decades of competition
-that could best be described as intramural, the startup funding
-business is finally getting some real competition.  That should
-last several years at least, and maybe a lot longer. Unless there's
-some huge market crash, the next couple years are going to be a
-good time for startups to raise money.  And that's exciting because
-it means lots more startups will happen.
-Notes[1]
-I've also heard them called "Mini-VCs" and "Micro-VCs." I
-don't know which name will stick.There were a couple predecessors.  Ron Conway had angel funds
-starting in the 1990s, and in some ways First Round Capital is closer to a
-super-angel than a VC fund.[2]
-It wouldn't cut their overall returns tenfold, because investing
-later would probably (a) cause them to lose less on investments
-that failed, and (b) not allow them to get as large a percentage
-of startups as they do now.  So it's hard to predict precisely what
-would happen to their returns.[3]
-The brand of an investor derives mostly from the success of
-their portfolio companies.  The top VCs thus have a big brand
-advantage over the super-angels.  They could make it self-perpetuating
-if they used it to get all the best new startups.  But I don't think
-they'll be able to.  To get all the best startups, you have to do
-more than make them want you.  You also have to want them; you have
-to recognize them when you see them, and that's much harder.
-Super-angels will snap up stars that VCs miss.  And that will cause
-the brand gap between the top VCs and the super-angels gradually
-to erode.[4]
-Though in a traditional series A round VCs put two partners
-on your board, there are signs now that VCs may begin to conserve
-board seats by switching to what used to be considered an angel-round
-board, consisting of two founders and one VC.  Which is also to the
-founders' advantage if it means they still control the company.[5]
-In a series A round, you usually have to give up more than
-the actual amount of stock the VCs buy, because they insist you
-dilute yourselves to set aside an "option pool" as well.  I predict
-this practice will gradually disappear though.[6]
-The best thing for founders, if they can get it, is a convertible
-note with no valuation cap at all.  In that case the money invested
-in the angel round just converts into stock at the valuation of the
-next round, no matter how large.  Angels and super-angels tend not
-to like uncapped notes. They have no idea how much of the company
-they're buying.  If the company does well and the valuation of the
-next round is high, they may end up with only a sliver of it.  So
-by agreeing to uncapped notes, VCs who don't care about valuations
-in angel rounds can make offers that super-angels hate to match.[7]
-Obviously signalling risk is also not a problem if you'll
-never need to raise more money.  But startups are often mistaken
-about that.Thanks to Sam Altman, John Bautista, Patrick Collison, James
-Lindenbaum, Reid Hoffman, Jessica Livingston and Harj Taggar
-for reading drafts
-of this.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/todo.txt

@@ -1,22 +0,0 @@
-April 2012A palliative care nurse called Bronnie Ware made a list of the
-biggest regrets
-of the dying.  Her list seems plausible.  I could see
-myself — can see myself — making at least 4 of these
-5 mistakes.If you had to compress them into a single piece of advice, it might
-be: don't be a cog.  The 5 regrets paint a portrait of post-industrial
-man, who shrinks himself into a shape that fits his circumstances,
-then turns dutifully till he stops.The alarming thing is, the mistakes that produce these regrets are
-all errors of omission.  You forget your dreams, ignore your family,
-suppress your feelings, neglect your friends, and forget to be
-happy.  Errors of omission are a particularly dangerous type of
-mistake, because you make them by default.I would like to avoid making these mistakes.  But how do you avoid
-mistakes you make by default?  Ideally you transform your life so
-it has other defaults.  But it may not be possible to do that
-completely. As long as these mistakes happen by default, you probably
-have to be reminded not to make them.  So I inverted the 5 regrets,
-yielding a list of 5 commands
-
-   Don't ignore your dreams; don't work too much; say what you
-   think; cultivate friendships; be happy.
-
-which I then put at the top of the file I use as a todo list.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/unions.txt

@@ -1,42 +0,0 @@
-May 2007People who worry about the increasing gap between rich and poor
-generally look back on the mid twentieth century as a golden age.
-In those days we had a large number of high-paying union manufacturing
-jobs that boosted the median income.  I wouldn't quite call the
-high-paying union job a myth, but I think people who dwell on it
-are reading too much into it.Oddly enough, it was working with startups that made me realize
-where the high-paying union job came from.  In a rapidly growing
-market, you don't worry too much about efficiency.  It's more
-important to grow fast.  If there's some mundane problem getting
-in your way, and there's a simple solution that's somewhat expensive,
-just take it and get on with more important things.  EBay didn't
-win by paying less for servers than their competitors.Difficult though it may be to imagine now, manufacturing was a
-growth industry in the mid twentieth century.  This was an era when
-small firms making everything from cars to candy were getting
-consolidated into a new kind of corporation with national reach and
-huge economies of scale.  You had to grow fast or die.  Workers
-were for these companies what servers are for an Internet startup.
-A reliable supply was more important than low cost.If you looked in the head of a 1950s auto executive, the attitude
-must have been: sure, give 'em whatever they ask for, so long as
-the new model isn't delayed.In other words, those workers were not paid what their work was
-worth.  Circumstances being what they were, companies would have
-been stupid to insist on paying them so little.If you want a less controversial example of this phenomenon, ask
-anyone who worked as a consultant building web sites during the
-Internet Bubble.  In the late nineties you could get paid huge sums
-of money for building the most trivial things.  And yet does anyone
-who was there have any expectation those days will ever return?  I
-doubt it.  Surely everyone realizes that was just a temporary
-aberration.The era of labor unions seems to have been the same kind of aberration, 
-just spread
-over a longer period, and mixed together with a lot of ideology
-that prevents people from viewing it with as cold an eye as they
-would something like consulting during the Bubble.Basically, unions were just Razorfish.People who think the labor movement was the creation of heroic union
-organizers have a problem to explain: why are unions shrinking now?
-The best they can do is fall back on the default explanation of
-people living in fallen civilizations.  Our ancestors were giants.
-The workers of the early twentieth century must have had a moral
-courage that's lacking today.In fact there's a simpler explanation.  The early twentieth century
-was just a fast-growing startup overpaying for infrastructure.  And
-we in the present are not a fallen people, who have abandoned
-whatever mysterious high-minded principles produced the high-paying
-union job.  We simply live in a time when the fast-growing companies
-overspend on different things.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/useful.txt

@@ -1,225 +0,0 @@
-February 2020What should an essay be? Many people would say persuasive. That's
-what a lot of us were taught essays should be. But I think we can
-aim for something more ambitious: that an essay should be useful.To start with, that means it should be correct. But it's not enough
-merely to be correct. It's easy to make a statement correct by
-making it vague. That's a common flaw in academic writing, for
-example. If you know nothing at all about an issue, you can't go
-wrong by saying that the issue is a complex one, that there are
-many factors to be considered, that it's a mistake to take too
-simplistic a view of it, and so on.Though no doubt correct, such statements tell the reader nothing.
-Useful writing makes claims that are as strong as they can be made
-without becoming false.For example, it's more useful to say that Pike's Peak is near the
-middle of Colorado than merely somewhere in Colorado. But if I say
-it's in the exact middle of Colorado, I've now gone too far, because
-it's a bit east of the middle.Precision and correctness are like opposing forces. It's easy to
-satisfy one if you ignore the other. The converse of vaporous
-academic writing is the bold, but false, rhetoric of demagogues.
-Useful writing is bold, but true.It's also two other things: it tells people something important,
-and that at least some of them didn't already know.Telling people something they didn't know doesn't always mean
-surprising them. Sometimes it means telling them something they
-knew unconsciously but had never put into words. In fact those may
-be the more valuable insights, because they tend to be more
-fundamental.Let's put them all together. Useful writing tells people something
-true and important that they didn't already know, and tells them
-as unequivocally as possible.Notice these are all a matter of degree. For example, you can't
-expect an idea to be novel to everyone. Any insight that you have
-will probably have already been had by at least one of the world's
-7 billion people. But it's sufficient if an idea is novel to a lot
-of readers.Ditto for correctness, importance, and strength. In effect the four
-components are like numbers you can multiply together to get a score
-for usefulness. Which I realize is almost awkwardly reductive, but
-nonetheless true._____
-How can you ensure that the things you say are true and novel and
-important? Believe it or not, there is a trick for doing this. I
-learned it from my friend Robert Morris, who has a horror of saying
-anything dumb. His trick is not to say anything unless he's sure
-it's worth hearing. This makes it hard to get opinions out of him,
-but when you do, they're usually right.Translated into essay writing, what this means is that if you write
-a bad sentence, you don't publish it. You delete it and try again.
-Often you abandon whole branches of four or five paragraphs. Sometimes
-a whole essay.You can't ensure that every idea you have is good, but you can
-ensure that every one you publish is, by simply not publishing the
-ones that aren't.In the sciences, this is called publication bias, and is considered
-bad. When some hypothesis you're exploring gets inconclusive results,
-you're supposed to tell people about that too. But with essay
-writing, publication bias is the way to go.My strategy is loose, then tight. I write the first draft of an
-essay fast, trying out all kinds of ideas. Then I spend days rewriting
-it very carefully.I've never tried to count how many times I proofread essays, but
-I'm sure there are sentences I've read 100 times before publishing
-them. When I proofread an essay, there are usually passages that
-stick out in an annoying way, sometimes because they're clumsily
-written, and sometimes because I'm not sure they're true. The
-annoyance starts out unconscious, but after the tenth reading or
-so I'm saying "Ugh, that part" each time I hit it. They become like
-briars that catch your sleeve as you walk past. Usually I won't
-publish an essay till they're all gone — till I can read through
-the whole thing without the feeling of anything catching.I'll sometimes let through a sentence that seems clumsy, if I can't
-think of a way to rephrase it, but I will never knowingly let through
-one that doesn't seem correct. You never have to. If a sentence
-doesn't seem right, all you have to do is ask why it doesn't, and
-you've usually got the replacement right there in your head.This is where essayists have an advantage over journalists. You
-don't have a deadline. You can work for as long on an essay as you
-need to get it right. You don't have to publish the essay at all,
-if you can't get it right. Mistakes seem to lose courage in the
-face of an enemy with unlimited resources. Or that's what it feels
-like. What's really going on is that you have different expectations
-for yourself. You're like a parent saying to a child "we can sit
-here all night till you eat your vegetables." Except you're the
-child too.I'm not saying no mistake gets through. For example, I added condition
-(c) in "A Way to Detect Bias" 
-after readers pointed out that I'd
-omitted it. But in practice you can catch nearly all of them.There's a trick for getting importance too. It's like the trick I
-suggest to young founders for getting startup ideas: to make something
-you yourself want. You can use yourself as a proxy for the reader.
-The reader is not completely unlike you, so if you write about
-topics that seem important to you, they'll probably seem important
-to a significant number of readers as well.Importance has two factors. It's the number of people something
-matters to, times how much it matters to them. Which means of course
-that it's not a rectangle, but a sort of ragged comb, like a Riemann
-sum.The way to get novelty is to write about topics you've thought about
-a lot. Then you can use yourself as a proxy for the reader in this
-department too. Anything you notice that surprises you, who've
-thought about the topic a lot, will probably also surprise a
-significant number of readers. And here, as with correctness and
-importance, you can use the Morris technique to ensure that you
-will. If you don't learn anything from writing an essay, don't
-publish it.You need humility to measure novelty, because acknowledging the
-novelty of an idea means acknowledging your previous ignorance of
-it. Confidence and humility are often seen as opposites, but in
-this case, as in many others, confidence helps you to be humble.
-If you know you're an expert on some topic, you can freely admit
-when you learn something you didn't know, because you can be confident
-that most other people wouldn't know it either.The fourth component of useful writing, strength, comes from two
-things: thinking well, and the skillful use of qualification. These
-two counterbalance each other, like the accelerator and clutch in
-a car with a manual transmission. As you try to refine the expression
-of an idea, you adjust the qualification accordingly. Something
-you're sure of, you can state baldly with no qualification at all,
-as I did the four components of useful writing. Whereas points that
-seem dubious have to be held at arm's length with perhapses.As you refine an idea, you're pushing in the direction of less
-qualification. But you can rarely get it down to zero. Sometimes
-you don't even want to, if it's a side point and a fully refined
-version would be too long.Some say that qualifications weaken writing. For example, that you
-should never begin a sentence in an essay with "I think," because
-if you're saying it, then of course you think it. And it's true
-that "I think x" is a weaker statement than simply "x." Which is
-exactly why you need "I think." You need it to express your degree
-of certainty.But qualifications are not scalars. They're not just experimental
-error. There must be 50 things they can express: how broadly something
-applies, how you know it, how happy you are it's so, even how it
-could be falsified. I'm not going to try to explore the structure
-of qualification here. It's probably more complex than the whole
-topic of writing usefully. Instead I'll just give you a practical
-tip: Don't underestimate qualification. It's an important skill in
-its own right, not just a sort of tax you have to pay in order to
-avoid saying things that are false. So learn and use its full range.
-It may not be fully half of having good ideas, but it's part of
-having them.There's one other quality I aim for in essays: to say things as
-simply as possible. But I don't think this is a component of
-usefulness. It's more a matter of consideration for the reader. And
-it's a practical aid in getting things right; a mistake is more
-obvious when expressed in simple language. But I'll admit that the
-main reason I write simply is not for the reader's sake or because
-it helps get things right, but because it bothers me to use more
-or fancier words than I need to. It seems inelegant, like a program
-that's too long.I realize florid writing works for some people. But unless you're
-sure you're one of them, the best advice is to write as simply as
-you can._____
-I believe the formula I've given you, importance + novelty +
-correctness + strength, is the recipe for a good essay. But I should
-warn you that it's also a recipe for making people mad.The root of the problem is novelty. When you tell people something
-they didn't know, they don't always thank you for it. Sometimes the
-reason people don't know something is because they don't want to
-know it. Usually because it contradicts some cherished belief. And
-indeed, if you're looking for novel ideas, popular but mistaken
-beliefs are a good place to find them. Every popular mistaken belief
-creates a dead zone of ideas around 
-it that are relatively unexplored because they contradict it.The strength component just makes things worse. If there's anything
-that annoys people more than having their cherished assumptions
-contradicted, it's having them flatly contradicted.Plus if you've used the Morris technique, your writing will seem
-quite confident. Perhaps offensively confident, to people who
-disagree with you. The reason you'll seem confident is that you are
-confident: you've cheated, by only publishing the things you're
-sure of.  It will seem to people who try to disagree with you that
-you never admit you're wrong. In fact you constantly admit you're
-wrong. You just do it before publishing instead of after.And if your writing is as simple as possible, that just makes things
-worse. Brevity is the diction of command. If you watch someone
-delivering unwelcome news from a position of inferiority, you'll
-notice they tend to use lots of words, to soften the blow. Whereas
-to be short with someone is more or less to be rude to them.It can sometimes work to deliberately phrase statements more weakly
-than you mean. To put "perhaps" in front of something you're actually
-quite sure of. But you'll notice that when writers do this, they
-usually do it with a wink.I don't like to do this too much. It's cheesy to adopt an ironic
-tone for a whole essay. I think we just have to face the fact that
-elegance and curtness are two names for the same thing.You might think that if you work sufficiently hard to ensure that
-an essay is correct, it will be invulnerable to attack. That's sort
-of true. It will be invulnerable to valid attacks. But in practice
-that's little consolation.In fact, the strength component of useful writing will make you
-particularly vulnerable to misrepresentation. If you've stated an
-idea as strongly as you could without making it false, all anyone
-has to do is to exaggerate slightly what you said, and now it is
-false.Much of the time they're not even doing it deliberately. One of the
-most surprising things you'll discover, if you start writing essays,
-is that people who disagree with you rarely disagree with what
-you've actually written. Instead they make up something you said
-and disagree with that.For what it's worth, the countermove is to ask someone who does
-this to quote a specific sentence or passage you wrote that they
-believe is false, and explain why. I say "for what it's worth"
-because they never do. So although it might seem that this could
-get a broken discussion back on track, the truth is that it was
-never on track in the first place.Should you explicitly forestall likely misinterpretations? Yes, if
-they're misinterpretations a reasonably smart and well-intentioned
-person might make. In fact it's sometimes better to say something
-slightly misleading and then add the correction than to try to get
-an idea right in one shot. That can be more efficient, and can also
-model the way such an idea would be discovered.But I don't think you should explicitly forestall intentional
-misinterpretations in the body of an essay. An essay is a place to
-meet honest readers. You don't want to spoil your house by putting
-bars on the windows to protect against dishonest ones. The place
-to protect against intentional misinterpretations is in end-notes.
-But don't think you can predict them all. People are as ingenious
-at misrepresenting you when you say something they don't want to
-hear as they are at coming up with rationalizations for things they
-want to do but know they shouldn't. I suspect it's the same skill._____
-As with most other things, the way to get better at writing essays
-is to practice. But how do you start? Now that we've examined the
-structure of useful writing, we can rephrase that question more
-precisely. Which constraint do you relax initially? The answer is,
-the first component of importance: the number of people who care
-about what you write.If you narrow the topic sufficiently, you can probably find something
-you're an expert on. Write about that to start with. If you only
-have ten readers who care, that's fine. You're helping them, and
-you're writing. Later you can expand the breadth of topics you write
-about.The other constraint you can relax is a little surprising: publication.
-Writing essays doesn't have to mean publishing them. That may seem
-strange now that the trend is to publish every random thought, but
-it worked for me. I wrote what amounted to essays in notebooks for
-about 15 years. I never published any of them and never expected
-to. I wrote them as a way of figuring things out. But when the web
-came along I'd had a lot of practice.Incidentally, 
-Steve 
-Wozniak did the same thing. In high school he
-designed computers on paper for fun. He couldn't build them because
-he couldn't afford the components. But when Intel launched 4K DRAMs
-in 1975, he was ready._____
-How many essays are there left to write though? The answer to that
-question is probably the most exciting thing I've learned about
-essay writing. Nearly all of them are left to write.Although the essay 
-is an old form, it hasn't been assiduously
-cultivated. In the print era, publication was expensive, and there
-wasn't enough demand for essays to publish that many. You could
-publish essays if you were already well known for writing something
-else, like novels. Or you could write book reviews that you took
-over to express your own ideas. But there was not really a direct
-path to becoming an essayist. Which meant few essays got written,
-and those that did tended to be about a narrow range of subjects.Now, thanks to the internet, there's a path. Anyone can publish
-essays online. You start in obscurity, perhaps, but at least you
-can start. You don't need anyone's permission.It sometimes happens that an area of knowledge sits quietly for
-years, till some change makes it explode. Cryptography did this to
-number theory. The internet is doing it to the essay.The exciting thing is not that there's a lot left to write, but
-that there's a lot left to discover. There's a certain kind of idea
-that's best discovered by writing essays. If most essays are still
-unwritten, most such ideas are still undiscovered.Notes[1] Put railings on the balconies, but don't put bars on the windows.[2] Even now I sometimes write essays that are not meant for
-publication. I wrote several to figure out what Y Combinator should
-do, and they were really helpful.Thanks to Trevor Blackwell, Daniel Gackle, Jessica Livingston, and
-Robert Morris for reading drafts of this.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/vb.txt

@@ -1,129 +0,0 @@
-January 2016Life is short, as everyone knows. When I was a kid I used to wonder
-about this. Is life actually short, or are we really complaining
-about its finiteness?  Would we be just as likely to feel life was
-short if we lived 10 times as long?Since there didn't seem any way to answer this question, I stopped
-wondering about it.  Then I had kids.  That gave me a way to answer
-the question, and the answer is that life actually is short.Having kids showed me how to convert a continuous quantity, time,
-into discrete quantities. You only get 52 weekends with your 2 year
-old.  If Christmas-as-magic lasts from say ages 3 to 10, you only
-get to watch your child experience it 8 times.  And while it's
-impossible to say what is a lot or a little of a continuous quantity
-like time, 8 is not a lot of something.  If you had a handful of 8
-peanuts, or a shelf of 8 books to choose from, the quantity would
-definitely seem limited, no matter what your lifespan was.Ok, so life actually is short.  Does it make any difference to know
-that?It has for me.  It means arguments of the form "Life is too short
-for x" have great force.  It's not just a figure of speech to say
-that life is too short for something.  It's not just a synonym for
-annoying.  If you find yourself thinking that life is too short for
-something, you should try to eliminate it if you can.When I ask myself what I've found life is too short for, the word
-that pops into my head is "bullshit." I realize that answer is
-somewhat tautological.  It's almost the definition of bullshit that
-it's the stuff that life is too short for.  And yet bullshit does
-have a distinctive character.  There's something fake about it.
-It's the junk food of experience.
-[1]If you ask yourself what you spend your time on that's bullshit,
-you probably already know the answer.  Unnecessary meetings, pointless
-disputes, bureaucracy, posturing, dealing with other people's
-mistakes, traffic jams, addictive but unrewarding pastimes.There are two ways this kind of thing gets into your life: it's
-either forced on you, or it tricks you.  To some extent you have to
-put up with the bullshit forced on you by circumstances.  You need
-to make money, and making money consists mostly of errands.  Indeed,
-the law of supply and demand insures that: the more rewarding some
-kind of work is, the cheaper people will do it.  It may be that
-less bullshit is forced on you than you think, though.  There has
-always been a stream of people who opt out of the default grind and
-go live somewhere where opportunities are fewer in the conventional
-sense, but life feels more authentic.  This could become more common.You can do it on a smaller scale without moving.  The amount of
-time you have to spend on bullshit varies between employers.  Most
-large organizations (and many small ones) are steeped in it.  But
-if you consciously prioritize bullshit avoidance over other factors
-like money and prestige, you can probably find employers that will
-waste less of your time.If you're a freelancer or a small company, you can do this at the
-level of individual customers.  If you fire or avoid toxic customers,
-you can decrease the amount of bullshit in your life by more than
-you decrease your income.But while some amount of bullshit is inevitably forced on you, the
-bullshit that sneaks into your life by tricking you is no one's
-fault but your own.  And yet the bullshit you choose may be harder
-to eliminate than the bullshit that's forced on you.  Things that
-lure you into wasting your time have to be really good at
-tricking you.  An example that will be familiar to a lot of people
-is arguing online.  When someone
-contradicts you, they're in a sense attacking you. Sometimes pretty
-overtly.  Your instinct when attacked is to defend yourself.  But
-like a lot of instincts, this one wasn't designed for the world we
-now live in.  Counterintuitive as it feels, it's better most of
-the time not to defend yourself.  Otherwise these people are literally
-taking your life.
-[2]Arguing online is only incidentally addictive. There are more
-dangerous things than that. As I've written before, one byproduct
-of technical progress is that things we like tend to become more
-addictive.  Which means we will increasingly have to make a conscious
-effort to avoid addictions — to stand outside ourselves and ask "is
-this how I want to be spending my time?"As well as avoiding bullshit, one should actively seek out things
-that matter.  But different things matter to different people, and
-most have to learn what matters to them.  A few are lucky and realize
-early on that they love math or taking care of animals or writing,
-and then figure out a way to spend a lot of time doing it.  But
-most people start out with a life that's a mix of things that
-matter and things that don't, and only gradually learn to distinguish
-between them.For the young especially, much of this confusion is induced by the
-artificial situations they find themselves in. In middle school and
-high school, what the other kids think of you seems the most important
-thing in the world.  But when you ask adults what they got wrong
-at that age, nearly all say they cared too much what other kids
-thought of them.One heuristic for distinguishing stuff that matters is to ask
-yourself whether you'll care about it in the future.  Fake stuff
-that matters usually has a sharp peak of seeming to matter.  That's
-how it tricks you.  The area under the curve is small, but its shape
-jabs into your consciousness like a pin.The things that matter aren't necessarily the ones people would
-call "important."  Having coffee with a friend matters.  You won't
-feel later like that was a waste of time.One great thing about having small children is that they make you
-spend time on things that matter: them. They grab your sleeve as
-you're staring at your phone and say "will you play with me?" And
-odds are that is in fact the bullshit-minimizing option.If life is short, we should expect its shortness to take us by
-surprise. And that is just what tends to happen.  You take things
-for granted, and then they're gone.  You think you can always write
-that book, or climb that mountain, or whatever, and then you realize
-the window has closed.  The saddest windows close when other people
-die. Their lives are short too.  After my mother died, I wished I'd
-spent more time with her.  I lived as if she'd always be there.
-And in her typical quiet way she encouraged that illusion.  But an
-illusion it was. I think a lot of people make the same mistake I
-did.The usual way to avoid being taken by surprise by something is to
-be consciously aware of it.  Back when life was more precarious,
-people used to be aware of death to a degree that would now seem a
-bit morbid.  I'm not sure why, but it doesn't seem the right answer
-to be constantly reminding oneself of the grim reaper hovering at
-everyone's shoulder.  Perhaps a better solution is to look at the
-problem from the other end. Cultivate a habit of impatience about
-the things you most want to do. Don't wait before climbing that
-mountain or writing that book or visiting your mother.  You don't
-need to be constantly reminding yourself why you shouldn't wait.
-Just don't wait.I can think of two more things one does when one doesn't have much
-of something: try to get more of it, and savor what one has.  Both
-make sense here.How you live affects how long you live.  Most people could do better.
-Me among them.But you can probably get even more effect by paying closer attention
-to the time you have.  It's easy to let the days rush by.  The
-"flow" that imaginative people love so much has a darker cousin
-that prevents you from pausing to savor life amid the daily slurry
-of errands and alarms.  One of the most striking things I've read
-was not in a book, but the title of one: James Salter's Burning
-the Days.It is possible to slow time somewhat. I've gotten better at it.
-Kids help.  When you have small children, there are a lot of moments
-so perfect that you can't help noticing.It does help too to feel that you've squeezed everything out of
-some experience.  The reason I'm sad about my mother is not just
-that I miss her but that I think of all the things we could have
-done that we didn't.  My oldest son will be 7 soon.  And while I
-miss the 3 year old version of him, I at least don't have any regrets
-over what might have been.  We had the best time a daddy and a 3
-year old ever had.Relentlessly prune bullshit, don't wait to do things that matter,
-and savor the time you have.  That's what you do when life is short.Notes[1]
-At first I didn't like it that the word that came to mind was
-one that had other meanings.  But then I realized the other meanings
-are fairly closely related.  Bullshit in the sense of things you
-waste your time on is a lot like intellectual bullshit.[2]
-I chose this example deliberately as a note to self.  I get
-attacked a lot online.  People tell the craziest lies about me.
-And I have so far done a pretty mediocre job of suppressing the
-natural human inclination to say "Hey, that's not true!"Thanks to Jessica Livingston and Geoff Ralston for reading drafts
-of this.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/vcsqueeze.txt

@@ -1,124 +0,0 @@
-November 2005In the next few years, venture capital funds will find themselves
-squeezed from four directions.  They're already stuck with a seller's
-market, because of the huge amounts they raised at the end of the
-Bubble and still haven't invested.  This by itself is not the end
-of the world.  In fact, it's just a more extreme version of the
-norm
-in the VC business: too much money chasing too few deals.Unfortunately, those few deals now want less and less money, because
-it's getting so cheap to start a startup.  The four causes: open
-source, which makes software free; Moore's law, which makes hardware
-geometrically closer to free; the Web, which makes promotion free
-if you're good; and better languages, which make development a lot
-cheaper.When we started our startup in 1995, the first three were our biggest
-expenses.  We had to pay $5000 for the Netscape Commerce Server,
-the only software that then supported secure http connections.  We
-paid $3000 for a server with a 90 MHz processor and 32 meg of
-memory.  And we paid a PR firm about $30,000 to promote our launch.Now you could get all three for nothing.  You can get the software
-for free; people throw away computers more powerful than our first
-server; and if you make something good you can generate ten times
-as much traffic by word of mouth online than our first PR firm got
-through the print media.And of course another big change for the average startup is that
-programming languages have improved-- or rather, the median language has.  At most startups ten years
-ago, software development meant ten programmers writing code in
-C++.  Now the same work might be done by one or two using Python
-or Ruby.During the Bubble, a lot of people predicted that startups would
-outsource their development to India.  I think a better model for
-the future is David Heinemeier Hansson, who outsourced his development
-to a more powerful language instead.  A lot of well-known applications
-are now, like BaseCamp, written by just one programmer.  And one
-guy is more than 10x cheaper than ten, because (a) he won't waste
-any time in meetings, and (b) since he's probably a founder, he can
-pay himself nothing.Because starting a startup is so cheap, venture capitalists now
-often want to give startups more money than the startups want to
-take.  VCs like to invest several million at a time.  But as one
-VC told me after a startup he funded would only take about half a
-million, "I don't know what we're going to do.  Maybe we'll just
-have to give some of it back." Meaning give some of the fund back
-to the institutional investors who supplied it, because it wasn't
-going to be possible to invest it all.Into this already bad situation comes the third problem: Sarbanes-Oxley.
-Sarbanes-Oxley is a law, passed after the Bubble, that drastically
-increases the regulatory burden on public companies. And in addition
-to the cost of compliance, which is at least two million dollars a
-year, the law introduces frightening legal exposure for corporate
-officers.  An experienced CFO I know said flatly: "I would not
-want to be CFO of a public company now."You might think that responsible corporate governance is an area
-where you can't go too far.  But you can go too far in any law, and
-this remark convinced me that Sarbanes-Oxley must have.  This CFO
-is both the smartest and the most upstanding money guy I know.  If
-Sarbanes-Oxley deters people like him from being CFOs of public  
-companies, that's proof enough that it's broken.Largely because of Sarbanes-Oxley, few startups go public now.  For
-all practical purposes, succeeding now equals getting bought.  Which
-means VCs are now in the business of finding promising little 2-3
-man startups and pumping them up into companies that cost $100
-million to acquire.   They didn't mean to be in this business; it's
-just what their business has evolved into.Hence the fourth problem: the acquirers have begun to realize they
-can buy wholesale.  Why should they wait for VCs to make the startups
-they want more expensive?  Most of what the VCs add, acquirers don't
-want anyway.  The acquirers already have brand recognition and HR
-departments.  What they really want is the software and the developers,
-and that's what the startup is in the early phase: concentrated
-software and developers.Google, typically, seems to have been the first to figure this out.
-"Bring us your startups early," said Google's speaker at the Startup School.  They're quite
-explicit about it: they like to acquire startups at just the point
-where they would do a Series A round.  (The Series A round is the
-first round of real VC funding; it usually happens in the first
-year.) It is a brilliant strategy, and one that other big technology
-companies will no doubt try to duplicate.  Unless they want to have 
-still more of their lunch eaten by Google.Of course, Google has an advantage in buying startups: a lot of the
-people there are rich, or expect to be when their options vest.
-Ordinary employees find it very hard to recommend an acquisition;
-it's just too annoying to see a bunch of twenty year olds get rich
-when you're still working for salary.  Even if it's the right thing   
-for your company to do.The Solution(s)Bad as things look now, there is a way for VCs to save themselves.
-They need to do two things, one of which won't surprise them, and  
-another that will seem an anathema.Let's start with the obvious one: lobby to get Sarbanes-Oxley  
-loosened.  This law was created to prevent future Enrons, not to
-destroy the IPO market.  Since the IPO market was practically dead
-when it passed, few saw what bad effects it would have.  But now 
-that technology has recovered from the last bust, we can see clearly
-what a bottleneck Sarbanes-Oxley has become.Startups are fragile plants—seedlings, in fact.  These seedlings
-are worth protecting, because they grow into the trees of the
-economy.  Much of the economy's growth is their growth.  I think
-most politicians realize that.  But they don't realize just how   
-fragile startups are, and how easily they can become collateral
-damage of laws meant to fix some other problem.Still more dangerously, when you destroy startups, they make very
-little noise.  If you step on the toes of the coal industry, you'll
-hear about it.  But if you inadvertantly squash the startup industry,
-all that happens is that the founders of the next Google stay in 
-grad school instead of starting a company.My second suggestion will seem shocking to VCs: let founders cash  
-out partially in the Series A round.  At the moment, when VCs invest
-in a startup, all the stock they get is newly issued and all the 
-money goes to the company.  They could buy some stock directly from
-the founders as well.Most VCs have an almost religious rule against doing this.  They
-don't want founders to get a penny till the company is sold or goes
-public.  VCs are obsessed with control, and they worry that they'll
-have less leverage over the founders if the founders have any money.This is a dumb plan.  In fact, letting the founders sell a little stock
-early would generally be better for the company, because it would
-cause the founders' attitudes toward risk to be aligned with the
-VCs'.  As things currently work, their attitudes toward risk tend
-to be diametrically opposed: the founders, who have nothing, would
-prefer a 100% chance of $1 million to a 20% chance of $10 million,
-while the VCs can afford to be "rational" and prefer the latter.Whatever they say, the reason founders are selling their companies
-early instead of doing Series A rounds is that they get paid up
-front.  That first million is just worth so much more than the
-subsequent ones.  If founders could sell a little stock early,
-they'd be happy to take VC money and bet the rest on a bigger
-outcome.So why not let the founders have that first million, or at least
-half million?  The VCs would get same number of shares for the   
-money.  So what if some of the money would go to the  
-founders instead of the company?Some VCs will say this is
-unthinkable—that they want all their money to be put to work
-growing the company.  But the fact is, the huge size of current VC
-investments is dictated by the structure
-of VC funds, not the needs of startups.  Often as not these large  
-investments go to work destroying the company rather than growing
-it.The angel investors who funded our startup let the founders sell
-some stock directly to them, and it was a good deal for everyone. 
-The angels made a huge return on that investment, so they're happy.
-And for us founders it blunted the terrifying all-or-nothingness
-of a startup, which in its raw form is more a distraction than a
-motivator.If VCs are frightened at the idea of letting founders partially
-cash out, let me tell them something still more frightening: you
-are now competing directly with Google.
-Thanks to Trevor Blackwell, Sarah Harlin, Jessica
-Livingston, and Robert Morris for reading drafts of this.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/vw.txt

@@ -1,75 +0,0 @@
-January 2012A few hours before the Yahoo acquisition was announced in June 1998
-I took a snapshot of Viaweb's
-site.  I thought it might be interesting to look at one day.The first thing one notices is is how tiny the pages are.  Screens
-were a lot smaller in 1998.  If I remember correctly, our frontpage
-used to just fit in the size window people typically used then.Browsers then (IE 6 was still 3 years in the future) had few fonts
-and they weren't antialiased.  If you wanted to make pages that
-looked good, you had to render display text as images.You may notice a certain similarity between the Viaweb and Y Combinator logos.  We did that
-as an inside joke when we started YC.  Considering how basic a red
-circle is, it seemed surprising to me when we started Viaweb how
-few other companies used one as their logo.  A bit later I realized
-why.On the Company
-page you'll notice a mysterious individual called John McArtyem.
-Robert Morris (aka Rtm) was so publicity averse after the 
-Worm that he
-didn't want his name on the site.  I managed to get him to agree
-to a compromise: we could use his bio but not his name.  He has
-since relaxed a bit
-on that point.Trevor graduated at about the same time the acquisition closed, so in the
-course of 4 days he went from impecunious grad student to millionaire
-PhD.  The culmination of my career as a writer of press releases
-was one celebrating
-his graduation, illustrated with a drawing I did of him during
-a meeting.(Trevor also appears as Trevino
-Bagwell in our directory of web designers merchants could hire
-to build stores for them.  We inserted him as a ringer in case some
-competitor tried to spam our web designers.   We assumed his logo
-would deter any actual customers, but it did not.)Back in the 90s, to get users you had to get mentioned in magazines
-and newspapers.  There were not the same ways to get found online
-that there are today.  So we used to pay a PR
-firm $16,000 a month to get us mentioned in the press.  Fortunately
-reporters liked
-us.In our advice about
-getting traffic from search engines (I don't think the term SEO
-had been coined yet), we say there are only 7 that matter: Yahoo,
-AltaVista, Excite, WebCrawler, InfoSeek, Lycos, and HotBot.  Notice
-anything missing?  Google was incorporated that September.We supported online transactions via a company called 
-Cybercash,
-since if we lacked that feature we'd have gotten beaten up in product
-comparisons.  But Cybercash was so bad and most stores' order volumes
-were so low that it was better if merchants processed orders like phone orders.  We had a page in our site trying to talk merchants
-out of doing real time authorizations.The whole site was organized like a funnel, directing people to the
-test drive.
-It was a novel thing to be able to try out software online.  We put
-cgi-bin in our dynamic urls to fool competitors about how our
-software worked.We had some well
-known users.  Needless to say, Frederick's of Hollywood got the
-most traffic.  We charged a flat fee of $300/month for big stores,
-so it was a little alarming to have users who got lots of traffic.
-I once calculated how much Frederick's was costing us in bandwidth,
-and it was about $300/month.Since we hosted all the stores, which together were getting just
-over 10 million page views per month in June 1998, we consumed what
-at the time seemed a lot of bandwidth.  We had 2 T1s (3 Mb/sec)
-coming into our offices.  In those days there was no AWS.  Even
-colocating servers seemed too risky, considering how often things
-went wrong with them.  So we had our servers in our offices.  Or
-more precisely, in Trevor's office.  In return for the unique
-privilege of sharing his office with no other humans, he had to
-share it with 6 shrieking tower servers.  His office was nicknamed
-the Hot Tub on account of the heat they generated.  Most days his
-stack of window air conditioners could keep up.For describing pages, we had a template language called RTML, which
-supposedly stood for something, but which in fact I named after
-Rtm.  RTML was Common Lisp augmented by some macros and libraries,
-and concealed under a structure editor that made it look like it
-had syntax.Since we did continuous releases, our software didn't actually have
-versions.  But in those days the trade press expected versions, so
-we made them up.  If we wanted to get lots of attention, we made
-the version number an
-integer.  That "version 4.0" icon was generated by our own
-button generator, incidentally.  The whole Viaweb site was made
-with our software, even though it wasn't an online store, because
-we wanted to experience what our users did.At the end of 1997, we released a general purpose shopping search
-engine called Shopfind.  It
-was pretty advanced for the time.  It had a programmable crawler
-that could crawl most of the different stores online and pick out
-the products.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/want.txt

@@ -1,43 +0,0 @@
-November 2022Since I was about 9 I've been puzzled by the apparent contradiction
-between being made of matter that behaves in a predictable way, and
-the feeling that I could choose to do whatever I wanted. At the
-time I had a self-interested motive for exploring the question. At
-that age (like most succeeding ages) I was always in trouble with
-the authorities, and it seemed to me that there might possibly be
-some way to get out of trouble by arguing that I wasn't responsible
-for my actions. I gradually lost hope of that, but the puzzle
-remained: How do you reconcile being a machine made of matter with
-the feeling that you're free to choose what you do?
-[1]The best way to explain the answer may be to start with a slightly
-wrong version, and then fix it. The wrong version is: You can do
-what you want, but you can't want what you want. Yes, you can control
-what you do, but you'll do what you want, and you can't control
-that.The reason this is mistaken is that people do sometimes change what
-they want. People who don't want to want something — drug addicts,
-for example — can sometimes make themselves stop wanting it. And
-people who want to want something — who want to like classical
-music, or broccoli — sometimes succeed.So we modify our initial statement: You can do what you want, but
-you can't want to want what you want.That's still not quite true. It's possible to change what you want
-to want. I can imagine someone saying "I decided to stop wanting
-to like classical music." But we're getting closer to the truth.
-It's rare for people to change what they want to want, and the more
-"want to"s we add, the rarer it gets.We can get arbitrarily close to a true statement by adding more "want
-to"s in much the same way we can get arbitrarily close to 1 by adding
-more 9s to a string of 9s following a decimal point. In practice
-three or four "want to"s must surely be enough. It's hard even to
-envision what it would mean to change what you want to want to want
-to want, let alone actually do it.So one way to express the correct answer is to use a regular
-expression. You can do what you want, but there's some statement
-of the form "you can't (want to)* want what you want" that's true.
-Ultimately you get back to a want that you don't control.
-[2]
-Notes[1]
-I didn't know when I was 9 that matter might behave randomly,
-but I don't think it affects the problem much. Randomness destroys
-the ghost in the machine as effectively as determinism.[2]
-If you don't like using an expression, you can make the same
-point using higher-order desires: There is some n such that you
-don't control your nth-order desires.
-Thanks to Trevor Blackwell,
-Jessica Livingston, Robert Morris, and
-Michael Nielsen for reading drafts of this.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/web20.txt

@@ -1,299 +0,0 @@
-
-
-Want to start a startup?  Get funded by
-Y Combinator.
-
-
-
-
-November 2005Does "Web 2.0" mean anything?  Till recently I thought it didn't,
-but the truth turns out to be more complicated.  Originally, yes,
-it was meaningless.  Now it seems to have acquired a meaning.  And
-yet those who dislike the term are probably right, because if it
-means what I think it does, we don't need it.I first heard the phrase "Web 2.0" in the name of the Web 2.0
-conference in 2004.  At the time it was supposed to mean using "the
-web as a platform," which I took to refer to web-based applications.
-[1]So I was surprised at a conference this summer when Tim O'Reilly
-led a session intended to figure out a definition of "Web 2.0."
-Didn't it already mean using the web as a platform?  And if it
-didn't already mean something, why did we need the phrase at all?OriginsTim says the phrase "Web 2.0" first
-arose in "a brainstorming session between
-O'Reilly and Medialive International." What is Medialive International?
-"Producers of technology tradeshows and conferences," according to
-their site.  So presumably that's what this brainstorming session
-was about.  O'Reilly wanted to organize a conference about the web,
-and they were wondering what to call it.I don't think there was any deliberate plan to suggest there was a
-new version of the web.  They just wanted to make the point
-that the web mattered again.  It was a kind of semantic deficit
-spending: they knew new things were coming, and the "2.0" referred
-to whatever those might turn out to be.And they were right.  New things were coming.  But the new version
-number led to some awkwardness in the short term.  In the process
-of developing the pitch for the first conference, someone must have
-decided they'd better take a stab at explaining what that "2.0"
-referred to.  Whatever it meant, "the web as a platform" was at
-least not too constricting.The story about "Web 2.0" meaning the web as a platform didn't live
-much past the first conference.  By the second conference, what
-"Web 2.0" seemed to mean was something about democracy.  At least,
-it did when people wrote about it online.  The conference itself
-didn't seem very grassroots.  It cost $2800, so the only people who
-could afford to go were VCs and people from big companies.And yet, oddly enough, Ryan Singel's article
-about the conference in Wired News spoke of "throngs of
-geeks."  When a friend of mine asked Ryan about this, it was news
-to him.  He said he'd originally written something like "throngs
-of VCs and biz dev guys" but had later shortened it just to "throngs,"
-and that this must have in turn been expanded by the editors into
-"throngs of geeks."  After all, a Web 2.0 conference would presumably
-be full of geeks, right?Well, no.  There were about 7.  Even Tim O'Reilly was wearing a   
-suit, a sight so alien I couldn't parse it at first.  I saw
-him walk by and said to one of the O'Reilly people "that guy looks
-just like Tim.""Oh, that's Tim.  He bought a suit."
-I ran after him, and sure enough, it was.  He explained that he'd
-just bought it in Thailand.The 2005 Web 2.0 conference reminded me of Internet trade shows
-during the Bubble, full of prowling VCs looking for the next hot
-startup.  There was that same odd atmosphere created by a large  
-number of people determined not to miss out.  Miss out on what?
-They didn't know.  Whatever was going to happen—whatever Web 2.0
-turned out to be.I wouldn't quite call it "Bubble 2.0" just because VCs are eager
-to invest again.  The Internet is a genuinely big deal.  The bust
-was as much an overreaction as
-the boom.  It's to be expected that once we started to pull out of
-the bust, there would be a lot of growth in this area, just as there
-was in the industries that spiked the sharpest before the Depression.The reason this won't turn into a second Bubble is that the IPO
-market is gone.  Venture investors
-are driven by exit strategies.  The reason they were funding all  
-those laughable startups during the late 90s was that they hoped
-to sell them to gullible retail investors; they hoped to be laughing
-all the way to the bank.  Now that route is closed.  Now the default
-exit strategy is to get bought, and acquirers are less prone to
-irrational exuberance than IPO investors.  The closest you'll get 
-to Bubble valuations is Rupert Murdoch paying $580 million for   
-Myspace.  That's only off by a factor of 10 or so.1. AjaxDoes "Web 2.0" mean anything more than the name of a conference
-yet?  I don't like to admit it, but it's starting to.  When people
-say "Web 2.0" now, I have some idea what they mean.  And the fact
-that I both despise the phrase and understand it is the surest proof
-that it has started to mean something.One ingredient of its meaning is certainly Ajax, which I can still
-only just bear to use without scare quotes.  Basically, what "Ajax"
-means is "Javascript now works."  And that in turn means that
-web-based applications can now be made to work much more like desktop
-ones.As you read this, a whole new generation
-of software is being written to take advantage of Ajax.  There
-hasn't been such a wave of new applications since microcomputers
-first appeared.  Even Microsoft sees it, but it's too late for them
-to do anything more than leak "internal"  
-documents designed to give the impression they're on top of this
-new trend.In fact the new generation of software is being written way too
-fast for Microsoft even to channel it, let alone write their own
-in house.  Their only hope now is to buy all the best Ajax startups
-before Google does.  And even that's going to be hard, because
-Google has as big a head start in buying microstartups as it did
-in search a few years ago.  After all, Google Maps, the canonical
-Ajax application, was the result of a startup they bought.So ironically the original description of the Web 2.0 conference
-turned out to be partially right: web-based applications are a big
-component of Web 2.0.  But I'm convinced they got this right by 
-accident.  The Ajax boom didn't start till early 2005, when Google
-Maps appeared and the term "Ajax" was coined.2. DemocracyThe second big element of Web 2.0 is democracy.  We now have several
-examples to prove that amateurs can   
-surpass professionals, when they have the right kind of system to 
-channel their efforts.  Wikipedia
-may be the most famous.  Experts have given Wikipedia middling
-reviews, but they miss the critical point: it's good enough.  And   
-it's free, which means people actually read it.  On the web, articles
-you have to pay for might as well not exist.  Even if you were    
-willing to pay to read them yourself, you can't link to them.    
-They're not part of the conversation.Another place democracy seems to win is in deciding what counts as
-news.  I never look at any news site now except Reddit.
-[2]
- I know if something major
-happens, or someone writes a particularly interesting article, it   
-will show up there.  Why bother checking the front page of any
-specific paper or magazine?  Reddit's like an RSS feed for the whole
-web, with a filter for quality.  Similar sites include Digg, a technology news site that's
-rapidly approaching Slashdot in popularity, and del.icio.us, the collaborative
-bookmarking network that set off the "tagging" movement.  And whereas
-Wikipedia's main appeal is that it's good enough and free, these
-sites suggest that voters do a significantly better job than human
-editors.The most dramatic example of Web 2.0 democracy is not in the selection
-of ideas, but their production.  
-I've noticed for a while that the stuff I read on individual people's
-sites is as good as or better than the stuff I read in newspapers
-and magazines.  And now I have independent evidence: the top links
-on Reddit are generally links to individual people's sites rather  
-than to magazine articles or news stories.My experience of writing
-for magazines suggests an explanation.  Editors.  They control the
-topics you can write about, and they can generally rewrite whatever
-you produce.  The result is to damp extremes.  Editing yields 95th
-percentile writing—95% of articles are improved by it, but 5% are
-dragged down.  5% of the time you get "throngs of geeks."On the web, people can publish whatever they want.  Nearly all of
-it falls short of the editor-damped writing in print publications.
-But the pool of writers is very, very large.  If it's large enough,
-the lack of damping means the best writing online should surpass  
-the best in print.
-[3]  
-And now that the web has evolved mechanisms
-for selecting good stuff, the web wins net.  Selection beats damping,
-for the same reason market economies beat centrally planned ones.Even the startups are different this time around.  They are to the  
-startups of the Bubble what bloggers are to the print media.  During
-the Bubble, a startup meant a company headed by an MBA that was   
-blowing through several million dollars of VC money to "get big
-fast" in the most literal sense.  Now it means a smaller, younger, more technical group that just      
-decided to make something great.  They'll decide later if they want  
-to raise VC-scale funding, and if they take it, they'll take it on
-their terms.3. Don't Maltreat UsersI think everyone would agree that democracy and Ajax are elements
-of "Web 2.0."  I also see a third: not to maltreat users.  During
-the Bubble a lot of popular sites were quite high-handed with users.
-And not just in obvious ways, like making them register, or subjecting
-them to annoying ads.  The very design of the average site in the   
-late 90s was an abuse.  Many of the most popular sites were loaded
-with obtrusive branding that made them slow to load and sent the
-user the message: this is our site, not yours.  (There's a physical
-analog in the Intel and Microsoft stickers that come on some
-laptops.)I think the root of the problem was that sites felt they were giving
-something away for free, and till recently a company giving anything
-away for free could be pretty high-handed about it.  Sometimes it
-reached the point of economic sadism: site owners assumed that the
-more pain they caused the user, the more benefit it must be to them.  
-The most dramatic remnant of this model may be at salon.com, where   
-you can read the beginning of a story, but to get the rest you have
-sit through a movie.At Y Combinator we advise all the startups we fund never to lord
-it over users.  Never make users register, unless you need to in
-order to store something for them.  If you do make users register,   
-never make them wait for a confirmation link in an email; in fact,
-don't even ask for their email address unless you need it for some
-reason.  Don't ask them any unnecessary questions.  Never send them
-email unless they explicitly ask for it.  Never frame pages you
-link to, or open them in new windows.  If you have a free version 
-and a pay version, don't make the free version too restricted.  And
-if you find yourself asking "should we allow users to do x?" just 
-answer "yes" whenever you're unsure.  Err on the side of generosity.In How to Start a Startup I advised startups
-never to let anyone fly under them, meaning never to let any other
-company offer a cheaper, easier solution.  Another way to fly low 
-is to give users more power.  Let users do what they want.  If you 
-don't and a competitor does, you're in trouble.iTunes is Web 2.0ish in this sense.  Finally you can buy individual
-songs instead of having to buy whole albums.  The recording industry
-hated the idea and resisted it as long as possible.  But it was
-obvious what users wanted, so Apple flew under the labels.
-[4]
-Though really it might be better to describe iTunes as Web 1.5.     
-Web 2.0 applied to music would probably mean individual bands giving
-away DRMless songs for free.The ultimate way to be nice to users is to give them something for
-free that competitors charge for.  During the 90s a lot of people   
-probably thought we'd have some working system for micropayments     
-by now.  In fact things have gone in the other direction.  The most   
-successful sites are the ones that figure out new ways to give stuff
-away for free.  Craigslist has largely destroyed the classified ad
-sites of the 90s, and OkCupid looks likely to do the same to the
-previous generation of dating sites.Serving web pages is very, very cheap.  If you can make even a   
-fraction of a cent per page view, you can make a profit.  And
-technology for targeting ads continues to improve.  I wouldn't be
-surprised if ten years from now eBay had been supplanted by an      
-ad-supported freeBay (or, more likely, gBay).Odd as it might sound, we tell startups that they should try to
-make as little money as possible.  If you can figure out a way to
-turn a billion dollar industry into a fifty million dollar industry,
-so much the better, if all fifty million go to you.  Though indeed,
-making things cheaper often turns out to generate more money in the
-end, just as automating things often turns out to generate more
-jobs.The ultimate target is Microsoft.  What a bang that balloon is going
-to make when someone pops it by offering a free web-based alternative 
-to MS Office.
-[5]
-Who will?  Google?  They seem to be taking their
-time.  I suspect the pin will be wielded by a couple of 20 year old
-hackers who are too naive to be intimidated by the idea.  (How hard
-can it be?)The Common ThreadAjax, democracy, and not dissing users.  What do they all have in  
-common?  I didn't realize they had anything in common till recently,
-which is one of the reasons I disliked the term "Web 2.0" so much.
-It seemed that it was being used as a label for whatever happened
-to be new—that it didn't predict anything.But there is a common thread.  Web 2.0 means using the web the way
-it's meant to be used.  The "trends" we're seeing now are simply
-the inherent nature of the web emerging from under the broken models
-that got imposed on it during the Bubble.I realized this when I read an  interview with
-Joe Kraus, the co-founder of Excite.
-[6]
-
-  Excite really never got the business model right at all.  We fell 
-  into the classic problem of how when a new medium comes out it
-  adopts the practices, the content, the business models of the old
-  medium—which fails, and then the more appropriate models get
-  figured out.
-
-It may have seemed as if not much was happening during the years
-after the Bubble burst.  But in retrospect, something was happening:
-the web was finding its natural angle of repose.  The democracy 
-component, for example—that's not an innovation, in the sense of
-something someone made happen.  That's what the web naturally tends
-to produce.Ditto for the idea of delivering desktop-like applications over the
-web.  That idea is almost as old as the web.  But the first time    
-around it was co-opted by Sun, and we got Java applets.  Java has
-since been remade into a generic replacement for C++, but in 1996
-the story about Java was that it represented a new model of software.
-Instead of desktop applications, you'd run Java "applets" delivered
-from a server.This plan collapsed under its own weight. Microsoft helped kill it,
-but it would have died anyway.  There was no uptake among hackers.
-When you find PR firms promoting
-something as the next development platform, you can be sure it's
-not.  If it were, you wouldn't need PR firms to tell you, because   
-hackers would already be writing stuff on top of it, the way sites    
-like Busmonster used Google Maps as a
-platform before Google even meant it to be one.The proof that Ajax is the next hot platform is that thousands of  
-hackers have spontaneously started building things on top
-of it.  Mikey likes it.There's another thing all three components of Web 2.0 have in common.
-Here's a clue.  Suppose you approached investors with the following
-idea for a Web 2.0 startup:
-
-  Sites like del.icio.us and flickr allow users to "tag" content
-  with descriptive tokens.  But there is also huge source of
-  implicit tags that they ignore: the text within web links.
-  Moreover, these links represent a social network connecting the   
-  individuals and organizations who created the pages, and by using
-  graph theory we can compute from this network an estimate of the
-  reputation of each member.  We plan to mine the web for these 
-  implicit tags, and use them together with the reputation hierarchy
-  they embody to enhance web searches.
-
-How long do you think it would take them on average to realize that
-it was a description of Google?Google was a pioneer in all three components of Web 2.0: their core
-business sounds crushingly hip when described in Web 2.0 terms, 
-"Don't maltreat users" is a subset of "Don't be evil," and of course
-Google set off the whole Ajax boom with Google Maps.Web 2.0 means using the web as it was meant to be used, and Google
-does.  That's their secret.    They're sailing with the wind, instead of sitting  
-becalmed praying for a business model, like the print media, or   
-trying to tack upwind by suing their customers, like Microsoft and 
-the record labels.
-[7]Google doesn't try to force things to happen their way.  They try   
-to figure out what's going to happen, and arrange to be standing 
-there when it does.  That's the way to approach technology—and 
-as business includes an ever larger technological component, the
-right way to do business.The fact that Google is a "Web 2.0" company shows that, while
-meaningful, the term is also rather bogus.  It's like the word
-"allopathic."  It just means doing things right, and it's a bad   
-sign when you have a special word for that.
-Notes[1]
-From the conference
-site, June 2004: "While the first wave of the Web was closely  
-tied to the browser, the second wave extends applications across    
-the web and enables a new generation of services and business
-opportunities."  To the extent this means anything, it seems to be
-about 
-web-based applications.[2]
-Disclosure: Reddit was funded by 
-Y Combinator.  But although
-I started using it out of loyalty to the home team, I've become a
-genuine addict.  While we're at it, I'm also an investor in
-!MSFT, having sold all my shares earlier this year.[3]
-I'm not against editing. I spend more time editing than
-writing, and I have a group of picky friends who proofread almost
-everything I write.  What I dislike is editing done after the fact  
-by someone else.[4]
-Obvious is an understatement.  Users had been climbing in through  
-the window for years before Apple finally moved the door.[5]
-Hint: the way to create a web-based alternative to Office may
-not be to write every component yourself, but to establish a protocol
-for web-based apps to share a virtual home directory spread across
-multiple servers.  Or it may be to write it all yourself.[6]
-In Jessica Livingston's
-Founders at
-Work.[7]
-Microsoft didn't sue their customers directly, but they seem 
-to have done all they could to help SCO sue them.Thanks to Trevor Blackwell, Sarah Harlin, Jessica Livingston, Peter
-Norvig, Aaron Swartz, and Jeff Weiner for reading drafts of this, and to the
-guys at O'Reilly and Adaptive Path for answering my questions.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/weird.txt

@@ -1,30 +0,0 @@
-August 2021When people say that in their experience all programming languages
-are basically equivalent, they're making a statement not about
-languages but about the kind of programming they've done.99.5% of programming consists of gluing together calls to library
-functions. All popular languages are equally good at this. So one
-can easily spend one's whole career operating in the intersection
-of popular programming languages.But the other .5% of programming is disproportionately interesting.
-If you want to learn what it consists of, the weirdness of weird
-languages is a good clue to follow.Weird languages aren't weird by accident. Not the good ones, at
-least. The weirdness of the good ones usually implies the existence
-of some form of programming that's not just the usual gluing together
-of library calls.A concrete example: Lisp macros. Lisp macros seem weird even to
-many Lisp programmers. They're not only not in the intersection of
-popular languages, but by their nature would be hard to implement
-properly in a language without turning it into a dialect of
-Lisp. And macros are definitely evidence of techniques that go
-beyond glue programming. For example, solving problems by first
-writing a language for problems of that type, and then writing
-your specific application in it. Nor is this all you can do with
-macros; it's just one region in a space of program-manipulating
-techniques that even now is far from fully explored.So if you want to expand your concept of what programming can be,
-one way to do it is by learning weird languages. Pick a language
-that most programmers consider weird but whose median user is smart,
-and then focus on the differences between this language and the
-intersection of popular languages. What can you say in this language
-that would be impossibly inconvenient to say in others? In the
-process of learning how to say things you couldn't previously say,
-you'll probably be learning how to think things you couldn't
-previously think.
-Thanks to Trevor Blackwell, Patrick Collison, Daniel Gackle, Amjad
-Masad, and Robert Morris for reading drafts of this.

recipes/experimental/long-context/H2O/data/PaulGrahamEssays/wisdom.txt

@@ -1,320 +0,0 @@
-February 2007A few days ago I finally figured out something I've wondered about
-for 25 years: the relationship between wisdom and intelligence.
-Anyone can see they're not the same by the number of people who are
-smart, but not very wise.  And yet intelligence and wisdom do seem
-related.  How?What is wisdom?  I'd say it's knowing what to do in a lot of
-situations.  I'm not trying to make a deep point here about the
-true nature of wisdom, just to figure out how we use the word.  A
-wise person is someone who usually knows the right thing to do.And yet isn't being smart also knowing what to do in certain
-situations?  For example, knowing what to do when the teacher tells
-your elementary school class to add all the numbers from 1 to 100?
-[1]Some say wisdom and intelligence apply to different types of
-problems—wisdom to human problems and intelligence to abstract
-ones.  But that isn't true.  Some wisdom has nothing to do with
-people: for example, the wisdom of the engineer who knows certain
-structures are less prone to failure than others.  And certainly
-smart people can find clever solutions to human problems as well
-as abstract ones. 
-[2]Another popular explanation is that wisdom comes from experience
-while intelligence is innate.  But people are not simply wise in
-proportion to how much experience they have.  Other things must
-contribute to wisdom besides experience, and some may be innate: a
-reflective disposition, for example.Neither of the conventional explanations of the difference between
-wisdom and intelligence stands up to scrutiny.  So what is the
-difference?  If we look at how people use the words "wise" and
-"smart," what they seem to mean is different shapes of performance.Curve"Wise" and "smart" are both ways of saying someone knows what to
-do.  The difference is that "wise" means one has a high average
-outcome across all situations, and "smart" means one does spectacularly
-well in a few.  That is, if you had a graph in which the x axis
-represented situations and the y axis the outcome, the graph of the
-wise person would be high overall, and the graph of the smart person
-would have high peaks.The distinction is similar to the rule that one should judge talent
-at its best and character at its worst.  Except you judge intelligence
-at its best, and wisdom by its average.  That's how the two are
-related: they're the two different senses in which the same curve
-can be high.So a wise person knows what to do in most situations, while a smart
-person knows what to do in situations where few others could.  We
-need to add one more qualification: we should ignore cases where
-someone knows what to do because they have inside information. 
-[3]
-But aside from that, I don't think we can get much more specific
-without starting to be mistaken.Nor do we need to.  Simple as it is, this explanation predicts, or
-at least accords with, both of the conventional stories about the
-distinction between wisdom and intelligence.  Human problems are
-the most common type, so being good at solving those is key in
-achieving a high average outcome.   And it seems natural that a
-high average outcome depends mostly on experience, but that dramatic
-peaks can only be achieved by people with certain rare, innate
-qualities; nearly anyone can learn to be a good swimmer, but to be
-an Olympic swimmer you need a certain body type.This explanation also suggests why wisdom is such an elusive concept:
-there's no such thing.  "Wise" means something—that one is
-on average good at making the right choice.  But giving the name
-"wisdom" to the supposed quality that enables one to do that doesn't
-mean such a thing exists.  To the extent "wisdom" means anything,
-it refers to a grab-bag of qualities as various as self-discipline,
-experience, and empathy.  
-[4]Likewise, though "intelligent" means something, we're asking for
-trouble if we insist on looking for a single thing called "intelligence."
-And whatever its components, they're not all innate.  We use the
-word "intelligent" as an indication of ability: a smart person can
-grasp things few others could.  It does seem likely there's some
-inborn predisposition to intelligence (and wisdom too), but this
-predisposition is not itself intelligence.One reason we tend to think of intelligence as inborn is that people
-trying to measure it have concentrated on the aspects of it that
-are most measurable.  A quality that's inborn will obviously be
-more convenient to work with than one that's influenced by experience,
-and thus might vary in the course of a study.  The problem comes
-when we drag the word "intelligence" over onto what they're measuring.
-If they're measuring something inborn, they can't be measuring
-intelligence.  Three year olds aren't smart.   When we describe one
-as smart, it's shorthand for "smarter than other three year olds."SplitPerhaps it's a technicality to point out that a predisposition to
-intelligence is not the same as intelligence.  But it's an important
-technicality, because it reminds us that we can become smarter,
-just as we can become wiser.The alarming thing is that we may have to choose between the two.If wisdom and intelligence are the average and peaks of the same
-curve, then they converge as the number of points on the curve
-decreases.  If there's just one point, they're identical: the average
-and maximum are the same.  But as the number of points increases,
-wisdom and intelligence diverge.  And historically the number of
-points on the curve seems to have been increasing: our ability is
-tested in an ever wider range of situations.In the time of Confucius and Socrates, people seem to have regarded
-wisdom, learning, and intelligence as more closely related than we
-do.  Distinguishing between "wise" and "smart" is a modern habit.
-[5]
-And the reason we do is that they've been diverging.  As knowledge
-gets more specialized, there are more points on the curve, and the
-distinction between the spikes and the average becomes sharper,
-like a digital image rendered with more pixels.One consequence is that some old recipes may have become obsolete.
-At the very least we have to go back and figure out if they were
-really recipes for wisdom or intelligence.  But the really striking
-change, as intelligence and wisdom drift apart, is that we may have
-to decide which we prefer.  We may not be able to optimize for both
-simultaneously.Society seems to have voted for intelligence.  We no longer admire
-the sage—not the way people did two thousand years ago.  Now
-we admire the genius.  Because in fact the distinction we began
-with has a rather brutal converse: just as you can be smart without
-being very wise, you can be wise without being very smart.  That
-doesn't sound especially admirable.  That gets you James Bond, who
-knows what to do in a lot of situations, but has to rely on Q for
-the ones involving math.Intelligence and wisdom are obviously not mutually exclusive.  In
-fact, a high average may help support high peaks.  But there are
-reasons to believe that at some point you have to choose between
-them.  One is the example of very smart people, who are so often
-unwise that in popular culture this now seems to be regarded as the
-rule rather than the exception.  Perhaps the absent-minded professor
-is wise in his way, or wiser than he seems, but he's not wise in
-the way Confucius or Socrates wanted people to be. 
-[6]NewFor both Confucius and Socrates, wisdom, virtue, and happiness were
-necessarily related.  The wise man was someone who knew what the
-right choice was and always made it; to be the right choice, it had
-to be morally right; he was therefore always happy, knowing he'd
-done the best he could.  I can't think of many ancient philosophers
-who would have disagreed with that, so far as it goes."The superior man is always happy; the small man sad," said Confucius.
-[7]Whereas a few years ago I read an interview with a mathematician
-who said that most nights he went to bed discontented, feeling he
-hadn't made enough progress.  
-[8]
-The Chinese and Greek words we
-translate as "happy" didn't mean exactly what we do by it, but
-there's enough overlap that this remark contradicts them.Is the mathematician a small man because he's discontented?  No;
-he's just doing a kind of work that wasn't very common in Confucius's
-day.Human knowledge seems to grow fractally.  Time after time, something
-that seemed a small and uninteresting area—experimental error,
-even—turns out, when examined up close, to have as much in
-it as all knowledge up to that point.  Several of the fractal buds
-that have exploded since ancient times involve inventing and
-discovering new things.  Math, for example, used to be something a
-handful of people did part-time.  Now it's the career of thousands.
-And in work that involves making new things, some old rules don't
-apply.Recently I've spent some time advising people, and there I find the
-ancient rule still works: try to understand the situation as well
-as you can, give the best advice you can based on your experience,
-and then don't worry about it, knowing you did all you could.  But
-I don't have anything like this serenity when I'm writing an essay.
-Then I'm worried.  What if I run out of ideas?  And when I'm writing,
-four nights out of five I go to bed discontented, feeling I didn't
-get enough done.Advising people and writing are fundamentally different types of
-work.  When people come to you with a problem and you have to figure
-out the right thing to do, you don't (usually) have to invent
-anything.  You just weigh the alternatives and try to judge which
-is the prudent choice.  But prudence can't tell me what sentence
-to write next.  The search space is too big.Someone like a judge or a military officer can in much of his work
-be guided by duty, but duty is no guide in making things.  Makers
-depend on something more precarious: inspiration.  And like most
-people who lead a precarious existence, they tend to be worried,
-not contented.  In that respect they're more like the small man of
-Confucius's day, always one bad harvest (or ruler) away from
-starvation. Except instead of being at the mercy of weather and
-officials, they're at the mercy of their own imagination.LimitsTo me it was a relief just to realize it might be ok to be discontented.
-The idea that a successful person should be happy has thousands of
-years of momentum behind it.  If I was any good, why didn't I have
-the easy confidence winners are supposed to have?  But that, I now
-believe, is like a runner asking "If I'm such a good athlete, why
-do I feel so tired?" Good runners still get tired; they just get
-tired at higher speeds.People whose work is to invent or discover things are in the same
-position as the runner.  There's no way for them to do the best
-they can, because there's no limit to what they could do.  The
-closest you can come is to compare yourself to other people.  But
-the better you do, the less this matters.  An undergrad who gets
-something published feels like a star.  But for someone at the top
-of the field, what's the test of doing well?  Runners can at least
-compare themselves to others doing exactly the same thing; if you
-win an Olympic gold medal, you can be fairly content, even if you
-think you could have run a bit faster.  But what is a novelist to
-do?Whereas if you're doing the kind of work in which problems are
-presented to you and you have to choose between several alternatives,
-there's an upper bound on your performance: choosing the best every
-time.  In ancient societies, nearly all work seems to have been of
-this type.  The peasant had to decide whether a garment was worth
-mending, and the king whether or not to invade his neighbor, but
-neither was expected to invent anything.  In principle they could
-have; the king could have invented firearms, then invaded his
-neighbor.  But in practice innovations were so rare that they weren't
-expected of you, any more than goalkeepers are expected to score
-goals. 
-[9]
-In practice, it seemed as if there was a correct decision
-in every situation, and if you made it you'd done your job perfectly,
-just as a goalkeeper who prevents the other team from scoring is
-considered to have played a perfect game.In this world, wisdom seemed paramount.  
-[10]
-Even now, most people
-do work in which problems are put before them and they have to
-choose the best alternative.  But as knowledge has grown more
-specialized, there are more and more types of work in which people
-have to make up new things, and in which performance is therefore
-unbounded.  Intelligence has become increasingly important relative
-to wisdom because there is more room for spikes.RecipesAnother sign we may have to choose between intelligence and wisdom
-is how different their recipes are.  Wisdom seems to come largely
-from curing childish qualities, and intelligence largely from
-cultivating them.Recipes for wisdom, particularly ancient ones, tend to have a
-remedial character.  To achieve wisdom one must cut away all the
-debris that fills one's head on emergence from childhood, leaving
-only the important stuff.  Both self-control and experience have
-this effect: to eliminate the random biases that come from your own
-nature and from the circumstances of your upbringing respectively.
-That's not all wisdom is, but it's a large part of it.  Much of
-what's in the sage's head is also in the head of every twelve year
-old.  The difference is that in the head of the twelve year old
-it's mixed together with a lot of random junk.The path to intelligence seems to be through working on hard problems.
-You develop intelligence as you might develop muscles, through
-exercise.  But there can't be too much compulsion here.  No amount
-of discipline can replace genuine curiosity.  So cultivating
-intelligence seems to be a matter of identifying some bias in one's
-character—some tendency to be interested in certain types of
-things—and nurturing it.  Instead of obliterating your
-idiosyncrasies in an effort to make yourself a neutral vessel for
-the truth, you select one and try to grow it from a seedling into
-a tree.The wise are all much alike in their wisdom, but very smart people
-tend to be smart in distinctive ways.Most of our educational traditions aim at wisdom. So perhaps one
-reason schools work badly is that they're trying to make intelligence
-using recipes for wisdom.  Most recipes for wisdom have an element
-of subjection.  At the very least, you're supposed to do what the
-teacher says.  The more extreme recipes aim to break down your
-individuality the way basic training does.  But that's not the route
-to intelligence.  Whereas wisdom comes through humility, it may
-actually help, in cultivating intelligence, to have a mistakenly
-high opinion of your abilities, because that encourages you to keep
-working.  Ideally till you realize how mistaken you were.(The reason it's hard to learn new skills late in life is not just
-that one's brain is less malleable.  Another probably even worse
-obstacle is that one has higher standards.)I realize we're on dangerous ground here.  I'm not proposing the
-primary goal of education should be to increase students' "self-esteem."
-That just breeds laziness.  And in any case, it doesn't really fool
-the kids, not the smart ones.  They can tell at a young age that a
-contest where everyone wins is a fraud.A teacher has to walk a narrow path: you want to encourage kids to
-come up with things on their own, but you can't simply applaud
-everything they produce.  You have to be a good audience: appreciative,
-but not too easily impressed.  And that's a lot of work.  You have
-to have a good enough grasp of kids' capacities at different ages
-to know when to be surprised.That's the opposite of traditional recipes for education.  Traditionally
-the student is the audience, not the teacher; the student's job is
-not to invent, but to absorb some prescribed body of material.  (The
-use of the term "recitation" for sections in some colleges is a
-fossil of this.) The problem with these old traditions is that
-they're too much influenced by recipes for wisdom.DifferentI deliberately gave this essay a provocative title; of course it's
-worth being wise.  But I think it's important to understand the
-relationship between intelligence and wisdom, and particularly what
-seems to be the growing gap between them.  That way we can avoid
-applying rules and standards to intelligence that are really meant
-for wisdom.  These two senses of "knowing what to do" are more
-different than most people realize.  The path to wisdom is through
-discipline, and the path to intelligence through carefully selected
-self-indulgence.  Wisdom is universal, and intelligence idiosyncratic.
-And while wisdom yields calmness, intelligence much of the time
-leads to discontentment.That's particularly worth remembering.  A physicist friend recently
-told me half his department was on Prozac.  Perhaps if we acknowledge
-that some amount of frustration is inevitable in certain kinds
-of work, we can mitigate its effects.  Perhaps we can box it up and
-put it away some of the time, instead of letting it flow together
-with everyday sadness to produce what seems an alarmingly large
-pool.  At the very least, we can avoid being discontented about
-being discontented.If you feel exhausted, it's not necessarily because there's something
-wrong with you.  Maybe you're just running fast.Notes[1]
-Gauss was supposedly asked this when he was 10.  Instead of
-laboriously adding together the numbers like the other students,
-he saw that they consisted of 50 pairs that each summed to 101 (100
-+ 1, 99 + 2, etc), and that he could just multiply 101 by 50 to get
-the answer, 5050.[2]
-A variant is that intelligence is the ability to solve problems,
-and wisdom the judgement to know how to use those solutions.   But
-while this is certainly an important relationship between wisdom
-and intelligence, it's not the distinction between them.  Wisdom
-is useful in solving problems too, and intelligence can help in
-deciding what to do with the solutions.[3]
-In judging both intelligence and wisdom we have to factor out
-some knowledge. People who know the combination of a safe will be
-better at opening it than people who don't, but no one would say
-that was a test of intelligence or wisdom.But knowledge overlaps with wisdom and probably also intelligence.
-A knowledge of human nature is certainly part of wisdom.  So where
-do we draw the line?Perhaps the solution is to discount knowledge that at some point
-has a sharp drop in utility.  For example, understanding French
-will help you in a large number of situations, but its value drops
-sharply as soon as no one else involved knows French.  Whereas the
-value of understanding vanity would decline more gradually.The knowledge whose utility drops sharply is the kind that has
-little relation to other knowledge.  This includes mere conventions,
-like languages and safe combinations, and also what we'd call
-"random" facts, like movie stars' birthdays, or how to distinguish
-1956 from 1957 Studebakers.[4]
-People seeking some single thing called "wisdom" have been
-fooled by grammar.  Wisdom is just knowing the right thing to do,
-and there are a hundred and one different qualities that help in
-that.  Some, like selflessness, might come from meditating in an
-empty room, and others, like a knowledge of human nature, might
-come from going to drunken parties.Perhaps realizing this will help dispel the cloud of semi-sacred
-mystery that surrounds wisdom in so many people's eyes.  The mystery
-comes mostly from looking for something that doesn't exist.  And
-the reason there have historically been so many different schools
-of thought about how to achieve wisdom is that they've focused on
-different components of it.When I use the word "wisdom" in this essay, I mean no more than
-whatever collection of qualities helps people make the right choice
-in a wide variety of situations.[5]
-Even in English, our sense of the word "intelligence" is
-surprisingly recent.  Predecessors like "understanding" seem to
-have had a broader meaning.[6]
-There is of course some uncertainty about how closely the remarks
-attributed to Confucius and Socrates resemble their actual opinions.
-I'm using these names as we use the name "Homer," to mean the
-hypothetical people who said the things attributed to them.[7]
-Analects VII:36, Fung trans.Some translators use "calm" instead of "happy."  One source of
-difficulty here is that present-day English speakers have a different
-idea of happiness from many older societies.  Every language probably
-has a word meaning "how one feels when things are going well," but
-different cultures react differently when things go well.  We react
-like children, with smiles and laughter.  But in a more reserved
-society, or in one where life was tougher, the reaction might be a
-quiet contentment.[8]
-It may have been Andrew Wiles, but I'm not sure.  If anyone
-remembers such an interview, I'd appreciate hearing from you.[9]
-Confucius claimed proudly that he had never invented
-anything—that he had simply passed on an accurate account of
-ancient traditions.  [Analects VII:1] It's hard for us now to
-appreciate how important a duty it must have been in preliterate
-societies to remember and pass on the group's accumulated knowledge.
-Even in Confucius's time it still seems to have been the first duty
-of the scholar.[10]
-The bias toward wisdom in ancient philosophy may be exaggerated
-by the fact that, in both Greece and China, many of the first
-philosophers (including Confucius and Plato) saw themselves as
-teachers of administrators, and so thought disproportionately about
-such matters.  The few people who did invent things, like storytellers,
-must have seemed an outlying data point that could be ignored.Thanks to Trevor Blackwell, Sarah Harlin, Jessica Livingston,
-and Robert Morris for reading drafts of this.

recipes/experimental/long-context/H2O/run_needle_haystack_test.py

@@ -32,6 +32,7 @@ if __name__ == '__main__':
     parser.add_argument("--enable_h2o_generation", action='store_true')
     parser.add_argument("--num_heavy_hitter_tokens", type=int, default=128)
     parser.add_argument("--num_window_length", type=int, default=256)
+    parser.add_argument("--num_chunk_size", type=int, default=2048)
 
     parser.add_argument("--enable_position_rolling", action='store_true')
 
@@ -75,6 +76,10 @@ if __name__ == '__main__':
 
             input = tokenizer(prompt, truncation=False, return_tensors="pt").to(model.device)
             context_length = input.input_ids.shape[-1]
+            if context_length > args.num_chunk_size:
+                # truncate the context to the maximum chunk size
+                input = {k: v[:, -args.num_chunk_size:] for k, v in input.items()}
+
             output = model.generate(
                 **input,
                 max_new_tokens=args.max_new_tokens,

recipes/experimental/long-context/H2O/src/h20.sh

@@ -0,0 +1,7 @@
+CUDA_VISIBLE_DEVICES=2 python -u run_needle_haystack_test.py \
+--input-path data/needle_test/Huggingface \
+--output-path needle_test_results/huggingface/llama-3-8b-instruct-h2o-4096/ \
+--model-name meta-llama/Meta-Llama-3-8B-Instruct \
+--enable_h2o_generation \
+--num_window_length 4096 \
+--num_heavy_hitter_tokens 2048