The Explore/Exploit-dilemma

Every time I go to an Indian restaurant, I'm faced with a dilemma: Should I order my favorite Tikka Massala or try something new from the menu? Many decisions in our daily lives boil down to such a choice between new and familiar. Should we have lunch at the regular place or try the newcomer on the corner? Should we choose a pizza Capricciosa or try the special? Should we take a trip to a new destination, or book another charter to Mallorca? This dilemma is known as explore/exploit and has occupied mathematicians and computer scientists since the 1950s. Not only is the dilemma notoriously difficult to solve; it also has several surprising applications.