In an art market that commands more than $150 million for an Amedeo Modigliani, or more than $90 million for a David Hockney—a living artist—how can even the most well-endowed museum compete?