My opinion is this: the cost involved does not justify the ends. Take it for what it's worth.
And that's a valid opinion.. hsr has to have enough of a benefit, either through convenience or cost, to compete with driving or flying...
Swish has made some good points, part of the problem is that these cities, once you get to them, are not convenient for people who rode in on the train. Are you going to ride the train in and then rent a car to get out to the stadium or to the suburbs where your family lives?
American people, and this is just collectively as a culture, are NOT going to ride a train from DC to NY, switch to a subway train to get closer, then switch to a bus to get to their final destination... they just aren't. People who work in downtown DC who are going to a meeting in downtown NYC will ride the train because it's convenient but not a lot of others will..
Let's look at the logistics of a person in Rockville, MD, who wants to go to a Wizards game at the Barclays Center..
Drive from your home in Rockville to DC Metro Station and park, time 30 minutes. Take metro from Rockville station to Amtrak station downtown, time 40 minutes. Amtrak from DC to NY, time 3 hours. NY subway from Penn Station to Barclays Center 50 minutes... total time (and this is assuming zero time at changovers) is almost 5 hours... Round trip cost, $220/person
Time to drive from Rockville, MD to Barclays Center with minimal traffic, 4 hours and 15 minutes... Round trip cost, $80/car load
So what would it take to make it a viable option? I don't know.