I recommend to start out with Heroku because it's free and the documentation is very clear on how to deploy the app. You don't need to configure and maintain the server. Try Heroku if you want to completely focus on developing Rails app.
However, as the app scale up, you will soon find out that Heroku become more and more expensive and sometimes it doesn't give you much flexibility. One big disadvantage of Heroku free plan is your app will "sleep" if no one use it. As the result, the first user who visit your app will experience a long wait for the server to kick up.
You can have the choice of learning Unix command and setup your own server on AWS or using managed hosting service such as Engine Yard, Rackspace.
I used AWS because it has one year free tier, enough for me to play around with S3, EC2, RDS etc. This blog shows the steps pretty clearly: http://dennissuratna.com/rails-deployment-aws1/
Spoiler Alert:
If you want to use AWS and avoid all of the hassle, try cloud66.com. I've tried it for 2 weeks now and couldn't recommend more of it. It gives me the flexibility of AWS and the convenience of the prebuilt Cloud66 toolbelt(similar to Heroku CLI).