Your Database is stored in the mysql database. You need to grab a database dump using your hosting services sql admin interface whatever that may be. eg. phpmyadmin.

Your web root is every file in your concrete installation stored server side. Everything that is in your public document root.

Since I don't have a public directory in the directory where Concrete and my site are sitting, I'm not 100% clear what I'm supposed to back up. If I just back up the entire folder will that work?

you mainly need to backup the /concrete directory, and the database, if you don't know how to backup the database ask your host