Sure! Remove the read permissions of that page. You'll find the Permissions by going to edit mode in that page and pressing Permissions button at the top of page. It's also available through the Sitemap.
If that's the only way to disable a page C5 should at least signify with the page image icon that the page is disabled. I think it would be better to have a enable/disable option for the pages so you can leave the permission settings alone and have an X on folder icon to signify it's been disabled.