Is there a way to set advanced permissions in the template for which blocks the user are allowed to use?

OR

Is there a way to set which user or group has permission to use which blocks?

It would be very nice if one could do this so that the users that are allowed to edit text doesn't get their hands on any of the other blockes and ruin the page. I already have enabled the advanced permissions but I can only find that you can set permission in edit mode on each block, and for a whole site that is a bit tedious.

Best regards
Fredrik