I think you need indeed a more flexible bond, super glue is indeed too hard.
Maybe a small drop of epoxy glue spread sparingly and evenly?

Problem with "epoxy 2 component" glue is that it is expensive for small quantities and if you want to use it wisely you are forced to find a whole range of things to glue in one go in order to spend it wisely.

If the buttons are getting white: stop using finger nails and keep those nails short?

Or maybe try a less destructive means: some square of high end double sided tape?