I’ve just used arrow keys on the keyboard for that type of use. I don’t see mouse usage unless a trackpad type of input device becomes standard.
Were those other "Android TV" boxes running the actual Android TV platform, or were they running regular Android? A lot of the cheap Android pirate TV boxes run plain Android and not the TV platform. There currently are no browsers available on the TV platform.