I have a mysterious component covered with yellow "glue" or something. It has very hard surface, but it seems its been poured into the plastic "cage". The shiny cover on the top is some kind of paper, I can rip it off.
Any idea how to remove this goo from the PCB? (Without damaging the components underneath)
That's a potted circuit. The shiny paper on top likely forms some sort of EMI shield to reduce interference it may cause or receive.
Potting is usually an epoxy, which usually cannot be removed either chemically (dissolution) or thermally (melting it), so you are left with mechanical (chip away at it). Some are soft, and while they stick fairly well, with a bit of time can be removed. If it's a hard variant, it may well be impossible, and if you really want to figure out what's under there, using an X-ray inspection machine would be a good idea.
Some rare epoxies are susceptible to attack by extremely aggressive organic solvents (dichloromethane, xylene, etc.), but you may well destroy the board in attempt to remove it. If you have access to some chemicals and can chip off small samples of the epoxy, you could give it a whirl.