i found out my mystery noise. my brother said it was a small exhaust leak and that it didn't make the noise when in park or reverse because of the stall converter.

anyway is this something i should be seriously worried about?

 

If you have an exhaust leak, changing the transmission gear should not change the leak. O2 readings can be influenced by leaks in the exhaust, so I would fix it, if I were you.