Do I need to keep my phone on to be connected ?

Yes you need to keep your phone connected to keep the connection active.