Hi laurent.
If you just leave the connection to the phone's system, it will simply connect to any BT signal available. You can give the App specific IDs to connect with, but if you don't scan for devices you do not know whether or not they are in range or even available. Less of an issue if all devices are BLE but in your scenario, problem.
If this set up is only for personal use (you are not going commercial with it), then corners can be cut, but you can't get away from the fact that a device has to be found before it can be connected to.
What about approaching your design in a different way. The esp32 has a constant power supply? If so, that can scan 24/7, right? That would make the need to use the car or bike's BT redundant. Just have a connection between the phone (App) and the esp32. If you approach the portal in your car, it will open if the esp32 connects with your Phone App.
Edit: By the way, any Bluetooth solution to automation in this sort of scenario will drain the battery. The only way to avoid that is to switch the device BT on when nearing the portal, or do not use Bluetooth.